1 # copyright 2003-2014 LOGILAB S.A. (Paris, FRANCE), all rights reserved. |
|
2 # contact http://www.logilab.fr/ -- mailto:contact@logilab.fr |
|
3 # |
|
4 # This file is part of CubicWeb. |
|
5 # |
|
6 # CubicWeb is free software: you can redistribute it and/or modify it under the |
|
7 # terms of the GNU Lesser General Public License as published by the Free |
|
8 # Software Foundation, either version 2.1 of the License, or (at your option) |
|
9 # any later version. |
|
10 # |
|
11 # CubicWeb is distributed in the hope that it will be useful, but WITHOUT |
|
12 # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS |
|
13 # FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more |
|
14 # details. |
|
15 # |
|
16 # You should have received a copy of the GNU Lesser General Public License along |
|
17 # with CubicWeb. If not, see <http://www.gnu.org/licenses/>. |
|
18 """ |
|
19 Generalities |
|
20 ------------ |
|
21 |
|
22 Paraphrasing the `emacs`_ documentation, let us say that hooks are an important |
|
23 mechanism for customizing an application. A hook is basically a list of |
|
24 functions to be called on some well-defined occasion (this is called `running |
|
25 the hook`). |
|
26 |
|
27 .. _`emacs`: http://www.gnu.org/software/emacs/manual/html_node/emacs/Hooks.html |
|
28 |
|
29 Hooks |
|
30 ~~~~~ |
|
31 |
|
32 In |cubicweb|, hooks are subclasses of the :class:`~cubicweb.server.hook.Hook` |
|
33 class. They are selected over a set of pre-defined `events` (and possibly more |
|
34 conditions, hooks being selectable appobjects like views and components). They |
|
35 should implement a :meth:`~cubicweb.server.hook.Hook.__call__` method that will |
|
36 be called when the hook is triggered. |
|
37 |
|
38 There are two families of events: data events (before / after any individual |
|
39 update of an entity / or a relation in the repository) and server events (such |
|
40 as server startup or shutdown). In a typical application, most of the hooks are |
|
41 defined over data events. |
|
42 |
|
43 Also, some :class:`~cubicweb.server.hook.Operation` may be registered by hooks, |
|
44 which will be fired when the transaction is commited or rolled back. |
|
45 |
|
46 The purpose of data event hooks is usually to complement the data model as |
|
47 defined in the schema, which is static by nature and only provide a restricted |
|
48 builtin set of dynamic constraints, with dynamic or value driven behaviours. |
|
49 For instance they can serve the following purposes: |
|
50 |
|
51 * enforcing constraints that the static schema cannot express (spanning several |
|
52 entities/relations, exotic value ranges and cardinalities, etc.) |
|
53 |
|
54 * implement computed attributes |
|
55 |
|
56 It is functionally equivalent to a `database trigger`_, except that database |
|
57 triggers definition languages are not standardized, hence not portable (for |
|
58 instance, PL/SQL works with Oracle and PostgreSQL but not SqlServer nor Sqlite). |
|
59 |
|
60 .. _`database trigger`: http://en.wikipedia.org/wiki/Database_trigger |
|
61 |
|
62 |
|
63 .. hint:: |
|
64 |
|
65 It is a good practice to write unit tests for each hook. See an example in |
|
66 :ref:`hook_test` |
|
67 |
|
68 Operations |
|
69 ~~~~~~~~~~ |
|
70 |
|
71 Operations are subclasses of the :class:`~cubicweb.server.hook.Operation` class |
|
72 that may be created by hooks and scheduled to happen on `precommit`, |
|
73 `postcommit` or `rollback` event (i.e. respectivly before/after a commit or |
|
74 before a rollback of a transaction). |
|
75 |
|
76 Hooks are being fired immediately on data operations, and it is sometime |
|
77 necessary to delay the actual work down to a time where we can expect all |
|
78 information to be there, or when all other hooks have run (though take case |
|
79 since operations may themselves trigger hooks). Also while the order of |
|
80 execution of hooks is data dependant (and thus hard to predict), it is possible |
|
81 to force an order on operations. |
|
82 |
|
83 So, for such case where you may miss some information that may be set later in |
|
84 the transaction, you should instantiate an operation in the hook. |
|
85 |
|
86 Operations may be used to: |
|
87 |
|
88 * implements a validation check which needs that all relations be already set on |
|
89 an entity |
|
90 |
|
91 * process various side effects associated with a transaction such as filesystem |
|
92 udpates, mail notifications, etc. |
|
93 |
|
94 |
|
95 Events |
|
96 ------ |
|
97 |
|
98 Hooks are mostly defined and used to handle `dataflow`_ operations. It |
|
99 means as data gets in (entities added, updated, relations set or |
|
100 unset), specific events are issued and the Hooks matching these events |
|
101 are called. |
|
102 |
|
103 You can get the event that triggered a hook by accessing its `event` |
|
104 attribute. |
|
105 |
|
106 .. _`dataflow`: http://en.wikipedia.org/wiki/Dataflow |
|
107 |
|
108 |
|
109 Entity modification related events |
|
110 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
|
111 |
|
112 When called for one of these events, hook will have an `entity` attribute |
|
113 containing the entity instance. |
|
114 |
|
115 - `before_add_entity`, `before_update_entity`: |
|
116 |
|
117 On those events, you can access the modified attributes of the entity using |
|
118 the `entity.cw_edited` dictionary. The values can be modified and the old |
|
119 values can be retrieved. |
|
120 |
|
121 If you modify the `entity.cw_edited` dictionary in the hook, that is before |
|
122 the database operations take place, you will avoid the need to process a whole |
|
123 new rql query and the underlying backend query (eg usually sql) will contain |
|
124 the modified data. For example: |
|
125 |
|
126 .. sourcecode:: python |
|
127 |
|
128 self.entity.cw_edited['age'] = 42 |
|
129 |
|
130 will modify the age before it is written to the backend storage. |
|
131 |
|
132 Similarly, removing an attribute from `cw_edited` will cancel its |
|
133 modification: |
|
134 |
|
135 .. sourcecode:: python |
|
136 |
|
137 del self.entity.cw_edited['age'] |
|
138 |
|
139 On a `before_update_entity` event, you can access the old and new values: |
|
140 |
|
141 .. sourcecode:: python |
|
142 |
|
143 old, new = entity.cw_edited.oldnewvalue('age') |
|
144 |
|
145 - `after_add_entity`, `after_update_entity` |
|
146 |
|
147 On those events, you can get the list of attributes that were modified using |
|
148 the `entity.cw_edited` dictionary, but you can not modify it or get the old |
|
149 value of an attribute. |
|
150 |
|
151 - `before_delete_entity`, `after_delete_entity` |
|
152 |
|
153 On those events, the entity has no `cw_edited` dictionary. |
|
154 |
|
155 .. note:: `self.entity.cw_set(age=42)` will set the `age` attribute to |
|
156 42. But to do so, it will generate a rql query that will have to be processed, |
|
157 hence may trigger some hooks, etc. This could lead to infinitely looping hooks. |
|
158 |
|
159 Relation modification related events |
|
160 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
|
161 |
|
162 When called for one of these events, hook will have `eidfrom`, `rtype`, `eidto` |
|
163 attributes containing respectively the eid of the subject entity, the relation |
|
164 type and the eid of the object entity. |
|
165 |
|
166 * `before_add_relation`, `before_delete_relation` |
|
167 |
|
168 On those events, you can still get the original relation by issuing a rql query. |
|
169 |
|
170 * `after_add_relation`, `after_delete_relation` |
|
171 |
|
172 Specific selectors are shipped for these kinds of events, see in particular |
|
173 :class:`~cubicweb.server.hook.match_rtype`. |
|
174 |
|
175 Also note that relations can be added or deleted, but not updated. |
|
176 |
|
177 Non data events |
|
178 ~~~~~~~~~~~~~~~ |
|
179 |
|
180 Hooks called on server start/maintenance/stop event (e.g. |
|
181 `server_startup`, `server_maintenance`, `before_server_shutdown`, |
|
182 `server_shutdown`) have a `repo` attribute, but *their `_cw` attribute |
|
183 is None*. The `server_startup` is called on regular startup, while |
|
184 `server_maintenance` is called on cubicweb-ctl upgrade or shell |
|
185 commands. `server_shutdown` is called anyway but connections to the |
|
186 native source is impossible; `before_server_shutdown` handles that. |
|
187 |
|
188 Hooks called on backup/restore event (eg `server_backup`, |
|
189 `server_restore`) have a `repo` and a `timestamp` attributes, but |
|
190 *their `_cw` attribute is None*. |
|
191 |
|
192 Hooks called on session event (eg `session_open`, `session_close`) have no |
|
193 special attribute. |
|
194 |
|
195 |
|
196 API |
|
197 --- |
|
198 |
|
199 Hooks control |
|
200 ~~~~~~~~~~~~~ |
|
201 |
|
202 It is sometimes convenient to explicitly enable or disable some hooks. For |
|
203 instance if you want to disable some integrity checking hook. This can be |
|
204 controlled more finely through the `category` class attribute, which is a string |
|
205 giving a category name. One can then uses the |
|
206 :meth:`~cubicweb.server.session.Connection.deny_all_hooks_but` and |
|
207 :meth:`~cubicweb.server.session.Connection.allow_all_hooks_but` context managers to |
|
208 explicitly enable or disable some categories. |
|
209 |
|
210 The existing categories are: |
|
211 |
|
212 * ``security``, security checking hooks |
|
213 |
|
214 * ``worfklow``, workflow handling hooks |
|
215 |
|
216 * ``metadata``, hooks setting meta-data on newly created entities |
|
217 |
|
218 * ``notification``, email notification hooks |
|
219 |
|
220 * ``integrity``, data integrity checking hooks |
|
221 |
|
222 * ``activeintegrity``, data integrity consistency hooks, that you should **never** |
|
223 want to disable |
|
224 |
|
225 * ``syncsession``, hooks synchronizing existing sessions |
|
226 |
|
227 * ``syncschema``, hooks synchronizing instance schema (including the physical database) |
|
228 |
|
229 * ``email``, email address handling hooks |
|
230 |
|
231 * ``bookmark``, bookmark entities handling hooks |
|
232 |
|
233 |
|
234 Nothing precludes one to invent new categories and use existing mechanisms to |
|
235 filter them in or out. |
|
236 |
|
237 |
|
238 Hooks specific predicates |
|
239 ~~~~~~~~~~~~~~~~~~~~~~~~~ |
|
240 .. autoclass:: cubicweb.server.hook.match_rtype |
|
241 .. autoclass:: cubicweb.server.hook.match_rtype_sets |
|
242 |
|
243 |
|
244 Hooks and operations classes |
|
245 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
|
246 .. autoclass:: cubicweb.server.hook.Hook |
|
247 .. autoclass:: cubicweb.server.hook.Operation |
|
248 .. autoclass:: cubicweb.server.hook.LateOperation |
|
249 .. autoclass:: cubicweb.server.hook.DataOperationMixIn |
|
250 """ |
|
251 from __future__ import print_function |
|
252 |
|
253 __docformat__ = "restructuredtext en" |
|
254 |
|
255 from warnings import warn |
|
256 from logging import getLogger |
|
257 from itertools import chain |
|
258 |
|
259 from logilab.common.decorators import classproperty, cached |
|
260 from logilab.common.deprecation import deprecated, class_renamed |
|
261 from logilab.common.logging_ext import set_log_methods |
|
262 from logilab.common.registry import (NotPredicate, OrPredicate, |
|
263 objectify_predicate) |
|
264 |
|
265 from cubicweb import RegistryNotFound, server |
|
266 from cubicweb.cwvreg import CWRegistry, CWRegistryStore |
|
267 from cubicweb.predicates import ExpectedValuePredicate, is_instance |
|
268 from cubicweb.appobject import AppObject |
|
269 |
|
270 ENTITIES_HOOKS = set(('before_add_entity', 'after_add_entity', |
|
271 'before_update_entity', 'after_update_entity', |
|
272 'before_delete_entity', 'after_delete_entity')) |
|
273 RELATIONS_HOOKS = set(('before_add_relation', 'after_add_relation' , |
|
274 'before_delete_relation','after_delete_relation')) |
|
275 SYSTEM_HOOKS = set(('server_backup', 'server_restore', |
|
276 'server_startup', 'server_maintenance', |
|
277 'server_shutdown', 'before_server_shutdown', |
|
278 'session_open', 'session_close')) |
|
279 ALL_HOOKS = ENTITIES_HOOKS | RELATIONS_HOOKS | SYSTEM_HOOKS |
|
280 |
|
281 def _iter_kwargs(entities, eids_from_to, kwargs): |
|
282 if not entities and not eids_from_to: |
|
283 yield kwargs |
|
284 elif entities: |
|
285 for entity in entities: |
|
286 kwargs['entity'] = entity |
|
287 yield kwargs |
|
288 else: |
|
289 for subject, object in eids_from_to: |
|
290 kwargs.update({'eidfrom': subject, 'eidto': object}) |
|
291 yield kwargs |
|
292 |
|
293 |
|
294 class HooksRegistry(CWRegistry): |
|
295 |
|
296 def register(self, obj, **kwargs): |
|
297 obj.check_events() |
|
298 super(HooksRegistry, self).register(obj, **kwargs) |
|
299 |
|
300 def call_hooks(self, event, cnx=None, **kwargs): |
|
301 """call `event` hooks for an entity or a list of entities (passed |
|
302 respectively as the `entity` or ``entities`` keyword argument). |
|
303 """ |
|
304 kwargs['event'] = event |
|
305 if cnx is None: # True for events such as server_start |
|
306 for hook in sorted(self.possible_objects(cnx, **kwargs), |
|
307 key=lambda x: x.order): |
|
308 hook() |
|
309 else: |
|
310 if 'entities' in kwargs: |
|
311 assert 'entity' not in kwargs, \ |
|
312 'can\'t pass "entities" and "entity" arguments simultaneously' |
|
313 assert 'eids_from_to' not in kwargs, \ |
|
314 'can\'t pass "entities" and "eids_from_to" arguments simultaneously' |
|
315 entities = kwargs.pop('entities') |
|
316 eids_from_to = [] |
|
317 elif 'eids_from_to' in kwargs: |
|
318 entities = [] |
|
319 eids_from_to = kwargs.pop('eids_from_to') |
|
320 else: |
|
321 entities = [] |
|
322 eids_from_to = [] |
|
323 pruned = self.get_pruned_hooks(cnx, event, |
|
324 entities, eids_from_to, kwargs) |
|
325 |
|
326 # by default, hooks are executed with security turned off |
|
327 with cnx.security_enabled(read=False): |
|
328 for _kwargs in _iter_kwargs(entities, eids_from_to, kwargs): |
|
329 hooks = sorted(self.filtered_possible_objects(pruned, cnx, **_kwargs), |
|
330 key=lambda x: x.order) |
|
331 debug = server.DEBUG & server.DBG_HOOKS |
|
332 with cnx.security_enabled(write=False): |
|
333 with cnx.running_hooks_ops(): |
|
334 for hook in hooks: |
|
335 if debug: |
|
336 print(event, _kwargs, hook) |
|
337 hook() |
|
338 |
|
339 def get_pruned_hooks(self, cnx, event, entities, eids_from_to, kwargs): |
|
340 """return a set of hooks that should not be considered by filtered_possible objects |
|
341 |
|
342 the idea is to make a first pass over all the hooks in the |
|
343 registry and to mark put some of them in a pruned list. The |
|
344 pruned hooks are the one which: |
|
345 |
|
346 * are disabled at the connection level |
|
347 |
|
348 * have a selector containing a :class:`match_rtype` or an |
|
349 :class:`is_instance` predicate which does not match the rtype / etype |
|
350 of the relations / entities for which we are calling the hooks. This |
|
351 works because the repository calls the hooks grouped by rtype or by |
|
352 etype when using the entities or eids_to_from keyword arguments |
|
353 |
|
354 Only hooks with a simple predicate or an AndPredicate of simple |
|
355 predicates are considered for disabling. |
|
356 |
|
357 """ |
|
358 if 'entity' in kwargs: |
|
359 entities = [kwargs['entity']] |
|
360 if len(entities): |
|
361 look_for_selector = is_instance |
|
362 etype = entities[0].__regid__ |
|
363 elif 'rtype' in kwargs: |
|
364 look_for_selector = match_rtype |
|
365 etype = None |
|
366 else: # nothing to prune, how did we get there ??? |
|
367 return set() |
|
368 cache_key = (event, kwargs.get('rtype'), etype) |
|
369 pruned = cnx.pruned_hooks_cache.get(cache_key) |
|
370 if pruned is not None: |
|
371 return pruned |
|
372 pruned = set() |
|
373 cnx.pruned_hooks_cache[cache_key] = pruned |
|
374 if look_for_selector is not None: |
|
375 for id, hooks in self.items(): |
|
376 for hook in hooks: |
|
377 enabled_cat, main_filter = hook.filterable_selectors() |
|
378 if enabled_cat is not None: |
|
379 if not enabled_cat(hook, cnx): |
|
380 pruned.add(hook) |
|
381 continue |
|
382 if main_filter is not None: |
|
383 if isinstance(main_filter, match_rtype) and \ |
|
384 (main_filter.frometypes is not None or \ |
|
385 main_filter.toetypes is not None): |
|
386 continue |
|
387 first_kwargs = next(_iter_kwargs(entities, eids_from_to, kwargs)) |
|
388 if not main_filter(hook, cnx, **first_kwargs): |
|
389 pruned.add(hook) |
|
390 return pruned |
|
391 |
|
392 |
|
393 def filtered_possible_objects(self, pruned, *args, **kwargs): |
|
394 for appobjects in self.values(): |
|
395 if pruned: |
|
396 filtered_objects = [obj for obj in appobjects if obj not in pruned] |
|
397 if not filtered_objects: |
|
398 continue |
|
399 else: |
|
400 filtered_objects = appobjects |
|
401 obj = self._select_best(filtered_objects, |
|
402 *args, **kwargs) |
|
403 if obj is None: |
|
404 continue |
|
405 yield obj |
|
406 |
|
407 class HooksManager(object): |
|
408 def __init__(self, vreg): |
|
409 self.vreg = vreg |
|
410 |
|
411 def call_hooks(self, event, cnx=None, **kwargs): |
|
412 try: |
|
413 registry = self.vreg['%s_hooks' % event] |
|
414 except RegistryNotFound: |
|
415 return # no hooks for this event |
|
416 registry.call_hooks(event, cnx, **kwargs) |
|
417 |
|
418 |
|
419 for event in ALL_HOOKS: |
|
420 CWRegistryStore.REGISTRY_FACTORY['%s_hooks' % event] = HooksRegistry |
|
421 |
|
422 |
|
423 # some hook specific predicates ################################################# |
|
424 |
|
425 @objectify_predicate |
|
426 def enabled_category(cls, req, **kwargs): |
|
427 if req is None: |
|
428 return True # XXX how to deactivate server startup / shutdown event |
|
429 return req.is_hook_activated(cls) |
|
430 |
|
431 @objectify_predicate |
|
432 def issued_from_user_query(cls, req, **kwargs): |
|
433 return 0 if req.hooks_in_progress else 1 |
|
434 |
|
435 from_dbapi_query = class_renamed('from_dbapi_query', |
|
436 issued_from_user_query, |
|
437 message='[3.21] ') |
|
438 |
|
439 |
|
440 class rechain(object): |
|
441 def __init__(self, *iterators): |
|
442 self.iterators = iterators |
|
443 def __iter__(self): |
|
444 return iter(chain(*self.iterators)) |
|
445 |
|
446 |
|
447 class match_rtype(ExpectedValuePredicate): |
|
448 """accept if the relation type is found in expected ones. Optional |
|
449 named parameters `frometypes` and `toetypes` can be used to restrict |
|
450 target subject and/or object entity types of the relation. |
|
451 |
|
452 :param \*expected: possible relation types |
|
453 :param frometypes: candidate entity types as subject of relation |
|
454 :param toetypes: candidate entity types as object of relation |
|
455 """ |
|
456 def __init__(self, *expected, **more): |
|
457 self.expected = expected |
|
458 self.frometypes = more.pop('frometypes', None) |
|
459 self.toetypes = more.pop('toetypes', None) |
|
460 assert not more, "unexpected kwargs in match_rtype: %s" % more |
|
461 |
|
462 def __call__(self, cls, req, *args, **kwargs): |
|
463 if kwargs.get('rtype') not in self.expected: |
|
464 return 0 |
|
465 if self.frometypes is not None and \ |
|
466 req.entity_metas(kwargs['eidfrom'])['type'] not in self.frometypes: |
|
467 return 0 |
|
468 if self.toetypes is not None and \ |
|
469 req.entity_metas(kwargs['eidto'])['type'] not in self.toetypes: |
|
470 return 0 |
|
471 return 1 |
|
472 |
|
473 |
|
474 class match_rtype_sets(ExpectedValuePredicate): |
|
475 """accept if the relation type is in one of the sets given as initializer |
|
476 argument. The goal of this predicate is that it keeps reference to original sets, |
|
477 so modification to thoses sets are considered by the predicate. For instance |
|
478 |
|
479 .. sourcecode:: python |
|
480 |
|
481 MYSET = set() |
|
482 |
|
483 class Hook1(Hook): |
|
484 __regid__ = 'hook1' |
|
485 __select__ = Hook.__select__ & match_rtype_sets(MYSET) |
|
486 ... |
|
487 |
|
488 class Hook2(Hook): |
|
489 __regid__ = 'hook2' |
|
490 __select__ = Hook.__select__ & match_rtype_sets(MYSET) |
|
491 |
|
492 Client code can now change `MYSET`, this will changes the selection criteria |
|
493 of :class:`Hook1` and :class:`Hook1`. |
|
494 """ |
|
495 |
|
496 def __init__(self, *expected): |
|
497 self.expected = expected |
|
498 |
|
499 def __call__(self, cls, req, *args, **kwargs): |
|
500 for rel_set in self.expected: |
|
501 if kwargs.get('rtype') in rel_set: |
|
502 return 1 |
|
503 return 0 |
|
504 |
|
505 |
|
506 # base class for hook ########################################################## |
|
507 |
|
508 class Hook(AppObject): |
|
509 """Base class for hook. |
|
510 |
|
511 Hooks being appobjects like views, they have a `__regid__` and a `__select__` |
|
512 class attribute. Like all appobjects, hooks have the `self._cw` attribute which |
|
513 represents the current connection. In entity hooks, a `self.entity` attribute is |
|
514 also present. |
|
515 |
|
516 The `events` tuple is used by the base class selector to dispatch the hook |
|
517 on the right events. It is possible to dispatch on multiple events at once |
|
518 if needed (though take care as hook attribute may vary as described above). |
|
519 |
|
520 .. Note:: |
|
521 |
|
522 Do not forget to extend the base class selectors as in: |
|
523 |
|
524 .. sourcecode:: python |
|
525 |
|
526 class MyHook(Hook): |
|
527 __regid__ = 'whatever' |
|
528 __select__ = Hook.__select__ & is_instance('Person') |
|
529 |
|
530 else your hooks will be called madly, whatever the event. |
|
531 """ |
|
532 __select__ = enabled_category() |
|
533 # set this in derivated classes |
|
534 events = None |
|
535 category = None |
|
536 order = 0 |
|
537 # stop pylint from complaining about missing attributes in Hooks classes |
|
538 eidfrom = eidto = entity = rtype = repo = None |
|
539 |
|
540 @classmethod |
|
541 @cached |
|
542 def filterable_selectors(cls): |
|
543 search = cls.__select__.search_selector |
|
544 if search((NotPredicate, OrPredicate)): |
|
545 return None, None |
|
546 enabled_cat = search(enabled_category) |
|
547 main_filter = search((is_instance, match_rtype)) |
|
548 return enabled_cat, main_filter |
|
549 |
|
550 @classmethod |
|
551 def check_events(cls): |
|
552 try: |
|
553 for event in cls.events: |
|
554 if event not in ALL_HOOKS: |
|
555 raise Exception('bad event %s on %s.%s' % ( |
|
556 event, cls.__module__, cls.__name__)) |
|
557 except AttributeError: |
|
558 raise |
|
559 except TypeError: |
|
560 raise Exception('bad .events attribute %s on %s.%s' % ( |
|
561 cls.events, cls.__module__, cls.__name__)) |
|
562 |
|
563 @classmethod |
|
564 def __registered__(cls, reg): |
|
565 cls.check_events() |
|
566 |
|
567 @classproperty |
|
568 def __registries__(cls): |
|
569 if cls.events is None: |
|
570 return [] |
|
571 return ['%s_hooks' % ev for ev in cls.events] |
|
572 |
|
573 known_args = set(('entity', 'rtype', 'eidfrom', 'eidto', 'repo', 'timestamp')) |
|
574 def __init__(self, req, event, **kwargs): |
|
575 for arg in self.known_args: |
|
576 if arg in kwargs: |
|
577 setattr(self, arg, kwargs.pop(arg)) |
|
578 super(Hook, self).__init__(req, **kwargs) |
|
579 self.event = event |
|
580 |
|
581 set_log_methods(Hook, getLogger('cubicweb.hook')) |
|
582 |
|
583 |
|
584 # abtract hooks for relation propagation ####################################### |
|
585 # See example usage in hooks of the nosylist cube |
|
586 |
|
587 class PropagateRelationHook(Hook): |
|
588 """propagate some `main_rtype` relation on entities linked as object of |
|
589 `subject_relations` or as subject of `object_relations` (the watched |
|
590 relations). |
|
591 |
|
592 This hook ensure that when one of the watched relation is added, the |
|
593 `main_rtype` relation is added to the target entity of the relation. |
|
594 Notice there are no default behaviour defined when a watched relation is |
|
595 deleted, you'll have to handle this by yourself. |
|
596 |
|
597 You usually want to use the :class:`match_rtype_sets` predicate on concrete |
|
598 classes. |
|
599 """ |
|
600 events = ('after_add_relation',) |
|
601 |
|
602 # to set in concrete class |
|
603 main_rtype = None |
|
604 subject_relations = None |
|
605 object_relations = None |
|
606 |
|
607 def __call__(self): |
|
608 assert self.main_rtype |
|
609 for eid in (self.eidfrom, self.eidto): |
|
610 etype = self._cw.entity_metas(eid)['type'] |
|
611 if self.main_rtype not in self._cw.vreg.schema.eschema(etype).subjrels: |
|
612 return |
|
613 if self.rtype in self.subject_relations: |
|
614 meid, seid = self.eidfrom, self.eidto |
|
615 else: |
|
616 assert self.rtype in self.object_relations |
|
617 meid, seid = self.eidto, self.eidfrom |
|
618 self._cw.execute( |
|
619 'SET E %s P WHERE X %s P, X eid %%(x)s, E eid %%(e)s, NOT E %s P' |
|
620 % (self.main_rtype, self.main_rtype, self.main_rtype), |
|
621 {'x': meid, 'e': seid}) |
|
622 |
|
623 |
|
624 class PropagateRelationAddHook(Hook): |
|
625 """Propagate to entities at the end of watched relations when a `main_rtype` |
|
626 relation is added. |
|
627 |
|
628 `subject_relations` and `object_relations` attributes should be specified on |
|
629 subclasses and are usually shared references with attributes of the same |
|
630 name on :class:`PropagateRelationHook`. |
|
631 |
|
632 Because of those shared references, you can use `skip_subject_relations` and |
|
633 `skip_object_relations` attributes when you don't want to propagate to |
|
634 entities linked through some particular relations. |
|
635 """ |
|
636 events = ('after_add_relation',) |
|
637 |
|
638 # to set in concrete class (mandatory) |
|
639 subject_relations = None |
|
640 object_relations = None |
|
641 # to set in concrete class (optionally) |
|
642 skip_subject_relations = () |
|
643 skip_object_relations = () |
|
644 |
|
645 def __call__(self): |
|
646 eschema = self._cw.vreg.schema.eschema(self._cw.entity_metas(self.eidfrom)['type']) |
|
647 execute = self._cw.execute |
|
648 for rel in self.subject_relations: |
|
649 if rel in eschema.subjrels and not rel in self.skip_subject_relations: |
|
650 execute('SET R %s P WHERE X eid %%(x)s, P eid %%(p)s, ' |
|
651 'X %s R, NOT R %s P' % (self.rtype, rel, self.rtype), |
|
652 {'x': self.eidfrom, 'p': self.eidto}) |
|
653 for rel in self.object_relations: |
|
654 if rel in eschema.objrels and not rel in self.skip_object_relations: |
|
655 execute('SET R %s P WHERE X eid %%(x)s, P eid %%(p)s, ' |
|
656 'R %s X, NOT R %s P' % (self.rtype, rel, self.rtype), |
|
657 {'x': self.eidfrom, 'p': self.eidto}) |
|
658 |
|
659 |
|
660 class PropagateRelationDelHook(PropagateRelationAddHook): |
|
661 """Propagate to entities at the end of watched relations when a `main_rtype` |
|
662 relation is deleted. |
|
663 |
|
664 This is the opposite of the :class:`PropagateRelationAddHook`, see its |
|
665 documentation for how to use this class. |
|
666 """ |
|
667 events = ('after_delete_relation',) |
|
668 |
|
669 def __call__(self): |
|
670 eschema = self._cw.vreg.schema.eschema(self._cw.entity_metas(self.eidfrom)['type']) |
|
671 execute = self._cw.execute |
|
672 for rel in self.subject_relations: |
|
673 if rel in eschema.subjrels and not rel in self.skip_subject_relations: |
|
674 execute('DELETE R %s P WHERE X eid %%(x)s, P eid %%(p)s, ' |
|
675 'X %s R' % (self.rtype, rel), |
|
676 {'x': self.eidfrom, 'p': self.eidto}) |
|
677 for rel in self.object_relations: |
|
678 if rel in eschema.objrels and not rel in self.skip_object_relations: |
|
679 execute('DELETE R %s P WHERE X eid %%(x)s, P eid %%(p)s, ' |
|
680 'R %s X' % (self.rtype, rel), |
|
681 {'x': self.eidfrom, 'p': self.eidto}) |
|
682 |
|
683 |
|
684 |
|
685 # abstract classes for operation ############################################### |
|
686 |
|
687 class Operation(object): |
|
688 """Base class for operations. |
|
689 |
|
690 Operation may be instantiated in the hooks' `__call__` method. It always |
|
691 takes a connection object as first argument (accessible as `.cnx` from the |
|
692 operation instance), and optionally all keyword arguments needed by the |
|
693 operation. These keyword arguments will be accessible as attributes from the |
|
694 operation instance. |
|
695 |
|
696 An operation is triggered on connections set events related to commit / |
|
697 rollback transations. Possible events are: |
|
698 |
|
699 * `precommit`: |
|
700 |
|
701 the transaction is being prepared for commit. You can freely do any heavy |
|
702 computation, raise an exception if the commit can't go. or even add some |
|
703 new operations during this phase. If you do anything which has to be |
|
704 reverted if the commit fails afterwards (eg altering the file system for |
|
705 instance), you'll have to support the 'revertprecommit' event to revert |
|
706 things by yourself |
|
707 |
|
708 * `revertprecommit`: |
|
709 |
|
710 if an operation failed while being pre-commited, this event is triggered |
|
711 for all operations which had their 'precommit' event already fired to let |
|
712 them revert things (including the operation which made the commit fail) |
|
713 |
|
714 * `rollback`: |
|
715 |
|
716 the transaction has been either rolled back either: |
|
717 |
|
718 * intentionally |
|
719 * a 'precommit' event failed, in which case all operations are rolled back |
|
720 once 'revertprecommit'' has been called |
|
721 |
|
722 * `postcommit`: |
|
723 |
|
724 the transaction is over. All the ORM entities accessed by the earlier |
|
725 transaction are invalid. If you need to work on the database, you need to |
|
726 start a new transaction, for instance using a new internal connection, |
|
727 which you will need to commit. |
|
728 |
|
729 For an operation to support an event, one has to implement the `<event |
|
730 name>_event` method with no arguments. |
|
731 |
|
732 The order of operations may be important, and is controlled according to |
|
733 the insert_index's method output (whose implementation vary according to the |
|
734 base hook class used). |
|
735 """ |
|
736 |
|
737 def __init__(self, cnx, **kwargs): |
|
738 self.cnx = cnx |
|
739 self.__dict__.update(kwargs) |
|
740 self.register(cnx) |
|
741 # execution information |
|
742 self.processed = None # 'precommit', 'commit' |
|
743 self.failed = False |
|
744 |
|
745 @property |
|
746 @deprecated('[3.19] Operation.session is deprecated, use Operation.cnx instead') |
|
747 def session(self): |
|
748 return self.cnx |
|
749 |
|
750 def register(self, cnx): |
|
751 cnx.add_operation(self, self.insert_index()) |
|
752 |
|
753 def insert_index(self): |
|
754 """return the index of the latest instance which is not a |
|
755 LateOperation instance |
|
756 """ |
|
757 # faster by inspecting operation in reverse order for heavy transactions |
|
758 i = None |
|
759 for i, op in enumerate(reversed(self.cnx.pending_operations)): |
|
760 if isinstance(op, (LateOperation, SingleLastOperation)): |
|
761 continue |
|
762 return -i or None |
|
763 if i is None: |
|
764 return None |
|
765 return -(i + 1) |
|
766 |
|
767 def handle_event(self, event): |
|
768 """delegate event handling to the opertaion""" |
|
769 getattr(self, event)() |
|
770 |
|
771 def precommit_event(self): |
|
772 """the observed connections set is preparing a commit""" |
|
773 |
|
774 def revertprecommit_event(self): |
|
775 """an error went when pre-commiting this operation or a later one |
|
776 |
|
777 should revert pre-commit's changes but take care, they may have not |
|
778 been all considered if it's this operation which failed |
|
779 """ |
|
780 |
|
781 def rollback_event(self): |
|
782 """the observed connections set has been rolled back |
|
783 |
|
784 do nothing by default |
|
785 """ |
|
786 |
|
787 def postcommit_event(self): |
|
788 """the observed connections set has committed""" |
|
789 |
|
790 # these are overridden by set_log_methods below |
|
791 # only defining here to prevent pylint from complaining |
|
792 info = warning = error = critical = exception = debug = lambda msg,*a,**kw: None |
|
793 |
|
794 set_log_methods(Operation, getLogger('cubicweb.session')) |
|
795 |
|
796 def _container_add(container, value): |
|
797 {set: set.add, list: list.append}[container.__class__](container, value) |
|
798 |
|
799 |
|
800 class DataOperationMixIn(object): |
|
801 """Mix-in class to ease applying a single operation on a set of data, |
|
802 avoiding to create as many as operation as they are individual modification. |
|
803 The body of the operation must then iterate over the values that have been |
|
804 stored in a single operation instance. |
|
805 |
|
806 You should try to use this instead of creating on operation for each |
|
807 `value`, since handling operations becomes costly on massive data import. |
|
808 |
|
809 Usage looks like: |
|
810 |
|
811 .. sourcecode:: python |
|
812 |
|
813 class MyEntityHook(Hook): |
|
814 __regid__ = 'my.entity.hook' |
|
815 __select__ = Hook.__select__ & is_instance('MyEntity') |
|
816 events = ('after_add_entity',) |
|
817 |
|
818 def __call__(self): |
|
819 MyOperation.get_instance(self._cw).add_data(self.entity) |
|
820 |
|
821 |
|
822 class MyOperation(DataOperationMixIn, Operation): |
|
823 def precommit_event(self): |
|
824 for bucket in self.get_data(): |
|
825 process(bucket) |
|
826 |
|
827 You can modify the `containercls` class attribute, which defines the |
|
828 container class that should be instantiated to hold payloads. An instance is |
|
829 created on instantiation, and then the :meth:`add_data` method will add the |
|
830 given data to the existing container. Default to a `set`. Give `list` if you |
|
831 want to keep arrival ordering. You can also use another kind of container |
|
832 by redefining :meth:`_build_container` and :meth:`add_data` |
|
833 |
|
834 More optional parameters can be given to the `get_instance` operation, that |
|
835 will be given to the operation constructor (for obvious reasons those |
|
836 parameters should not vary accross different calls to this method for a |
|
837 given operation). |
|
838 |
|
839 .. Note:: |
|
840 For sanity reason `get_data` will reset the operation, so that once |
|
841 the operation has started its treatment, if some hook want to push |
|
842 additional data to this same operation, a new instance will be created |
|
843 (else that data has a great chance to be never treated). This implies: |
|
844 |
|
845 * you should **always** call `get_data` when starting treatment |
|
846 |
|
847 * you should **never** call `get_data` for another reason. |
|
848 """ |
|
849 containercls = set |
|
850 |
|
851 @classproperty |
|
852 def data_key(cls): |
|
853 return ('cw.dataops', cls.__name__) |
|
854 |
|
855 @classmethod |
|
856 def get_instance(cls, cnx, **kwargs): |
|
857 # no need to lock: transaction_data already comes from thread's local storage |
|
858 try: |
|
859 return cnx.transaction_data[cls.data_key] |
|
860 except KeyError: |
|
861 op = cnx.transaction_data[cls.data_key] = cls(cnx, **kwargs) |
|
862 return op |
|
863 |
|
864 def __init__(self, *args, **kwargs): |
|
865 super(DataOperationMixIn, self).__init__(*args, **kwargs) |
|
866 self._container = self._build_container() |
|
867 self._processed = False |
|
868 |
|
869 def __contains__(self, value): |
|
870 return value in self._container |
|
871 |
|
872 def _build_container(self): |
|
873 return self.containercls() |
|
874 |
|
875 def union(self, data): |
|
876 """only when container is a set""" |
|
877 assert not self._processed, """Trying to add data to a closed operation. |
|
878 Iterating over operation data closed it and should be reserved to precommit / |
|
879 postcommit method of the operation.""" |
|
880 self._container |= data |
|
881 |
|
882 def add_data(self, data): |
|
883 assert not self._processed, """Trying to add data to a closed operation. |
|
884 Iterating over operation data closed it and should be reserved to precommit / |
|
885 postcommit method of the operation.""" |
|
886 _container_add(self._container, data) |
|
887 |
|
888 def remove_data(self, data): |
|
889 assert not self._processed, """Trying to add data to a closed operation. |
|
890 Iterating over operation data closed it and should be reserved to precommit / |
|
891 postcommit method of the operation.""" |
|
892 self._container.remove(data) |
|
893 |
|
894 def get_data(self): |
|
895 assert not self._processed, """Trying to get data from a closed operation. |
|
896 Iterating over operation data closed it and should be reserved to precommit / |
|
897 postcommit method of the operation.""" |
|
898 self._processed = True |
|
899 op = self.cnx.transaction_data.pop(self.data_key) |
|
900 assert op is self, "Bad handling of operation data, found %s instead of %s for key %s" % ( |
|
901 op, self, self.data_key) |
|
902 return self._container |
|
903 |
|
904 |
|
905 |
|
906 class LateOperation(Operation): |
|
907 """special operation which should be called after all possible (ie non late) |
|
908 operations |
|
909 """ |
|
910 def insert_index(self): |
|
911 """return the index of the lastest instance which is not a |
|
912 SingleLastOperation instance |
|
913 """ |
|
914 # faster by inspecting operation in reverse order for heavy transactions |
|
915 i = None |
|
916 for i, op in enumerate(reversed(self.cnx.pending_operations)): |
|
917 if isinstance(op, SingleLastOperation): |
|
918 continue |
|
919 return -i or None |
|
920 if i is None: |
|
921 return None |
|
922 return -(i + 1) |
|
923 |
|
924 |
|
925 |
|
926 class SingleLastOperation(Operation): |
|
927 """special operation which should be called once and after all other |
|
928 operations |
|
929 """ |
|
930 |
|
931 def register(self, cnx): |
|
932 """override register to handle cases where this operation has already |
|
933 been added |
|
934 """ |
|
935 operations = cnx.pending_operations |
|
936 index = self.equivalent_index(operations) |
|
937 if index is not None: |
|
938 equivalent = operations.pop(index) |
|
939 else: |
|
940 equivalent = None |
|
941 cnx.add_operation(self, self.insert_index()) |
|
942 return equivalent |
|
943 |
|
944 def equivalent_index(self, operations): |
|
945 """return the index of the equivalent operation if any""" |
|
946 for i, op in enumerate(reversed(operations)): |
|
947 if op.__class__ is self.__class__: |
|
948 return -(i+1) |
|
949 return None |
|
950 |
|
951 def insert_index(self): |
|
952 return None |
|
953 |
|
954 |
|
955 class SendMailOp(SingleLastOperation): |
|
956 def __init__(self, cnx, msg=None, recipients=None, **kwargs): |
|
957 # may not specify msg yet, as |
|
958 # `cubicweb.sobjects.supervision.SupervisionMailOp` |
|
959 if msg is not None: |
|
960 assert recipients |
|
961 self.to_send = [(msg, recipients)] |
|
962 else: |
|
963 assert recipients is None |
|
964 self.to_send = [] |
|
965 super(SendMailOp, self).__init__(cnx, **kwargs) |
|
966 |
|
967 def register(self, cnx): |
|
968 previous = super(SendMailOp, self).register(cnx) |
|
969 if previous: |
|
970 self.to_send = previous.to_send + self.to_send |
|
971 |
|
972 def postcommit_event(self): |
|
973 self.cnx.repo.threaded_task(self.sendmails) |
|
974 |
|
975 def sendmails(self): |
|
976 self.cnx.vreg.config.sendmails(self.to_send) |
|
977 |
|
978 |
|
979 class RQLPrecommitOperation(Operation): |
|
980 # to be defined in concrete classes |
|
981 rqls = None |
|
982 |
|
983 def precommit_event(self): |
|
984 execute = self.cnx.execute |
|
985 for rql in self.rqls: |
|
986 execute(*rql) |
|
987 |
|
988 |
|
989 class CleanupNewEidsCacheOp(DataOperationMixIn, SingleLastOperation): |
|
990 """on rollback of a insert query we have to remove from repository's |
|
991 type/source cache eids of entities added in that transaction. |
|
992 |
|
993 NOTE: querier's rqlst/solutions cache may have been polluted too with |
|
994 queries such as Any X WHERE X eid 32 if 32 has been rolled back however |
|
995 generated queries are unpredictable and analysing all the cache probably |
|
996 too expensive. Notice that there is no pb when using args to specify eids |
|
997 instead of giving them into the rql string. |
|
998 """ |
|
999 data_key = 'neweids' |
|
1000 |
|
1001 def rollback_event(self): |
|
1002 """the observed connections set has been rolled back, |
|
1003 remove inserted eid from repository type/source cache |
|
1004 """ |
|
1005 try: |
|
1006 self.cnx.repo.clear_caches(self.get_data()) |
|
1007 except KeyError: |
|
1008 pass |
|
1009 |
|
1010 class CleanupDeletedEidsCacheOp(DataOperationMixIn, SingleLastOperation): |
|
1011 """on commit of delete query, we have to remove from repository's |
|
1012 type/source cache eids of entities deleted in that transaction. |
|
1013 """ |
|
1014 data_key = 'pendingeids' |
|
1015 def postcommit_event(self): |
|
1016 """the observed connections set has been rolled back, |
|
1017 remove inserted eid from repository type/source cache |
|
1018 """ |
|
1019 try: |
|
1020 eids = self.get_data() |
|
1021 self.cnx.repo.clear_caches(eids) |
|
1022 self.cnx.repo.app_instances_bus.publish(['delete'] + list(str(eid) for eid in eids)) |
|
1023 except KeyError: |
|
1024 pass |
|