1 # copyright 2003-2014 LOGILAB S.A. (Paris, FRANCE), all rights reserved.

     2 # contact http://www.logilab.fr/ -- mailto:contact@logilab.fr

     3 #

     4 # This file is part of CubicWeb.

     5 #

     6 # CubicWeb is free software: you can redistribute it and/or modify it under the

     7 # terms of the GNU Lesser General Public License as published by the Free

     8 # Software Foundation, either version 2.1 of the License, or (at your option)

     9 # any later version.

    10 #

    11 # CubicWeb is distributed in the hope that it will be useful, but WITHOUT

    12 # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS

    13 # FOR A PARTICULAR PURPOSE.  See the GNU Lesser General Public License for more

    14 # details.

    15 #

    16 # You should have received a copy of the GNU Lesser General Public License along

    17 # with CubicWeb.  If not, see <http://www.gnu.org/licenses/>.

    18 """

    19 Generalities

    20 ------------

    21 

    22 Paraphrasing the `emacs`_ documentation, let us say that hooks are an important

    23 mechanism for customizing an application. A hook is basically a list of

    24 functions to be called on some well-defined occasion (this is called `running

    25 the hook`).

    26 

    27 .. _`emacs`: http://www.gnu.org/software/emacs/manual/html_node/emacs/Hooks.html

    28 

    29 Hooks

    30 ~~~~~

    31 

    32 In |cubicweb|, hooks are subclasses of the :class:`~cubicweb.server.hook.Hook`

    33 class. They are selected over a set of pre-defined `events` (and possibly more

    34 conditions, hooks being selectable appobjects like views and components).  They

    35 should implement a :meth:`~cubicweb.server.hook.Hook.__call__` method that will

    36 be called when the hook is triggered.

    37 

    38 There are two families of events: data events (before / after any individual

    39 update of an entity / or a relation in the repository) and server events (such

    40 as server startup or shutdown).  In a typical application, most of the hooks are

    41 defined over data events.

    42 

    43 Also, some :class:`~cubicweb.server.hook.Operation` may be registered by hooks,

    44 which will be fired when the transaction is commited or rolled back.

    45 

    46 The purpose of data event hooks is usually to complement the data model as

    47 defined in the schema, which is static by nature and only provide a restricted

    48 builtin set of dynamic constraints, with dynamic or value driven behaviours.

    49 For instance they can serve the following purposes:

    50 

    51 * enforcing constraints that the static schema cannot express (spanning several

    52   entities/relations, exotic value ranges and cardinalities, etc.)

    53 

    54 * implement computed attributes

    55 

    56 It is functionally equivalent to a `database trigger`_, except that database

    57 triggers definition languages are not standardized, hence not portable (for

    58 instance, PL/SQL works with Oracle and PostgreSQL but not SqlServer nor Sqlite).

    59 

    60 .. _`database trigger`: http://en.wikipedia.org/wiki/Database_trigger

    61 

    62 

    63 .. hint::

    64 

    65    It is a good practice to write unit tests for each hook. See an example in

    66    :ref:`hook_test`

    67 

    68 Operations

    69 ~~~~~~~~~~

    70 

    71 Operations are subclasses of the :class:`~cubicweb.server.hook.Operation` class

    72 that may be created by hooks and scheduled to happen on `precommit`,

    73 `postcommit` or `rollback` event (i.e. respectivly before/after a commit or

    74 before a rollback of a transaction).

    75 

    76 Hooks are being fired immediately on data operations, and it is sometime

    77 necessary to delay the actual work down to a time where we can expect all

    78 information to be there, or when all other hooks have run (though take case

    79 since operations may themselves trigger hooks). Also while the order of

    80 execution of hooks is data dependant (and thus hard to predict), it is possible

    81 to force an order on operations.

    82 

    83 So, for such case where you may miss some information that may be set later in

    84 the transaction, you should instantiate an operation in the hook.

    85 

    86 Operations may be used to:

    87 

    88 * implements a validation check which needs that all relations be already set on

    89   an entity

    90 

    91 * process various side effects associated with a transaction such as filesystem

    92   udpates, mail notifications, etc.

    93 

    94 

    95 Events

    96 ------

    97 

    98 Hooks are mostly defined and used to handle `dataflow`_ operations. It

    99 means as data gets in (entities added, updated, relations set or

   100 unset), specific events are issued and the Hooks matching these events

   101 are called.

   102 

   103 You can get the event that triggered a hook by accessing its `event`

   104 attribute.

   105 

   106 .. _`dataflow`: http://en.wikipedia.org/wiki/Dataflow

   107 

   108 

   109 Entity modification related events

   110 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

   111 

   112 When called for one of these events, hook will have an `entity` attribute

   113 containing the entity instance.

   114 

   115 - `before_add_entity`, `before_update_entity`:

   116 

   117   On those events, you can access the modified attributes of the entity using

   118   the `entity.cw_edited` dictionary. The values can be modified and the old

   119   values can be retrieved.

   120 

   121   If you modify the `entity.cw_edited` dictionary in the hook, that is before

   122   the database operations take place, you will avoid the need to process a whole

   123   new rql query and the underlying backend query (eg usually sql) will contain

   124   the modified data. For example:

   125 

   126   .. sourcecode:: python

   127 

   128      self.entity.cw_edited['age'] = 42

   129 

   130   will modify the age before it is written to the backend storage.

   131 

   132   Similarly, removing an attribute from `cw_edited` will cancel its

   133   modification:

   134 

   135   .. sourcecode:: python

   136 

   137      del self.entity.cw_edited['age']

   138 

   139   On a `before_update_entity` event, you can access the old and new values:

   140 

   141   .. sourcecode:: python

   142 

   143      old, new = entity.cw_edited.oldnewvalue('age')

   144 

   145 - `after_add_entity`, `after_update_entity`

   146 

   147   On those events, you can get the list of attributes that were modified using

   148   the `entity.cw_edited` dictionary, but you can not modify it or get the old

   149   value of an attribute.

   150 

   151 - `before_delete_entity`, `after_delete_entity`

   152 

   153   On those events, the entity has no `cw_edited` dictionary.

   154 

   155 .. note:: `self.entity.cw_set(age=42)` will set the `age` attribute to

   156   42. But to do so, it will generate a rql query that will have to be processed,

   157   hence may trigger some hooks, etc. This could lead to infinitely looping hooks.

   158 

   159 Relation modification related events

   160 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

   161 

   162 When called for one of these events, hook will have `eidfrom`, `rtype`, `eidto`

   163 attributes containing respectively the eid of the subject entity, the relation

   164 type and the eid of the object entity.

   165 

   166 * `before_add_relation`, `before_delete_relation`

   167 

   168   On those events, you can still get the original relation by issuing a rql query.

   169 

   170 * `after_add_relation`, `after_delete_relation`

   171 

   172 Specific selectors are shipped for these kinds of events, see in particular

   173 :class:`~cubicweb.server.hook.match_rtype`.

   174 

   175 Also note that relations can be added or deleted, but not updated.

   176 

   177 Non data events

   178 ~~~~~~~~~~~~~~~

   179 

   180 Hooks called on server start/maintenance/stop event (e.g.

   181 `server_startup`, `server_maintenance`, `before_server_shutdown`,

   182 `server_shutdown`) have a `repo` attribute, but *their `_cw` attribute

   183 is None*.  The `server_startup` is called on regular startup, while

   184 `server_maintenance` is called on cubicweb-ctl upgrade or shell

   185 commands. `server_shutdown` is called anyway but connections to the

   186 native source is impossible; `before_server_shutdown` handles that.

   187 

   188 Hooks called on backup/restore event (eg `server_backup`,

   189 `server_restore`) have a `repo` and a `timestamp` attributes, but

   190 *their `_cw` attribute is None*.

   191 

   192 Hooks called on session event (eg `session_open`, `session_close`) have no

   193 special attribute.

   194 

   195 

   196 API

   197 ---

   198 

   199 Hooks control

   200 ~~~~~~~~~~~~~

   201 

   202 It is sometimes convenient to explicitly enable or disable some hooks. For

   203 instance if you want to disable some integrity checking hook. This can be

   204 controlled more finely through the `category` class attribute, which is a string

   205 giving a category name.  One can then uses the

   206 :meth:`~cubicweb.server.session.Connection.deny_all_hooks_but` and

   207 :meth:`~cubicweb.server.session.Connection.allow_all_hooks_but` context managers to

   208 explicitly enable or disable some categories.

   209 

   210 The existing categories are:

   211 

   212 * ``security``, security checking hooks

   213 

   214 * ``worfklow``, workflow handling hooks

   215 

   216 * ``metadata``, hooks setting meta-data on newly created entities

   217 

   218 * ``notification``, email notification hooks

   219 

   220 * ``integrity``, data integrity checking hooks

   221 

   222 * ``activeintegrity``, data integrity consistency hooks, that you should **never**

   223   want to disable

   224 

   225 * ``syncsession``, hooks synchronizing existing sessions

   226 

   227 * ``syncschema``, hooks synchronizing instance schema (including the physical database)

   228 

   229 * ``email``, email address handling hooks

   230 

   231 * ``bookmark``, bookmark entities handling hooks

   232 

   233 

   234 Nothing precludes one to invent new categories and use existing mechanisms to

   235 filter them in or out.

   236 

   237 

   238 Hooks specific predicates

   239 ~~~~~~~~~~~~~~~~~~~~~~~~~

   240 .. autoclass:: cubicweb.server.hook.match_rtype

   241 .. autoclass:: cubicweb.server.hook.match_rtype_sets

   242 

   243 

   244 Hooks and operations classes

   245 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~

   246 .. autoclass:: cubicweb.server.hook.Hook

   247 .. autoclass:: cubicweb.server.hook.Operation

   248 .. autoclass:: cubicweb.server.hook.LateOperation

   249 .. autoclass:: cubicweb.server.hook.DataOperationMixIn

   250 """

   251 from __future__ import print_function

   252 

   253 __docformat__ = "restructuredtext en"

   254 

   255 from warnings import warn

   256 from logging import getLogger

   257 from itertools import chain

   258 

   259 from logilab.common.decorators import classproperty, cached

   260 from logilab.common.deprecation import deprecated, class_renamed

   261 from logilab.common.logging_ext import set_log_methods

   262 from logilab.common.registry import (NotPredicate, OrPredicate,

   263                                      objectify_predicate)

   264 

   265 from cubicweb import RegistryNotFound, server

   266 from cubicweb.cwvreg import CWRegistry, CWRegistryStore

   267 from cubicweb.predicates import ExpectedValuePredicate, is_instance

   268 from cubicweb.appobject import AppObject

   269 

   270 ENTITIES_HOOKS = set(('before_add_entity',    'after_add_entity',

   271                       'before_update_entity', 'after_update_entity',

   272                       'before_delete_entity', 'after_delete_entity'))

   273 RELATIONS_HOOKS = set(('before_add_relation',   'after_add_relation' ,

   274                        'before_delete_relation','after_delete_relation'))

   275 SYSTEM_HOOKS = set(('server_backup', 'server_restore',

   276                     'server_startup', 'server_maintenance',

   277                     'server_shutdown', 'before_server_shutdown',

   278                     'session_open', 'session_close'))

   279 ALL_HOOKS = ENTITIES_HOOKS | RELATIONS_HOOKS | SYSTEM_HOOKS

   280 

   281 def _iter_kwargs(entities, eids_from_to, kwargs):

   282     if not entities and not eids_from_to:

   283         yield kwargs

   284     elif entities:

   285         for entity in entities:

   286             kwargs['entity'] = entity

   287             yield kwargs

   288     else:

   289         for subject, object in eids_from_to:

   290             kwargs.update({'eidfrom': subject, 'eidto': object})

   291             yield kwargs

   292 

   293 

   294 class HooksRegistry(CWRegistry):

   295 

   296     def register(self, obj, **kwargs):

   297         obj.check_events()

   298         super(HooksRegistry, self).register(obj, **kwargs)

   299 

   300     def call_hooks(self, event, cnx=None, **kwargs):

   301         """call `event` hooks for an entity or a list of entities (passed

   302         respectively as the `entity` or ``entities`` keyword argument).

   303         """

   304         kwargs['event'] = event

   305         if cnx is None: # True for events such as server_start

   306             for hook in sorted(self.possible_objects(cnx, **kwargs),

   307                                key=lambda x: x.order):

   308                 hook()

   309         else:

   310             if 'entities' in kwargs:

   311                 assert 'entity' not in kwargs, \

   312                        'can\'t pass "entities" and "entity" arguments simultaneously'

   313                 assert 'eids_from_to' not in kwargs, \

   314                        'can\'t pass "entities" and "eids_from_to" arguments simultaneously'

   315                 entities = kwargs.pop('entities')

   316                 eids_from_to = []

   317             elif 'eids_from_to' in kwargs:

   318                 entities = []

   319                 eids_from_to = kwargs.pop('eids_from_to')

   320             else:

   321                 entities = []

   322                 eids_from_to = []

   323             pruned = self.get_pruned_hooks(cnx, event,

   324                                            entities, eids_from_to, kwargs)

   325 

   326             # by default, hooks are executed with security turned off

   327             with cnx.security_enabled(read=False):

   328                 for _kwargs in _iter_kwargs(entities, eids_from_to, kwargs):

   329                     hooks = sorted(self.filtered_possible_objects(pruned, cnx, **_kwargs),

   330                                    key=lambda x: x.order)

   331                     debug = server.DEBUG & server.DBG_HOOKS

   332                     with cnx.security_enabled(write=False):

   333                         with cnx.running_hooks_ops():

   334                             for hook in hooks:

   335                                 if debug:

   336                                     print(event, _kwargs, hook)

   337                                 hook()

   338 

   339     def get_pruned_hooks(self, cnx, event, entities, eids_from_to, kwargs):

   340         """return a set of hooks that should not be considered by filtered_possible objects

   341 

   342         the idea is to make a first pass over all the hooks in the

   343         registry and to mark put some of them in a pruned list. The

   344         pruned hooks are the one which:

   345 

   346         * are disabled at the connection level

   347 

   348         * have a selector containing a :class:`match_rtype` or an

   349           :class:`is_instance` predicate which does not match the rtype / etype

   350           of the relations / entities for which we are calling the hooks. This

   351           works because the repository calls the hooks grouped by rtype or by

   352           etype when using the entities or eids_to_from keyword arguments

   353 

   354         Only hooks with a simple predicate or an AndPredicate of simple

   355         predicates are considered for disabling.

   356 

   357         """

   358         if 'entity' in kwargs:

   359             entities = [kwargs['entity']]

   360         if len(entities):

   361             look_for_selector = is_instance

   362             etype = entities[0].__regid__

   363         elif 'rtype' in kwargs:

   364             look_for_selector = match_rtype

   365             etype = None

   366         else: # nothing to prune, how did we get there ???

   367             return set()

   368         cache_key = (event, kwargs.get('rtype'), etype)

   369         pruned = cnx.pruned_hooks_cache.get(cache_key)

   370         if pruned is not None:

   371             return pruned

   372         pruned = set()

   373         cnx.pruned_hooks_cache[cache_key] = pruned

   374         if look_for_selector is not None:

   375             for id, hooks in self.items():

   376                 for hook in hooks:

   377                     enabled_cat, main_filter = hook.filterable_selectors()

   378                     if enabled_cat is not None:

   379                         if not enabled_cat(hook, cnx):

   380                             pruned.add(hook)

   381                             continue

   382                     if main_filter is not None:

   383                         if isinstance(main_filter, match_rtype) and \

   384                            (main_filter.frometypes is not None  or \

   385                             main_filter.toetypes is not None):

   386                             continue

   387                         first_kwargs = next(_iter_kwargs(entities, eids_from_to, kwargs))

   388                         if not main_filter(hook, cnx, **first_kwargs):

   389                             pruned.add(hook)

   390         return pruned

   391 

   392 

   393     def filtered_possible_objects(self, pruned, *args, **kwargs):

   394         for appobjects in self.values():

   395             if pruned:

   396                 filtered_objects = [obj for obj in appobjects if obj not in pruned]

   397                 if not filtered_objects:

   398                     continue

   399             else:

   400                 filtered_objects = appobjects

   401             obj = self._select_best(filtered_objects,

   402                                     *args, **kwargs)

   403             if obj is None:

   404                 continue

   405             yield obj

   406 

   407 class HooksManager(object):

   408     def __init__(self, vreg):

   409         self.vreg = vreg

   410 

   411     def call_hooks(self, event, cnx=None, **kwargs):

   412         try:

   413             registry = self.vreg['%s_hooks' % event]

   414         except RegistryNotFound:

   415             return # no hooks for this event

   416         registry.call_hooks(event, cnx, **kwargs)

   417 

   418 

   419 for event in ALL_HOOKS:

   420     CWRegistryStore.REGISTRY_FACTORY['%s_hooks' % event] = HooksRegistry

   421 

   422 

   423 # some hook specific predicates #################################################

   424 

   425 @objectify_predicate

   426 def enabled_category(cls, req, **kwargs):

   427     if req is None:

   428         return True # XXX how to deactivate server startup / shutdown event

   429     return req.is_hook_activated(cls)

   430 

   431 @objectify_predicate

   432 def issued_from_user_query(cls, req, **kwargs):

   433     return 0 if req.hooks_in_progress else 1

   434 

   435 from_dbapi_query = class_renamed('from_dbapi_query',

   436                                  issued_from_user_query,

   437                                  message='[3.21] ')

   438 

   439 

   440 class rechain(object):

   441     def __init__(self, *iterators):

   442         self.iterators = iterators

   443     def __iter__(self):

   444         return iter(chain(*self.iterators))

   445 

   446 

   447 class match_rtype(ExpectedValuePredicate):

   448     """accept if the relation type is found in expected ones. Optional

   449     named parameters `frometypes` and `toetypes` can be used to restrict

   450     target subject and/or object entity types of the relation.

   451 

   452     :param \*expected: possible relation types

   453     :param frometypes: candidate entity types as subject of relation

   454     :param toetypes: candidate entity types as object of relation

   455     """

   456     def __init__(self, *expected, **more):

   457         self.expected = expected

   458         self.frometypes = more.pop('frometypes', None)

   459         self.toetypes = more.pop('toetypes', None)

   460         assert not more, "unexpected kwargs in match_rtype: %s" % more

   461 

   462     def __call__(self, cls, req, *args, **kwargs):

   463         if kwargs.get('rtype') not in self.expected:

   464             return 0

   465         if self.frometypes is not None and \

   466                req.entity_metas(kwargs['eidfrom'])['type'] not in self.frometypes:

   467             return 0

   468         if self.toetypes is not None and \

   469                req.entity_metas(kwargs['eidto'])['type'] not in self.toetypes:

   470             return 0

   471         return 1

   472 

   473 

   474 class match_rtype_sets(ExpectedValuePredicate):

   475     """accept if the relation type is in one of the sets given as initializer

   476     argument. The goal of this predicate is that it keeps reference to original sets,

   477     so modification to thoses sets are considered by the predicate. For instance

   478 

   479     .. sourcecode:: python

   480 

   481       MYSET = set()

   482 

   483       class Hook1(Hook):

   484           __regid__ = 'hook1'

   485           __select__ = Hook.__select__ & match_rtype_sets(MYSET)

   486           ...

   487 

   488       class Hook2(Hook):

   489           __regid__ = 'hook2'

   490           __select__ = Hook.__select__ & match_rtype_sets(MYSET)

   491 

   492     Client code can now change `MYSET`, this will changes the selection criteria

   493     of :class:`Hook1` and :class:`Hook1`.

   494     """

   495 

   496     def __init__(self, *expected):

   497         self.expected = expected

   498 

   499     def __call__(self, cls, req, *args, **kwargs):

   500         for rel_set in self.expected:

   501             if kwargs.get('rtype') in rel_set:

   502                 return 1

   503         return 0

   504 

   505 

   506 # base class for hook ##########################################################

   507 

   508 class Hook(AppObject):

   509     """Base class for hook.

   510 

   511     Hooks being appobjects like views, they have a `__regid__` and a `__select__`

   512     class attribute. Like all appobjects, hooks have the `self._cw` attribute which

   513     represents the current connection. In entity hooks, a `self.entity` attribute is

   514     also present.

   515 

   516     The `events` tuple is used by the base class selector to dispatch the hook

   517     on the right events. It is possible to dispatch on multiple events at once

   518     if needed (though take care as hook attribute may vary as described above).

   519 

   520     .. Note::

   521 

   522       Do not forget to extend the base class selectors as in:

   523 

   524       .. sourcecode:: python

   525 

   526           class MyHook(Hook):

   527             __regid__ = 'whatever'

   528             __select__ = Hook.__select__ & is_instance('Person')

   529 

   530       else your hooks will be called madly, whatever the event.

   531     """

   532     __select__ = enabled_category()

   533     # set this in derivated classes

   534     events = None

   535     category = None

   536     order = 0

   537     # stop pylint from complaining about missing attributes in Hooks classes

   538     eidfrom = eidto = entity = rtype = repo = None

   539 

   540     @classmethod

   541     @cached

   542     def filterable_selectors(cls):

   543         search = cls.__select__.search_selector

   544         if search((NotPredicate, OrPredicate)):

   545             return None, None

   546         enabled_cat = search(enabled_category)

   547         main_filter = search((is_instance, match_rtype))

   548         return enabled_cat, main_filter

   549 

   550     @classmethod

   551     def check_events(cls):

   552         try:

   553             for event in cls.events:

   554                 if event not in ALL_HOOKS:

   555                     raise Exception('bad event %s on %s.%s' % (

   556                         event, cls.__module__, cls.__name__))

   557         except AttributeError:

   558             raise

   559         except TypeError:

   560             raise Exception('bad .events attribute %s on %s.%s' % (

   561                 cls.events, cls.__module__, cls.__name__))

   562 

   563     @classmethod

   564     def __registered__(cls, reg):

   565         cls.check_events()

   566 

   567     @classproperty

   568     def __registries__(cls):

   569         if cls.events is None:

   570             return []

   571         return ['%s_hooks' % ev for ev in cls.events]

   572 

   573     known_args = set(('entity', 'rtype', 'eidfrom', 'eidto', 'repo', 'timestamp'))

   574     def __init__(self, req, event, **kwargs):

   575         for arg in self.known_args:

   576             if arg in kwargs:

   577                 setattr(self, arg, kwargs.pop(arg))

   578         super(Hook, self).__init__(req, **kwargs)

   579         self.event = event

   580 

   581 set_log_methods(Hook, getLogger('cubicweb.hook'))

   582 

   583 

   584 # abtract hooks for relation propagation #######################################

   585 # See example usage in hooks of the nosylist cube

   586 

   587 class PropagateRelationHook(Hook):

   588     """propagate some `main_rtype` relation on entities linked as object of

   589     `subject_relations` or as subject of `object_relations` (the watched

   590     relations).

   591 

   592     This hook ensure that when one of the watched relation is added, the

   593     `main_rtype` relation is added to the target entity of the relation.

   594     Notice there are no default behaviour defined when a watched relation is

   595     deleted, you'll have to handle this by yourself.

   596 

   597     You usually want to use the :class:`match_rtype_sets` predicate on concrete

   598     classes.

   599     """

   600     events = ('after_add_relation',)

   601 

   602     # to set in concrete class

   603     main_rtype = None

   604     subject_relations = None

   605     object_relations = None

   606 

   607     def __call__(self):

   608         assert self.main_rtype

   609         for eid in (self.eidfrom, self.eidto):

   610             etype = self._cw.entity_metas(eid)['type']

   611             if self.main_rtype not in self._cw.vreg.schema.eschema(etype).subjrels:

   612                 return

   613         if self.rtype in self.subject_relations:

   614             meid, seid = self.eidfrom, self.eidto

   615         else:

   616             assert self.rtype in self.object_relations

   617             meid, seid = self.eidto, self.eidfrom

   618         self._cw.execute(

   619             'SET E %s P WHERE X %s P, X eid %%(x)s, E eid %%(e)s, NOT E %s P'

   620             % (self.main_rtype, self.main_rtype, self.main_rtype),

   621             {'x': meid, 'e': seid})

   622 

   623 

   624 class PropagateRelationAddHook(Hook):

   625     """Propagate to entities at the end of watched relations when a `main_rtype`

   626     relation is added.

   627 

   628     `subject_relations` and `object_relations` attributes should be specified on

   629     subclasses and are usually shared references with attributes of the same

   630     name on :class:`PropagateRelationHook`.

   631 

   632     Because of those shared references, you can use `skip_subject_relations` and

   633     `skip_object_relations` attributes when you don't want to propagate to

   634     entities linked through some particular relations.

   635     """

   636     events = ('after_add_relation',)

   637 

   638     # to set in concrete class (mandatory)

   639     subject_relations = None

   640     object_relations = None

   641     # to set in concrete class (optionally)

   642     skip_subject_relations = ()

   643     skip_object_relations = ()

   644 

   645     def __call__(self):

   646         eschema = self._cw.vreg.schema.eschema(self._cw.entity_metas(self.eidfrom)['type'])

   647         execute = self._cw.execute

   648         for rel in self.subject_relations:

   649             if rel in eschema.subjrels and not rel in self.skip_subject_relations:

   650                 execute('SET R %s P WHERE X eid %%(x)s, P eid %%(p)s, '

   651                         'X %s R, NOT R %s P' % (self.rtype, rel, self.rtype),

   652                         {'x': self.eidfrom, 'p': self.eidto})

   653         for rel in self.object_relations:

   654             if rel in eschema.objrels and not rel in self.skip_object_relations:

   655                 execute('SET R %s P WHERE X eid %%(x)s, P eid %%(p)s, '

   656                         'R %s X, NOT R %s P' % (self.rtype, rel, self.rtype),

   657                         {'x': self.eidfrom, 'p': self.eidto})

   658 

   659 

   660 class PropagateRelationDelHook(PropagateRelationAddHook):

   661     """Propagate to entities at the end of watched relations when a `main_rtype`

   662     relation is deleted.

   663 

   664     This is the opposite of the :class:`PropagateRelationAddHook`, see its

   665     documentation for how to use this class.

   666     """

   667     events = ('after_delete_relation',)

   668 

   669     def __call__(self):

   670         eschema = self._cw.vreg.schema.eschema(self._cw.entity_metas(self.eidfrom)['type'])

   671         execute = self._cw.execute

   672         for rel in self.subject_relations:

   673             if rel in eschema.subjrels and not rel in self.skip_subject_relations:

   674                 execute('DELETE R %s P WHERE X eid %%(x)s, P eid %%(p)s, '

   675                         'X %s R' % (self.rtype, rel),

   676                         {'x': self.eidfrom, 'p': self.eidto})

   677         for rel in self.object_relations:

   678             if rel in eschema.objrels and not rel in self.skip_object_relations:

   679                 execute('DELETE R %s P WHERE X eid %%(x)s, P eid %%(p)s, '

   680                         'R %s X' % (self.rtype, rel),

   681                         {'x': self.eidfrom, 'p': self.eidto})

   682 

   683 

   684 

   685 # abstract classes for operation ###############################################

   686 

   687 class Operation(object):

   688     """Base class for operations.

   689 

   690     Operation may be instantiated in the hooks' `__call__` method. It always

   691     takes a connection object as first argument (accessible as `.cnx` from the

   692     operation instance), and optionally all keyword arguments needed by the

   693     operation. These keyword arguments will be accessible as attributes from the

   694     operation instance.

   695 

   696     An operation is triggered on connections set events related to commit /

   697     rollback transations. Possible events are:

   698 

   699     * `precommit`:

   700 

   701       the transaction is being prepared for commit. You can freely do any heavy

   702       computation, raise an exception if the commit can't go. or even add some

   703       new operations during this phase. If you do anything which has to be

   704       reverted if the commit fails afterwards (eg altering the file system for

   705       instance), you'll have to support the 'revertprecommit' event to revert

   706       things by yourself

   707 

   708     * `revertprecommit`:

   709 

   710       if an operation failed while being pre-commited, this event is triggered

   711       for all operations which had their 'precommit' event already fired to let

   712       them revert things (including the operation which made the commit fail)

   713 

   714     * `rollback`:

   715 

   716       the transaction has been either rolled back either:

   717 

   718        * intentionally

   719        * a 'precommit' event failed, in which case all operations are rolled back

   720          once 'revertprecommit'' has been called

   721 

   722     * `postcommit`:

   723 

   724       the transaction is over. All the ORM entities accessed by the earlier

   725       transaction are invalid. If you need to work on the database, you need to

   726       start a new transaction, for instance using a new internal connection,

   727       which you will need to commit.

   728 

   729     For an operation to support an event, one has to implement the `<event

   730     name>_event` method with no arguments.

   731 

   732     The order of operations may be important, and is controlled according to

   733     the insert_index's method output (whose implementation vary according to the

   734     base hook class used).

   735     """

   736 

   737     def __init__(self, cnx, **kwargs):

   738         self.cnx = cnx

   739         self.__dict__.update(kwargs)

   740         self.register(cnx)

   741         # execution information

   742         self.processed = None # 'precommit', 'commit'

   743         self.failed = False

   744 

   745     @property

   746     @deprecated('[3.19] Operation.session is deprecated, use Operation.cnx instead')

   747     def session(self):

   748         return self.cnx

   749 

   750     def register(self, cnx):

   751         cnx.add_operation(self, self.insert_index())

   752 

   753     def insert_index(self):

   754         """return the index of the latest instance which is not a

   755         LateOperation instance

   756         """

   757         # faster by inspecting operation in reverse order for heavy transactions

   758         i = None

   759         for i, op in enumerate(reversed(self.cnx.pending_operations)):

   760             if isinstance(op, (LateOperation, SingleLastOperation)):

   761                 continue

   762             return -i or None

   763         if i is None:

   764             return None

   765         return -(i + 1)

   766 

   767     def handle_event(self, event):

   768         """delegate event handling to the opertaion"""

   769         getattr(self, event)()

   770 

   771     def precommit_event(self):

   772         """the observed connections set is preparing a commit"""

   773 

   774     def revertprecommit_event(self):

   775         """an error went when pre-commiting this operation or a later one

   776 

   777         should revert pre-commit's changes but take care, they may have not

   778         been all considered if it's this operation which failed

   779         """

   780 

   781     def rollback_event(self):

   782         """the observed connections set has been rolled back

   783 

   784         do nothing by default

   785         """

   786 

   787     def postcommit_event(self):

   788         """the observed connections set has committed"""

   789 

   790     # these are overridden by set_log_methods below

   791     # only defining here to prevent pylint from complaining

   792     info = warning = error = critical = exception = debug = lambda msg,*a,**kw: None

   793 

   794 set_log_methods(Operation, getLogger('cubicweb.session'))

   795 

   796 def _container_add(container, value):

   797     {set: set.add, list: list.append}[container.__class__](container, value)

   798 

   799 

   800 class DataOperationMixIn(object):

   801     """Mix-in class to ease applying a single operation on a set of data,

   802     avoiding to create as many as operation as they are individual modification.

   803     The body of the operation must then iterate over the values that have been

   804     stored in a single operation instance.

   805 

   806     You should try to use this instead of creating on operation for each

   807     `value`, since handling operations becomes costly on massive data import.

   808 

   809     Usage looks like:

   810 

   811     .. sourcecode:: python

   812 

   813         class MyEntityHook(Hook):

   814             __regid__ = 'my.entity.hook'

   815             __select__ = Hook.__select__ & is_instance('MyEntity')

   816             events = ('after_add_entity',)

   817 

   818             def __call__(self):

   819                 MyOperation.get_instance(self._cw).add_data(self.entity)

   820 

   821 

   822         class MyOperation(DataOperationMixIn, Operation):

   823             def precommit_event(self):

   824                 for bucket in self.get_data():

   825                     process(bucket)

   826 

   827     You can modify the `containercls` class attribute, which defines the

   828     container class that should be instantiated to hold payloads. An instance is

   829     created on instantiation, and then the :meth:`add_data` method will add the

   830     given data to the existing container. Default to a `set`. Give `list` if you

   831     want to keep arrival ordering. You can also use another kind of container

   832     by redefining :meth:`_build_container` and :meth:`add_data`

   833 

   834     More optional parameters can be given to the `get_instance` operation, that

   835     will be given to the operation constructor (for obvious reasons those

   836     parameters should not vary accross different calls to this method for a

   837     given operation).

   838 

   839     .. Note::

   840         For sanity reason `get_data` will reset the operation, so that once

   841         the operation has started its treatment, if some hook want to push

   842         additional data to this same operation, a new instance will be created

   843         (else that data has a great chance to be never treated). This implies:

   844 

   845         * you should **always** call `get_data` when starting treatment

   846 

   847         * you should **never** call `get_data` for another reason.

   848     """

   849     containercls = set

   850 

   851     @classproperty

   852     def data_key(cls):

   853         return ('cw.dataops', cls.__name__)

   854 

   855     @classmethod

   856     def get_instance(cls, cnx, **kwargs):

   857         # no need to lock: transaction_data already comes from thread's local storage

   858         try:

   859             return cnx.transaction_data[cls.data_key]

   860         except KeyError:

   861             op = cnx.transaction_data[cls.data_key] = cls(cnx, **kwargs)

   862             return op

   863 

   864     def __init__(self, *args, **kwargs):

   865         super(DataOperationMixIn, self).__init__(*args, **kwargs)

   866         self._container = self._build_container()

   867         self._processed = False

   868 

   869     def __contains__(self, value):

   870         return value in self._container

   871 

   872     def _build_container(self):

   873         return self.containercls()

   874 

   875     def union(self, data):

   876         """only when container is a set"""

   877         assert not self._processed, """Trying to add data to a closed operation.

   878 Iterating over operation data closed it and should be reserved to precommit /

   879 postcommit method of the operation."""

   880         self._container |= data

   881 

   882     def add_data(self, data):

   883         assert not self._processed, """Trying to add data to a closed operation.

   884 Iterating over operation data closed it and should be reserved to precommit /

   885 postcommit method of the operation."""

   886         _container_add(self._container, data)

   887 

   888     def remove_data(self, data):

   889         assert not self._processed, """Trying to add data to a closed operation.

   890 Iterating over operation data closed it and should be reserved to precommit /

   891 postcommit method of the operation."""

   892         self._container.remove(data)

   893 

   894     def get_data(self):

   895         assert not self._processed, """Trying to get data from a closed operation.

   896 Iterating over operation data closed it and should be reserved to precommit /

   897 postcommit method of the operation."""

   898         self._processed = True

   899         op = self.cnx.transaction_data.pop(self.data_key)

   900         assert op is self, "Bad handling of operation data, found %s instead of %s for key %s" % (

   901             op, self, self.data_key)

   902         return self._container

   903 

   904 

   905 

   906 class LateOperation(Operation):

   907     """special operation which should be called after all possible (ie non late)

   908     operations

   909     """

   910     def insert_index(self):

   911         """return the index of  the lastest instance which is not a

   912         SingleLastOperation instance

   913         """

   914         # faster by inspecting operation in reverse order for heavy transactions

   915         i = None

   916         for i, op in enumerate(reversed(self.cnx.pending_operations)):

   917             if isinstance(op, SingleLastOperation):

   918                 continue

   919             return -i or None

   920         if i is None:

   921             return None

   922         return -(i + 1)

   923 

   924 

   925 

   926 class SingleLastOperation(Operation):

   927     """special operation which should be called once and after all other

   928     operations

   929     """

   930 

   931     def register(self, cnx):

   932         """override register to handle cases where this operation has already

   933         been added

   934         """

   935         operations = cnx.pending_operations

   936         index = self.equivalent_index(operations)

   937         if index is not None:

   938             equivalent = operations.pop(index)

   939         else:

   940             equivalent = None

   941         cnx.add_operation(self, self.insert_index())

   942         return equivalent

   943 

   944     def equivalent_index(self, operations):

   945         """return the index of the equivalent operation if any"""

   946         for i, op in enumerate(reversed(operations)):

   947             if op.__class__ is self.__class__:

   948                 return -(i+1)

   949         return None

   950 

   951     def insert_index(self):

   952         return None

   953 

   954 

   955 class SendMailOp(SingleLastOperation):

   956     def __init__(self, cnx, msg=None, recipients=None, **kwargs):

   957         # may not specify msg yet, as

   958         # `cubicweb.sobjects.supervision.SupervisionMailOp`

   959         if msg is not None:

   960             assert recipients

   961             self.to_send = [(msg, recipients)]

   962         else:

   963             assert recipients is None

   964             self.to_send = []

   965         super(SendMailOp, self).__init__(cnx, **kwargs)

   966 

   967     def register(self, cnx):

   968         previous = super(SendMailOp, self).register(cnx)

   969         if previous:

   970             self.to_send = previous.to_send + self.to_send

   971 

   972     def postcommit_event(self):

   973         self.cnx.repo.threaded_task(self.sendmails)

   974 

   975     def sendmails(self):

   976         self.cnx.vreg.config.sendmails(self.to_send)

   977 

   978 

   979 class RQLPrecommitOperation(Operation):

   980     # to be defined in concrete classes

   981     rqls = None

   982 

   983     def precommit_event(self):

   984         execute = self.cnx.execute

   985         for rql in self.rqls:

   986             execute(*rql)

   987 

   988 

   989 class CleanupNewEidsCacheOp(DataOperationMixIn, SingleLastOperation):

   990     """on rollback of a insert query we have to remove from repository's

   991     type/source cache eids of entities added in that transaction.

   992 

   993     NOTE: querier's rqlst/solutions cache may have been polluted too with

   994     queries such as Any X WHERE X eid 32 if 32 has been rolled back however

   995     generated queries are unpredictable and analysing all the cache probably

   996     too expensive. Notice that there is no pb when using args to specify eids

   997     instead of giving them into the rql string.

   998     """

   999     data_key = 'neweids'

  1000 

  1001     def rollback_event(self):

  1002         """the observed connections set has been rolled back,

  1003         remove inserted eid from repository type/source cache

  1004         """

  1005         try:

  1006             self.cnx.repo.clear_caches(self.get_data())

  1007         except KeyError:

  1008             pass

  1009 

  1010 class CleanupDeletedEidsCacheOp(DataOperationMixIn, SingleLastOperation):

  1011     """on commit of delete query, we have to remove from repository's

  1012     type/source cache eids of entities deleted in that transaction.

  1013     """

  1014     data_key = 'pendingeids'

  1015     def postcommit_event(self):

  1016         """the observed connections set has been rolled back,

  1017         remove inserted eid from repository type/source cache

  1018         """

  1019         try:

  1020             eids = self.get_data()

  1021             self.cnx.repo.clear_caches(eids)

  1022             self.cnx.repo.app_instances_bus.publish(['delete'] + list(str(eid) for eid in eids))

  1023         except KeyError:

  1024             pass