more / cleaner / in code documentation of vreg, selectors and appobject

add note about running repository / client code

added information about the naming conventions in schema.py also: * some typo fix * note about ObjectRelation soon being deprecated * clarified a few hazy points

[doc/book] a new chapter on how to use the ORM

improved doc on selectors an vregistry

fix rest formatting

typo fix

merge

catch and log errors caused by bad json data

list allowed values for ldap protocol setting in help, so that the generated file includes this in a comment

[doc/book] enhance i18n section #656191

[doc] more (doc oriented) feature in list

[doc] enhanced setup section: fix sections'level, more postgres tips, merge environment doc with cwconfig.py module docstring using custom autodocstring directive

[doc] enhanced concepts section

remove ref to no more existing file

[cleaning] simpler implementation of cmd_create_entity

[storages] fix fs_importing side-effect on entity.data When creating a new File object, if fs_importing is set, we want entity.data to be the file content instead of the filepath for the rest of the transaction. (see test_bfss_fs_importing_transparency) for test implementation To make this possible, the storage hooks (entity_added / entity_updated) must return the correct value to set in the entity dict.

nicer warning

[doc] started full features list

[python] take care to this detail of @contextmanager: if an unhandled exception occurs in the block, it is reraised inside the generator at the point where the yield occurred

[entity] explain why the assert failed

[test] fix expected error

fix bad merge

fix #822890, missing argument on abstract source backup method make backup failed on multi-sources instances

merge

[schema] ensure we don't remove has_text relation type unfortunatly

merge

Added tag cubicweb-debian-version-3.7.3-1 for changeset 44c7bf90df71

Added tag cubicweb-version-3.7.3 for changeset 768beb8e15f1

[distrib] fix pkginfo

3.7.3

[hooks] better message on bad .event class attribute

[repository] forbid usage of set_attributes() in before_add_entity set_attributes() will generate a RQL query that will silently fail. An explicit error is better. You can still use the ``entity[attr] = value`` notation that won't generate a SQL query but still update the inner state of the entity and its edited_attributes attribute.

merge

[cleanup] tb already printed by self.exception; add note

[facet] fix 'bookmark this search': * vid wasn't included in the link generated by the facet box * proper url quoting in both js / box code (try an has_text search with '&' inside...)

[widget] more wide textarea on bookmark edition

[distribution] the web/data/images directory has to be handled in pkginfo

[test] use the new temporary_appojects CM to simplfy unittest_storage

[session] try to reconnect on unexpected error in system_sql. We need this for proper reconnection of pool used by hooks or looping task doing sql queries (such as cw_cleanup_transaction)

[source] log attempt to reconnect using warning level. Also, don't relog failed sql (already logged)

[test] avoid critical log by informing the source we consider Personne as a multi-sources type

[test] use new temporary_appobjects context manager

[test] that doesn't smell good but fix the test...

[vreg] we do want the opposite...

[testlib] simple helper method to register temporary appobjects (hooks for instance) with a context manager

[undo] fix transactions table name in cleanup operation

Add context management methode to dbapi.Connection the connection is automaticly closed when exiting the context.

i18n update

[i18n] use named substitution to avoid gettext warnings

[dataimport] print transaction id when we get one

[cleanup] add some notes

[optimisation] _select_best is used very intensivly. Peanuts optimization may count :)

[optimization] traced_selection is used very intensively in debug mode optimizing it is worth the pain. Basically : don't do stuff unnecessary when we don't trace anything.

[optimization] improve massive write performance by optimizing hooks selection profiling on some massive deletion showed up that 2/3 of the time was spent in hooks selection. Those changes make it much more acceptable (through selection is still not negligeable): * use one registry for each event, so we've much less hooks to check when emiting an event as well as no more need for the match_event selector. This required ability to put one appobject into several registries, using a __registries__ class attribute. * check for deprecated .enabled at registry initialization time instead of at selection time A very simple HooksManager class has been reintroduce to choose the right registry on call_hooks. Those optimisations leads to a ~x3 factor of time necessary to delete 16000 entities.

[upgrade] after upgrade, fork command to start instance avoid grave uicfg pb (explicit rules all removed...)

merge

[repo creation] removing existing entities of 'single' cardinality relatino should be considered as 'activeintegrity' hook. Also don't disable that category during repo creation to avoid pb such as two default workflows for one entity types

merge

handle Service Close request correctly (closes #736039) The windows event self._stop_event was not used, and twisted was not getting a chance of terminating cleanly by calling its shutdown event handlers (which would eventually call pyro_unregister). Fixed by calling reactor.stop()

[js form validation] fix case where we've None/null keys in the errors dictionnary

[cleanup] exists imported, use it

[form] CompoundField should not return itself in actual_fields, this avoid error on iliane with 3.7.2 where it's marked as required (from the schema) while value are provided by subfields

[repo] we want a hooks manager and hooks initialized event on quick start (we trigger events such as server_backup / server_restore for instance)

[distribution] sql extensions files now have proper extension, fix manifest file

backport stable

[events manager] onevent decorator may be given extra arguments to give to em.bind

[cleanup] some notes

[reload] on registry reloading, we should change class of users of opened session

[appobject imports] don't include __init__ in package's name. See comment for more info

[rset] cubes such as iliane hi-jack rset description by poping some lines from it. Handle this on RepeatList

[undo] basic support for undoing of entity creation / relation addition

[undo] init entity cache when undoing an entity deletion

[undo] consistent is/is_instance_of processing

[repo] don't need rset description on those queries

[cleanup] style fixes, add nodes, 0.2 cents refactorings

[undo] during undoing, call hooks in the [active]integrity and undo categories

[repo] refactor fti operation to use set_operation and a single operation whatever the number of entities to ft index

[querier] introduce RepeatList class, used to optimize size of data returned for result set description When rql query has no ambiguity, we used to return the same description * N where N is the size of the result set. Returning RepeatList class avoid that multiplication. According to quick benchmark this improve performance for result set whose size is ~ > 50, has very small penalty for rset < 50, and in any case improve the size of data to be transfered over the network through pyro connection.

[repo] on add entity, set cache as soon as possible + fill type/source cache

[repo] more efficient eid cache operations handling based on set_operation; refactor * split CleanupEidTypeCache operation into one operation for deleted eids and another for added eids, and put them in cw.server.hook * for consistency, move fti unindexing handling from the repository to the system source * handling of deleted eids is done in plan execution (ssplanner), no needs to redo it in *delete_info -> no more need for _prepare_delete_info method, killed

[repo] move eschema_eid function from hooks.metadata to server.utils

[primary view] skip virtual rtypes anyway

[test] CubicWebTC.create_user expect an actual query, and we don't need to give session here

[dataimport] stop disabling undo ; commit return transaction id

[logging] demote static file served to debug

[cleanup]

[repo] optimize massive insertion/deletion by using the new set_operation function Idea is that on massive insertion, cost of handling the list of operation become non negligeable, so we should minimize the number of operations in that list. The set_operation function ease usage of operation associated to data in session.transaction_data, and we only add the operation when data set isn't initialized yet, else we simply add data to the set. The operation then simply process accumulated data.

[repo] make etype which should go in deleted_entities configurable: we only need this for types imported from other multi-sources instances

[repo] don't execute child step twice in EntitiesDeleteStep

[form] consider autoform_field_kwargs for meta fields; allow required customization

[workflow] support for automatic transition starting from the initial state

[testlib] create_user support extra kwargs to give to create_entity(CWUser)

fix dataimport for 3.7.2

use lgc.testlib.TestCase

backport stable