[testlib] rework request building in init_authentication The previous code was building a full authenticated request and tried to undo the authentication afterward. The new code just create a bare Request with no authenticated session linked yet. This is much closer of what the actual authentication process does.

[request] drop the user argument for set_session I see no code nor test that use this optional argument. removing it help to clean the session code. The set_session function will soon be deprecated anyway (at the same time than the dbapi) related to #2503918

[web-request] handle default language earlier We now read language negotiation header at initialisation time (previously done during ``set_session``). This simplify the set_session code for later reworking. There is no change in behavior, the user selected language continue to overwrite the one picked from the header/default when applicable. related to #2503918

[etwist] pass the repository to the root resource This is another step toward a cleaner instantiation scheme for the repo. The http test now pass an already created object to the etwist server instead of relying on the config cache. Related to #2249513

[repo] move repo.gc_stats to Service API (closes #2951068) We currently have a method on the repo that anyone can call. `call_service` gives a similar thing, modulo: * can be restricted access to manager only (thanks to selection on services), * less unrelated code on the repository class, * no need to fetch a reference to a repo object from the client side * (`req._cnx.repo` is not an API and config.repository() is on its way out). The old way to access this information (repo.gc_stats()) is deprecated.

[repo] move repo stats to Service (closes #2951067) We currently add a new method on repo that anyone can call. Call service allows to reach the same result with: * Restricted access to manager only (thanks to selection on services), * Less unrelated code on the repo. * No need to fetch a reference to a repo object from client side. `req._cnx.repo` is not an API and config.repository() is on its way out. The old way to access this information (repo.stats()) is deprecated.

[service] drop the asynchronous execution possibility Call_service was able of both sync and sync execution making the API confusing. There is not user of the async case. We drop the async argument in favor of synchronous execution only. This makes call_service the official API to call server side code from the client side. This is a remplacement for the usual monkey patching of the repo object. The zmq notification bus is a solid alternative for codes that needs to start an async execution.

[auth] pass `repo` instead of `vreg` to SessionManager and AuthenticationManager Those object actually need a repo object to connect and create new session object. They used to retrieve the repo object using ``vreg.config.repository()``. I'm trying to sanitise the initialisation stack and get ride of ``config.repository()``, a function that mix factory and cache. Retrieving the vreg from the repo is trivial (``repo.vreg``) so we now pass the repo object instead of the vreg.

web/application: instantiate the repository outside of CubicWebPublisher This improves decoupling and allows ``CubicWebPublisher`` user to choose which argument are used during ``Repository`` instantiation. In particular is allows caller to provide a ``TaskManager``. The `vreg` argument of publisher is made mandatory. See CubicWebPublisher docstring for details. Related to #2249513

remove vreg argument to CWPublisher The passing a vreg to CWPublisher was used by test to allow reused of an existing vreg. We dropped this feature two commit ago and can safely removes it. Removing this argument allows later cleanup and improvement on the CWPublisher and Repository front. closes #2944813

[devtools] vreg becomes a class ppty After multiple devtools refactoring, the vreg used for test is now *always* got from the repository. Making this explicit is a step toward simplification of the vreg handling during various initialisation phases. Related to #2944813

[devtools] drop unused `reset_schema` and `reset_vreg` class attribute They have no effect and no user... Related to #2944813

[connection] move security control logic on Connection The _security_enabled context manager now operate on connection. We have to keep a specific entry point in Session to ensure Connection object are properly cleaned up when using session to "manage" Connection live cycle. Related to #2503918

[connection] move hook control logic on Connection The _hook_control context manager now operate on connection. We have to keep a specific entry point in Session to ensure Connection object are properly cleaned up when using session to "manage" Connection live cycle. Related to #2503918

[session] update _hook_control docstring

[connection] reinstall cnx.data as cnx.transaction_data Too much code expects self._cw.data to be session data. backout 9b2f68916474 Related to #2503918 Related to #2912807

[connection] give access to is_internal_session boolean Needed if we are to use this object like we currently do with session. Related to #2503918

[req] drop from_controller on non WebRequest object (Closes #2901079) The `controller` concept is purely a web things. It does not belong to generic `req` object. The `build_url` code is slightly changed to handle that. This probably requires extended cleanup later. This is a barely used and very internal API. No deprecation period

[connection] add logging method on connection If we are to use this object like we currently do with session, we need to have the logging method as a lot of code use them.

rename server.session.transaction into server.session.connection The ongoing rework of the API to access the database include the splitting of Session in two objects: The ``Session`` object will only hold credential (user) and global session data. A new object will publicly emerge to handle a database access. This object is named ``Connection`` since that the way database accessors with the same property are named in other system. So we renamed the ``Transaction`` object into ``Connection``. The ``Transaction`` object have already grown in the direction of something directly usable by generic code, but the name ``Transaction`` is ill suited as such object can be used for multiple database transaction in a row. Related to #2503918

[application] call req.set_session in application.main_handle_request The Session handling chain is no more responsible for calling req.set_session. It just returns a valid session and lets the caller link it to the Request. This opens the way to explicitly creating and closing a connection/transaction in ``application.main_handle_request``. Related to #2503918

[session-handler] use session directly to update last usage We don't really need the WebRequest for that. Not using the WebRequest to access the cubicweb repository here will allow a delayed set_session. Related to #2503918

[application/connect] simplify connection logic ``application.connect`` now either sets a full featured ``DBAPISession`` to the ``WebRequest`` object or raises ``AuthenticationError``. The creation and usage of a fake DBAPISession is now handled by ``main_handle_request`` when needed. This means that fake DBAPISession are no longer tracked by the session manager and that user are not given anyway to retrieve them for a later request. This fake DBAPISession is still passed to ``core_handle`` because multiple cubes like registration or forgotten password need this behavior. We would like to get ride of it in the future. This clarification of the connection API greatly simplifies ``DBAPISession`` retrieval//creation process opening the way to improvements in this area. Related to #2503918

Drop hijack user (closes #2901093) Deprecated since 3.17.0. Dropping this function help the session cleanup business.

[sources] drop support for ldapuser source (closes #2936496) The ldapfeed source is a replacement for ldapuser. Use it instead.

merge stable back into default

3.17 is the new stable

Added tag cubicweb-version-3.16.5, cubicweb-debiann-version-3.16.5-1, cubicweb-centos-version-3.16.5-1 for changeset 810a05fba1a4

[pkg] prepare 3.16.5

Added tag cubicweb-version-3.17.2, cubicweb-debian-version-3.17.2-1 for changeset 195e519fe97c

[pkg] prepare 3.17.2

repository: make tests pass again The tests use mono-threaded pyro, which breaks assumptions made in e27337dfec8c. To fix that, remove sessions from the _pyro_sessions dict when they're closed, and force the test pyro client to actually disconnect when it's done.

repository: monkey patch pyro connection handling to detect clients going away When a pyro client goes away we need to close their session (to release their cnxset). Add a dict to the repository mapping pyro client threads to session object. Closes #2932058

notification: use viewargs for notif_entity_updated instead of transaction_data Now that notifications use separate sessions, they can't rely on the original transaction_data being around, so pass the data through view arguments instead, so the notification view knows what changed on updated entities. Closes #2936180

notification: properly handle cnx and lifetime of the hijacked session When we create a session for the notification rendering, we need to 1) give it a cnxset 2) commit and close it after we're done 3) restore the original session for subsequent notifications This changes the ordering of actual mail sending, since there are several different commits involved, but I don't see a way to fix that short of restoring hijack_user. Related to #2934523

notification: fix session creation (closes #2934523) Session() takes a user and repo, not repo and user.

Added tag cubicweb-version-3.17.1, cubicweb-debian-version-3.17.1-1, cubicweb-centos-version-3.17.1-1 for changeset f98d1c46ed9f

[cwctl] add configure command to cw-ctl (closes #2709702) usage: cw-ctl configure <appid> --param key=value,key2=value2 --param key3=value3

[cwctl] rename option no-post-create to no-db-create

[pkg] version 3.17.1 for rpm packages

[session] upgrade session closed error from Exception to SessionClosedError Exception is far too wide and we already have a SessionClosedError anyway. Closes #2897696

[pkg] prepare 3.17.1

[web/application] add some minimal documentation Some documentation to the Application main entry point.

[session] raise proper exception with get_tx is called on a closed session Prevent attribute error on _txs Closes #2897836

[view] typo in a docstring

[cwctl] pass missing inputlevel argument to postcreate() (closes #2901037)

[server] fix documentation of ZMQ options harder The syntax for zmq addresses is tcp://host:port, no zmqpickle involved.

merge with 3.16.x fixes

[server] fix documentation of ZMQ options

[zmq] set order for ZMQStartHook so other hooks don't need to If an application wants to subscribe to some messages on the zmq bus, that needs to happen after ZMQStartHook.

import merge_options directly from logilab.common

[server] fix error messages in write_sources_file()

[doc] a couple almost-typos

[schemaserial] mitigate critical message during migration We can't fetch extra_props because it is not yet there.

[doc] typo

[cw-ctl] fix help message for schemadiff In global help (closes #2888538) The leading space make it appears on multiple line. The help content is a bit improved in the process.

[dbapi] fix connect backwards compat harder Passing None as group and host should result in "pyro:///cubicweb.appid", not "pyro://None/None.appid". Followup for 2091d275fe5c. Closes #2882666.

[zmq] make publish address optional It should be possible to use the zmq communication bus in a read-only manner. Allow setting zmq-address-sub without zmq-address-pub. Closes #2897178.

Fix documentation for zmq pub/sub hooks and configuration Closes #2897177

[entity] improve deprecation messages

[devtools] use self._parse so AssertionError is properly raised instead of lxml error (test failure introduced in 6711f78c18be)

[testlib] fix page validator selection. Closes #2869456 * use a validator that checks XML well-formness for the html5 doctype (as a convenient side-effect, it does not choke on e.g. a canvas tag like the HTMLParser does) * use a DTD validator if there is an xml declaration * else use the HTMLValidator

[testlib] introduce a validator that check xml-well formness This validator simply check XML is well formed and accept any entities (think HTML defines much more entities than bare XML)

[view] return HTML5 doctype. Closes #2869426 We dropped xhtml support in 3.17 but we kept providing documents using the XHTML strict doctype, while our content wasn't conform to the DTD since we dropped proper namespaces definition as well as declaration of cubicweb DTD extensions.

[testlib] repreprocess content so contextual display has a chance to show the proper line. Closes #2869481 the context is calculated using position information relative to a preprocessed content, not the original source

[web request] drop no more necessary request.document_surrounding_div method This is awful and no more necessary since we dropped xhtml support

[testlib] unspaghettify Validator / PageInfo api

[testlib] update htmlparsers.VALMAP: stop using SaxOnlyValidator and add an entry for html

[testlib] SaxOnlyValidator is deprecated, use XMLValidator instead

[dataimport] minor typo in error handling

[views] attempt to clarify the inline help system documentation

[sobject] fix notification operation regression introduced in bf4003760e02 Operation use self. session. not self._cw. I'm a bit surprised that its not tested. closes ##2870077

Added tag cubicweb-debian-version-3.17.0-2 for changeset 09a0c7ea6c3c

[pkg/debian] prepare release 3.17.0-2

[pkg/debian] update yams runtime depends Only the built depends were updated to 0.37.0.

[pkg/debian] expand all depends and co on multiple line This is more readable and simplifies patching.

Added tag cubicweb-debian-version-3.17.0-1 for changeset 22be40c492e9

Added tag cubicweb-version-3.17.0 for changeset cc1a0aad580c

[test/view] exclude some Blog entity from automatic view testing Some Blog view requires the sioc cubes that we extracted in 3.17 we do not really want to test cubes view here. So we exclude CWEtype that really requires the sioc cubes.

[doc] fix: cubicweb.gmap.js has moved to the geocoding cube

[test/ldap] do not assume order Do no assume a specific order is stable when testing group members

Added tags for version 3.16.4 on changeset 041804bc48e9

[notification] ensure official API is test By using the official API in the hook we ensure it is tested.

fix typo in notification causing NameError introduced by: f5b40b66d36e

[test/notification] merge related test classe They have no reason to be distinct.

merge with stable

[server] add a looping task to free unused cnxsets capture by dead pyro When a pyro client disconnects, we're not necessarily notified so they can hang on to their session/cnxset forever. Don't let them do that. The hacky looping task is only installed when pyro is enabled. This optional activation save this very hacky patch from being rejected. closes #2825380

[pkg/debian] update changelog to 3.17.0

[devtools] Use XMLValidator instead deprecated SaxOnlyValidator The renaming was introduced in 198fdadafed6.o This prevent a meta class conflict introduced by logilab-common 0fcd2e050621.

[pkg] update changelog

[pkg] prepare 3.17.0

[pkg] prepare 3.16.4

[repo] straightforward bootstrap sequence. Closes #2841188 * kill fill_schema method which was doing to much different things, in favor of a deserialize_method which is simply reading schema from the db and returning it * set_schema doesn't bother with cubes This allows to easilly follow what's going on and so to cleanup init_cnxset_pool to get the following boostrap order: 1. load cubes (implies loading site_cubicweb) 2. load schema 3. init data sources 4. init cnx sets

[repo] minor cleanups to bootstrap sequence * fix some docstring/comments * move initialization of HooksManager to the __init__.py, this has nothing to do with pool initialization / boostraping

[schemaserial] serialize additional yams parameter for customs type Yams have a new custom type feature. We add support for extra parameter on CWAttribute in schema serialization handling. Closes #2847151

[sql] preprocess_entity uses lgdb helper's SQL converters. The preprocess_entity does not yield anymore the conversion logic for different base types, but relies on a new SQL_CONVERTERS dictionary of the db helper. This will allow later inclusion of new base types.

[repo] kill rebuildinfered feature from Repository.set_schema This clutters thing and is useless in most cases: when schema just have been readen from the database or from the file-system, infered relations have just be calculated. The only use-case where it should be rebuilt is on migration (ie schema modification by editing the meta-model using rql), so move this feature there.

[test] stop deepcopying for nothing

drop xhtml content-type support (closes #2065651) * HTMLStream does not care about xml any more * reqquest.demote_to_html and .xhtml_browser are deprecated * web config: drop force-html-content-type option * adjust tests

[htmlparser] exclude <script> tag from html source lxml are confused by them :-(

[htmlparser] store unaltered source in pageinfo.source Do not store the parser to preserve the source aspect. Store the initial input in the source argument instead. This is very useful when parsing html. In such case we need to drop ``<script>`` tag.

[htmlparser] rename SaxOnlyValidator to XMLValidator

[htmlparser] add missing deprecation message

[form] remove hard coded id html id should be unique. It is clearly not achieved here.

[server/test] use server config in migration This allows a lighter initialisation (no rtag needed). Execution of those test move from 300 secondes to 220 seconds here. A 25% speedup

[session] deprecate `hijack_user` method The semantic of this method is wicked and lead to very hard violation of sanity. (multiple transaction using the very same cursors). We deprecated the API to be able to drop it as soon as possible. The method was added long ago for some notification trick. It is not needed since we sent notification on ``postcommit_event``. (closes #2781782)

[mail] drop hijack_user usage The transaction is now commited before notification. We can create a new session directly and drop `hijack_user` usage. This prepare #2781782

[notification] merge BaseNotificationView and NotificationView They are in the same module now.

[notification] move notification view in ``sobject.notification`` It has no user outside this module. This enforce serversideness of notification and allow future cleanup. No backward compat is set up to prevent circular import. The class has no other user anyway. (closes 2845144)

remove unused import pyflakes say they are unused. Tests suite confirms.

one more merge

[notification] introduce an official `notify_on_commit` function It provides a proper notification api instead of replacing it by yet another clumsy one. Closes #2837251

[cubicweb/doc] Replace dc_type() by cw_etype dc_type() is may be translated. Leading to terrible terrible damage.

[dataimport] backout 6947201033be (related to #2788402) (and add a try: except to cache the intended error) The problem actually comes from the ``MassiveObjectStore`` in the ``dataio`` cube, so it should be corrected there. Here, we only protect it with a ``RuntimeWarning`` so that the user can see the problem. ``value`` is set to ``None`` (whence to ``NULL`` from a database standpoint), so that the data can be nevertheless inserted in the database. However, only the keys present in ``row`` are actually non-'``NULL``'. The real solution is to work out the issue in ``MassiveObjectStore`` directly. The current try/except should only be a temporary hack.

merge default heads

[c-c serverctl] import locally to speed up c-c

[facet js] minor refactoring and cleanups * extract facetCheckBoxSelect and facetCheckBoxUnselect * use $ as prefix for jQuery variable (facet -> $facet) * other minor rephrasing

[ldapfeed] Add support for LDAP groups (closes #2528116) Groups from the LDAP server are imported as CWGourp entities, and in_group relationships are created between existing CWUsers and CWGroups Unit tests have been refactored a bit, especially ti add helper methods to manage LDAP database.

[test/ldap] small improvement to ldapfeed unit tests better group testing and further testing of imported user life cycle.

[ldap] prepare import of CWGroup All CWUser specific code is put in dedicated sections.

[ldap] handle modification date We are now able to import modification date from ldap.

[ldap] refactor attributes mapping handling * use explicit attributes lists when calling ldap search * direct and reverse attributes maps contains now the same elements.

[ldapfeed] add support for multiple email addresses from ldap Multiple emails case was ignored before this change. email test

merge 3.16.x fix in 3.17.x branch

[session] fix hooks_control backward compat Session object have no ``hooks_control`` control method. it has ``allow_all_hooks_but`` and ``deny_all_hooks_but`` ones. Regression introduced in bd5b5759c9b3.

[facet js] fix reordering of facet check boxes. Closes #2732947 Before this patch, when one select an element, it's moved to the top of the select content. Fine. But when it's later deselected, it stays there instead of moving back to its original location. This patch fixes that by introducing a facetCheckBoxReorder function which properly reorder the whole facet, instead of buggy attempt to locally reorder.

[vreg] (re)build entity classes cache after registry initialization. This fix potential memory fragmentation issue. Closes #2719113

[task] allow negative intervals for add_looping_task (closes #2818280) Instead of rejecting values < 0 with a ValueError, we ignore tasks with a negative interval. This introduce a simple way to disable task from configuration.

[task] allow interval=0 for looping tasks There is valid usecase for perpetual running task.

[portable dump] skip virtual relations, they have no table associated. Closes #2841199

[test] fix skipIf import, currently failing if unittest2 is not there

[test] fix bad merge

merge with other default head

[ldap] an empty 'user-base-dn' disable the user importation process, The same semantic will be used for group. We start by using it for user for the sake of consistency.

drop commented print

[ldap] Major test refactoring Mostly fix tests (previous refactoring), but also refactor a bit the base class to have better/simpler interaction with the slapd server.

[test/ldap] have RQL2LDAPFilterTC use its own database I do not know what RQL2LDAPFilterTC is about and why it does this strange reuse of LDAPUserSourceTC pre_setup_database but it is certainly a good idea to have it use a dedicated database-cache.

[ldap] quieter tests Capture slapadd and slapd output if everything goes well.

[test/ldap source] better source naming, some cleanups (prepares #2528116) ldap uri becomes "ldap" in all cases prepare separation of CONFIGs by source type

use cw_etype instead of __regid__ We have a clean public API now.

[entity] add an official ``cw_etype`` property for entity This property holds the CWEtype id of the entity. This information was previously available through __regid__ but this is an ugly public API. So we have a new clean public API. The ``dc_type()`` method could not be used as it is translated. (closes #2793792)

[notification] use new style operation for notification This results in far less operations created. Operation creation is expensive. We keep some compatibility with the previous API because some cubes have been reported to use it.

[cleanup] drop code deprecated since 3.6 The 3.6 API have been dropped some time ago. Those three lines survived the purge. One of them is even deprecated since 3.5.

[notification] fix documentation The doc say precommit, the code says postcommit. The code is right. We also clarify the documentation of another related operation looking suspiciously wrong but actually correct.

[ldapuser] Deprecated the source We have ldapfeed for some time. maintaining both is cumbersome. This old style source will be dropped in 3.18. closes #2567712

[rql2sql] Remove an XXX by explaining why Int non-substitution is desired (in the current state of the API)

[rql2sql] Boolean values should be substitued like others. Closes #2829481 Furthermore lgdb 1.10 deprecate boolean_value method and this simplify testing.

[toward-py3k] rewrite __cmp__ (closes #2715115) In Python 3k `__cmp__` special method is no longer supported. So we split it in `__lt__` and `__eq__`. details on Python3k behavior here: http://docs.python.org/3.0/whatsnew/3.0.html#ordering-comparisons

skeleton: add RPM spec template (closes #2800884)

[clean_session] revert itervalues usage from 0bb18407c053 self.close(session) changes the self._sessions content. We need to create a full list of value before starting closing them.

merge 3.16.x fix

[ldap] Use correct API for logging message The good practice cat prefer logging to do the substitution himself.

Fix obsolete docstring for add_looping_task The docstring was telling lies since changeset cae198371548 "add_looping_task can be used any time"

[devtools] prevent Xvfb resets Seems like firefox opens more than one connection to the X server in some cases, and things get utterly confused.

[LDAP] rename unittest ldapuser to ldapsource (prepares #2528116) the tests involved are not only for ldapuser module but also for the more general ldapfeed Necessary to prepare LDAP patch for CWgroup

[devtools] protect xvfb-run.sh against clean_up being called twice

Added tag cubicweb-version-3.16.3, cubicweb-debian-version-3.16.3-1, cubicweb-centos-version-3.16.3-1 for changeset ee860c51f56b

3.16.3

[service] make sure synchronous call_service has a cnxset Fixes regression in 3.16.2 (closes #2807720)

Merge testfunc testfunc resurrection Closes #2535377

[web] move qunit test back into web and fix the hanging qunit test can not work outside web. a lot of try/finally clause have been added to prevent qunit to hang on error. A not a test a indeed broken. But we at least put in them a running state again..

[test] drop forsaken windmill test (Closes 2535376) * not run since multiple years * windmill itself is in poor shape not even packaged in Debian. The peace which other seek they find; The heaviest storms not longet last; Heaven grants even to the guiltiest mind An amnesty for what is past; When will my sentence be reversed? I only pray to know the worst; And wish as if my heart would burst.

Added tag cubicweb-debian-version-3.16.2-1 for changeset b3c1ad0cbf00

Added tag cubicweb-version-3.16.2 for changeset a2b4f245aa57

3.16.2

[server/test] remove dependency on a pyro name server (closes #2801737) Use a direct pyroloc:// url instead of going through a NS.

[packaging] add spec file to build RPMs (closes #2800850)

[predicates] keep making `yes` predicate importable from cubicweb (closes #2790319) The `yes` predicate was moved out in CubicWeb 3.15 but it should have remained importable from the cubicweb.predicates module, for bw compat reasons and also because logilab.common.registry is an implementation detail.

[views/tables|css] backout rev 895e34d9ae0d This changeset introduced a GUI behaviour change that has usability side-effects (eg. shows a non-undoable massive action as a first-class action). The idea is good, but it requires a better implementation; tabs are not the proper GUI solution to the original usability issue.

[webconfig] cleanly handles ``fckeditor_installed`` when no uiprops The previous version of the code relied on a default value with a dummy ``FCKEDITOR_PATH`` entry. The new code check if uiprops is initialized or not.

[webconfig] fallback to default configs if https ones are not defined When cubicweb serves a request with an https scheme, it tries to access ``https_uiprops`` and ``https_datadir_urls``. However if those are not defined we should fallback to unsecure url instead of crashing with a traceback. There are very valid usecases for undefined https url. For example if your ``base_url``, ``datadir_url`` and uiprops content are already https urls. Closes #2786643

[webconfig] explicit some webconfig attribute Explicit declaration of attribute are much clearer. Just do it.

merge 3.16.x fix in 3.17.x branch

merge with 3.15.x fix

Added tag cubicweb-debian-version-3.15.11-1 for changeset 09d65bc1f025

Added tag cubicweb-version-3.15.11 for changeset 38c6a3ea8252

[querier] improve 'Password selection not allowed' error message to report the involved variable name whose type is Password, it usually help understanding the problem.

[predicates] use select_or_none rather than select+try/except, expected to be more efficient

[pkg] prepare version 3.15.11 ?

merge 3.14.x fix into 3.15.x

[facet] ensure a facet DOM id is a valid jQuery identifier. Closes #2789089

[predicates] fix 'adaptable' implementation, used to return 0 on a rset with individualy adaptable entities of different types. Closes #2709663

[cmd] fix exlog documentation (closes #2715913) The table documentation was out of sync since 8a075bc94563.

Added tag cubicweb-debian-version-3.14.10-1 for changeset 197bcd087c87

Added tag cubicweb-version-3.14.10 for changeset 0ff798f80138

[pkg] prepare 3.14.10 release

[querier] fix eid relations handling in SET queries (closes #2797052) _extract_eid_consts() implementation must consider arithmetic operators when trying to associate rql variables to specific eids.

[ext/rest] add directive bookmark to rest (closes #2545595)

[c-c create] make post-create step optional (closes #2712041) In this context the "post-create" step refers to the database's creation and initialization. We want to be able to create a new instance but skip the database creation (`db-create` and `db-init` commands) in automatic setup.

[cwconfig] Add optional configuration param to load a UI cube before other cubes Needed later on to load UI cubes such as legacyui, and one day squareui.

[web/views] Move massmailing to its own cube (closes #2788086)

merge with stable fix

[req] explicit `user` attribute The `RequestSessionBase` object needs a `user` attribute. It is initialized by sub class, but having it explicitly declared is much cleaner.

[test] use the right order in some assertEqual The unittest2 function use ``(expected, got)`` order.

[session/transaction] move resource accessors on Transaction

[session/transaction] move multiple utility on Transaction

[session/transaction] handle cnxset repository logic in transaction Interacting with the repo is now handled by the Transaction code directly.

[transaction] keep a reference to the repo object

[session] make security_enabled API private The doc already says "internal API". The old entry point is now officially deprecated and the context manager have been made private.

[session] make hook_control API private The doc already says "internal API". The old entry point is now officially deprecated and the context manager have been made private.

[service] split session retrieval and service execution Internal session are not tracker by the repository and can not be retrieved using _getsession. Moreover their session id is not unique… Closes #2735700

[service] handle cnxset in the async case only The service may be called by a session, we really do not want the call to service to mess with the transaction state. This does not apply to async were thread magic ensure a dedicated transaction.

[service] extract session retrieval from "task" closure The session was retrieved there because `set_cnxset` must be called in the same thread as the code using it. We now retrieve the session beforehand and explicitly call `set_cnxset`.

[cubicweb/doc] Add tutorial on data import in CubicWeb. This involves creating the "tutorials/dataimport" directory structure under "cubicweb/doc" and, inside the "dataimport" directory, putting several files: - a ResT file containing the tutorial *per se*; this tutorial addresses the following issues: * creating a CubicWeb schema for representing a given data set (here, the Diseasome RDF data, for illustration purposes); * parsing the data; * importing the data, by using several stores: + the ``RQLObjectStore``, ``NoHookRQLObjectStore`` and ``SQLGenObjectStore`` from the ``dataimport`` module in CubicWeb; + the ``MassiveObjectStore`` from the ``dataimport`` module in the ``dataio`` cube. The tutorial also provides timing benchmarks of the various stores. - a set of Python files illustrating the data import, in the context of Diseasome RDF data parsing: * a Diseasome RDF data parse module, * a Diseasome data import module, * a CubicWeb schema for representing Diseasome data.

[dataimport] Slight message modification in exception handling code.

[dataimport] Handle various data formats when creating buffers from data. :note: On the long run, this should be addressed before, when these data are created.

[dataimport] Add a ``flush`` method for all stores. For ``ObjectStore``, ``RQLObjectStore`` and ``NoHookRQLObjectStore`` the method tries to call the ``commit`` method. For ``SQLGenObjectStore``, the ``flush`` method is basically unchanged with respect to previous changesets.

[dataimport] Uniformize the API across the different stores. This is achieved by modifying the ``relate`` method so that it takes an extra ``**kwargs``. More specifically, ``SQLGenObjectStore``'s ``relate`` method needs the the type of the subject entity which is passed through ``**kwargs`` as the ``subjtype`` keyword argument. Actually, it is the ``add_relation`` method of the ``SQLGenObjectStore`` who needs this argument. However, as this method is not called directly (but via the ``relate`` method), the ``subjtype`` argument is passed to ``add_relation`` via ``relate``. The other stores' ``relate`` methods do not need this extra argument, hence for the other stores ``**kwargs`` is empty. In this manner, the API is unified across the different stores.

[data] Add a CSS for undo UI rendering The nesting of ordered list and unordered list was badly handled. This has gone unoticed becaused it was not used. But the undo feature use this markup to render the changesets. That's why this minor change to ths CSS is introduced.

[web/views] extract cube geocoding (closes #353000)

[c-c list] add an optional argument to the list command (closes #2709703) This optional argument allows to restrict listing to only a given type of items (cubes, instances, configurations).

default base-url now uses fdqn instead of hostname only (closes #2542815)

[editcontroller] a small debugging help (closes #2518980)

[doc] small doc improvement

[doc] fix Pyro url

[doc] fix Narval project url

[tutorial] minor documentation fixes

merge 3.16.x fix in 3.17.x

merge 3.15.x into 3.16.x

server/source/native: fix wrong usage of .lstrip that produce garbled error messages (closes #2777641) Fixed an .rstrip(...) for the general case (untested). Fixed an .lstrip(...) for the sqlite path (tested).

[doc] update 3.17 release note

[migration] fix bug in `CWAttributeAddOp.revertprecommit_event` The `rdefdef` attribute is set during the operation execution. In some the operation crash before this assignment. In such case `revertprecommit` raise an attribute error crashing the whole process and shadowing the original error. This changeset detect and Avoid this situation.

[session] security context directly use transaction object when applicable Most of the security logic have been moved on Transaction anyway.

[session] move context manager logic in the security context manager The `init_security` and `reset_security` method have no other user that the context manager itself. Having dedicated public function is not useful.

[session] hook control context directly use transaction object when applicable Most of the hooks control logic have been moved on Transaction anyway.

[session] move context manager logic in the hook controls context manager The `init_hooks_mode_categories` and `reset_hooks_mode_categories` have no other user that the context manager itself. Having dedicated public function is not useful.

[session] drop the useless set_hooks_mode method It has not external user and trivial code

[session/transaction] move most undo support into transaction A small part that need repo access has not been moved into transaction yet.

[session] simplify mode property We can delegate very thing to transaction utility

[session/transaction] pass the session object to the Transaction __init__ This simplify access to variables. As we are moving more and more logic and states in the Transaction it will greatly helps in future commit.

[session] use a dedicated class to track cnxset We introduce a new CnxSetTracker to track `cnxset` used by Transaction and allows to wait for them. This new class does not use Thread ID not thread joining to work. This allows to use multiple transaction per thread and a transaction in multiple thread. The class itself is totally threadsafe by the Transaction is still not thread safe. The old _threads_in_transaction attribute is dropped in favor of a new logic based on this object. The registration of cnxset used is not done by the Transaction itself. tx.cnset is a property handling the Consistency of its value with the CnxSetTracker instance. Note: The CnxSetTracker instance only track transaction id, not transaction itself, So not reference cycle are created.

[session] cleanup Transaction.cnxset The attribute should always be there. It is None when not cnxset exists.

[session] drop useless getter and setter for security Those function have very few user and does not anything fancy. Internal user now access tx.read/write_security directly. External user have been blessed with a writable property. They are only interested in the write part of they did not used the return of the removed function. They access it through the session since I prefer the external world to not be aware of Transaction yet.

[session/transaction] move read_security subtlety in transaction This is transaction data, itshould be handled by the transaction itself.

[transaction] initialize dbapi_request in __init__ There is not reason to have it implicitly declared. The attribute is renamed to match its public name.

[whatsnew] notify drop of deprecated code

[web/views] extract cube embed (closes #1916015)

[skeleton] Update debian packaging to support only python 2.6 and up

Drop iprogress code (closes #2777628)

[web/views] extract cube sioc (closes #1916018)