[connection] give access to is_internal_session boolean Needed if we are to use this object like we currently do with session. Related to #2503918

[req] drop from_controller on non WebRequest object (Closes #2901079) The `controller` concept is purely a web things. It does not belong to generic `req` object. The `build_url` code is slightly changed to handle that. This probably requires extended cleanup later. This is a barely used and very internal API. No deprecation period

[connection] add logging method on connection If we are to use this object like we currently do with session, we need to have the logging method as a lot of code use them.

rename server.session.transaction into server.session.connection The ongoing rework of the API to access the database include the splitting of Session in two objects: The ``Session`` object will only hold credential (user) and global session data. A new object will publicly emerge to handle a database access. This object is named ``Connection`` since that the way database accessors with the same property are named in other system. So we renamed the ``Transaction`` object into ``Connection``. The ``Transaction`` object have already grown in the direction of something directly usable by generic code, but the name ``Transaction`` is ill suited as such object can be used for multiple database transaction in a row. Related to #2503918

[application] call req.set_session in application.main_handle_request The Session handling chain is no more responsible for calling req.set_session. It just returns a valid session and lets the caller link it to the Request. This opens the way to explicitly creating and closing a connection/transaction in ``application.main_handle_request``. Related to #2503918

[session-handler] use session directly to update last usage We don't really need the WebRequest for that. Not using the WebRequest to access the cubicweb repository here will allow a delayed set_session. Related to #2503918

[application/connect] simplify connection logic ``application.connect`` now either sets a full featured ``DBAPISession`` to the ``WebRequest`` object or raises ``AuthenticationError``. The creation and usage of a fake DBAPISession is now handled by ``main_handle_request`` when needed. This means that fake DBAPISession are no longer tracked by the session manager and that user are not given anyway to retrieve them for a later request. This fake DBAPISession is still passed to ``core_handle`` because multiple cubes like registration or forgotten password need this behavior. We would like to get ride of it in the future. This clarification of the connection API greatly simplifies ``DBAPISession`` retrieval//creation process opening the way to improvements in this area. Related to #2503918

Drop hijack user (closes #2901093) Deprecated since 3.17.0. Dropping this function help the session cleanup business.

[sources] drop support for ldapuser source (closes #2936496) The ldapfeed source is a replacement for ldapuser. Use it instead.

merge stable back into default

3.17 is the new stable

Added tag cubicweb-version-3.16.5, cubicweb-debiann-version-3.16.5-1, cubicweb-centos-version-3.16.5-1 for changeset 810a05fba1a4

[pkg] prepare 3.16.5

Added tag cubicweb-version-3.17.2, cubicweb-debian-version-3.17.2-1 for changeset 195e519fe97c

[pkg] prepare 3.17.2

repository: make tests pass again The tests use mono-threaded pyro, which breaks assumptions made in e27337dfec8c. To fix that, remove sessions from the _pyro_sessions dict when they're closed, and force the test pyro client to actually disconnect when it's done.

repository: monkey patch pyro connection handling to detect clients going away When a pyro client goes away we need to close their session (to release their cnxset). Add a dict to the repository mapping pyro client threads to session object. Closes #2932058

notification: use viewargs for notif_entity_updated instead of transaction_data Now that notifications use separate sessions, they can't rely on the original transaction_data being around, so pass the data through view arguments instead, so the notification view knows what changed on updated entities. Closes #2936180

notification: properly handle cnx and lifetime of the hijacked session When we create a session for the notification rendering, we need to 1) give it a cnxset 2) commit and close it after we're done 3) restore the original session for subsequent notifications This changes the ordering of actual mail sending, since there are several different commits involved, but I don't see a way to fix that short of restoring hijack_user. Related to #2934523

notification: fix session creation (closes #2934523) Session() takes a user and repo, not repo and user.

Added tag cubicweb-version-3.17.1, cubicweb-debian-version-3.17.1-1, cubicweb-centos-version-3.17.1-1 for changeset f98d1c46ed9f

[cwctl] add configure command to cw-ctl (closes #2709702) usage: cw-ctl configure <appid> --param key=value,key2=value2 --param key3=value3

[cwctl] rename option no-post-create to no-db-create

[pkg] version 3.17.1 for rpm packages

[session] upgrade session closed error from Exception to SessionClosedError Exception is far too wide and we already have a SessionClosedError anyway. Closes #2897696

[pkg] prepare 3.17.1

[web/application] add some minimal documentation Some documentation to the Application main entry point.

[session] raise proper exception with get_tx is called on a closed session Prevent attribute error on _txs Closes #2897836

[view] typo in a docstring

[cwctl] pass missing inputlevel argument to postcreate() (closes #2901037)

[server] fix documentation of ZMQ options harder The syntax for zmq addresses is tcp://host:port, no zmqpickle involved.

merge with 3.16.x fixes

[server] fix documentation of ZMQ options

[zmq] set order for ZMQStartHook so other hooks don't need to If an application wants to subscribe to some messages on the zmq bus, that needs to happen after ZMQStartHook.

import merge_options directly from logilab.common

[server] fix error messages in write_sources_file()

[doc] a couple almost-typos

[schemaserial] mitigate critical message during migration We can't fetch extra_props because it is not yet there.

[doc] typo

[cw-ctl] fix help message for schemadiff In global help (closes #2888538) The leading space make it appears on multiple line. The help content is a bit improved in the process.

[dbapi] fix connect backwards compat harder Passing None as group and host should result in "pyro:///cubicweb.appid", not "pyro://None/None.appid". Followup for 2091d275fe5c. Closes #2882666.

[zmq] make publish address optional It should be possible to use the zmq communication bus in a read-only manner. Allow setting zmq-address-sub without zmq-address-pub. Closes #2897178.

Fix documentation for zmq pub/sub hooks and configuration Closes #2897177

[entity] improve deprecation messages

[devtools] use self._parse so AssertionError is properly raised instead of lxml error (test failure introduced in 6711f78c18be)

[testlib] fix page validator selection. Closes #2869456 * use a validator that checks XML well-formness for the html5 doctype (as a convenient side-effect, it does not choke on e.g. a canvas tag like the HTMLParser does) * use a DTD validator if there is an xml declaration * else use the HTMLValidator

[testlib] introduce a validator that check xml-well formness This validator simply check XML is well formed and accept any entities (think HTML defines much more entities than bare XML)

[view] return HTML5 doctype. Closes #2869426 We dropped xhtml support in 3.17 but we kept providing documents using the XHTML strict doctype, while our content wasn't conform to the DTD since we dropped proper namespaces definition as well as declaration of cubicweb DTD extensions.

[testlib] repreprocess content so contextual display has a chance to show the proper line. Closes #2869481 the context is calculated using position information relative to a preprocessed content, not the original source

[web request] drop no more necessary request.document_surrounding_div method This is awful and no more necessary since we dropped xhtml support

[testlib] unspaghettify Validator / PageInfo api

[testlib] update htmlparsers.VALMAP: stop using SaxOnlyValidator and add an entry for html

[testlib] SaxOnlyValidator is deprecated, use XMLValidator instead

[dataimport] minor typo in error handling

[views] attempt to clarify the inline help system documentation

[sobject] fix notification operation regression introduced in bf4003760e02 Operation use self. session. not self._cw. I'm a bit surprised that its not tested. closes ##2870077

Added tag cubicweb-debian-version-3.17.0-2 for changeset 09a0c7ea6c3c

[pkg/debian] prepare release 3.17.0-2

[pkg/debian] update yams runtime depends Only the built depends were updated to 0.37.0.

[pkg/debian] expand all depends and co on multiple line This is more readable and simplifies patching.

Added tag cubicweb-debian-version-3.17.0-1 for changeset 22be40c492e9

Added tag cubicweb-version-3.17.0 for changeset cc1a0aad580c

[test/view] exclude some Blog entity from automatic view testing Some Blog view requires the sioc cubes that we extracted in 3.17 we do not really want to test cubes view here. So we exclude CWEtype that really requires the sioc cubes.

[doc] fix: cubicweb.gmap.js has moved to the geocoding cube

[test/ldap] do not assume order Do no assume a specific order is stable when testing group members

Added tags for version 3.16.4 on changeset 041804bc48e9

[notification] ensure official API is test By using the official API in the hook we ensure it is tested.

fix typo in notification causing NameError introduced by: f5b40b66d36e

[test/notification] merge related test classe They have no reason to be distinct.

merge with stable

[server] add a looping task to free unused cnxsets capture by dead pyro When a pyro client disconnects, we're not necessarily notified so they can hang on to their session/cnxset forever. Don't let them do that. The hacky looping task is only installed when pyro is enabled. This optional activation save this very hacky patch from being rejected. closes #2825380

[pkg/debian] update changelog to 3.17.0

[devtools] Use XMLValidator instead deprecated SaxOnlyValidator The renaming was introduced in 198fdadafed6.o This prevent a meta class conflict introduced by logilab-common 0fcd2e050621.

[pkg] update changelog

[pkg] prepare 3.17.0

[pkg] prepare 3.16.4

[repo] straightforward bootstrap sequence. Closes #2841188 * kill fill_schema method which was doing to much different things, in favor of a deserialize_method which is simply reading schema from the db and returning it * set_schema doesn't bother with cubes This allows to easilly follow what's going on and so to cleanup init_cnxset_pool to get the following boostrap order: 1. load cubes (implies loading site_cubicweb) 2. load schema 3. init data sources 4. init cnx sets

[repo] minor cleanups to bootstrap sequence * fix some docstring/comments * move initialization of HooksManager to the __init__.py, this has nothing to do with pool initialization / boostraping

[schemaserial] serialize additional yams parameter for customs type Yams have a new custom type feature. We add support for extra parameter on CWAttribute in schema serialization handling. Closes #2847151

[sql] preprocess_entity uses lgdb helper's SQL converters. The preprocess_entity does not yield anymore the conversion logic for different base types, but relies on a new SQL_CONVERTERS dictionary of the db helper. This will allow later inclusion of new base types.

[repo] kill rebuildinfered feature from Repository.set_schema This clutters thing and is useless in most cases: when schema just have been readen from the database or from the file-system, infered relations have just be calculated. The only use-case where it should be rebuilt is on migration (ie schema modification by editing the meta-model using rql), so move this feature there.

[test] stop deepcopying for nothing

drop xhtml content-type support (closes #2065651) * HTMLStream does not care about xml any more * reqquest.demote_to_html and .xhtml_browser are deprecated * web config: drop force-html-content-type option * adjust tests

[htmlparser] exclude <script> tag from html source lxml are confused by them :-(

[htmlparser] store unaltered source in pageinfo.source Do not store the parser to preserve the source aspect. Store the initial input in the source argument instead. This is very useful when parsing html. In such case we need to drop ``<script>`` tag.

[htmlparser] rename SaxOnlyValidator to XMLValidator

[htmlparser] add missing deprecation message

[form] remove hard coded id html id should be unique. It is clearly not achieved here.

[server/test] use server config in migration This allows a lighter initialisation (no rtag needed). Execution of those test move from 300 secondes to 220 seconds here. A 25% speedup

[session] deprecate `hijack_user` method The semantic of this method is wicked and lead to very hard violation of sanity. (multiple transaction using the very same cursors). We deprecated the API to be able to drop it as soon as possible. The method was added long ago for some notification trick. It is not needed since we sent notification on ``postcommit_event``. (closes #2781782)

[mail] drop hijack_user usage The transaction is now commited before notification. We can create a new session directly and drop `hijack_user` usage. This prepare #2781782

[notification] merge BaseNotificationView and NotificationView They are in the same module now.

[notification] move notification view in ``sobject.notification`` It has no user outside this module. This enforce serversideness of notification and allow future cleanup. No backward compat is set up to prevent circular import. The class has no other user anyway. (closes 2845144)

remove unused import pyflakes say they are unused. Tests suite confirms.

one more merge

[notification] introduce an official `notify_on_commit` function It provides a proper notification api instead of replacing it by yet another clumsy one. Closes #2837251

[cubicweb/doc] Replace dc_type() by cw_etype dc_type() is may be translated. Leading to terrible terrible damage.

[dataimport] backout 6947201033be (related to #2788402) (and add a try: except to cache the intended error) The problem actually comes from the ``MassiveObjectStore`` in the ``dataio`` cube, so it should be corrected there. Here, we only protect it with a ``RuntimeWarning`` so that the user can see the problem. ``value`` is set to ``None`` (whence to ``NULL`` from a database standpoint), so that the data can be nevertheless inserted in the database. However, only the keys present in ``row`` are actually non-'``NULL``'. The real solution is to work out the issue in ``MassiveObjectStore`` directly. The current try/except should only be a temporary hack.

merge default heads

[c-c serverctl] import locally to speed up c-c

[facet js] minor refactoring and cleanups * extract facetCheckBoxSelect and facetCheckBoxUnselect * use $ as prefix for jQuery variable (facet -> $facet) * other minor rephrasing

[ldapfeed] Add support for LDAP groups (closes #2528116) Groups from the LDAP server are imported as CWGourp entities, and in_group relationships are created between existing CWUsers and CWGroups Unit tests have been refactored a bit, especially ti add helper methods to manage LDAP database.

[test/ldap] small improvement to ldapfeed unit tests better group testing and further testing of imported user life cycle.

[ldap] prepare import of CWGroup All CWUser specific code is put in dedicated sections.

[ldap] handle modification date We are now able to import modification date from ldap.

[ldap] refactor attributes mapping handling * use explicit attributes lists when calling ldap search * direct and reverse attributes maps contains now the same elements.

[ldapfeed] add support for multiple email addresses from ldap Multiple emails case was ignored before this change. email test

merge 3.16.x fix in 3.17.x branch

[session] fix hooks_control backward compat Session object have no ``hooks_control`` control method. it has ``allow_all_hooks_but`` and ``deny_all_hooks_but`` ones. Regression introduced in bd5b5759c9b3.

[facet js] fix reordering of facet check boxes. Closes #2732947 Before this patch, when one select an element, it's moved to the top of the select content. Fine. But when it's later deselected, it stays there instead of moving back to its original location. This patch fixes that by introducing a facetCheckBoxReorder function which properly reorder the whole facet, instead of buggy attempt to locally reorder.

[vreg] (re)build entity classes cache after registry initialization. This fix potential memory fragmentation issue. Closes #2719113

[task] allow negative intervals for add_looping_task (closes #2818280) Instead of rejecting values < 0 with a ValueError, we ignore tasks with a negative interval. This introduce a simple way to disable task from configuration.

[task] allow interval=0 for looping tasks There is valid usecase for perpetual running task.

[portable dump] skip virtual relations, they have no table associated. Closes #2841199

[test] fix skipIf import, currently failing if unittest2 is not there

[test] fix bad merge

merge with other default head

[ldap] an empty 'user-base-dn' disable the user importation process, The same semantic will be used for group. We start by using it for user for the sake of consistency.

drop commented print

[ldap] Major test refactoring Mostly fix tests (previous refactoring), but also refactor a bit the base class to have better/simpler interaction with the slapd server.