[client-connection] add an auto-close property for ClientConnection The next commit introduce a connect function that open a new Session and return an associated ClientConnection. The Session should not be used by anything else than the created ClientConnection, so we want to close it at the same time than the ClientConnection. The implementation in this changeset is simplistic. We probably want to move this notion in the session itself. This be improved once we have server side Connection

[cwuser] make CWUser callable, returning self for dbapi compatibility In the dbapi, Connection and Session have a ``user`` method to generated a user for a request In the repo api, Connection and Session have a user attribute inherited from SessionRequestBase prototype. This ugly hack allows to not break user of the user method. To ease transition from dbapi to repoapi we need the ClientConnection to be as compatible as possible with the dbapi. Adding this method goes in this direction. It'll get deprecated in the deprecation wave that will conclude the repoapi refactoring. related to #2503918

[client-connect] drop rqlst on rset returned client side The DBAPI used to drop the RQL syntax tree on rset to save bandwidth. We could stop doing it for In-memory client connection. However keeping the syntax tree leads to multiple tests failures. So we keep dropping it as the DBAPI always did. related to #2503918

[client-connection] add a repo property for dbapi compatibility To ease transition from dbapi to repoapi we need the ClientConnection to be as compatible as possible with the dbapi. Adding this method goes in this direction. It'll get deprecated in the deprecation wave that will conclude the repoapi refactoring. related to #2503918

[client-connection] add a connection property for dbapi compatibility To ease transition from dbapi to repoapi we need the ClientConnection to be as compatible as possible with the dbapi. Adding this method goes in this direction. It'll get deprecated in the deprecation wave that will conclude the repoapi refactoring. related to #2503918

[client-connection] add a sessionid property for dbapi compatibility To ease transition from dbapi to repoapi we need the ClientConnection to be as compatible as possible with the dbapi. Adding this method goes in this direction. It'll get deprecated in the deprecation wave that will conclude the repoapi refactoring. related to #2503918

[client-connection] add a cursor() method for dbapi compatibility To ease transition from dbapi to repoapi we need the ClientConnection to be as compatible as possible with the dbapi. Adding this method goes in this direction. It'll get deprecated in the deprecation wave that will conclude the repoapi refactoring. related to #2503918

[client-connection] add a request() method for dbapi compatibility To ease transition from dbapi to repoapi we need the ClientConnection to be as compatible as possible with the dbapi. Adding this method goes in this direction. It'll get deprecated in the deprecation wave that will conclude the repoapi refactoring. related to #2503918

[repoapi] introduce a basic ClientConnection class This is the new official way to access the repo from client side. It still access Session object directly as the server side connection is not up yet (and it's not up because it would have no user). Multiple follow up commit will install compatibility with the DBAPI. This will ease the migration from dbapi to repoapi. ClientConnection has no user yet but later commit will use it in the whole Web stack. related to #2503918

[req] add a _set_user method RequestSessionBase usally need to recreate a new CWuser appobject linked to themself. We add it to the base class to avoid multiple redifinition.

[server/session] add a login property session.login is a DBAPISession attribute. Having it on server side session will helps the rework of the API to access repository. The new schema drop the concept of DBAPISession and use server side session for the same purpose. related to #2503918

[server/session] allow access to session id using sessionid session.sessionid is a DBAPISession attribute. Having it on server side session will helps the rework of the API to access repository. The new schema drop the concept of DBAPISession and use server side session for the same purpose. related to #2503918

[server/session] Implement anonymous_session Now we have a simple rule to compute if a session is anonymous we can implement the property for server session too. Having it on server side session will helps the rework of the API to access repository. The new schema drop the concept of DBAPISession and use server side session for the same purpose. Related to #2953943 Related to #2503918

[server/session] do not clear session.cnx if unrelated to cnx actually cleared The cnx parameter may be different from the one actually loaded for the current thread. This will be possible in future commit when you close a client connection for example.

[dbapi] move ProgrammingError into cubicweb module A new ``repoapi`` will be introduced as a replacement for the dbapi. It will need ProgrammingError too. Related to #2503918

[repoapi] move get_repository function into a new repoapi module This new module aims to host the function of the new API replacing the old DBAPI. `get_repository` is still needed hence moved to the new module. Related to #2503918

[testlib] gather all repository access logic in one place Refactoring of the repository access API in test is imminent. We plan to move from the "old" dbapi to the new repoapi. Gathering all impacted method in one place help to understand how all those method interact and help readability for both patch and resulting code. No code change is done at all in this changeset. The refactoring will code later.

[testlib] move repo and related attribute back on Instance instead of Class The repo and cnx was hold by the TestCase class to work as a cache and avoid recreation of the repo from scratch of each test. However since bad26a22fe29 the caching is done by the DatabaseHandler object and it is not necessary have a second layer of cache on the TestCase itself. We move the repo and cnx attribute back on the TestCase instance itself and make several class methods instance methods again. This will helps to change dbapi access in test from dbapi to repoapi. related to #2503918

[dbapi] makes anonymous_connection a computed property The current implementation is a boolean flag set manually by client code after connection creation. This led to different way to decide a anonymous_connection should be True (eg. different in the test than in the actual application code). It should not be client responsibility to set this flag. ``cnx.anonymous_connection`` is now a purely computed property. Connection with user in the "guests" group are anonymous, the other ain't. ``Session.anonymous_session`` is computed from ``cnx.anonymous_connection`` and get the updated behavior transparently. Closes #2953943

[webrequest] simplify set_session code Thanks to the previous changeset we are assured that session handed to set_session is full featured one. This allows a simpler code for this method. related to #2503918

[webrequest] set DBAPISession without cnx at initialisation time Such session are necessary for minimal use of a Request. Setting on by default allow simplification later linking with a full featured DBAPISession or equivalent. This was not possible before as all session was tracked by session manager. They are not tracked anymore since aa709bc6b6c1 and we can safely create them by default. related to #2503918

[testlib] rework request building in init_authentication The previous code was building a full authenticated request and tried to undo the authentication afterward. The new code just create a bare Request with no authenticated session linked yet. This is much closer of what the actual authentication process does.

[request] drop the user argument for set_session I see no code nor test that use this optional argument. removing it help to clean the session code. The set_session function will soon be deprecated anyway (at the same time than the dbapi) related to #2503918

[web-request] handle default language earlier We now read language negotiation header at initialisation time (previously done during ``set_session``). This simplify the set_session code for later reworking. There is no change in behavior, the user selected language continue to overwrite the one picked from the header/default when applicable. related to #2503918

[etwist] pass the repository to the root resource This is another step toward a cleaner instantiation scheme for the repo. The http test now pass an already created object to the etwist server instead of relying on the config cache. Related to #2249513

[repo] move repo.gc_stats to Service API (closes #2951068) We currently have a method on the repo that anyone can call. `call_service` gives a similar thing, modulo: * can be restricted access to manager only (thanks to selection on services), * less unrelated code on the repository class, * no need to fetch a reference to a repo object from the client side * (`req._cnx.repo` is not an API and config.repository() is on its way out). The old way to access this information (repo.gc_stats()) is deprecated.

[repo] move repo stats to Service (closes #2951067) We currently add a new method on repo that anyone can call. Call service allows to reach the same result with: * Restricted access to manager only (thanks to selection on services), * Less unrelated code on the repo. * No need to fetch a reference to a repo object from client side. `req._cnx.repo` is not an API and config.repository() is on its way out. The old way to access this information (repo.stats()) is deprecated.

[service] drop the asynchronous execution possibility Call_service was able of both sync and sync execution making the API confusing. There is not user of the async case. We drop the async argument in favor of synchronous execution only. This makes call_service the official API to call server side code from the client side. This is a remplacement for the usual monkey patching of the repo object. The zmq notification bus is a solid alternative for codes that needs to start an async execution.

[auth] pass `repo` instead of `vreg` to SessionManager and AuthenticationManager Those object actually need a repo object to connect and create new session object. They used to retrieve the repo object using ``vreg.config.repository()``. I'm trying to sanitise the initialisation stack and get ride of ``config.repository()``, a function that mix factory and cache. Retrieving the vreg from the repo is trivial (``repo.vreg``) so we now pass the repo object instead of the vreg.

web/application: instantiate the repository outside of CubicWebPublisher This improves decoupling and allows ``CubicWebPublisher`` user to choose which argument are used during ``Repository`` instantiation. In particular is allows caller to provide a ``TaskManager``. The `vreg` argument of publisher is made mandatory. See CubicWebPublisher docstring for details. Related to #2249513

remove vreg argument to CWPublisher The passing a vreg to CWPublisher was used by test to allow reused of an existing vreg. We dropped this feature two commit ago and can safely removes it. Removing this argument allows later cleanup and improvement on the CWPublisher and Repository front. closes #2944813

[devtools] vreg becomes a class ppty After multiple devtools refactoring, the vreg used for test is now *always* got from the repository. Making this explicit is a step toward simplification of the vreg handling during various initialisation phases. Related to #2944813

[devtools] drop unused `reset_schema` and `reset_vreg` class attribute They have no effect and no user... Related to #2944813

[connection] move security control logic on Connection The _security_enabled context manager now operate on connection. We have to keep a specific entry point in Session to ensure Connection object are properly cleaned up when using session to "manage" Connection live cycle. Related to #2503918

[connection] move hook control logic on Connection The _hook_control context manager now operate on connection. We have to keep a specific entry point in Session to ensure Connection object are properly cleaned up when using session to "manage" Connection live cycle. Related to #2503918

[session] update _hook_control docstring

[connection] reinstall cnx.data as cnx.transaction_data Too much code expects self._cw.data to be session data. backout 9b2f68916474 Related to #2503918 Related to #2912807

[connection] give access to is_internal_session boolean Needed if we are to use this object like we currently do with session. Related to #2503918

[req] drop from_controller on non WebRequest object (Closes #2901079) The `controller` concept is purely a web things. It does not belong to generic `req` object. The `build_url` code is slightly changed to handle that. This probably requires extended cleanup later. This is a barely used and very internal API. No deprecation period

[connection] add logging method on connection If we are to use this object like we currently do with session, we need to have the logging method as a lot of code use them.

rename server.session.transaction into server.session.connection The ongoing rework of the API to access the database include the splitting of Session in two objects: The ``Session`` object will only hold credential (user) and global session data. A new object will publicly emerge to handle a database access. This object is named ``Connection`` since that the way database accessors with the same property are named in other system. So we renamed the ``Transaction`` object into ``Connection``. The ``Transaction`` object have already grown in the direction of something directly usable by generic code, but the name ``Transaction`` is ill suited as such object can be used for multiple database transaction in a row. Related to #2503918

[application] call req.set_session in application.main_handle_request The Session handling chain is no more responsible for calling req.set_session. It just returns a valid session and lets the caller link it to the Request. This opens the way to explicitly creating and closing a connection/transaction in ``application.main_handle_request``. Related to #2503918

[session-handler] use session directly to update last usage We don't really need the WebRequest for that. Not using the WebRequest to access the cubicweb repository here will allow a delayed set_session. Related to #2503918

[application/connect] simplify connection logic ``application.connect`` now either sets a full featured ``DBAPISession`` to the ``WebRequest`` object or raises ``AuthenticationError``. The creation and usage of a fake DBAPISession is now handled by ``main_handle_request`` when needed. This means that fake DBAPISession are no longer tracked by the session manager and that user are not given anyway to retrieve them for a later request. This fake DBAPISession is still passed to ``core_handle`` because multiple cubes like registration or forgotten password need this behavior. We would like to get ride of it in the future. This clarification of the connection API greatly simplifies ``DBAPISession`` retrieval//creation process opening the way to improvements in this area. Related to #2503918

Drop hijack user (closes #2901093) Deprecated since 3.17.0. Dropping this function help the session cleanup business.

[sources] drop support for ldapuser source (closes #2936496) The ldapfeed source is a replacement for ldapuser. Use it instead.

merge stable back into default

3.17 is the new stable

Added tag cubicweb-version-3.16.5, cubicweb-debiann-version-3.16.5-1, cubicweb-centos-version-3.16.5-1 for changeset 810a05fba1a4

[pkg] prepare 3.16.5

Added tag cubicweb-version-3.17.2, cubicweb-debian-version-3.17.2-1 for changeset 195e519fe97c

[pkg] prepare 3.17.2

repository: make tests pass again The tests use mono-threaded pyro, which breaks assumptions made in e27337dfec8c. To fix that, remove sessions from the _pyro_sessions dict when they're closed, and force the test pyro client to actually disconnect when it's done.

repository: monkey patch pyro connection handling to detect clients going away When a pyro client goes away we need to close their session (to release their cnxset). Add a dict to the repository mapping pyro client threads to session object. Closes #2932058

notification: use viewargs for notif_entity_updated instead of transaction_data Now that notifications use separate sessions, they can't rely on the original transaction_data being around, so pass the data through view arguments instead, so the notification view knows what changed on updated entities. Closes #2936180

notification: properly handle cnx and lifetime of the hijacked session When we create a session for the notification rendering, we need to 1) give it a cnxset 2) commit and close it after we're done 3) restore the original session for subsequent notifications This changes the ordering of actual mail sending, since there are several different commits involved, but I don't see a way to fix that short of restoring hijack_user. Related to #2934523

notification: fix session creation (closes #2934523) Session() takes a user and repo, not repo and user.

Added tag cubicweb-version-3.17.1, cubicweb-debian-version-3.17.1-1, cubicweb-centos-version-3.17.1-1 for changeset f98d1c46ed9f

[cwctl] add configure command to cw-ctl (closes #2709702) usage: cw-ctl configure <appid> --param key=value,key2=value2 --param key3=value3

[cwctl] rename option no-post-create to no-db-create

[pkg] version 3.17.1 for rpm packages

[session] upgrade session closed error from Exception to SessionClosedError Exception is far too wide and we already have a SessionClosedError anyway. Closes #2897696

[pkg] prepare 3.17.1

[web/application] add some minimal documentation Some documentation to the Application main entry point.

[session] raise proper exception with get_tx is called on a closed session Prevent attribute error on _txs Closes #2897836

[view] typo in a docstring

[cwctl] pass missing inputlevel argument to postcreate() (closes #2901037)

[server] fix documentation of ZMQ options harder The syntax for zmq addresses is tcp://host:port, no zmqpickle involved.

merge with 3.16.x fixes

[server] fix documentation of ZMQ options

[zmq] set order for ZMQStartHook so other hooks don't need to If an application wants to subscribe to some messages on the zmq bus, that needs to happen after ZMQStartHook.

import merge_options directly from logilab.common

[server] fix error messages in write_sources_file()

[doc] a couple almost-typos

[schemaserial] mitigate critical message during migration We can't fetch extra_props because it is not yet there.

[doc] typo

[cw-ctl] fix help message for schemadiff In global help (closes #2888538) The leading space make it appears on multiple line. The help content is a bit improved in the process.

[dbapi] fix connect backwards compat harder Passing None as group and host should result in "pyro:///cubicweb.appid", not "pyro://None/None.appid". Followup for 2091d275fe5c. Closes #2882666.

[zmq] make publish address optional It should be possible to use the zmq communication bus in a read-only manner. Allow setting zmq-address-sub without zmq-address-pub. Closes #2897178.

Fix documentation for zmq pub/sub hooks and configuration Closes #2897177

[entity] improve deprecation messages

[devtools] use self._parse so AssertionError is properly raised instead of lxml error (test failure introduced in 6711f78c18be)

[testlib] fix page validator selection. Closes #2869456 * use a validator that checks XML well-formness for the html5 doctype (as a convenient side-effect, it does not choke on e.g. a canvas tag like the HTMLParser does) * use a DTD validator if there is an xml declaration * else use the HTMLValidator

[testlib] introduce a validator that check xml-well formness This validator simply check XML is well formed and accept any entities (think HTML defines much more entities than bare XML)

[view] return HTML5 doctype. Closes #2869426 We dropped xhtml support in 3.17 but we kept providing documents using the XHTML strict doctype, while our content wasn't conform to the DTD since we dropped proper namespaces definition as well as declaration of cubicweb DTD extensions.

[testlib] repreprocess content so contextual display has a chance to show the proper line. Closes #2869481 the context is calculated using position information relative to a preprocessed content, not the original source

[web request] drop no more necessary request.document_surrounding_div method This is awful and no more necessary since we dropped xhtml support

[testlib] unspaghettify Validator / PageInfo api

[testlib] update htmlparsers.VALMAP: stop using SaxOnlyValidator and add an entry for html

[testlib] SaxOnlyValidator is deprecated, use XMLValidator instead

[dataimport] minor typo in error handling

[views] attempt to clarify the inline help system documentation

[sobject] fix notification operation regression introduced in bf4003760e02 Operation use self. session. not self._cw. I'm a bit surprised that its not tested. closes ##2870077

Added tag cubicweb-debian-version-3.17.0-2 for changeset 09a0c7ea6c3c

[pkg/debian] prepare release 3.17.0-2

[pkg/debian] update yams runtime depends Only the built depends were updated to 0.37.0.

[pkg/debian] expand all depends and co on multiple line This is more readable and simplifies patching.

Added tag cubicweb-debian-version-3.17.0-1 for changeset 22be40c492e9

Added tag cubicweb-version-3.17.0 for changeset cc1a0aad580c

[test/view] exclude some Blog entity from automatic view testing Some Blog view requires the sioc cubes that we extracted in 3.17 we do not really want to test cubes view here. So we exclude CWEtype that really requires the sioc cubes.

[doc] fix: cubicweb.gmap.js has moved to the geocoding cube

[test/ldap] do not assume order Do no assume a specific order is stable when testing group members

Added tags for version 3.16.4 on changeset 041804bc48e9

[notification] ensure official API is test By using the official API in the hook we ensure it is tested.

fix typo in notification causing NameError introduced by: f5b40b66d36e

[test/notification] merge related test classe They have no reason to be distinct.

merge with stable

[server] add a looping task to free unused cnxsets capture by dead pyro When a pyro client disconnects, we're not necessarily notified so they can hang on to their session/cnxset forever. Don't let them do that. The hacky looping task is only installed when pyro is enabled. This optional activation save this very hacky patch from being rejected. closes #2825380

[pkg/debian] update changelog to 3.17.0

[devtools] Use XMLValidator instead deprecated SaxOnlyValidator The renaming was introduced in 198fdadafed6.o This prevent a meta class conflict introduced by logilab-common 0fcd2e050621.

[pkg] update changelog

[pkg] prepare 3.17.0

[pkg] prepare 3.16.4

[repo] straightforward bootstrap sequence. Closes #2841188 * kill fill_schema method which was doing to much different things, in favor of a deserialize_method which is simply reading schema from the db and returning it * set_schema doesn't bother with cubes This allows to easilly follow what's going on and so to cleanup init_cnxset_pool to get the following boostrap order: 1. load cubes (implies loading site_cubicweb) 2. load schema 3. init data sources 4. init cnx sets

[repo] minor cleanups to bootstrap sequence * fix some docstring/comments * move initialization of HooksManager to the __init__.py, this has nothing to do with pool initialization / boostraping

[schemaserial] serialize additional yams parameter for customs type Yams have a new custom type feature. We add support for extra parameter on CWAttribute in schema serialization handling. Closes #2847151

[sql] preprocess_entity uses lgdb helper's SQL converters. The preprocess_entity does not yield anymore the conversion logic for different base types, but relies on a new SQL_CONVERTERS dictionary of the db helper. This will allow later inclusion of new base types.

[repo] kill rebuildinfered feature from Repository.set_schema This clutters thing and is useless in most cases: when schema just have been readen from the database or from the file-system, infered relations have just be calculated. The only use-case where it should be rebuilt is on migration (ie schema modification by editing the meta-model using rql), so move this feature there.

[test] stop deepcopying for nothing

drop xhtml content-type support (closes #2065651) * HTMLStream does not care about xml any more * reqquest.demote_to_html and .xhtml_browser are deprecated * web config: drop force-html-content-type option * adjust tests