[hook] remove assumption about entity cache vs cw_edited We were assuming that if an entity is in the connection's cache, it has a cw_edited attribute. This doesn't seem to always be the case. At least in cubicweb 3.21, merging the web and repo-side caches makes this more likely, but Denis says he's also seen this on 3.20.

[hook] don't open-code Connection.entity_cache

[tox] Add missing dependency on webtest for devtools tests

[tox] Add missing dependency on webtest for wsgi tests

[tox] Remove set of PATH environment variable The : concatenation does not work actually so the resulting PATH was just wrong (it did not contain the previous PATH). Anyways, the PATH environment variable is passed by default by tox so it's probably better to let the caller set it (thus avoiding to hard-code the specific postgres version and path). http://tox.readthedocs.org/en/latest/example/basic.html#passing-down-environment-variables

[doc] generalize info about debian/ubuntu repositories That section was not kept up to date with recent distro releases; stop trying.

[doc] fix link to setup info in README ... and fix URL to the repository's gpg key.

[server/session] remove unused cnx_meth function

[server] don't issue a warning for something expected Creating an instance happens all the time in tests, there's no reason to warn every time.

[server] stick a _cw on internal connnection's user Entity._fetch_restrictions goes through user._cw.vreg :( Closes #5705769

[migration/3.21] fix stupid error in migration script str.join doesn't like a list of ints. Closes #5706359.

Added tag 3.21.0, debian/3.21.0-1, centos/3.21.0-1 for changeset 887c6eef8077

[pkg] various fixes - add missing files to MANIFEST - exclude doc/changes from dh_installchangelogs (it's a directory, which dh_installchangelogs doesn't expect) - fix doc build in debian package

[doc] fix broken symlinks

[pkg] update 3.21.0 release date

[doc/3.21] mention the postgresql-only state of the upgrade script

[datafeed] also catch EnvironmentError when trying to load the cwclientlib config file means the config file has not been found, so fall back to the old implementation instead of crashing.

[schema] Make syntax compatible with print_function yams seems to exec() us sometimes (instead of __import__()), and yams.reader uses print_function nowadays.

[pkg] 3.21.0

[test] use FakeFile instead of File in test_fulltextindex File.data is no longer fulltextindexed.

i18n update

[serverctl] fix schema-diff command It hadn't been ported away from dbapi.

[serverct] call server_{maintenance,shutdown} hooks around source-sync command

[serverctl] call server_{maintenance,shutdown} hooks around add-source

Update hgignore

delete unmaintained clone_deps script Closes #2517580.

merge 3.20.9 into 3.21 Fix conflict in unittest_postgres.py by changing sources configuration in PostgresTimeoutConfiguration.__init__ so it happens *after* setUpModule().

Added tag 3.20.9, debian/3.20.9-1, centos/3.20.9-1 for changeset d477e6447582

[pkg] 3.20.9

[forms] don't display 'remove' link of inlined form for the first entity related using a '+' cardinality to its parent entity Closes #5227394

[tox] Update doc testenv to account for documentation restructuring * Build doc from `doc` directory. * Use `sphinx-build` directly instead of `make` as this now works and allows to build within .tox temporary environment (instead of doc/_build), preferable for a "test" action. Related to #5447161.

[schema] drop old backward compat code preventing yams to be itself fully cleaned

[datafeed] allow to use cwclientlib for datafeed's queries (closes #5456849) HTTP based URLs will be handled by a CWProxy if: - cwclientlib is available and - the URL matches a cwclientlibrc config entry Otherwise, fallback to previous implementation. See https://www.cubicweb.org/project/cwclientlib :Warning: This comes with an API modification of DataFeedParser.retrieve_url: it used to accept data and headers arguments to build GET or POST (if data was given) queries, but this was not used by any known code and imply more complicated code.

[cwxml parser] refactor url handling code to use urlparse and urlencode (related to #5456849)

add use_extid_as_cwuri ext entity transform, that will be often necessary and not so easy to write at once

[dataimport] dispatch and deprecate old code previously, the whole dataimport.py content has been copied in dataimport/__init__.py, then dataimport/importer.py API has been introduced. Following these changes, this cset: * dispatch __init__ code into sub-modules * dispatch unittest_dataimport accordingly * deprecates functions and classes that are deprecated by the new API Related to #5414753.

Add tox.ini file * Factor out `commands` for most environments using {envname} substitution to build pytest commande. * Positional arguments are passed on to `pytest` (others may also be passed after a `--`). * Set PATH env variable for tests requiring PostgreSQL. * Also added a `doc` target to build the book. Also update .hgignore exclude .tox directory (where virtualenvs will be created and cubicweb.egg-info created by sdist command). Closes #5447161.

Add requirements.txt files in test directories These files, located in each test directory (when needed), list test dependencies supplementary to install requirements. Some requirements.txt files include psycopg2 because it won't get installed by logilab-database apparently. Related to #5447161.

Spelling fixes in comments and docstrings

[web/views] Clean up confusing backwards compatibility code Dead code since fbb370195199, related to #4943392.

[server/test] avoid lgdb warnings from str to unicode conversion

[web/test] Modernize unittest_propertysheet.py - remove call to os.makedirs in global scope - fold CACHEDIR as an instance attribute - bonus: use unittest directly instead of lgc.testlib

[web/webconfig] Drop pre-3.9 backward compatibility method "has_resource"

[web] Stop using negative indices with os.stat results The python doc says: More items may be added at the end by some implementations. .st_mtime is also more readable than [-2].

[doc] small rst fixes and cleanups in changelog files Related to #4832808

[doc] make the changelog chapter a one html page Related to #4832808

[doc] reformat changes files headers to remove the "what's new" phrasing and add the publication date instead. Related to #4832808

[doc] remove mentions of pyro and reposity instance type from the concepts chapter Related to #4832808

[doc] put the tutorials between the intro and the `Repository development` toc of the book Related to #4832808

[doc] move the documentations from docstring directly to the book. and improve a bit API documentations. Related to #4832808

[predicates] remove deprecated sentences in docstrings

[predicates] objectify the anonymous_user predicate for the sake of consistency.

[doc] more fixes of warnings/errors in doc build - move 3.21.rst changelog description at it's proper place - include changelogs in the main index - few typos - add :noindex: for autoxxx directives not in api/ Related to #4832808

[doc] fix warnings/errors in doc build - fix links to images - fix a couple of typos - re-add IDownloadableOneLineView doc - rename documenting.rst back to .txt, it's intended as a doc of how to write rst, not part of the rst doc Related to #4832808

[doc] Start documenting the API Related to #4832808

[doc] Remove useless indexes Related to #4832808

[doc] Insert the dev docs in the toc Related to #4832808

[doc] Restructure the documentation * Create a new index file * Move the sphinx configuration files do the documentation root * Move book/README to dev/documenting.rst * Move book/mode_plan.py to tools/ * Move book/en/images to images * Move book/en/* to book/ * Move changelogs to changes/* * Adapt the Makefile * Add a title to the javascript api index Related to #4832808

[migration/3.21.0] performance and reliability fixes - use EXCEPT instead of NOT IN, seems to improve performance dramatically on postgresql - delete rows from cw_<etype> tables whose eid is unknown - deal with missing/dangling required inline relations (by deleting the subject entities) instead of asserting - fix/improve print statements - don't ask for confirmation before running pure select queries

[serverctl] fix source-sync command Closes #5491700

[server] missing import Fixes NameError in an unlikely error case.

[server/test] stop using email cube - make unittest_migractions use a separate datadir, and add a fake email cube there (we use it to test {add,drop}_cube, so need something that looks like a cube) - for the other test modules, just copy what we need from cubicweb-email's schema to our test schema

[web/views] Remove unused variable

[server/session] bring anonymous_connection back Fixes cubicweb-comment.

[migration/3.21] idempotency fixes Drop constraints before (re)creating them This is postgresql-specific syntax, sqlserver needs work.

[migration/3.21] Make index and table creation idempotent

[migration/3.21] Add missing foreign keys when upgrading Try to delete dangling relations, since there's no point in keeping them. Related to #4846892.

[schema2sql] properly consider skip_relations parameters. This is the same bug as yams #286912. Also, introducing a unique function telling if the relation should have a table or not unhide some other problems related to computed relation, or missing skipped relations information.

[hooks/test] don't rely on RelationDefinition implicitly Its presence in globals is deprecated and going away.

[web] write css_class to attrs on rendering and not on button creation Because css_class attribute may be overriden by bootstrap after button creation and this modification has to be taken into account. closes #5457548

[statsd_logger] handle ipv6 addresses

implement a simple statsd logger (closes #5488711) and add a few timing probes on: - etwist.server.CubicWebRootResource.render_request() - server.querier.QuerierHelper.execute() - server.sources.native.NativeSQLSource: - sqlexec() - compile_rql() - authenticate() - doexec() - doexecmany() as well as a pair of counters on server.querier.QuerierHelper cache (hit/miss).

[predicates] ExpectedValuePredicate now accepts a dict parameter ``match_form_params`` and ``match_kwargs`` benefit from that. For instance, the following statements are valid:: __select__ = match_form_params('vid', 'subvid') __select__ = match_form_params(vid='list', subvid='incontext') __select__ = match_form_params(vid=('list', 'tsearch')) In the latter cases, not only the parameters (``vid``/``subvid``) must be in the request form but their corresponding value must also match the expected values (or one of possible choices if `values` is a sequence). closes #5484070

[rql2sql] Fix iter_exists_sols() excessive cleaning of _state.tables Closes #5503548

[cwctl] allow overriding config settings from the command line (closes #5557656) eg. cubicweb-ctl start -p port:8082 myapp to overload the port parameter defined in the all-in-one.conf file.

avoid sanitizing warnings

[devtools] avoid warning about source['global-db-name'] sqlite and postgres db handlers store the db template name there.

[skeleton] remove no-break space from skeleton setup.py

[server/test] drop unused import

[doc] update 3.21 release notes

[devtools] Sort _cw_fields in fake_form Try to generate a predictable string, to help testing with PYTHONHASHSEED=random.

[entity] sort fetchattrs when generating rql Try to generate a predictable rql query, to help testing with PYTHONHASHSEED=random.

Update hgignore for temporary test databases

[js] removed unused and broken SuggestForm Widget (closes #5213487)

[test] Fix unittest_i18n to run properly with "pytest -t" (closes #5576169) When running "pytest -t", many tests modules will be imported and run within a single python process. unittest_i18n fails when run after tests such as unittest_webtest.py or unittest_httptest.py. The main reason is that unittest_i18n does various tricks to add a test cube to the search path, which badly fails when a previous test module has already loaded "stuff" (for lack of a better word). Instead, let's just call the 'i18ncube' command with subprocess. The change from logilab.common.testlib to unittest is not directly needed, but comes as a free cleanup with the above changes.

[server/test] fix random error in unittest_security When entering a new Connection, we cache the user and its 'login' attribute (with no permissions checking). This test makes 'CWUser.login' unreadable by guests, and then proceeds to make sure the 'anon' user can actually not read any 'login' attribute. However, due to the above cnx initialization, anon's login is actually cached, hence readable. This happens to make the test fail sometimes depending on the order in which CWUser entities are returned, because one of them has .complete() called, which as a side effect sets the attribute cache to None for unreadable attributes. Call .complete() on both entities to reset the login cache. While this is still highly debatable, at least it's consistent.

[entity] fix typo in comment

[doc] Add a tutorial and extend documentation for ExtEntityImporter Related to #5414753.

[dataimport] introduce the importer and extentity classes This introduces the ``ExtEntity`` class which is a transitional state between data at external source and the actual CubicWeb entities. ``ExtEntitiesImporter`` is then in charge to turn a bunch of ext entities into CW entities in repository, using a given store. This changeset also introduces ``SimpleImportLog`` and ``HTMLImportLog`` which implement the CW DataImportLog interface in order to show log messages in UI using simple text and HTML formats respectively, instead of storing these messages in database. Both have mostly been backported from cubes.skos.dataimport. Closes #5414753.

[test] Use store.prepare_insert_relation instead of deprecated relate method Follow-up for 1f5026e7d848. Related to #5040344.

[dataimport] since we have foreign key constraints in the database, 'entities' table should be updated first

[dataimport] Move stores to new API. Here is the final store API: * ``prepare_insert_entity(<entity type>, **kwargs) -> eid``: given an entity type, attributes and inlined relations, return the eid of the entity to be inserted, *with no guarantee that anything has been inserted in database*, * ``prepare_update_entity(<entity type>, eid, **kwargs) -> None``: given an entity type and eid, promise for update given attributes and inlined relations *with no guarantee that anything has been inserted in database*, * ``prepare_insert_relation(eid_from, rtype, eid_to) -> None``: indicate that a relation ``rtype`` should be added between entities with eids ``eid_from`` and ``eid_to``. Similarly to ``prepare_insert_entity()``, *there is no guarantee that the relation will be inserted in database*, * ``flush() -> None``: flush any temporary data to database. May be called several times during an import, * ``finish() -> None``: additional stuff to do after import is terminated. **Warning:** ``prepare_update_entity()`` still needs to be implemented for NoHookRQLObjectStore. Related to #5040344

[js] Using appropriate http method when calling a remote function to respect REST principles. closes #5355952

[server/test] Add a test for db-statement-timeout option Sadly I don't think there's a way to test that functionality without adding sleeps which make the test suite longer :/ Related to #2547026

[datafeed] give an error message if a source is missing a parser id Closes #3484231

[storage] use mkstemp to create files in bfss Avoids race condition, at the cost of slightly less predictable file names.

[test] Skip spa2rql tests when fyzz is not available Closes #5456750.

[web/staticcontrollers] fix docstring spelling

[server/rql2sql] fix spelling in comment Not that I understand what that sentence means.

[doc/book] fix rql syntax examples "Any", not "ANY". "EXISTS" is a function.

[migration] add sql constraints on upgrade Related to #5154406.

[schema] neuter check method for constraints we implement in sql Closes #5154406.

add IUserFriendlyError adapter for violation of check constraints This way we get back the same error messages we get from the python check. Related to #5154406

[hooks/syncschema] drop/add check constraints as appropriate Related to #5154406

[schema2sql] support NOW and TODAY in check constraints Related to #5154406.

[schema2sql] insert some constraints into the backend BoundaryConstraint, IntervalBoundConstraint and StaticVocabularyConstraint can be implemented in SQL with CHECK constraints. Next steps, not part of this changeset: - migrations - removing no longer necessary pre-insertion work on the python side - translating backend exception due to constraint violation into ValidationError Note: this means that these constraints can no longer be ignored by disabling integrity hooks. Related to #5154406

[test] run unittest_syncschema with postgresql Lets us test more things, since we can actually do ALTER TABLE.

[test] run unittest_migractions on postgresql sqlite's lack of support for a bunch of alter table functionality means some thing just don't get tested otherwise.

[devtools] Put pg sockets in /tmp Makes for shorter paths (limited by sockaddr_un). Also use the default port, since we're not listening on TCP sockets.

[devtools] change the way we start/stop postgresql Instead of having the test db handler start a cluster on demand, use the test module's setUp/tearDown callbacks to do it. This allows to have one data directory (and thus cluster) per test module, allowing different test modules to run in parallel, each using its own database cluster whose path is based on the test module.

[devtools] make test db name for postgresql process-specific In case we run two tests concurrently on the same cluster, better not use the same db name for both.

[devtools] extract functions to start/stop a postgresql cluster

[test] don't leave NULL columns around when making an attribute required It doesn't matter on sqlite (it doesn't do ALTER COLUMN), but when running this test on postgresql it fails to add the 'NOT NULL' constraint otherwise.

[server/session] Roll back in connection exit This was changed in commit 8b35a898b334 "[server] remove cnxset tracking, it is now unneeded". When exiting a CubicWeb Connection, the database connection is put back into the common pool. At that point anything done during the lifetime of the CubicWeb Connection needs to be committed or rolled back. Do a proper rollback to finish the transaction, as was happening before the above-mentionned commit. This showed up on testing with postgresql, when a DB schema modification was blocked by an old connection that was still "in transaction".

[web/views/formrenderer] do not use `cubicweb:target` attribute on form (closes #5534074) The `cubicweb:target` is a flag indicating that the form data (with file input) shall be posted inside an iframe - this is a well known "ajax-like" workaround to post files with browsers that do not support `FormData` (a.k.a IE<10). The `cubicweb:target` was introduced when CW used the "xhtml strict" doctype. Now that CW uses the "html5" doctype, this namespaced attribute is no longer necessary and the iframe can be generated directly. Before this patch, CW inserts the `cubicweb:target` attribute in the form DOM element (server side) and `setFormsTarget()` updates the DOM with a new `<iframe>` element (client side). Now, CW inserts the `<iframe>` DOM element directly (server side), making `setFormsTarget` useless.

[js] lint : make it explicit that typeof is an operator, not a function. Also, use === / !== except for null / undefined comparison

[devtools] qunit: show helpful data when an assertion fails Related to #5533333.

[web/test] Use shorter $() instead of jQuery() Consistency FTW. Related to #5533333.

[devtools] qunit: use new async testing APIs http://qunitjs.com/cookbook/#asynchronous-callbacks QUnit keeps track of all the assert.async() objects created inside the test functions and expects all done() functions to be called. Failure to do so will result in the test being failed. Unlike .start and .stop which were internal APIs, assert.async() is stricter and fails tests if assert methods are used *after* all done() functions are called (see "test callback execution order"). Related to #5533333.

[devtools] qunit: use new assert APIs test functions now accept an "assert" argument which exposes QUnit's assert methods: assert.{expect,equal,deepEqual,ok} Related to #5533333.

[devtools] qunit: stop using global variables test -> QUnit.test module -> QUnit.module Related to #5533333.

[devtools] qunit: update to 1.18.0 (closes #5533333) * the logging API has changed (register callbacks instead of setting attributes on the QUnit object) * the logging callbacks all have a single "details" object argument * the "main" div was renamed to "qunit-fixture" (this change is responsible for all the churn in our own JS test files) * QUnit can now build the DOM elements it needs for reporting from a single "qunit" div

[devtools] qunit: stop using deprecated assert functions equals -> equal same -> deepEqual Related to #5533333.

[devtools] qunit: remove useless window.close() Modern browsers don't allow it anymore. Related to #5533333.

[devtools] qunit: refactor functions into QUnitView Related to #5533333.

[devtools] qunit: only start firefox once per test The separate profile initialization step doesn't seem necessary anymore. This was due to the system-wide installation of the "Sage" Firefox extensions which breaks -url on a fresh profile. Removing this obnoxious extension allows us to launch Firefox in a standard way. Related to #5533333.

[devtools] qunit: decrease test timeout Related to #5533333.

[devtools] qunit: increase resolution and depth of Xvfb screen Related to #5533333.

[web/test] Always call QUnit.ok with a message Related to #5533333.

[web/test] Fix remaining JS tests (closes #5533303)

[web/test] Stop testing that exceptions raised in Deferred callbacks call errback They no longer do as of changeset 7f1f7f710b16. Related to #5533303.

[web/test] stop calling loadxhtml with a form.callback It was deprecated in 3.9 (changeset 7b9553a9db65) and removed in 3.18. Related to #5533303.

[web/test] Fix reset of loaded JS/CSS files Name changed in commit ede740bd7077. Related to #5533303.

[web/test] Use proper JSON syntax Related to #5533303.

[web/test] Update mock ajax reply from "render" ajaxfunc CubicWeb transforms add_js() calls in AJAX context to <cubicweb:script> tags since ede740bd7077 and cd5738fc440f. Related to #5533303.

[devtools] Serve JS tests over HTTP (closes #5533285) Instead of mixing file:// URLs with XHR.

[web/data] Support Deferred objects outside XHR JS tests actually use this. Related to #5533303.

merge 3.20.8 in 3.21

[views] Revert previous changeset 8f05aaabf355 Breaks cubicweb/web/test/test_views.py as some views don't have a 'vid' kwarg.

Added tag 3.20.8, debian/3.20.8-1, centos/3.20.8-1 for changeset ec284980ed9e

[pkg] Prepare 3.20.8

[js] simplify cw.log implementation There's no need to rebuild and array and join its content, just call console.log with ``arguments``. The only trick is that console.log must be called on a console instance.

[web/views] don't force http redirect on logout We no longer forbid anonymous browsing on https, so we can keep the same scheme when logged out. Closes #5507479.

[views/schema] extract a hardcoded css class attribute

[formwidgets] move TextArea constants to the class level This will make it easy to override them.

[server/test] fix broken test The first bad revision is: changeset: 10396:82071f767cb8 user: Sylvain Thénault <sylvain.thenault@logilab.fr> date: Thu Jun 11 17:35:11 2015 +0200 summary: [schemas] cwuri should be read-only This changed the number of RQLExpressions to be more than the number of CWRelations in the test's schema. Related to #5457566.

[test] typo in entities test comment

[cwconfig] typo in comment

[serverctl] allow changing the log level for source-sync command When calling this from cron we only want to see output if something fails.

[web, urlrewrite] remove unused `cachekey` argument from `rgx_action` (closes #5355967) this argument prevent from generating the resulting rset descripition

[web] Fix the path to the `static directory` (closes #5355965)

[web/json] Controller.publish must return encoded data We were mixing binary from super() and text from our params, thus wrongly returning text. Closes #5507573.

[entities] fix dc_title output for bool(value) == False Closes #5194853.

[hooks] manhandle the parsed rql query, not its string representation ... and add a groupby clause to make it actually work (the selection might well use an aggregate function, so we need to tell the backend to group on our target entity). Closes #5362574

merge 3.19.12 in 3.20

Added tag 3.19.12, debian/3.19.12-1, centos/3.19.12-1 for changeset 5932de3d50bf

[pkg] prepare 3.19.12

[web/views/formrenderer] remove `cubicweb:target` attribute on form once the target <iframe> is inserted For some unknown reason, IE9 does not want to submit the form inside the <iframe> when the ``cubicweb:target`` attribute exists. (closes #5377672)

[views] implement a custom json error view If the client asked for ``(e)jsonexport`` view, answer him with a json response rather than an error formatted in a big HTML blob. add a custom _requested_vid predicate. In cw 3.21, it will be deprecated in favor of ``match_form_params(vid=('jsonexport', 'ejsonexport'))``

don't override req.form['vid'] in the publisher's error_handler closes #5359234

when some inlined relation is set using cw_edited, its security shouldn't be checked. It's currently checked anyway because upon modification of `cw_edited`, `entity.skip_security` is updated to avoid security checking, but this is only considered for attributes, not relations. Closes #5477315

[test] stop depending on the email cube for hooks test Backport necessary part of its schema.

[schemas] cwuri should be read-only Closes #5457566

fix EditController._ordered_formparams method in case some mandatory relation isn't specified If you have an HTML form which doesn't carry information about some mandatory relation (maybe because this relation is set by a hook), you ends up with a KeyError when the form is posted. Using a get instead of subscription protocol fixes the problem. Closes #5481617.

[views] take into account subvid form param in SameETypeListView closes #5098473

[doc] fix errors in test_visibility_propagation example * forget to use the access instructions * wrong use of the function entity_from_eid * some indentation problems

i18n typo

[predicates] Use proper deprecation mechanisms

[doc] loadRemote does a POST by default

[doc] how to change logo using css

[doc] escape | operator so it will be visible in the docs

[doc] add missing import for FourOhFour class

[doc] the instance is sytweb_instance and sytweb is the cube

[server] Do not use progress bar when stdout is not a TTY Closes #5460552.

[wf] Fix Workflow.replace_state() Closes #5359131

[tags] add properly formated tags for 3.19 and 3.20 releases

[reledit] also check for cardinality before displaying the delete relation action. Closes #5334750

[schema] add a unique index on cwuri Closes #4985752

[web] move AbstractAuthManager near its immediate concrete subclass Related to #1381328.

[web] move abstract session manager to web/sessions Related to #1381328.

[repository] rename session -> cnx Related to #3933480.

[server/test] delete commented-out test Related to #3933480.

[connection] remove the `mode` attribute It is now unused. While removing mode, we also drop some sqlserver-specific reconnection logic snippets. These had several downsides: * untested * partial coverage * done at the wrong level Related to #2919309.

[connection] remove ensure_cnx_set context manager uses It has been deprecated in a previous changeset and is now a no-op. Related to #2919309.

[server] remove cnxset tracking, it is now unneeded Indeed, all cnxsets will now be acquired using the following API:: with session.new_cnx() as cnx: cnx.execute(...) # do stuff cnx.commit() These well-scoped blocks are the only place a cnxset can be acquired. The old use-case for un-delimited cnxsets (pyro/zmqpickle protocols) have been removed. The "mode" of connection objects becomes "write" forever (it will be removed in a couple of changesets). Related to #2919309.

[devtools] remove the remaining bw compat for old-style tests It is just: * dangerously untested right now * on the path of further bw compat removal (cnxset handling) webtest drops an apparently pointless .login call Related to #3933480.

[session] remove session-related bw compat code Related to #3933480.

[session/transaction] removed unused transaction class Related to #3933480.

[transactions] cleanup after the dbapi removal * end req vs cnx dichotomy * Transaction takes a cnx at __init__ time * a couple of super calls are fixed Related to #3933480.

[session/cnx] remove `session_handled` logic Related to #3837233 .

[web/sessions] reduce repoapi usage where possible Using session.new_cnx() should be the new preferred way. Related to #3837233.

[entity,storage] remove hackish code from the bfss path The ClientConnection/Connection duality brought some painful hacks that are no longer needed. Hence we can send entity._cw_update_attr_cache and ._cw_dont_cache_attribute to the grave. Related to #3837233

[connection] eliminate ClientConnection leftovers * in the documentation * in the docstrings * in code notations (e.g. clt_cnx -> cnx) Related to #3837233. From now, there should be just one ClientConnection left in repoapi.py.

[session] all cnx._session become cnx.session The _session belonged to ClientConnection, which is just gone. Related to #3837233.

[repoapi] fold ClientConnection into Connection Connection replaces ClientConnection everywhere. Some notes: * testlib: .client_cnx and .repo_cnx become aliases of .cnx (we might not want to tell people to update their tests again for just no real benefit, so we'll live with these aliases for a while) * entity.as_rset must not be cached because we risk caching result sets having a Connection object as .req (helps unittest_breadcrumbs) * entity._cw_dont_cache_attributes loses its repo/request special paths and only keeps its storage/bfss user (this helps unittest_wfobjs) * moreover, entity.cw_instantiate and .cw_set stop overriding the attributes cache *after* the before_*_entity hooks have run, because there is no need to (it is now actually harmful to do it and unittest_hooks.test_html_tidy* tests remain green because of this) * rset._build_entity sticks its .req onto the entity just fetched from the cache, because otherwise it might carry a _cw that is a Connection object where a Request is expected (helps unittest_views_actions) * we get overall better cache usages (entity caches were split over Request + ClientConnection and Connection), hence the changes unittest_entity and unittest_wfobjs * void the ecache when providing the cnx to a request object Having the entity cache pre-filled when we bind it to the request object hurts because these entities are bound to Connection objects, that lack e.g. `.form` or `.add_js` and crash the views subsystem. Thus, the unittest_testlib.test_error_raised test will are kept green. Closes #3837233

[migractions] remove any session related leftovers Also, the migration handler really wants a "repo" session anyway. This highlights a bug in unittest_wfobjs.test_duplicated_transition where an old connection is reused. It probably used to work because it was made of session + set_cnxset magic dust. Related to #3933480.

[session] provide missing apis to Connection From this point, Connection provides a superset of ClientConnection. Related to #3837233.

[connection] replace .running_dbapi_query with .hooks_in_progress The thing was badly named. It tries to help distinguish between queries issued directly by the programmer (e.g in the views: cnx.execute(...)) from queries issued from the hooks, operations ... or even the repository or the native source objects. It worked heuristically being associated with the security being disabled. We provide a better name and an implementation distinct from the security management methods. Related to #3933480.

[dataimport] Turn the module into a package Just `hg mv` dataimport.py and test/unittest_dataimport.py.

[dataimport] Properly escape strings sent to COPY FROM (closes #5278743) See http://www.postgresql.org/docs/9.1/static/sql-copy.html#AEN64296 for escaping codes.

[http] fix cache-control syntax in EtagHTTPCacheManager use , to separate cache response directive closes #5307539

[connection] provide some missing documentation bits The previous changesets remove information about session data and transaction data. We provide the necessary entries in the docstring of the Connection class. Related to #3933480.

[repoapi,session] remove all session-as-cnx backward compat The `dbapi` being gone, we now can drop the session object bw-compatibility layer. This will allow further simplifications, such as folding ClientConnection and Connection (without too much pain), and then having persistent sessions. Related to #3933480.

[server] drop repo.internal_session and InternalSession

Added tag cubicweb-version-3.20.7, cubicweb-debian-version-3.20.7-1, cubicweb-centos-version-3.20.7-1 for changeset 359d68bc1260

[pkg] 3.20.7

[dbapi] retire repo.[gs]et_shared_data which were used by the dbapi Related to #3933480

[dbapi] retire repository transaction methods which were used by the dbapi Related to #3933480.

[dbapi] retire repo.call_service which was used by the dbapi Related to #3933480.

[dbapi] retire repo.commit/rollback which was used by the dbapi Related to #3933480.

[dbapi] retire repo.user_info which was used by the dbapi Related to #3933480.

[dbapi] retire repo.entity_metas which was used by the dbapi Related to #3933480.

[dbapi] retire repo.execute, which was used by the dbapi We will retire or reform pieces, mostly of the Repository class, that were related to the old dbapi way of life. Related to #3933480.

[devtools/testlib] remove dbapi backward-compat Related to #3933480.

[test] fix copy-pasta in doc string

[book] remove dbapi documentation Move some still-relevant doc related to the RQL connection API to the 'sessions' section. It might make sense to move this back outside of the "Repository development" chapter at some point since this is relevant in other contexts.

[uilib] `uilib.js` helper now honors explicit JSString (closes #4959538) Previously, if a JSString was inside a nested container (i.e. list or dict), it was handled as a string, not a verbatim js chunk (see docstring and test). So the following code:: >>> str(js.cw.pouet(1, {'callback': JSString('cw.mycallback')}) resulted in:: "cw.pouet(1, {'callback': 'cw.mycallback')})" while we expect (note the removed quotes):: "cw.pouet(1, {'callback': cw.mycallback)})" We use ``cubiweb.utils.js_dumps`` instead of ``cubicweb.utils.json_dumps`` which is aware of JSString.

[dbapi] remove the dbapi module and its immediate remaining users We suppress toolsutils.config_connect, which has currently only one known user (the email cube), which is being patched. It can be replaced with utils.admincnx function for a local access. Next will come a series to: * remove the session backward compatibility * fold ClientConnection into Connection Closes #3933480.

[web/views] remove dbapi usage from 'gc' view

[sqlite] Fix integrity error parsing Up to sqlite 3.7, integrity error message may be in the singular form if only one column is in the unique index. Closes #5224632

[postgres] fix limit_size regexp to properly remove one char tags like <p>. Closes #5221847

[postgres] _regproc.postgres.sql should be reloadable

[mail] include Date header in generated mails (closes #5271058) Mails without Date gets flagged as spam. Plus it's wrong anyway.

[doc] mention datadir-url configuration option in release notes Related to #5204550