avoid sanitizing warnings

[devtools] avoid warning about source['global-db-name'] sqlite and postgres db handlers store the db template name there.

[skeleton] remove no-break space from skeleton setup.py

[server/test] drop unused import

[doc] update 3.21 release notes

[devtools] Sort _cw_fields in fake_form Try to generate a predictable string, to help testing with PYTHONHASHSEED=random.

[entity] sort fetchattrs when generating rql Try to generate a predictable rql query, to help testing with PYTHONHASHSEED=random.

Update hgignore for temporary test databases

[js] removed unused and broken SuggestForm Widget (closes #5213487)

[test] Fix unittest_i18n to run properly with "pytest -t" (closes #5576169) When running "pytest -t", many tests modules will be imported and run within a single python process. unittest_i18n fails when run after tests such as unittest_webtest.py or unittest_httptest.py. The main reason is that unittest_i18n does various tricks to add a test cube to the search path, which badly fails when a previous test module has already loaded "stuff" (for lack of a better word). Instead, let's just call the 'i18ncube' command with subprocess. The change from logilab.common.testlib to unittest is not directly needed, but comes as a free cleanup with the above changes.

[server/test] fix random error in unittest_security When entering a new Connection, we cache the user and its 'login' attribute (with no permissions checking). This test makes 'CWUser.login' unreadable by guests, and then proceeds to make sure the 'anon' user can actually not read any 'login' attribute. However, due to the above cnx initialization, anon's login is actually cached, hence readable. This happens to make the test fail sometimes depending on the order in which CWUser entities are returned, because one of them has .complete() called, which as a side effect sets the attribute cache to None for unreadable attributes. Call .complete() on both entities to reset the login cache. While this is still highly debatable, at least it's consistent.

[entity] fix typo in comment

[doc] Add a tutorial and extend documentation for ExtEntityImporter Related to #5414753.

[dataimport] introduce the importer and extentity classes This introduces the ``ExtEntity`` class which is a transitional state between data at external source and the actual CubicWeb entities. ``ExtEntitiesImporter`` is then in charge to turn a bunch of ext entities into CW entities in repository, using a given store. This changeset also introduces ``SimpleImportLog`` and ``HTMLImportLog`` which implement the CW DataImportLog interface in order to show log messages in UI using simple text and HTML formats respectively, instead of storing these messages in database. Both have mostly been backported from cubes.skos.dataimport. Closes #5414753.

[test] Use store.prepare_insert_relation instead of deprecated relate method Follow-up for 1f5026e7d848. Related to #5040344.

[dataimport] since we have foreign key constraints in the database, 'entities' table should be updated first

[dataimport] Move stores to new API. Here is the final store API: * ``prepare_insert_entity(<entity type>, **kwargs) -> eid``: given an entity type, attributes and inlined relations, return the eid of the entity to be inserted, *with no guarantee that anything has been inserted in database*, * ``prepare_update_entity(<entity type>, eid, **kwargs) -> None``: given an entity type and eid, promise for update given attributes and inlined relations *with no guarantee that anything has been inserted in database*, * ``prepare_insert_relation(eid_from, rtype, eid_to) -> None``: indicate that a relation ``rtype`` should be added between entities with eids ``eid_from`` and ``eid_to``. Similarly to ``prepare_insert_entity()``, *there is no guarantee that the relation will be inserted in database*, * ``flush() -> None``: flush any temporary data to database. May be called several times during an import, * ``finish() -> None``: additional stuff to do after import is terminated. **Warning:** ``prepare_update_entity()`` still needs to be implemented for NoHookRQLObjectStore. Related to #5040344

[js] Using appropriate http method when calling a remote function to respect REST principles. closes #5355952

[server/test] Add a test for db-statement-timeout option Sadly I don't think there's a way to test that functionality without adding sleeps which make the test suite longer :/ Related to #2547026

[datafeed] give an error message if a source is missing a parser id Closes #3484231

[storage] use mkstemp to create files in bfss Avoids race condition, at the cost of slightly less predictable file names.

[test] Skip spa2rql tests when fyzz is not available Closes #5456750.

[web/staticcontrollers] fix docstring spelling

[server/rql2sql] fix spelling in comment Not that I understand what that sentence means.

[doc/book] fix rql syntax examples "Any", not "ANY". "EXISTS" is a function.

[migration] add sql constraints on upgrade Related to #5154406.

[schema] neuter check method for constraints we implement in sql Closes #5154406.

add IUserFriendlyError adapter for violation of check constraints This way we get back the same error messages we get from the python check. Related to #5154406

[hooks/syncschema] drop/add check constraints as appropriate Related to #5154406

[schema2sql] support NOW and TODAY in check constraints Related to #5154406.

[schema2sql] insert some constraints into the backend BoundaryConstraint, IntervalBoundConstraint and StaticVocabularyConstraint can be implemented in SQL with CHECK constraints. Next steps, not part of this changeset: - migrations - removing no longer necessary pre-insertion work on the python side - translating backend exception due to constraint violation into ValidationError Note: this means that these constraints can no longer be ignored by disabling integrity hooks. Related to #5154406

[test] run unittest_syncschema with postgresql Lets us test more things, since we can actually do ALTER TABLE.

[test] run unittest_migractions on postgresql sqlite's lack of support for a bunch of alter table functionality means some thing just don't get tested otherwise.

[devtools] Put pg sockets in /tmp Makes for shorter paths (limited by sockaddr_un). Also use the default port, since we're not listening on TCP sockets.

[devtools] change the way we start/stop postgresql Instead of having the test db handler start a cluster on demand, use the test module's setUp/tearDown callbacks to do it. This allows to have one data directory (and thus cluster) per test module, allowing different test modules to run in parallel, each using its own database cluster whose path is based on the test module.

[devtools] make test db name for postgresql process-specific In case we run two tests concurrently on the same cluster, better not use the same db name for both.

[devtools] extract functions to start/stop a postgresql cluster

[test] don't leave NULL columns around when making an attribute required It doesn't matter on sqlite (it doesn't do ALTER COLUMN), but when running this test on postgresql it fails to add the 'NOT NULL' constraint otherwise.

[server/session] Roll back in connection exit This was changed in commit 8b35a898b334 "[server] remove cnxset tracking, it is now unneeded". When exiting a CubicWeb Connection, the database connection is put back into the common pool. At that point anything done during the lifetime of the CubicWeb Connection needs to be committed or rolled back. Do a proper rollback to finish the transaction, as was happening before the above-mentionned commit. This showed up on testing with postgresql, when a DB schema modification was blocked by an old connection that was still "in transaction".

[web/views/formrenderer] do not use `cubicweb:target` attribute on form (closes #5534074) The `cubicweb:target` is a flag indicating that the form data (with file input) shall be posted inside an iframe - this is a well known "ajax-like" workaround to post files with browsers that do not support `FormData` (a.k.a IE<10). The `cubicweb:target` was introduced when CW used the "xhtml strict" doctype. Now that CW uses the "html5" doctype, this namespaced attribute is no longer necessary and the iframe can be generated directly. Before this patch, CW inserts the `cubicweb:target` attribute in the form DOM element (server side) and `setFormsTarget()` updates the DOM with a new `<iframe>` element (client side). Now, CW inserts the `<iframe>` DOM element directly (server side), making `setFormsTarget` useless.

[js] lint : make it explicit that typeof is an operator, not a function. Also, use === / !== except for null / undefined comparison

[devtools] qunit: show helpful data when an assertion fails Related to #5533333.

[web/test] Use shorter $() instead of jQuery() Consistency FTW. Related to #5533333.

[devtools] qunit: use new async testing APIs http://qunitjs.com/cookbook/#asynchronous-callbacks QUnit keeps track of all the assert.async() objects created inside the test functions and expects all done() functions to be called. Failure to do so will result in the test being failed. Unlike .start and .stop which were internal APIs, assert.async() is stricter and fails tests if assert methods are used *after* all done() functions are called (see "test callback execution order"). Related to #5533333.

[devtools] qunit: use new assert APIs test functions now accept an "assert" argument which exposes QUnit's assert methods: assert.{expect,equal,deepEqual,ok} Related to #5533333.

[devtools] qunit: stop using global variables test -> QUnit.test module -> QUnit.module Related to #5533333.

[devtools] qunit: update to 1.18.0 (closes #5533333) * the logging API has changed (register callbacks instead of setting attributes on the QUnit object) * the logging callbacks all have a single "details" object argument * the "main" div was renamed to "qunit-fixture" (this change is responsible for all the churn in our own JS test files) * QUnit can now build the DOM elements it needs for reporting from a single "qunit" div

[devtools] qunit: stop using deprecated assert functions equals -> equal same -> deepEqual Related to #5533333.