[warnings] put an end to warnings in the sqlite driver over `str` being sent instead of unicode strings

[web/request] clearly mark user_callback-related methods as deprecated The deprecated decorator not only emits a warning at run-time but above all makes the deprecation immediately apparent in the source. closes #3567793.

[book] document __unique_together__, remove bad RQLUniqueConstraint example RQLUniqueConstraint should be avoided whenever possible. Related to #3753250.

[schema] stop using RQLUniqueConstraint (closes #3753250) The last uses are replaced with unique together constraints.

[dataimport] Have ucsvreader's API match that of csv.reader (closes #3705701) 'separator' and 'quote' are replaced with 'delimiter' and 'quotechar'.

[utils] Add a '_cwtracehtml' GET parameter to trace self._cw.w() calls (closes #4601327) The core of this patch is in UStringIO.write(). When tracing is enabled, write() doesn't just append the 'value' argument to the underlying list. Instead, a stack trace is recorded and a special HTML "source" is formatted. The output with tracing enabled is an HTML page, with the original HTML escaped, and made clickable to show the stack trace when the write() call was done. This allows answering the recurring question: "who wrote this tag here?!"

[datafeed] Add a raise_on_error parameter to DataFeedSource.extid2entity And pass the option from various `process_` methods in existing parsers. This makes debugging in tests easier. Closes #4601191.

[test/view forms] use the official, undeprecated API

[rset] kill the rset._rqlst cache Right now it "works" for the standard, internal uses. However when we will fold ClientConnection and Connection, it will hurt, because suddenly we get more cache hits, and the following situation would become commonplace: * there is an un-annotated _rqlst given by the querier * some view (e.g. facets) requests the .syntax_tree, which takes a copy of _rqlst * the view actually expects the rql syntax tree to be annotated, but it was not, hence we crash. Related to #3837233.

[source/native] session -> cnx Also swap process_results arguments order to make cnx come earlier.

[hookstests] change got/expected order

[config] kill a getattr

[tests/web] switch previous commit (136b5f995f8) to the new test api Related to #3670209.

[web] remove unused argument from treeview._init_headers

[doc/3.20] mention CWEP-002

[web] stop accepting the magic __message form parameter This has been deprecated for a while, and replaced by _cwmsgid, which doesn't allow arbitrary content.

[web/request] add security_enabled method Just forward to the underlying repo connection. This can be useful in the ORM when dealing with a "code-only" attribute, i.e. something that shouldn't be directly user-visible but must be accessible from a web request.

[dataimport] Fix use of _create_copyfrom_buffer() (related to #3845572) This is used indirectly by SQLGenObjectStore when running on a PostgreSQL database. Regression introduced by commit 70056633085c.

[c-c configure] make it possible to specify the section for sources configuration (closes #3477678)

[server] fix 'cnx' variable confusion in DBG_SQL exception case The rollback handling expects 'cnx' to be the cubicweb Connection, but the DBG_SQL block was replacing it with an sql connection, leading to lulz down the line. Also remove obsolete getattr (the sqlite wrapping is now done at the cnxset level, so cnx.cnxset.cnx should be the right thing already).

Provide sufficient context to check 'delete' permission in AjaxEditRelationCtxComponent Call rdef.check only when both fromeid and toeid are available. Though only call it once (for the first encountered related entity). Factorize a bit to keep handling of CSS/JS addition the same. Closes #3670209.

merge 3.19.6 into 3.20 branch

Added tag cubicweb-version-3.19.6, cubicweb-debian-version-3.19.6-1, cubicweb-centos-version-3.19.6-1 for changeset 934341b848a6

[test] missing unittest.main() call in unittest_http_headers.py

[test] make unittest_webconfig independant of CWD if apphome is unset in ApptestConfiguration constructor, this later looks for a 'data' directory in CWD, making this test runnable only from web/test (and pytest do chdir before executing tests, so we did not noticed this problem).

[webctl] do not ask questions if verbosity is at 0 Closes #4641960.

[web] Slight tweak on jQueryUI's colors on links (related to #4564046) Follow up to commit 16f550b48d57 which was somewhat brutal. This change only overrides jQueryUI's CSS rules inside tab panes and leaves actual tab buttons alone.

[view] Remove unused imports

Remove obsolete __future__ imports Generators have been available since 2.3, "with" statements since 2.6.

[devtools] Remove unused local function Unused since cw's first commit.

[uihelper] Fix a possible NameError in meta_formconfig could occur when using FormConfig with a custom uicfg object. Closes #3802298.

[devtools] Avoid db_cache collisions and mis-loading This is done by stopping to share the db_cache between TestDataBaseHandler instances It may seem overkill, but is the only way I could get it to work properly. I think the whole db caching code needs some rework to better work with different db handlers. Closes #4601328

[devtools][pg] Remove a wrong assertion in _backup_database When using pre_setup_database, a repo gets initialised before calling backup_database. Hence self._repo is NOT None it that case.

[pkg] prepare 3.19.6

[merge] backport 3.18 fixes

Accept '==' operator in cubes dependencies '==' is the operator used in the python dependencies specifications. See PEP 440 http://legacy.python.org/dev/peps/pep-0440/#version-specifiers Using the same operator in cubes dependencies allow to make setuptools handle the cubes dependencies while keeping cubicweb happy about the dependencies. Closes #4375144

[session] don't silently ignore commits commits while in the precommit or postcommit phase are silently ignored. This is OK in precommit since there'll be a commit afterwards anyway; OTOH in postcommit it may surprise the developer who didn't read the documentation for postcommit operations carefully enough. Related to #4383922.

[migration/bootstrap] add explicit index removal for sqlserver, be tolerant to migration replay Closes #4618944.

Added tag cubicweb-version-3.18.7, cubicweb-debian-version-3.18.7-1, cubicweb-centos-version-3.18.7-1 for changeset cb96f4403cf2

[pkg] prepare 3.18.7

[migration/3.18] speed up the migration and be a little bit more informative Closes #4619277.

[schemaserial] be robust against duplicated CWUniqueTogetherConstrain entities arising from bug in the pre 3.18 era Closes #4619278.

[server] Handle unique constraint violations under recent sqlite The error message changed from "columns foo, bar, baz are not unique" to "UNIQUE constraint failed: table.foo, table.bar, table.baz". Closes #3564510

[merge] backport 3.17 fixes

Added tag cubicweb-version-3.17.18, cubicweb-debian-version-3.17.18-1, cubicweb-centos-version-3.17.18-1 for changeset cda4b066933f

[pkg] prepare 3.17.18

[source/native] handle newer Integrity error messages from sqlite > 3.7 Similar to a0cf2993b6d3 that was done for 3.18.

[source/native/backup_restore] have a unique tunable blocksize for the dump phase Thus we make memory errors much less likely. Closes #4618949.

[urlpublishing] handle sub-types in RestPathEvaluator Since commit 31a1813d53f3 "[entity/url publishing] fetch_rqlst should use 'is_instance_of' instead of 'is'", the RestPathEvaluator's input rset can have more than one etype, so it can't always use sameetypelist. Related to #3825488

[views] fix ecsvexport selector introduced by e48e5a597ccc Fix the view and make web/test/test_views.py pass again.

Add custom checker for Password values We override the default converter to pass Binary values through, but don't do anything about the checker. This worked previously because yams allowed StringIO instances, although its converter didn't do the right thing for them. Fixing this in yams requires that we properly register a checker.

[component] give Links a __repr__

[debian] move markdown dependency from cubicweb-web to cubicweb-common Pointed out by David Douard.

[web/auth] stop playing games with locals() We only want to know if any retriever found something.

[entity] rather than crashing with an IndexError on entity creation, raise a meaningful exception

[web] partial backout of #8391bf718485 to restore RelatedObjectsVComponent Apparently the intended deprecation did not work, and this component is used in at least the tracker cube.

[server] typo fix

[devtools][pg] Activate the non-durability settings of Postgresql. It speed up the PG tests by a factor of 3. See http://www.postgresql.org/docs/current/static/non-durability.html.

[server] fix LoginPasswordAuthentifier.authenticate some kind of typo there.

[web] Override jQueryUI's colors on links (closes #4564046)

[hooks] Fix precommit event logging message

[service] allow repo_stats for users It's used by the siteinfo view, which is available to managers and users. This change prevents a crash in that view.

[views] Escape class attribute value in CWGroup incontext view

[devtools] use a specific test_db_id when disabling anon The anonymous user is created (or not) in postcreate, which for tests means when creating the cached empty database. The anonymous_allowed=False test classes should thus not share their template db with other test classes, otherwise either they end up with an unexpected anonymous user, or the others miss theirs and things fall apart.

Restrict yams version CW 3.19 is not compatible with yams >= 0.40

[facets] Honor 'start_unfolded' facet attribute (closes #4502799)

[facets] Correctly look for inputs of type "hidden" (closes #4502768) jQuery ':hidden' selector looks at CSS visual properties (eg, 'display', 'visibility'). The intent here was probably to look for inputs of type "hidden", which many facets use to store user selection data (eg, FacetRangeWidget). The problem is that regular text inputs (eg the "has_text" facet which has a '<input type="text"/>') will be picked up by this selector if they are inside a folded facet. Chaos and destruction ensue.

[web] Cache results from 'i18n' ajax controller (closes #4487856) This is a simple transient cache which will be forgotten as soon as the user leaves the current page.

[web] There is no global noop(), use jQuery's instead (closes #4487832)

[server] Add missing import of logilab.database Closes #4469407.

[migration] hackish black magic to bootstrap addition of formula attr during migration Turns out we can't add an attribute to CWAttribute before the attribute exists. add_attribute generates rql with a 'formula' relation, which CW doesn't know about yet, so things get unhappy. Once that is fixed, we need to make the CWAttribute addition operation deal with CWAttribute entities that don't have a formula yet. Finally, move the migration to bootstrapmigration_repository so it happens early on, and use add_entity_type rather than add_relation_type for CWComputedRType.

[server] commit after serializing schema Use separate transactions for schema serialization and postcreate scripts execution, so that the latter can rely on the schema being persistently stored. This matters e.g. if something in a postcreate violates a unique_together constraint, we need the constraint persisted in order to raise a meaningful error. See #4525069.

[book] fix docstrings to please sphinx example rst roles in docstring must be protected (in a literal).

[book] several fixes in the rst files - prevent duplicate markup definitions, - use existing Workflow markup instead of redefining one (in baseschema.rst) - include dataimport - fix non-existant directive autodocfunction - remove the empty 'embedding.rst' file

[book] activate the viewcode extension Automagically make link to class/modules/etc. it finds in the documents.

[book] new theme based on pyramid theme (closes #4291287)

Remove unused lgc.interface imports and leftovers

[repository] provide a .new_session entry point The current .connect only returns a `sessionid` which must be given to repo._get_session(...) to get the real session object. Related to #4151635.

[devtools/testlib] Use actual 'admin' user configuration values Previously ``db-user`` and ``db-password`` configuration values were used to fill ``admlogin`` and `admpassword`` attributes of CubicWebTC. The correct data for these should read from `[admin]` section. Closes #4065070.

[forms] closes #2437859 - Detect and prevent concurrent edition of the same entity. Add the timestamp of form generation to each entity's meta-information fields. On form validation, check that no concurrent change is overwritten and raises a ValidationError in case of concurrent change. A nicer handling with a message and a link to the new version of the entity would be a good thing...

[migration] make sure the repo knows about all constraint types Adding new constraint types is cumbersome. It's easy to forget to do it in a migration script. So sync the constraint types at the beginning of each migration. Closes #3724892

[migration] stop caching the mapping from constraint type name to eid It's not so frequent that a cache seems necessary, and we were not invalidating that cache when adding a new constraint type. Related to #3724892.

[web/json] an empty rset is just fine for ejsonexport The ejsonexport view can just return an empty list for an empty rset. Closes #4005518

[RichString] Add markdown support Supporting markdown requires the 'Markdown' python packages, and since it widely available on the target platforms adding it as a strong requirements will not be a big constraint. Closes #3814302

[ldapfeed] Reduce default value for user-attrs-map option (closes #3824889) This is needed because lgc.configuration.Configuration does not allow removing key/value pairs from the default (due to its use of dict.update() internally). Since CWUser.login is required, users of the add-source command will always be able to override it.

[serverctl] Ask for URL when adding a new source (closes #3824868)

[serverctl] Ask for parser type when adding a new source (closes #3484231) Remove 'quick_start' option to load all AppObjects, including feed parsers.

[doc/3.20] more details on removed code related to #3799117

remove 3.11 bw compat Closes #3799117.

remove most 3.10 bw compat Related to #3799117. The boxes and entityvcomponent objects cannot really be removed as they are still used throughout the code base (and possible cubes). This may be caused by a non-working deprecation and needs some more work.

remove leftover pre 3.9 deprecation warnings

[utils] Remove function-in-the-middle call

[web] set Vary response header to "Accept-Language" when using content negotiation This is slightly annoying because the response actually only varies based on the language we decide to send, which has much fewer possible values than Accept-Language, but that's not in the request, so we can't easily use it. Deployments using varnish or similar and controlling the set of available languages will likely want to override this to allow reasonable amounts of caching. Closes #2105812

[web/cors] don't overwrite other Vary headers Vary is a list of request headers, we shouldn't override others.

[web] add support for HttpOnly cookie flag And use it for session cookies. Closes #4142521.

merge 3.19.5 into 3.20 branch