[merge] backport 3.17 fixes

Added tag cubicweb-version-3.17.18, cubicweb-debian-version-3.17.18-1, cubicweb-centos-version-3.17.18-1 for changeset cda4b066933f

[pkg] prepare 3.17.18

[source/native] handle newer Integrity error messages from sqlite > 3.7 Similar to a0cf2993b6d3 that was done for 3.18.

[source/native/backup_restore] have a unique tunable blocksize for the dump phase Thus we make memory errors much less likely. Closes #4618949.

[urlpublishing] handle sub-types in RestPathEvaluator Since commit 31a1813d53f3 "[entity/url publishing] fetch_rqlst should use 'is_instance_of' instead of 'is'", the RestPathEvaluator's input rset can have more than one etype, so it can't always use sameetypelist. Related to #3825488

[views] fix ecsvexport selector introduced by e48e5a597ccc Fix the view and make web/test/test_views.py pass again.

Add custom checker for Password values We override the default converter to pass Binary values through, but don't do anything about the checker. This worked previously because yams allowed StringIO instances, although its converter didn't do the right thing for them. Fixing this in yams requires that we properly register a checker.

[component] give Links a __repr__

[debian] move markdown dependency from cubicweb-web to cubicweb-common Pointed out by David Douard.

[web/auth] stop playing games with locals() We only want to know if any retriever found something.

[entity] rather than crashing with an IndexError on entity creation, raise a meaningful exception

[web] partial backout of #8391bf718485 to restore RelatedObjectsVComponent Apparently the intended deprecation did not work, and this component is used in at least the tracker cube.

[server] typo fix

[devtools][pg] Activate the non-durability settings of Postgresql. It speed up the PG tests by a factor of 3. See http://www.postgresql.org/docs/current/static/non-durability.html.

[server] fix LoginPasswordAuthentifier.authenticate some kind of typo there.

[web] Override jQueryUI's colors on links (closes #4564046)

[hooks] Fix precommit event logging message

[service] allow repo_stats for users It's used by the siteinfo view, which is available to managers and users. This change prevents a crash in that view.

[views] Escape class attribute value in CWGroup incontext view

[devtools] use a specific test_db_id when disabling anon The anonymous user is created (or not) in postcreate, which for tests means when creating the cached empty database. The anonymous_allowed=False test classes should thus not share their template db with other test classes, otherwise either they end up with an unexpected anonymous user, or the others miss theirs and things fall apart.

Restrict yams version CW 3.19 is not compatible with yams >= 0.40

[facets] Honor 'start_unfolded' facet attribute (closes #4502799)

[facets] Correctly look for inputs of type "hidden" (closes #4502768) jQuery ':hidden' selector looks at CSS visual properties (eg, 'display', 'visibility'). The intent here was probably to look for inputs of type "hidden", which many facets use to store user selection data (eg, FacetRangeWidget). The problem is that regular text inputs (eg the "has_text" facet which has a '<input type="text"/>') will be picked up by this selector if they are inside a folded facet. Chaos and destruction ensue.

[web] Cache results from 'i18n' ajax controller (closes #4487856) This is a simple transient cache which will be forgotten as soon as the user leaves the current page.

[web] There is no global noop(), use jQuery's instead (closes #4487832)

[server] Add missing import of logilab.database Closes #4469407.

[migration] hackish black magic to bootstrap addition of formula attr during migration Turns out we can't add an attribute to CWAttribute before the attribute exists. add_attribute generates rql with a 'formula' relation, which CW doesn't know about yet, so things get unhappy. Once that is fixed, we need to make the CWAttribute addition operation deal with CWAttribute entities that don't have a formula yet. Finally, move the migration to bootstrapmigration_repository so it happens early on, and use add_entity_type rather than add_relation_type for CWComputedRType.

[server] commit after serializing schema Use separate transactions for schema serialization and postcreate scripts execution, so that the latter can rely on the schema being persistently stored. This matters e.g. if something in a postcreate violates a unique_together constraint, we need the constraint persisted in order to raise a meaningful error. See #4525069.

[book] fix docstrings to please sphinx example rst roles in docstring must be protected (in a literal).

[book] several fixes in the rst files - prevent duplicate markup definitions, - use existing Workflow markup instead of redefining one (in baseschema.rst) - include dataimport - fix non-existant directive autodocfunction - remove the empty 'embedding.rst' file

[book] activate the viewcode extension Automagically make link to class/modules/etc. it finds in the documents.

[book] new theme based on pyramid theme (closes #4291287)

Remove unused lgc.interface imports and leftovers

[repository] provide a .new_session entry point The current .connect only returns a `sessionid` which must be given to repo._get_session(...) to get the real session object. Related to #4151635.

[devtools/testlib] Use actual 'admin' user configuration values Previously ``db-user`` and ``db-password`` configuration values were used to fill ``admlogin`` and `admpassword`` attributes of CubicWebTC. The correct data for these should read from `[admin]` section. Closes #4065070.

[forms] closes #2437859 - Detect and prevent concurrent edition of the same entity. Add the timestamp of form generation to each entity's meta-information fields. On form validation, check that no concurrent change is overwritten and raises a ValidationError in case of concurrent change. A nicer handling with a message and a link to the new version of the entity would be a good thing...

[migration] make sure the repo knows about all constraint types Adding new constraint types is cumbersome. It's easy to forget to do it in a migration script. So sync the constraint types at the beginning of each migration. Closes #3724892

[migration] stop caching the mapping from constraint type name to eid It's not so frequent that a cache seems necessary, and we were not invalidating that cache when adding a new constraint type. Related to #3724892.

[web/json] an empty rset is just fine for ejsonexport The ejsonexport view can just return an empty list for an empty rset. Closes #4005518

[RichString] Add markdown support Supporting markdown requires the 'Markdown' python packages, and since it widely available on the target platforms adding it as a strong requirements will not be a big constraint. Closes #3814302

[ldapfeed] Reduce default value for user-attrs-map option (closes #3824889) This is needed because lgc.configuration.Configuration does not allow removing key/value pairs from the default (due to its use of dict.update() internally). Since CWUser.login is required, users of the add-source command will always be able to override it.

[serverctl] Ask for URL when adding a new source (closes #3824868)

[serverctl] Ask for parser type when adding a new source (closes #3484231) Remove 'quick_start' option to load all AppObjects, including feed parsers.

[doc/3.20] more details on removed code related to #3799117

remove 3.11 bw compat Closes #3799117.

remove most 3.10 bw compat Related to #3799117. The boxes and entityvcomponent objects cannot really be removed as they are still used throughout the code base (and possible cubes). This may be caused by a non-working deprecation and needs some more work.

remove leftover pre 3.9 deprecation warnings

[utils] Remove function-in-the-middle call

[web] set Vary response header to "Accept-Language" when using content negotiation This is slightly annoying because the response actually only varies based on the language we decide to send, which has much fewer possible values than Accept-Language, but that's not in the request, so we can't easily use it. Deployments using varnish or similar and controlling the set of available languages will likely want to override this to allow reasonable amounts of caching. Closes #2105812

[web/cors] don't overwrite other Vary headers Vary is a list of request headers, we shouldn't override others.

[web] add support for HttpOnly cookie flag And use it for session cookies. Closes #4142521.

merge 3.19.5 into 3.20 branch

Added tag cubicweb-version-3.19.5, cubicweb-debian-version-3.19.5-1, cubicweb-centos-version-3.19.5-1 for changeset 3ac86df519af

i18n update 4 strings have disappeared.

[pkg] 3.19.5

[server] hold connection to the db in tx_actions We can be called without a cnxset (e.g. from repoapi).

[wsgi] If multipart cannot parse the POST content, let it pass. multipart can only parse html form data. It the content_type is, for example, "application/json", get_posted_data should not fail but just stop trying to read the content. Closes #4421845

[devtools] Fix JS tests' HTML code

[devtools] "Keep" some temporary files/dirs around to help with debugging The whole QUnitTestCase runs with an @with_tempdir so it's redundant anyway.

[devtools] Fix Firefox launcher in QUnitTestCase (closes #4294727) The main changes are: - stop creating the profile, firefox will create it - point firefox to a profile directory instead of giving it a profile name (this has the added bonus of not polluting the user's profile list) - start firefox once and kill it 5 seconds later to let it finish its profile creation (along with system-wide extensions setup)

[devtools] allow cross-origin requests for qunit We have a mix of file:// html and http:// ajax calls. Which we should at some point fix to all be http, but. Related to #4294727.

merge 3.19.4 in 3.20 branch

[cors] Fix CORS headers generators The Access-Control-Allow-* and Access-Control-Expose-Headers are response headers, not request headers. Hence, we need generators for them. Closes #4412575.

[wsgi] Fix posted files filename reading The filenames are parsed by multipart.parse_form_data, which does the unicode decoding. Trying to re-decode the filename was leading to an error.

[pkg] Depend on Pillow instead of PIL The Pillow library is becoming the de-facto replacement for PIL. It also is way simpler to install with pip than PIL. Closes #4411354.

Added tag cubicweb-version-3.19.4, cubicweb-debian-version-3.19.4-1, cubicweb-centos-version-3.19.4-1 for changeset c4e740e50fc7

[pkg] 3.19.4

merge 3.18.6 into 3.19

Added tag cubicweb-version-3.18.6, cubicweb-debian-version-3.18.6-1, cubicweb-centos-version-3.18.6-1 for changeset d91501356742

[pkg] 3.18.6

[hooks/security] allow edition of attributes with permissive permissions If an attribute has more permissive security rules than the entity type itself, we should be green and not deny action because of an early global entity permission check (with the more restrictive rules). Only if one attribute with the entity-level permission rules is edited will the global check be performed. Note: * the "if action == 'delete'" check at the entry of check_entity_attributes is a guard for a condition currently not happening in cubicweb itself (but application hooks could conceivably call this function with a 'delete' action) Closes #3489895.

Almost backout afcd46716d6a which breaks _select_best raising an ambiguity exception in debug mode. The problem is, before afcd4, *tests* ran in debug mode and we want this (e.g. we want to show, rather than swallow, select ambigüities). We juste replace the bogus __init__(vreg.config) by __init__(True), which is practically equivalent and also much more clear.

[server] fix anonymous_user predicate in tests devtools' TestServerConfiguration overrides the anonymous_user method, but not the anonymous-user config option, so testing for the latter would give the wrong result. Closes #3996664.

[entities] cw_rest_attr_info() should only consider required attributes (closes #3766717) This prevents CW from choosing unique but non-required attributes. None/NULL is a poor choice for RESTful URIs.

[views] csvexport accept an empty rset (closes #4236928) When you tried to apply the 'csvexport' view on an empty rset, the view couldn't be selected and you got a HTTP 500 error. Also add two new test cases.

[views] Display attributes in entity creation form based on "add" permission Previously, the "update" permission was used. Hence in case the latter is more restrictive that the "add" permission, an user may not be able to set such an attribute, despite she may have "add" permission on it. As a result of the change of permissions action in `editable_attributes` method (add/update depending on whether the entity is being created or modified), the "eid" attribute would have shown up in the edition form. To avoid this, it is moved in the "hidden" section (where it should arguably belong anyways). Closes #4342844.

[datafeed] Commit after all deletions in datafeed parser This avoids misleading validation error because schema constraints could be temporarily broken depending on the deletion order. Closes #4372127.

[schema] CWComputedRType is a schema type Hide it from the default views.

[doc] proofreading CWEP002 section

[CWEP002] document computed relations and attributes Related to #3546717.

[CWEP002 migration] support sync_schema_props_perms for computed attribute

[CWEP002 migration] support drop_relation_type/drop_relation_definition/drop_attribute for computed attributes

[CWEP002 migration] support add_relation_type/add_attribute for computed attributes Related to #3546717.

[CWEP002] properly handle serialization of computed attributes Until this cset, only serialization of computed attribute's formula was handled (thanks to the relation definition's properties dict). We now: * properly serialize/deserialize attribute's formula * test first if the database has been migrated and contains related column or not Related to #3546717

[CWEP002] Add support for computed attribute synchronization Related to #3546717. Test and handle the behaviour with several formulas and identified use cases. To do so add a birth year and a computed attribute age to Person in the test schema.

[CWEP002] Add schema finalization checks for computed attributes ``finalize_computed_attributes`` essentially checks that computed attributes types match with the type declaration of the attribute. Related to #3546717.

[CWEP002] Account for attribute formula in schema bootstrap This isn't enough to have computed attribute support, it is only in order not to crash when yams 0.40 is used. Related to #3546717.

[CWEP002 migration] support sync_schema_props_perms for computed relations

[CWEP002 migration] support add_relation_type for computed relations Related to #3546717.

[CWEP002 migration] support drop_relation_type for computed relations Related to #3546717.

[CWEP002 migration] properly raise exception on (add|drop)_relation_definition for computed relation Related to #3546717

[hooks] do not abuse of inheritance for CWRType hooks Just copy the selector to ease readability.

[schema serial] add some comments

[schema] define full_rql on RQLExpression class without it, we may have an error while displaying error which occured during some expression initialization because this attribute is not yet defined while used in the __str__ method.

[schema] properly raise BadSchemaDefinition when some expression mains variables may not be guessed

[CWEP002] properly handle serialization of computed relations We now: * have CWComputedRelation in the bootstrap schema to store computed relations * properly serialize/deserialize it * test first if the database has been migrated and contains the related table Related to #3546717 [jcr: adjust unittest_querier to pass with the added entity type]

[CWEP002] Plug the computed relation rewriter in the querier Related to #3546717.

[CWEP002] refactor rql read security checking Split 'check_read_perms' into 'check_relations_perms' which checks relations 'read' permissions and 'get_local_checks' which build dictionary of local security checks (rql expression) for variables. This allows to check relations 'read' permissions earlier in the process and so to prepare insertion of the rql rewriter: we want to check permissions of the computed relation, not permissions of relations introduced by the associated rule, to conform to the CWEP. Related to #3546717

[CWEP002] introduce RQLRelationRewriter Refactor existing RQLRewriter for later reuse of rewriting relation as specified by CWEP002. Work is different because we simply want to replace a relation by another rql snippet and we don't have to bother with EXISTS, subqueries and all. This rewriter is not yet plugged into the querier. Depends on yams 0.40 API. Related to #3546717

[CWEP002] Add schema finalization checks for computed relations (rules) Related to #3546717

[schema] add utility function to build a CubicWebSchema from a namespace Will be useful for testing computed attributes and relations.

Fix test migration crash waiting to happen due to inferred relations Avoid crashing when a relation definition which is already known (due to being inferred) gets explicitly added. As explained in a comment, this would deserve more thinking, but at least this fixes a test failure with computed relations.

fix typo in syncschema hooks

[test] Make test_undo_api less random The order in which hooks are run is not predictable if they have the same 'order' attribute, which is the case for SetOwnershipHook and SetInitialStateHook. So don't assume in_state will be set before created_by.

merge 3.17.17 into 3.18 branch

[wsgi] Fix unicode decoding in POST The application/x-www-form-urlencoded sent by firefox or chrome is utf-8 encoded BEFORE being urlencoded. Hence, decoding must urldecode THEN decode the utf-8 string, and not the other way around.

[wsgi] add the --debug / -D option to the wsgi command

[wsgi] Fix multiple variables reading in params Closes #4306081

[wsgi] Set self.vreg Closes #4306049

[wsgi] Add missing import Closes #4305988

[wsgi] Fix https detection - 'HTTPS' is not part of wsgi, it is a CGI variable. The right wsgi variable is 'wsgi.url_scheme' - Force https if url starts with '/https/' (and remove the prefix from the path. Closes #4305985

[wsgi] Re-set the request content after calling the inherited constructor. Closes #4191813

[wsgi] Honor the 'CONTENT_TYPE' wsgi variable See http://legacy.python.org/dev/peps/pep-0333/#environ-variables Closes #4191812

Added tag cubicweb-version-3.17.17, cubicweb-debian-version-3.17.17-1, cubicweb-centos-version-3.17.17-1 for changeset 57e9d1c70512

[pkg] prepare 3.17.17

[web/headers] don't list a request header twice in "Vary"

[test] silence a few more deprecation warnings in unittest_hooks self.session → cnx.

[session] call rollback in Connection.__exit__ If we just free the cnxset and clear the Connection, we're missing the pending operation's rollback_event callbacks, which may be needed for cleanup.

[devtools] pre_setup_database takes a Connection, not a Session

[source/native] cPickle is available in all supported pythons

devtools: deprecate .req_from_url Which uses .request() And instead provide .admin_request_from_url, which is a context manager yielding the request.

[tests] Add a webtest based TestCase class Closes #4002134

[tests] Move 'anonymous_allowed' property from CubicWebServerTC to CubicWebTC The effective set of the property on the config is now done later that before, ie at the end of CubicWebTC.init_config instead of before. It is believed not to have an impact on the test suites that uses `anonymous_allowed`. Related to #4002134

[compat] Remove imports of "any" and "all" from lgc (closes #4306044) They're builtin since python 2.5.

[css] Remove the `div` tag specification of the pendingDelete css rule The HTML tag was changed by e2f96b16c3bd from a div to a span without changing the css rule accordingly. Closes #4285248.

[cwvreg] cleanup the event manager when reloading modules Closes #3848995 The event manager callbacks are not cleaned during reloading. So the callback defined in the reloaded module appears twice (old and new version). This may cause problem when the old version is called.

[test] Add missing attribute 'add' permission in test schema Silences yams warning.

[facet html] Add surrounding div with "facetBody" class to "has_text" facet. This makes its html structure consistent with the other facets and fixes both css and a javascript behaviour (hide the facet body when clicking on its title). Closes #3797501.

[cwctl] don't prompt for cube options in automatic mode Closes #3984223

[serverctl] fix default value for db-init config-level Otherwise cubicweb-ctl db-init --automatic complains that "--automatic and --config-level should not be used together" when they weren't. Closes #3984336

[web] Return useful error messages when exceptions arise in ajax controllers (closes #3932503) The call to super() in RemoteCallException passes 'reason' all the way to python's Exception.__init__() which is then stored in Exception.args. This way, CubicWebPublisher.ajax_error_handler() gets a useful error message when calling unicode(exc). The change to uilib is just to prepend the exception type name. Just a small improvement.

[test] Fix test breakage uncovered by previous changeset Now that Binary.__eq__ doesn't always return true, this test started failing. CWAttribute.defaultval is zpickled, and the description_format attribute is a String, meaning unicode, so adjust the expected test result.

[etwist] use more specific REQUEST_ENTITY_TOO_LARGE instead of BAD_REQUEST When a request body exceeds the configured limit, return 413 to the client instead of a generic 400.

[entities, view] delete dead code The auto_unwrap_bw_compat metaclass calls unwrap_adapter_compat, which was removed in changeset 697a8181ba30 "remove 3.9 bw compat". So this can't possibly work anymore, meaning we can get rid of it.

[book] stop talking about the hg `forest` extension

[book] update sections about dependencies This is probably incomplete, but it's a start.

[web/component] move unicode() call around Makes Aurélien happier.

[web/js] replace callAddOrDeleteThenReload with more generic callAjaxFuncThenReload That's going to be a more useful replacement for callUserCallbackThenReload. Closes #4178566.

[hooks/syncsession] try to remove cnx vs session confusion

Fix Binary.__eq__ (closes #4172701) Due to a typo we always returned true, which was unhelpful.

[dataimport] stop ignoring errors on flush That can only result in database corruption.

[dataimport] don't commit on flush Changeset 26cdfc6dd6f8 introduced this confusion for no apparent reason, and it doesn't make much sense.

[test] Add test for dataimport's RQLObjectStore

[dataimport] Stop swallowing errors from commit/flush Silent DB corruption is not OK. Also drop comment from ObjectStore.commit that I don't understand.

[dataimport] Update RQLObjectStore to Connection API Take a connection instead of a session. Don't play games with set_cnxset.

[dataimport] remove _rql heresy 1) ObjectStore knows nothing about rql, it just drops it on the floor 2) RQLObjectStore has a session, it can run rql that way 3) setting an object's "_rql" attribute is not a reasonable user-facing interface Also drop _commit attribute from base ObjectStore, it doesn't use it.

[dataimport] remove dead code The only caller of ObjectStore._put is ObjectStore.create_entity, which RQLObjectStore overrides (and doesn't call up).

[dataimport] do not use sys.exit() to raise missing argument error Related to #3845572

[dataimport] _create_copyfrom_buffer: add the tests Related to #3845572

[dataimport] _create_copyfrom_buffer: do not ignore columns if data is a list Related to #3845572

[dataimport] _create_copyfrom_buffer: fix datetime converter + add time converter Include second and microsecond in the output instead of dropping them on the floor. The time zone is not supported for now, though. Related to #3845572

[dataimport] _create_copyfrom_buffer: raise ValueError if conversion cannot be performed If the conversion fails, there is no reason to continue execution. One should notify the error. Continuing after a misconverted data could corrupt the database. Related to #3845572

[dataimport] _create_copyfrom_buffer: fix separator check in string converter The empty string is a valid replace_sep. Related to #3845572

[dataimport] _create_copyfrom_buffer: put converters into a list Cleans up the code to avoid a succession of ifs. Related to #3845572

merge 3.19.3 into 3.20 branch

Added tag cubicweb-version-3.19.3, cubicweb-debian-version-3.19.3-1, cubicweb-centos-version-3.19.3-1 for changeset 37f7c60f89f1

[pkg] prepare 3.19.3

merge from 3.18 branch

merge 3.17.16 into 3.18 branch

Remove uses of logilab.common.compat.{all,any} They're just aliases to the builtin ones on python 2.5+. If anyone needs convincing: >>> from logilab.common import compat >>> compat.any <built-in function any> >>> compat.all <built-in function all>

remove references to global environment variable APYCOT_ROOT

Add warning messages when enabling remote pickle-based repository access Warn when starting a pyro or zmq-only repo, or when one of these access methods is enabled. Closes #2919295

[web] Fix expiry of anonymous sessions (closes #4154479) Things like: try: foo except: bar else: baz doesn't work very well if you expect baz to run even in the exception case. Plus, session.cnx.check() is a dbapi remnant, so drop it and just look at session.mtime.

[hooks/syncschema] avoid altering a dropped table If the constraint's rdef is being removed, CWConstraintDelOp doesn't need to do anything. Otherwise it may try to alter a removed table/column. Closes #4154549

[js] fix name error

[devtools] improve error message when postgresql tools are missing By default in at least Debian, some pg tools are not present in the PATH. Or they may not be installed. But the tests tools expects them to be in the PATH, and give an unhelpful 'No such file or directory' backtrace if they're not found. To help devs using the pg tests we improve the error message.

[server] Replace non portable strftime formatter (closes #4132161)

[web/request] use a picklable Counter object for tab index counters Related to #1381328.

[web/auth] The auth retriever's authenticated() callback takes a session, not a cnx We don't have a cnx at that point.

[test] Fix the query-log-file test Testing this functionality in a CubicWebTC is awkward because the wrapping happens in handle_request, which tests basically bypass (they call core_handle directly, and do their own magic for linking the request with a cnx). The test method worked when ran on its own, but not together with the rest of the test class. So use a CubicWebServerTC instead, which goes through the whole stack.

[doc/book] spelling fixes in security tutorial

[doc/book] spelling fixes in "testing" chapter

[datafeed sources] finish the session -> cnx switch Related to #3933480.

[doc/book] update examples, using the new connection api

[test/entities] use the new connection api

[web] restore query logging functionality (closes #3972561) The query-log-file option stopped working when the web stack was moved from dbapi to repoapi.

[cwctl] make cubicweb-ctl versions lighter (closes #4002158) Set config.quick_start to avoid unnecessary appobjects and schema loading.

Added tag cubicweb-version-3.17.16, cubicweb-debian-version-3.17.16-1, cubicweb-centos-version-3.17.16-1 for changeset a979d1594af6

[forms] consider inline creation form information as linkto info. Closes #3161121

[pkg] prepare 3.17.16

[reledit] Fix reledit icons jumpiness (closes #4106867) Introduce an "invisible" class (copied verbatim from Bootstrap) that only sets the "visibility" property. "display: none" is the reason for the whole jumpiness as it actually detaches the element from the rendered layout. "visibility: hidden" only makes it transparent while keeping its original box properties (width, height, etc).

[views] Replace poorly named "invisible" class with "list-unstyled" "li.invisible" actually means "hide bullet". Use a reasonable name instead, such as bootstrap's "list-unstyled" class. This patch also changes the DOM element the class is applied to. "li.invisible" had to be enabled on every "li" tag, whereas the "list-unstyled" class only needs to be applied to the parent "ul" element.

[views] a 'div' with 'display: inline' screams 'span' (grafted from af6e3db801fcfbd6f562c4a7cfdc89f187042792)

[pkg] remove simplejson dependency We no longer use it, the stdlib's json module is fine.

Added tag cubicweb-version-3.19.2, cubicweb-debian-version-3.19.2-1, cubicweb-centos-version-3.19.2-1 for changeset 8ac2202866e7

[pkg] Fix version number Note to self: don't phase -p before coffee.

[pkg] 3.19.2

[webtests] finish to give all self.view(....) a req=req parameter

[tests/syncsession] use the new connection api

[webtests/application] remove unused imports

[webtests/application] .user(...) really wants a request object

[tests/datafeed] use the new connection api (a small leftover) Test assertions get indented because of self.session -> scoped cnx.

[entitiestests/base] don't store an entity on the test case Make do with an eid.

[entitiestests/base] use the new connection api

[testlib] use the new connection api

[exttests/rest] use the new connection api

[webtests/bookmarks] use the new connection api

[hookstests/integrity] use the new connection api

[hookstests/base] use connections instead of web requests

[hookstests/base] use the new connection api

[hookstests/syncschema] use the new connection api

[tests/rqlannotation,querier] use the new connection api

[webtests/web] use the new connection api

[webtests/navigation] use the new connection api

[webtests/basetemplates] use the new connection api

[webtests/basecontrollers] use the new connection api

[testlib] deprecate .remote_call and provide new connection api friendly .remote_calling

[webtests/propertysheet] kill an "import *"

[webtests/breadcrumbs] break long lines

[webtests/views_editforms] use the new connection api

[webtests/urlrewrite] do not store a plain entity on the test, only an eid

[webtests/urlrewrite] use the new connection api

[webtests/urlpublisher] use the new connection api

[webtests/baseviews] use the new connection api

[devtools/testlib,fill] use the new connection api (for auto_populate) Note that this changes API of CubicWebTC's custom_populate and post_populate methods to take a connection instead of a cursor, which may affect some cubes.

[webtests/automatic views tests] use the new connection api

[devtools/repotest] simplify a very small helper

[tests/repotest] use the new connection api

[tests/querier] use the new connection api (part 3/3) Some adaptations to devtools/repotest: * dead code removal * remove session related code In the tests: * many tests actually don't check the querier but some generic rql property -- when such a test needs several statements to complete a commit, switch to session.new_cnx * provide an assertRQLEqual helper

[test/querier] use the new connection api (part 2/3) A couple of tests need a schema constraint to be dropped to pass with `.qexecute`.

[test/querier] use the new connection api (part 1/3) In devtools/repotest, we provide a simple `.qexecute` with autocommit, and we use it (where possible) in unittest_querier. Note that the .session object is a real session, not the magic property found in CubicWebTC test classes.

[dbapi] allow talking to a pre-3.19 remote repository Closes #4071285

[migration] don't add new elements to config._cubes in remove_cube We're expanding dependencies based on the current versions of the cubes, which may well not be the same as the ones known by the db (we're in the middle of an upgrade, after all). Listing them in config._cubes prevents us from calling add_cube() later to actually add them. Closes #4002156

[web test] repair test broken by 0872ac2a1db0 after the above cset, during test instance's url is set before the creation of core entities (ie created at database initialization), hence they have now a valid cwuri.

[dataimport] Drop reference to the 'source' column (closes #4067694). CW 3.19 drops the multi-source support. This column does not exist in 3.19.

[notification] drop on commit sending of message feature of NotificationView It duplicates feature already provided by cw.hooks.notification.notify_on_commit function. Simplify the API on the way.

[datafeed parser] enhance retrieve_url to support POSTing data and custom HTTP headers Closes #3682069

[datafeed parser] fix retrieve_url to always return urllib2.urlopen compatible output as one should expect to be able to call geturl/getcode/info on the returned object whatever the url he gave. Related to #3682069

[datafeed parser] factor out retrieve_url method from DataFeedXMLParser.parse Related to #3682069

[datafeed] Allow to override use_cwuri_as_url in configuration of a datafeed source This is necessary because in some case one will want to write parser which create entities not one-to-one related to entities from the distant source (for instance see the `vcrs` cube which create entities holding statistics for code review, while not importing review entities themselves). Ideally we should provide a way to the parser to indicate the source that cwuri should not be used, but that's a start... Closes #3670210

merge from 3.19 branch

[entity/rset] replace scary warnings about ambiguous rdefs by XXX in the code users can't do anything about that and shouldn't see those warnings.

[web/js] Deprecate baseuri() and replace with BASE_URL (closes #3955372) Added by commit 8bf2337a6f3b and publicly available since 3.11.2 released more than 3 years ago.

[request] Ensure base_url() always has a trailing '/' (closes #3955093) Just as an example, cubicweb.ajax.js is clearly written under the assumption that BASE_URL/baseuri() - which simply contains req.base_url() - has a trailing /.

[schema] spelling fixes in doc strings

[views] a 'div' with 'display: inline' screams 'span'

[views] Rewrite cw.utils.formContents() with jQuery (closes #3747244) Use jQuery's serializeArray() method instead. Just keep a bit of compatibility with FCKEditor...

[web/data] Remove broken backward compatibility function popupLoginBox() was deprecated in favor of cw.htmlhelpers.popupLoginBox() However, the use of cw.utils.deprecatedFunction() is broken. In firebug: >>> popupLoginBox() TypeError: newfunc is undefined cubicweb.js:113 (ligne 13) This function was modified back in 2010, so it's been broken for 4 years. Let's just get rid of it. Closes #4042978

[repoapi] make ClientConnection call RequestSessionBase's init Not calling up means we're missing some attributes, like encoding.

[webtests/views_staticcontrollers] use the new connection api

[webtests/views_selectors] use the new connection api

[webtests/views_json] use the new connection api

[webtests/views_errorform] use the new connection api

[webtests/views_actions] use the new connection api

[webtests/views_xmlrss] use the new connection api

[webtest/pyviews] use the new connection api

[tests/sqlutils] use the new connection api

[tests/postgres] use the new connection api

[tests/repository] use the new connection api (part 2/2) Some notes: * module imports cleanup * the unimplemented `test_transaction_interleaved` is wiped

[repository,tests] use the new connection api (part 1/2) `add_info` should not play with the cnxset nor the mode. Static analysis showed that: * add_info is an internal api called with a non-none cnx.cnxset and also in write mode * only the altered test violated this assumption, which is now somewhat documented

[book] fix various parts of the rql section * syntax error * erroneous query * bad sphinx :func: usage

[repository] session -> cnx renaming (already done in the source object)

[tests/session] remove unused import

[tests/datafeed] use the new connection api

[tests/checkintegrity] use the new connection api * serverctl: the db-check command also uses an internal connection object, which makes more sense in the context of `checkintegrity`. * checkintegrity: rename srvcnx to just cnx (a bit because in the near future, there will be one cnx to rule them all)

[tests/schemaserial] kill star import

[tests/schemaserial] break long lines

[tests/ldap] don't leak sessions

[tests/ldap] use the new connection api

[test/migration] fix crash on execution This assertion is there for unclear reasons. ===================== unittest_migration.py ======================= -> creating tables [====================] -> inserting default user and default groups. -> storing the schema in the database [====================] -> database for instance data initialized. F ====================================================================== FAIL: test_db_creation (unittest_migration.BaseCreationTC) make sure database can be created ---------------------------------------------------------------------- Traceback (most recent call last) File "/home/auc/confs/forges/logilab/common/testlib.py", line 661, in _proceed testfunc(*args, **kwargs) File "/home/auc/confs/forges/cubicweb/test/unittest_migration.py", line 106, in test_db_creation handler.build_db_cache() File "/home/auc/confs/forges/cubicweb/devtools/__init__.py", line 469, in build_db_cache self.init_test_database() File "/home/auc/confs/forges/cubicweb/devtools/__init__.py", line 729, in init_test_database init_repository(self.config, interactive=False) File "/home/auc/confs/forges/cubicweb/server/__init__.py", line 217, in init_repository repo = Repository(config, vreg=vreg) File "/home/auc/confs/forges/cubicweb/server/repository.py", line 197, in __init__ self.init_cnxset_pool() File "/home/auc/confs/forges/cubicweb/server/repository.py", line 220, in init_cnxset_pool config.bootstrap_cubes() File "/home/auc/confs/forges/cubicweb/devtools/__init__.py", line 176, in bootstrap_cubes super(TestServerConfiguration, self).bootstrap_cubes() File "/home/auc/confs/forges/cubicweb/server/serverconfig.py", line 279, in bootstrap_cubes self.init_cubes(self.expand_cubes(splitstrip(line))) File "/home/auc/confs/forges/cubicweb/cwconfig.py", line 1022, in init_cubes super(CubicWebConfiguration, self).init_cubes(cubes) File "/home/auc/confs/forges/cubicweb/cwconfig.py", line 798, in init_cubes assert self._cubes is None, repr(self._cubes) AssertionError: ('card', 'file', 'localperms', 'tag')

[web] Remove unreachable and broken code

[testlib] call init_config once the config has been properly bootstraped This should be done right after repository instantiation and as such involves giving CWTC.init_config as a callback to the test database handler (which will itself gives it to the repository initialization function). This unfortunatly requires to pass the init_config method to the server.init_repository function because it has to be called after the config has been initialized (which is done in Repository.__init__) but before the migration handler is instantiated (which will call 'server_maintainance' hook, hence may require the proper config). Another way to fix this would be to change the initialization sequence, but this is another story. Closes #3749378

[devtools] even better: use a plain Connection (server-side) object for pre_setup The previous commit switched from passing a Session to a ClientConnection. This fixes unittest_ldapsource's pre_setup_database, which relied on a server-side object to get at the repo.

[devtools] pre_setup_database should take a connection object

[tests/migration] use the new connection api

[tests/undo] use the new connection api

[js/ajax] deprecate user-callback support Related to #3567793.

EditRelationMixIn: stop using user_rql_callback Add generic add_relation/delete_relation ajax functions and use those. Related to #3567793

[tests/storage] use the new connection api

[tests/security] use the new connection api

[htmlwidgets] Remove unreachable code

Remove redundant __nonzero__ methods From https://docs.python.org/2/reference/datamodel.html#specialnames : When this method is not defined, __len__() is called, if it is defined, and the object is considered true if its result is nonzero. Also helpful for an eventual migration to py3k, as __nonzero__ has been replaced with __bool__, which also defaults to looking at __len__ if undefined.

[etwist] Remove unused host_prefixed_baseurl()

[basetemplate] Remove misleading comment The <base> tag was removed in commit 8bf2337a6f3b back in 2011.

[tests/security] break lines > 100 chars

[doc/3.19] Mention [gs]et_shared_data deprecation

[hooks] internal_session -> internal_cnx (including a typo)

deprecate get/set_shared_data API Session or transaction data must be used instead. We must forward .transaction_data to web/request from which it was missing (this is indeed a _cw object API). Closes #3799036.

[migration] Stop setting session.data['rebuild-infered'] It's no longer read anywhere since fe267b7336f3 "[migration] always rebuild infered relation"

Import PIL modules from the PIL namespace to be Pillow-compatible Closes #3924726

merge two stable heads

merge 3.18.5 into 3.19 branch

merge 3.17.15 into 3.18 branch

[vreg] fix bad super call to lgc.registry.Registry() Registry() expects a debugmode parameter, ovbiously not a cubicweb config object.

Added tag cubicweb-version-3.18.5, cubicweb-debian-version-3.18.5-1, cubicweb-centos-version-3.18.5-1 for changeset 5071b69b6b0b

[pkg] prepare 3.18.5

[sobjects] update notifications system to 3.19 api

[connection] set language on new Connection for the internal manager For regular users this is handled by _set_user, but for internal connections (e.g. used for notifications) we need something else.

Added tag cubicweb-version-3.19.1, cubicweb-debian-version-3.19.1-1, cubicweb-centos-version-3.19.1-1 for changeset 1fe4bc4a8ac8

[pkg] Add missing data files to MANIFEST Fonts and svg were missing from web/data.

[pkg] prepare 3.19.1

[views] new cubicweb logo (closes #3803681) - add the new CubicWeb logo - use it as a css (background-image) - change the color of the header bar (gradient made with css)

[serverctl] rename 'add-source' to 'source-add' for consistency's sake (closes #3732169)

[test/hook] use the new connection API

[testlib] complete the RepoAccess object (closes #3843614) It was missing the open_session call.

[debian] version cubicweb-bootstrap Breaks Apparently 0.6.x is OK with cw 3.19.

[wsgi] Delete unused instance_uri and get_full_path methods on CubicWebWsgiRequest

[test] drop unused import

[sobjects] silent 3.19 warnings in sobjects/supervising.py

[test] update entities/test/unittest_wfobjs to cw 3.19 api

[test] Add missing import to fix old yams warning

[dataimport, migration] silence find_entities / find_one_entity warning Deprecate associated methods to unify the API

[repo] fix extid2entity to ensure connection's has a cnxset else we'll call source.doexec, which attempt to access to cnxset.cu, and crash in some case (eg unittest_cwxmlparser in sobjects/test). Also, manually switch to 'write' mode to ensure cnxset is not released until commit/rollback when some entity is inserted.

[test] update sobjects/test/unittest_cwxmlparser to cw 3.19 api

[repo] make it clear in internal_cnx that security is disabled as explained in its docstring. It was previously relying on internal connection's fake user (InternalManager) for read security and on disabling security hooks for write security. Using the security_enabled context manager is more readable and more reliable: while the older implementation works thanks to the InternalManager associated to the session, custom hooks should rely on session.[read|write]_security being correctly set. This change also allows selecting Password attributes in internal connections without explicitly disabling read security.

[datafeed] update datafeed internals to use connection instead of session

[test] update sobjects/test/unittest_notification to cw 3.19 api

[test] simplify setup of sobjects/test/unittest_supervising.py

[test] update sobjects/test/unittest_supervising to cw 3.19 api

[test] update sobjects/test/unittest_email to cw 3.19 api

[wsgi] the spec says wsgi.url_scheme *must* be provided No need to be defensive, anything else is a bug in the gateway.

[shell] provide autocompletion for rql queries in cubicweb-ctl shell (closes #3560541) Use the ``rql.suggestions`` component to complete user's input - readline calls the ``complete`` method with user's input, - the ``complete`` method asks for each known completers if it can suggest completions for user's input.

[book] avoid '~' in all-in-one.conf, it is not expanded Closes #3753213

[book] uicfg moved from web to web.views a while ago

[wsgi] look at wsgi.url_scheme to decide if we're on https The server/gateway is supposed to fill it in for us.

[web] Fix HEAD request handling (closes #3677949) Don't special-case HEAD in CubicWebRequestBase.is_client_cache_valid so we don't return garbage. HEAD is supposed to return the actual headers that a GET would return, which we can't do if we bypass all the normal cubicweb logic. In particular, we no longer always return 200, and return the right Content-Length/Content-Type/...

[wsgi] avoid reading the entire request body in memory Import POST form handling code from https://raw.github.com/defnull/multipart/master/multipart.py to avoid reading arbitrary amounts of data from the network in memory. NOTES: - In the twisted case we limit the max request content-length to 100MB (by default), which seems kind of arbitrary, but avoids this issue - werkzeug.formparser has suitable code as well, but I don't know if we want to add it as a dependency

Added tag cubicweb-version-3.17.15, cubicweb-debian-version-3.17.15-1, cubicweb-centos-version-3.17.15-1 for changeset ee413076752b

[pkg] also bump __pkginfo__

[pkg] prepare 3.17.15

[cache] fix a cache issue with an entity created without relation cache prefill Prefilling the relation cache has its benefits sometimes, but for massive imports it may be a memory hog. The caches should be kept coherent even if there was no prefill. Closes #3828155.

[web/test] add a test for some http response codes Related to #3648809

[views] remove reference to the MAPPING_CHECKERS dict (closes #3810219) Was removed by by the changeset 65d93a4fd11c ([multi-sources-removal] Drop pyrorql and zmqrql sources)

[web/ajax] Always return a json dict with a 'reason' key in case of ajax errors Sending sometimes a bare unicode string and sometimes a json-encoded dict seems like a bad idea.

[web/ajax] don't override any status code with 500 in ajax_error_handler If the request produced an error such as Unauthorized or NotFound, core_handle sets an appropriate status_out. Which is then overridden by the ajax error handler.

[web] Add asserts to the raw header conversion functions This API is terrible, but at least this might help us catch some errors in our conversion definitions.

[view] Delete dead(?) code from ReloadableMixIn Nothing seems to ever call user_callback on the Component, only on the request. Related to #3567793

Add a db-statement-timeout option for postgresql sources Use that new option to set the statement_timeout option when creating connections, except for maintenance operations (shell and upgrade). Closes #2547026

[test] use assertNotIsInstance instead of assert_(not isinstance(...))

[services] register_user now returns the new CWUser Closes #3804444

[server/session] remove json dumps around call_service This backouts changeset b151beea9cb6. With call_service being in-process and synchronous, there's no RPC use case.

[req.find] Use vreg.schema.eschema for eschema lookup If an etype is non-existant, using vreg.schema[etype] raises a confusing error complaining about a non-existing relation. This is because of the implementation of vreg.schema.__getitem__ that look first in entities and then in relations. Using directly vreg.schema.eschema restrict the lookup to etypes only, hence raises a meaningful error when the etype does not exist.

[doc] Typo

[formfields] Handle 'placeholder' attribute for text and textarea See #636930. Closes #3406713.

[forms] Add autocomplete attribute for formrenderers If you set form.autocomplete = False, the form renderer in the open form tag will add the html option 'autocomplete=off'. It allows to disable the functionality of some browsers to fill in some forms automatically (for instance a profile with login/password) Closes #3747294

[widget] Addition of EmailInput Simple input widget with type = 'email' Closes #3741921

close branch

Added tag cubicweb-version-3.19.0, cubicweb-debian-version-3.19.0-1, cubicweb-centos-version-3.19.0-1 for changeset 1141927b8494

[debian] set release date

[entity] add a new `cw_linkable_rql` method * the new method returns the entities that are or may be related to the current one by the relation passed as an argument. Closes #3738011.

[schema] fix buggy composite_rdef_roles and associated test it should only look at rdefs involving the considered entity type. Closes #3712982

[doc] update 3.19 release notes

[login form] Fix BaseLogForm.form_action() The method implementation was calling super() with the wrong class

[migration] move 'entities' table changes from 3.19.0 to bootstrap script The 'mtime' and 'source' columns need to go away before we attempt to do anything else with the repo, otherwise any addition of an entity is going to explode.

[server/session] add missing rollback when freeing a db connection When we let go of a cnxset and give it back to the pool, we need to make sure it's not in the middle of a transaction. We got bitten by a migration where the connection used by admin's repo.connect was not rolled back before attempting the migration, causing a deadlock because it had a lock on a table. This could potentially also be an issue for e.g. hooks or looping tasks that didn't explicitly rollback before calling free_cnxset or moving out of a ensure_cnx_set block.

[debian] add Breaks on cubicweb-bootstrap Need to be versioned once we know which version fixes https://www.cubicweb.org/ticket/3728939

Drop duplicated content in ITreeAdapter docstring

an -> a (misc)

an utf-8 -> a utf-8

an URI -> a URI

an unicode string -> a unicode string

an URL -> a URL

Remove a misleading comment about self.render() behavior. The former comment suggested that View.render called with 'row' or 'col' would set cw_row or cw_col, which is not (anymore ?) accurate. Related to #3696871.

[migractions] add sanity check in rename_relation_type If the relation is still present in the fs schema, we probably shouldn't be renaming it (because it's still in use somewhere, possibly in a different cube). Closes #3608172

[dataimport] Add safety belt on "gen_rtype" in MetaGenerator, closes #3712892 Some cubes may well add to the META_RTYPES set, hence crashing the gen_<somecubemeta> call.

[dataimport] Correctly serialize datetime objects in _create_copyfrom_buffer, closes #3712885

[dataimport] Prevent ucsvreader from skipping the first line when ignore_errors is True (closes #3705791)

[dataimport] Avoid double unicode decoding in ucsvreader (closes #3705752)

[doc] Document the user_data dictionnary content

[debian] break old cubicweb-vcsfile We removed call_service's async argument, which it used.

[services] catch missing group early in register_user Better an assertion than a rql syntax error for such a programming error.

[services] rename register_user kwargs to cwuserkwargs They're attributes/relations for the created CWUser, this makes it clearer.

[services] don't hardcode 'users' group for register_user Let the caller choose which group the user should be in. Related to #3020639 Caveat: this is a possible security hole if untrusted values are allowed.

[sobjects/test] check that register_user properly sets user name and email address Related to #3020639

[server] Refactor Repository.register_user into a CubicWeb service (closes #3020639) [jcr: move commit to the caller; add item in release notes; fix NameErrors]

[migration] set a cnxset before process_script() Not pretty, but I don't have a better idea right now and this at least ensures migration scripts get run in the expected environment and can access sql stuff as before.

[server] make internal_cnx get and keep a connection to the db - internal_session worked that way - it's less surprising for users if their internal connections "just work", rather than having to deal with the cnxset when using low-level APIs Includes a backout of 3f62606c01a2 "[repo] Fix register_user" which is no longer needed. Depend on logilab-database 1.12.1 to avoid errors due to pointless check in python's sqlite3 binding.

[server/session] make commit/rollback obey ensure_cnx_set Inside the ensure_cnx_set context manager, commit and rollback aren't allowed to dispose of the cnxset.

[repo] fix deprecation messages for stats and gc_stats

[cwconfig] use "foo not in bar" instead of "not foo in bar"

[debian] Add Breaks on a few cubes that need changes for 3.19 - mercurial_server is affected by composite deletion changes - forgotpwd relied on session data to be forwarded to notification view - registration hooks into the authentication stack

[hooks] run cascading delete hook later Applications may have their own before_delete_entity hooks that rely on the composite relations still existing.

[devtools] set PGPORT/PGHOST when calling pg_ctl start Helps pg_ctl find the socket on squeeze, otherwise it looks in /var/run/postgresql.

[server/test] update to deal with lgdb sqlserver fix See https://www.logilab.org/ticket/235748

[test] Fix regressions from 96549de9dd70 "[test] use assertIn where appropriate"

[repo] Fix register_user Make sure we have a connection to the db around user creation.

[test] use assertNotIn where appropriate Automatically generated with: find -name unittest_*.py -o -name test_*.py | xargs sed -i -e 's/assertFalse(\([^,]*\) in \([^,]*\))/assertNotIn(\1, \2)/' plus revert of a couple false matches.

[test] use assertIn where appropriate Automatically generated with: find -name unittest_*.py -o -name test_*.py | xargs sed -i -e 's/assertTrue(\([^,]*\) in \([^,]*\))/assertIn(\1, \2)/'

[doc] Add removal of old multi-sources to 3.19 release notes

[pkg] Bump to 3.19.0

[web/debug] remove code broken by session refactoring session.cnx and cnx.check() no longer exist.

[web/debug] Fix check for open sessions itervalues returns an iterator, which never evaluates to False. Regression from 0bb18407c053 "[toward py3k] rewrite dict.keys() and dict.values() (part of #2711624)".

[cwconfig] Remove useless 'ui-cube' configuration option (closes #3728904) The mechanism introduced could never actually be used as it brings dependency loops to the cube graph. Just get rid of it. This reverts commit 4032499c701e.

[test] Use RepoAccess.shell() in unittest_predicates

[devtools] add 'shell' method to RepoAccess, deprecate CubicWebTC's Lets us control the connection's lifetime.

[repo] make extid2eid work with either a session or a connection The only place it should matter is for restoring pending_operations in case of failure. Which is not exercised by the test suite. Here be dragons.

[repository] extid2eid's 'commit' argument no more necessary This removes the need for creating an internal session.

[repository] complete argument is no more needed since we have no more 'true' multisource. It is always false.

[repository] 'session' argument is always given to type[_and_source]_from_eid, make it mandatory and simplify code accordingly

[repository] 'session' argument is always given to extid2eid, make it mandatory and simplify code accordingly

[web/data] Remove broken jquery-json plugin (closes #3590335) Conversion of Date() objects is somehow broken (don't know why beyond that it's due to jquery-json's monkeypatched Date.toJSON() method). Use the now-standard JSON.stringify() instead. http://caniuse.com/#feat=json says it's supported by all browsers, and even IE going back to version 8.

Add missing relation types items in schema global variables Some of these global variables are used, in particular, to control schema views (web/views/schema.py). While they are currently not properly documented (see #3360868), their definition are currently incomplete (e.g. one could reasonably expect to find ``cw_schema`` in ``SYSTEM_RTYPES``). Closes #3486114.

[web] return 403 for Unauthorized, not 401 401 is reserved to HTTP authentication. Just because it's also called "unauthorized" doesn't mean it's the same as cubicweb's Unauthorized exception. Closes #3648809.

[test] update unittest_repoapi to 3.19 api (sic) [jcr: while at it, fix typos and add missing __main__ section]

[test] update unittest_utils to 3.19 api

[test] update unittest_rset to 3.19 api [jcr: use self.admin_access instead of creating a new one each time]

[test] update unittest_rqlrewrite to 3.19 api

[test] update unittest_req to 3.19 api [jcr: use self.admin_access instead of creating a new one each time]

[test] update unittest_predicates to 3.19 api [jcr: use self.admin_access instead of creating a new one each time]

[web/test] port unittest_reledit to RepoAccess API

[web/test] port unittest_magicsearch to RepoAccess API request created in setUp makes context manager usage painful, so create a context manager to do the setup for each test class.

[web/test] use real_error_handling context manager instead of open-coding it

[web/test] port unittest_idownloadable to RepoAccess API

[web/test] remove useless calls to self.request() from CORS tests

[web/test] port unittest_formfields to RepoAccess API

[web/test] port unittest_form to RepoAccess API

[web/test] port unittest_facet to RepoAccess API

[web/test] move unittest_controller to RepoAccess test API

[web/test] port unittest_breadcrumbs to 3.19 test api

[web/test] avoid deprecation warnings from find_entities()

[web/test] port unittest_application to new 3.19 API

[test] silence warnings for unittest_schema by upgrading to new 3.19 API

[test] silence warnings for unittest_cwctl by upgrading to new 3.19 API

[test] replace some assert{True,False} with more specific method

[test] silence warnings for unittest_entity by upgrading to new 3.19 API

Fix constraint sync during migration - restore constraints lost during merge in test schema. - use constraint_by_eid in BeforeDeleteCWConstraintHook as done in 3.17.14 for BeforeDeleteConstrainedByHook. Fixes handling of multiple constraints of the same type. - make sync_schema_props_perms() delete the CWConstraint entity instead of the constrained_by relation. In 3.19, the latter doesn't automatically result in the former just because the relation is composite. Simplify the constraint migration to delete all removed constraints and recreate new ones even if they share the same type; that optimization made the code more complicated for (AFAICT) no significant reason.

merge 3.18.4 into default server/test/unittest_migractions fails due to interaction between new constraint test (from 3.18) and composite deletion change (in 3.19).

Added tag cubicweb-version-3.18.4, cubicweb-debian-version-3.18.4-1, cubicweb-centos-version-3.18.4-1 for changeset 0176da9bc752

[test/data] Add missing 'add' permission on attribute Silences a warning introduced in 3.18.

[hooks/notification] Fix indentation Follow-up for changeset 1b8552265f3b "[hooks/notification] use a cnx not a session"

[devtools] deprecate CWTC.user()

minor cleanups

some copyright updates

[devtools] properly close open access on tearDown avoiding further warning about living sessions while the repo is turned off. The doc string says: The RepoAccess need to be closed to destroy the associated Session. TestCase usually take care of this aspect for the user. but that wasn't true until now.

[devtools] make comment (a bit) more readable

[devtools] avoid an internal deprecation warning websession and session are now one and the same.

[server/session] undeprecate session.anonymous_session It's not connection-specific.

[web/sessions] use session.sessionid instead of deprecated session.id

[repo] don't set cnxset when getting the session to close it and avoid warning about that. session._cnx.rollback won't do anything if it's not set and a newer connection is now created to run the session_close hook, so there is not need to set cnxset. Notice we have to take care about not trying to rollback the underlying cnx when the session is not 'session_handled' (i.e. bw compat mode). This case only occurs due to hackish things to keep bw compat in testlib.

[repo] handle connection explicitly when calling session open/close hooks

[hooks/syncschema] work with connections not sessions

[server/sources] {before,after}_entity_insertion wants a cnx not a session

[hooks/syncsources] use a cnx not a session

[hooks/notification] use a cnx not a session

[hooks/metadata] use a cnx not a session

[hooks/bookmark] use a cnx not a session

[hooks/workflow] use a cnx not a session

[hooks/email] use a cnx not a session

[hooks/integrity] use a cnx not a session

[hooks/security] let's use a connection, not a session

[server/sources/native] deal with connections, not sessions

[server] eschema_eid needs a connection, not a session

[repository] operations get a connection instead of a session Left extid2eid unchanged because that function scares the shit out of me.

[server/hook] operations get a connection instead of a session

[repository] Use an internal connection in register_user

avoid to internally raise 3.19 warnings

[sources] ensure we have a cnxset in undoable_transactions

[server] move security/integrity hook management away from InternalSession.__init__ and to Repository.internal_{session,cnx} instead. Lets internal_cnx avoid deprecation warnings.

[migration] make 'session' object be a server-side Connection, not ClientConnection I think this better corresponds to the previous behaviour. However, that Connection does not have a cnxset, which should probably be changed (for both interactive_shell and cmd_process_script).

[sources] Skip problematic sources when starting shell instead of crashing. Closes #3670208 At least this gives an opportunity to fix the problem in using c-c shell, which is currently not possible because it crashes during initialization. Notice that since sources are copy based, it should not be a problem at all to have some disabled, beside for services such as authentication sources as ldapfeed.

[web] fix language negotiation Regression caused by commit 6fd0ac6506cb "[web-request] handle default language earlier". We would handle language negotiation at request init time, but overwrite the selected language with the site default when calling set_cnx.

[merge] bring the 3.17.14 fixes to 3.18

[test/utils] repair concat-resources test That was forgotten in 240a620b9cd3. Related to #3670503.

[pkg] bump cubicweb.spec

[dataimport] Correctly call rschema(rtype) in SqlGenObjectStore, closes #3694139

[pkg] prepare 3.18.4

[web] Disable 'concat-resources' by default (closes #3670503) It makes debugging harder and it can actually *break* working JS or CSS files. This is the first step towards complete removal.

[testlib] Drop override of assertItemsEqual method The override is not compatible with underlying (standard) method because it builds sets and rely on eid. Closes #3641111.

[rewrite] Fix crash when the main variable doesn't appear in the snippet's vargraph Also, the added test discover another bug once the first has been fixed (erroneous argument given to .rewrite recursive call). Closes #3661918

[ext/rest] Catch SystemExit raised by docutils docutils publisher may raise SystemExit which is not caught by Exception. Closes #3648763.

[entity] User-defined relation to skip for copy has precedence Otherwise permission problems can occur on rtypes not yet skipped Closes #3653459

Make EditController edit_entity method always return an eid In cases the entity was to be created or copied, eid was None hence the method returned None despite the dosctring says the method should always return an eid. Now if the eid variable is None, it is assigned to the newly created entity eid. Closes #3593029.

[migration] Improve update of in-memory schema during 3.18 CWAttribute.defaultval change We weren't updating the schema properly so a bunch of structures still had the old rdef with 'String' as object. That caused failures to add new attributes later in the migration as their default value would be a Binary object but the schema would expect a String. Closes #3683640

[migration] always rebuild infered relation This was skipped for some bad reason (see 12ad88615a12 which introduced the change). Fix for #231956 in Yams is necessary to allow this cset: during a migration, we want to always reinfer relations while allowing explicit redefinition of an infered relation later. Else, we may run the migration with missing parts of the schema (the one that should have been infered). Closes #3685463.

Backout "[web/navigation] use add_onload instead of inline javascript href" This reverts changeset 170e1437948d, for at least two reasons: the PageNavigationSelect component was not updated for the new type of navigation links, and external cubes defining View.page_navigation_url exist in the wild and also got broken navigation links. This reopens #3501626 and closes #3677945.

[test] those are rql tests, not cubicweb (and they break with rql 0.31.5). Remove them.

[source/native] allow many eid creation per .create_eid call (closes #3526594) [jcr: move migration code to bootstrapmigration_repository.py to make sure we stop using the old sequence ASAP]

[pkg] bump the dependency on logilab.database This will allow use of an extended API. Related to #3526594.

[source/native] refactor eid generation code This set of related methods clutters the native source API. It will be better served with a dedicated eid genrator object. Related to #3526594. [jcr: allocate the eid generator in __init__]

Move entity cache from web.request.ConnectionCubicWebRequestBase to repoapi.ClientConnection ResultSet._build_entity relies on an entity cache to avoid going round in circles (stack overflow due to infinite recursion) e.g. in a postcreate script. Clear the cache at commit/rollback time to avoid unexpected side effects; and explicitly clear the cache in a few tests where we mix ORM and RQL using the same connection.

[server] Fix AttributeError in extid2eid Session._tx was renamed in cb87e831c183, use _cnx instead.

[book] Update documentation for new repoapi Quite a few things change in 3.19: - repoapi instead of dbapi - ClientConnection / Connection / Session rework - web authentication process - test APIs Closes #3638793

[utils] the json module is always available It's included since python 2.6, and we don't support earlier versions.

[fti] properly close the ProgressBar Ensures cubicweb-ctl db-rebuild-fti ends its output with a newline.

[serverctl] use repoapi for db-check, add-source, rebuild-fti commands One more step towards getting rid of dbapi.

[devtools] make get_repo_and_cnx return a repoapi ClientConnection And adjust callers accordingly (they need to use it as a context manager to open/close it properly).

[repoapi] Avoid deprecation warnings from session.id Apparently it's called session.sessionid now.

[server] some s/session/cnx/ For merely accessing the database we need a connection, not a session.

Move setting session.mtime from dbapi to web session manager clean_session was broken since the switch away from dbapi on the web side, since session.mtime wasn't being set anywhere.

[web] whitespace cleanup in http_headers.py

[web] implement cross origin resource sharing (CORS) (closes #2491768) Partial implementation that is enough to get started but leaves out some of the advanced features like caching and non-simple methods and headers.

[devtools] add a 'method' argument to RepoAccess.web_request so that one can easily forge a request with any HTTP method. Also a bunch of clone cleanups.

[web] Fix handling of some http headers The generator takes as argument a list of raw string values, and is supposed to return the parsed version. Calling str on that list makes no sense.

[web/request] deprecate user_callback Storing closures in session data considered harmful. Closes #3567793

[doc] undocument user_callback Related to #3567793

Added tag cubicweb-centos-version-3.17.14-1, cubicweb-version-3.17.14, cubicweb-debian-version-3.17.14-1 for changeset fa00fc251d57

[migractions] Better handle removal of RQLConstraint in sync_schema In rdef synchronisation, filter unmodified constraints before processing the sets of old and new constraints. Add a new `constraint_by_eid` method on RelationDefinitionSchema, which eliminates the ambiguity of `constraint_by_type` usage when several constraints of the same type are involved in the same migration. Closes #3633644.

[test] when the config change, we should cleanup sys.path and sys.modules. Closes #3634164 Without this, modules imported within instance home will be messed up.

[wsgi] make sure request.content is available for consumption The wsgi.input stream is not seekable, so make a copy either to memory or disk so it's still available to the user even after we've parsed it (for POST form processing). Closes #3554996

[facet] create a RangeRQLPathFacet (closes #2852512) This facet is a mix between a RQLPathFacet and a RangeFacet, allowing to use the FacetRangeWidget on the RQLPathFacet target attribute

[request] Make sure set_message() actually displays its given message (closes #3003425) The default automatic form sets a standard submit message as session data. However in some cases, such as the cancellation of a form edition, another message is intended to be printed on-screen. This patch makes sure that set_message() really scrubs all previous messages (and ids) from the request object and from the session data.

[pkg] prepare 3.17.14

[hooks/syncschema] do not crash on double removal of a constraint Some complicated migrations may entail a double constraint removal. It is not yet known whether this is a genuine cw bug or the result of a complicated migration story, but we should nevertheless not crash. Rather, emit a CRITICAL warning and proceed. Closes #3595309.

[doc/book] Fix typo in import

[server/session] fix a couple typos in doc strings

[server/migractions] finish migration to repoapi objects Changeset 241b1232ed7f (Use repoapi instead of dbapi for cwctl shell, upgrade and db-init) only did half of the job. It left the migration handler with both a session (cubicweb.server.session.Session) and a cnx (cubicweb.repoapi.ClientConnection) attribute with different ideas of what Connection they were talking to. With this change, we: - make the caller responsible of disabling security on the Connection if it wants to - turn mih.session into an actual attribute, set on __init__ - same for cnx (the client connection) - drop the "lazy connection" logic, and establish the connection up-front - always go through the connection instead of the session when we need to talk to the db

[server] use the ClientConnection directly now that it has more methods available Gets rid of ugly self.cnx._cnx.

[repoapi] add security/hook control and system_sql Just redirect these methods to the repo-side Connection.

[predicates] fix thinko in doc strings Replace "e.g." with more appropriate "i.e.". These are not examples.

[web/basecomponents] remove EtypeRestrictionComponent Closes #3119951

[ldapparser, book] document additional error causes

[js] remove leftover dead code in loadxhtml form.callback support was removed in changeset 68cde7431c2c, but some signs of it remained.

[web] Avoid using an empty dict as default parameter of 'CubicWebRequestBase'. Also update the related docstring of the __init__ method.

[schema] fix composite deletion handling Do not delete component entities if the composite is not deleted in the same transaction. Deletion semantics are thus: if the composite is deleted, the components are deleted. Closes #3463112.

[schema] add composite handling helpers on EntitySchema (related to #3463112) These will help write predicates, hooks, etc.

[server] Handle unique constraint violations under recent sqlite The error message changed from "columns foo, bar, baz are not unique" to "UNIQUE constraint failed: table.foo, table.bar, table.baz". Closes #3564510

[i18n] update de: 1065 translated messages, 1 fuzzy translation, 259 untranslated messages. en: 656 translated messages, 669 untranslated messages. es: 1325 translated messages. fr: 1314 translated messages, 11 untranslated messages.

[i18n] es.po updated

merge 3.18.x in 3.19 branch

3.18 is stable

[schema] explicitly declare 'add' permissions for a couple attributes

Added tag cubicweb-version-3.17.13, cubicweb-debian-version-3.17.13-1, cubicweb-centos-version-3.17.13-1 for changeset 09b4ebb9b0f1

Added tag cubicweb-version-3.18.3, cubicweb-debian-version-3.18.3-1, cubicweb-centos-version-3.18.3-1 for changeset afd21fea201a

Update translations

[pkg] prepare 3.18.3

merge 3.17.13

[ldapfeed] fix encode error during initial user import Closes #3539196.

[web/data] Ignore disabled widgets in cw.utils.formContents() (closes #3544492) 17.12.1 Disabled controls : http://www.w3.org/TR/REC-html40/interact/forms.html#h-17.12.1 When set, the disabled attribute has the following effects on an element: * Disabled controls do not receive focus. * Disabled controls are skipped in tabbing navigation. * Disabled controls cannot be successful. The third one is the important one. 17.13.2 Successful controls : http://www.w3.org/TR/REC-html40/interact/forms.html#successful-controls A successful control is "valid" for submission. Every successful control has its control name paired with its current value as part of the submitted form data set. Bottom line, disable widgets should not be part of the names and values lists returned by formContents().

[etwist] Fix an empty request content after a Twisted processing (closes #3546795). The content of a POST request could be empty when the Twisted server is used.

[web/data] remove images that aren't used anywhere

[web/data] Add missing images from jquery-ui 1.10.3 Closes #3175933

[web/navigation] use add_onload instead of inline javascript href This way our javascript code isn't thrown out by the html cleaner e.g. when using the rql rest directive and a table view. To make things simpler, we now always use ajax URLs for navigation, even when we would previously have used regular links. Closes #3501626

[web/data] fix treeview regression (closes #3526466) Changeset 68cde7431c2c "[js] remove 3.9 bw compat (where apparently unused)" removed the use of form.callback from loadxhtml, which treeview relied on. Update to add a callback on the loadxhtml return value instead.

backout 8f3e963501e2, which is not ready yet

[pkg] also bump it there

[pkg] prepare 3.17.13

[navigation] use add_onload instead of inline javascript href This way our javascript code isn't thrown out by the html cleaner e.g. when using the rql rest directive and a table view. Closes #3501626

[uilib] allow canvas tags in the html cleaner Used by the iprogress cube. Closes #3524254.

[ajax] use a custom tag to handle dynamically loaded js Using <pre class="script"> makes it trivial for a malicious user to inject arbitrary javascript into a html or rest text element (because it looks innocent to the html sanitizer). Using a custom tag we can be sure that it actually comes from our code and not from untrusted user data. IE ignores custom tags, though, so we put it in its own namespace. https://extranet.logilab.fr/1530578

[dataimport] fix comment

[hooks/security, devtools/fill] silence yams 0.38.0 warnings

Drop 3.13 incomplete backward compat in edit controller. It is very old and broken (there is another non-backward-copmpatible usage of `_cw_entity_fields`), better to remove it instead of fixing. Closes #3515223.

[devtools] force locale to C for postgresql test clusters Avoids initdb failure with e.g. iso8859-X locale and utf-8 encoding.

[rql2sql] remove special behaviour of symmetric relation vs DISTINCT 0542a85fe667 replacing SQL OR by hooks for symmetric relations allows that. This involve a single test value change for a border case: when querying a symmetric relation without specifying the subject nor the object, you may get some duplicated result. IMO this is fine to let the user explicitly use DISTINCT or not and to remove the dedicated handling we had which didn't let any choice. Related to #3259713

Typo in comments and error messages

Fix typo in a setup.py comment

[dataimport, migration] more fixes in the spirit of a6c32edabc8d: [entity, metadata] huuum, use resolvable url as cwuri... And fix existing ones. Closes #3390388

[doc] Fix typo in devrepo/fti

[doc/3.19] Clarify repoapi.get_repository usage Followup for 9a62c52d167e.

[server] use a connection instead of a session for user authentication

Use repoapi instead of dbapi for cwctl shell, upgrade and db-init Hopefully nobody uses dbapi-specific API in their migration scripts. I guess we'll find out.

[server/repo] use internal_cnx in init_sources_from_database

[server/session] avoid spurious warnings from Session.close Add internal non-noisy _rollback method.

[server] rename session to cnx in querier and plan They don't actually need a session, so let's make that clear.

[repo] Use a connection instead of a session for repo.stats()

[repo] Use a connection instead of a session for repo.gc_stats()

[repo] Use a connection instead of a session for repo.get_versions()

[webconfig] get_repository moved to repoapi

[schemaserial] Replace 'cursor' with 'cnx' We don't have cursors in rql, since execute is synchronous.

[book] basic documentation for LDAPfeed

[testlib] Use session.sessionid instead of deprecated session.id

Add data/pgdb to hgignore patterns PostgresTestDataBaseHandler uses it as datadir.

[test] make unittest_postgres pass Use PostgresApptestConfiguration which starts a postgresql cluster locally for the duration of the test.

[devtools] make PostgresTestDataBaseHandler start its own postgresql Don't depend on postgresql already running on the system, or on a static config. To use this, set the configcls attribute of your CubicWebTC-derived test class to PostgresApptestConfiguration. Caveats: - this won't work with several tests running concurrently from the same directory, since each would start its own cluster with a shared data directory and stomp on each other's toes - you need initdb/pg_ctl in $PATH, e.g /usr/lib/postgresql/$version/bin on Debian/Ubuntu systems. Closes #3489631

[web/staticcontrollers] create a unique temporary file for concat handling Predictable names means different processes can race against each other. Let's avoid that and use mkstemp to get a real unique temporary file, and atomically rename it when we're done. Closes #3524182

[server/repo] Use session.sessionid instead of session.id Silences deprecation warning.

[server/session] use _free_cnxset instead of free_cnxset to avoid warnings No need to make DeprecationWarning noise for something the user doesn't have any control over.

[server] Use internal_cnx instead of internal_session in init_repository

[server/session] session.id → session.sessionid The former causes deprecation warnings.

English typography Get rid of extra space before `!' and `?'

[repo] restore python 2.6 compatibility Broken in 12dfce15c8ea.

[server] use internal_cnx instead of internal_session in deserialize_schema

[server/schemaserial] rename cursor argument to cnx cursor is a sql concept it has no meaning for rql.

[repo] Use a connection instead of a session in Repository.properties()

[server/sources] replace references to dbapi with repoapi

[doc/3.19] Make repoapi description match code

[serverctl] remove broken schema-sync command As far as I can tell the migration handler hasn't had a cmd_synchronize_schema method since cubicweb 3.2.0.

[server/session] Fix docstring typo for InternalSession

[book] typo

[server] fix TypeErrors With multisource gone, cnxset.reconnect() doesn't take a source argument. Broken since d62e13eba033.

[notification] use an InternalManager object for notifications if we have an (email, lang) tuple The non-uniform handling of CWUser vs (email, lang) tuple has caused too many bugs; this change mostly unifies the two code paths. Closes #3381521

[req] if a user has a None property, fall back to the default This can happen for InternalManager; the normal CWUser implementation of property_value already has a similar fallback. Related to #3381521

[server] Allow setting an InternalManager's preferred language Related to #3381521

merge 3.18.2 into 3.19 branch

[migration/3.18] protect against attributes in db missing from fsschema Happens if a previous migration forgot a drop_attribute. Closes #3471609

[doc] Update copyright date to 2014 (closes #3463136)

[repoapi] Fix typos in doc strings and comments

[cwxml parser] minor pylint fixes

[devtools] make CubicWebTC.login as admin undo previous logins Fixes things when login is not used as a context manager.

[devtools] fix self.session._cnx vs self.cnx._cnx confusion after self.login Make sure the client connection and the session refer to the same server connection.

[devtools] _orig_cnx does not exist any more (missing change)

[c-c schema] option to show single etype (closes #3404831)

[multi-sources-removal] Drop entities.source column Since we remove federated multi-sources support, the same value ('system') is always stored in this column and so could be removed. Also, cleanup repository caches and a few api where the very same useless information could be removed. Closes #2919300, at last [jcr: restore sanity check of etype against schema in checkintegrity.has_eid]

[repo] pylint cleanup, mainly of imports, with a bit of style

[multi-sources-removal] Drop no more used federated ('true') multi-sources related code Related to #2919300 (almost done!)

[multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper also, SqliteConnectionWrapper becomes a subclass. This is allowed since now ConnectionsSet responsability has been reduced to handling the system source only. This changeset also: * drops useless cnxset_set / check_connection api * deprecates former .source / .connection / container api but it renames neither the session's cnxset attribute, nor the related repository method. Related to #2919300

[multi-sources-removal] Drop cnxset_freed from source interface Though a hack is still needed when using sqlite Related to #2919300

[cnxset] add backwards compat for cnxset.source_cnxs It's used at least in a vcsfile migration script.

[multi-sources-removal] Simplify ConnectionsSet internal structures and public methods since it now handles a connection to the system source only Related to #2919300 [jcr: adjust 3.17.11 migration, fix a number of bugs in new ConnectionsSet implementation, fix source.{open,close}_source_connections]

[ldap] simplify connection handling since we deleted ldapuser source, we don't need anymore the get_connection and ConnectionWrapper stuff (that was used to put the ldap connection into the cnxset). Also, we should simply let connection errors propagate so this is properly reported to import logs.

[ldap] merge cw.server.ldaputils back into ldapfeed source we don't need separated modules since ldapuser source has been removed

[config] cleanup/refactor server sources file values handling * kill former `sources` method, misnamed since we've only the system source configuration in the sources file (and some default admin account information) * introduce new system_source_config and default_admin_config properties to access to the two information in this file Then use one or the other when appropriate

[repoapi] drop get_option_value's foreid argument repoapi was introduced in 3.19, so no need to introduce legacy deprecated stuff at the same time.

[multi-sources-removal] Drop foreid argument of repo.get_option_value It doesn't need it anymore. Related to #2919300

[c-c restore] drop useless restore-all option, and related systemonly argument

[multi-sources-removal] Kill repo.sources there is only the system source in there now! Related to #2919300

[multi-sources-removal] Drop no more necessary scleanup (source cleanup) argument Related to #2919300

[multi-sources-removal] drop source location search on glob_*[entity|relation] methods source is now always the system source Related to #2919300 [jcr: fix wrong rebase in symmetric relation removal]

[server test] simplify data-schemaserial test instance's schema

[multi-sources-removal] Drop (dont_)cross relation related code That was only needed for now deleted 'true' multisources. Related to #2919300 [jcr: also remove it from server/test/data-schemaserial/schema.py]

[multi-sources-removal] Simplify repo.delete_info_multi arguments, uri is no more used Related to #2919300

[multi-sources-removal] Drop deleted_entities system table and entities.mtime column since they were only used by the entities_modified_since api of the repository which has been dropped. Along with them, the multi-sources-etypes configuration variable and some sql queries at modification/deletion time of entities. Bon vent ! Related to #2919300

[multi-sources-removal] drop repository entities_modified_since public api actually only used by pyrorql/zmqrql sources which have been dropped. This will allow to drop a bunch of system meta-data (eg deleted_entities) Related to #2919300

[multi-sources-removal] Drop the "true" multi-sources planner Only keep copy-based sources. This will soon allow for greater code simplification. Related to #2919300

[multi-sources-removal] Drop cw.server.sources.extlite no more used for a while, beside an utility class Related to #2919300

[multi-sources-removal] Move cw.server.sources.ConnectionWrapper where it belongs It's only used by the ldap utilities. Related to #2919300

[multi-sources-removal] Drop pyrorql and zmqrql sources After a few years experementing "true" multi-sources, we're now moving to "copy-based" source à la datafeed. As pyro and zmq sources have no more known customers and the related code is in the way of future refactoring of cubicweb's core, we decided to drop support for those sources without backward compatibility. If you're still using a zmqrql or pyrorql source and you want to upgrade, ask support to move it to datafeed using a pre-3.19 version first. Related to #2919300 (first step)

[web/formfields] Make CompoundField inherit needs_multipart from its subfields If there's a FileField within a CompoundField, the latter will not have the enctype multipart set, even though the FileField needs it. CompoundField should inspect its children fields to determine whether it needs multipart. Closes #3406930.

[wsgi/handler] fix api change from 3.18

[req] deprecate get_cache (closes #3181499)

Added tag cubicweb-version-3.18.2, cubicweb-debian-version-3.18.2-1, cubicweb-centos-version-3.18.2-1 for changeset 6880674c1a26

merge 3.17.12 into 3.18 branch

[editcontroller] fix cardinality computation (closes #3120495)

Added tag cubicweb-version-3.17.12, cubicweb-debian-version-3.17.12-1, cubicweb-centos-version-3.17.12-1 for changeset 838d58a30f7e

[debian] move yams max version to Depends And fix a syntax error and wrong package name in the process.

[pkg] prepare 3.17.12

[web] Fix typo in docstring

[test] remove custom implementation of FakeRequest.header_accept_language() (closes #3375021) ``Accept-Language`` should be overridable just as any other request header. Default value is "nothing" since the application should be able the empty header case.

[adapters] fix a name stomping error (entity) Also give proper name to subject (role, not target), and kill unused variable.

[entities] unconditionnally sanitize the html output of printable_value Closes jpl##49466

[doc] Update the debian stable name to wheezy

[i18n] Fix two typos

fix deprecation info

[web error] exception may not have a 'status' attribute, generating an AttributeError that hides the original error. Closes #3381670

[test api] extract a context manager to temporarily disable app.error_handler monkey patch

[views/workflow/tmppng] fix console crash on nth call to this one-shot view The underlying design issue is not addressed here, only a cosmetic and "we understand and document what's happening" note. Related to #3400448.

[dataimport] The subjtype should be the subject of a relation, not the object, closes #3365113

[testlib] Fix assertMessageEqual assertEqual(msg, msg) had a hard time failing.

[doc] Pass field to value callback in FieldsForm example This avoids a deprecation warning when following the documentation.

[pkg] prepare 3.18.2

[migractions] Don't silently ignore errors when installing sql procedures If we're going to ignore errors, at least be loud about it. sqlexec already wraps the sql execution in a try/except block, so we were never going to catch any exceptions in the caller. Check the return value instead. Closes #3459618.

[schema] execute the new stored procs/funcs at migration time (closes #3450368)

[schema/stored procs] fix syntax error (closes #3450362)

Added tag cubicweb-version-3.18.1, cubicweb-debian-version-3.18.1-1, cubicweb-centos-version-3.18.1-1 for changeset 60322cb8636c

[web/data] Update fullcalendar to version 1.6.4 Fixes compat with jquery 1.10 (closes #3445689)

[debian] declare breaks on cubicweb-invoice (closes #3444985) It tries to access a 'state' attribute on an entity without going through the IWorkflowable adapter.

[hooks/security] silence yams warning (closes #3440707) If the rdef is not final, yams will complain.

[migration/3.18] recover from bad CWSource.in_synchronization default value The addition of this attribute in 3.13.8 had default=False, which made no sense, and was fixed in 3.13.9. However no migration was done so apps that ever saw the 3.13.8 code could still have '' as defaultval for this attribute.

[pkg] prepare 3.18.1

[security] Add comment to check_entity_attributes shortcut Make it clear that it is only an optimization. Related to #3444095.

Disable security hooks for internal sessions Fixes semantic change introduced in 96dba2efd16d where internal sessions started to check attribute permissions. Closes #3444095

[migration/3.18] Explicitly delete constraint on defaultval Make sure the SizeConstraint on defaultval is removed, and avoid more unnecessary commits.

[migration/3.18] Idempotency fixes Try to make partially-upgraded instances migrate properly.

[migration/3.18] Fix NameError

[migration/3.18] Add add_permission relation definitions earlier Otherwise sync_schema_props_perms('defaultval') does not work because the later relation is not in schema.

[migration/3.18] Only commit when we did something Avoids countless confirmation prompts. Closes #3438804.

[doc/3.19] fix (mostly) grammar

fix version number: we still target cw 3.19

Fix a couple merge errors

merge 3.18.0 in 3.19 branch

Added tag cubicweb-version-3.18.0, cubicweb-debian-version-3.18.0-1, cubicweb-centos-version-3.18.0-1 for changeset db37bf35a147

[book] drop reference to removed cubicweb.predicates.implements

3.18.0

[i18n] update French and English translations

i18n update

merge stable

[hooks/security] provide attribute "add" permission As of today, the update permission on attributes is checked at entity *creation* time. This forbids using update permissions the proper way. We set it to be checked at entity update time only. We introduce a specific 'add' permission rule for attributes. For backward compatibility, its default value will be the same as the current 'update' permission. Notes: * needs a new yams version (ticket #149216) * introduces two new 'add_permissions' rdefs (attribute - group|rqlexpr) * if the update permission was () and the bw compat kicks in, the rule is not enforced, to avoid un-creatable entity types -- this restriction will be lifted when the bw compat is gone * small internal refactoring on check_entity_attributes * one small pre 3.6.1 bw compat snippet must be removed from schemaserial Closes #2965518.

[server/test] reorganize data/migratedapp/schema.py to allow comparison with data/schema.py Pure reorganization to make it possible to visually compare both schemas.

[schema] move definition of RQLExpression classes before default permission dicts

[server] remove old backwards compat for PropagateSubjectRelation*Hook Those classes were removed from server/hook.py.

[book] replace PropagateSubjectRelation*Hook references PropagateSubjectRelationHook, PropagateSubjectRelationAddHook and PropagateSubjectRelationDelHook were deprecated by PropagateRelationHook, PropagateRelationAddHook, PropagateRelationDelHook respectively in v3.9, and removed in 3.18. Closes #3420672

[doc] update 3.18 release notes

[pkg] bump logilab-common requirement Necessary since 4b3e657d17ab "[source configuration] silent lgc 0.60 deprecation warning"

update jquery to 1.10 (closes #2786674) Includes: * base jquery * jquery.migrate to help for the transition * jquery.ui The jquery.ui.tabs api has slightly changed hance the change in web/views/tabs. The autocompletion widget is slightly adapted to follow an API change and now extends the base functionality in a simpler way.

[notification] Use https url when available in notification view context Closes #3376252.

[req] Return base-url in case https-url is None in base_url() Closes #3376322.

[server/session] fix typo in keep_cnxset_mode docstring

[breadcrumbs] Small breadcrumb components refactoring * use `open_breadcrumbs` and `close_breadcrumbs` methods where applicable in the BreadCrumbEntityVComponent hierarchy * include the first separator related code in `open_breadcrumbs` to avoid code duplication Closes #3223911.

[views/dot] use an inlined svg for schema and workflow graphs These are currently served as "temporary" pngs which are actually consummed immediately. Which means they cannot be requested twice and any further attempt will yield an error in the logs and some end-user surprise. There is no known acceptable workaround for IE-8 (and previous versions). SVGWEB could be workable but it's not trivial to integrate it properly. Closes #3400448.

[release notes] Documents one() and find() addition

[js] noop is undefined in plain javascript (related to #2786674)

[css] drop shadow.gif & use a couple of rules (closes #3381435)

[css] replace old rounded corners hack by css rules Closes #3381426.

[views/templates] provide a modal main template (closes #3274672) The current NonTemplatableViewTemplate may seem to provide the desired feature if a '__notemplate' is provided in the form, unfortunately, it: * wraps the view in a spurious div * calls view.set_stream(), which has the effect of wiping any js/css resource recorded until then by the view, leading to hard-to-track missing resources symptoms

[js/widgets] fix the InOut widget with modern jQuery versions Several things are done there: * reduction in size and complexity of the code * the unused defaultsettings are removed * the initial `unlinked` list is now correctly populated from python-side * the unit test is adjusted because it tested an irrelevant implementation detail which is no longer true (but the widget of course still handles correctly the linkto information) Tested with ie7, ie9, chromium, firefox. Tested with jQuery 1.6 (cw 3.17.x) and 1.10. Closes #3154531.

[autoform] Allow overridding of permissions checks Closes #3063930

[schema] fix unique together index handling We now provide a more compact indexname, using the schema constraint entity type and the position of the columns set in the entity type unique constraints list. This avoids a nasty name truncation issue. The UniqueTogetherError object is made smarter: it computes the rtypes, abstracting the underlying backend (pg/sqlserver vs sqlite). The `user friendly` adapter is much simplified since there is no longer any truncation issue. Uses a new logilab.database version (ticket #151507) and a new yams version (ticket #189299) Closes #2514939 [jcr: disable hooks when temporarily dropping CWUniqueTogetherConstraint entities]

[pyrorql] deprecate pyrorql source (closes #2919299)

[web/facet] do NOT drop offset/limit of the base rql (closes #3344579)

[webctl] Generate static data directory on upgrade (closes #2167873) - if the folder already exists, ``upgrade`` asks for deletion, - add an option (``generate-staticdir``) to allow skipping this task. - add an option (``staticdir-path``) to specify the static data folder path. The ``gen-static-datadir`` command allows to specify the target folder but there is otherwise no way to retrieve this information during upgrade.

[cwctl] Don't check for old lgc versions Support for check_duplicated_command in CommandLine was added in 2011 for 0.55, and we already depend on 0.59.0.

[migration/3.18] add sanity checks before changing symmetric relations Delete supposedly impossible relations from the db. Related to #3259713.

[migration/3.18] disable all hooks when setting up symmetric relations This operation is semantically a no-op, so it should be invisible for the application. Related to #3259713.

[devtools] bases for instrumenting / debugging standard propagation hooks * a class that helps writing command to check propagation, detecting standard errors and picture the propagation as a graph * a class to instrument sets used to configure standard propagation hooks closes #3171980

instrument cwvreg, so we may know what's being loaded by asking the registry store

set proper PACKAGE information for yams 0.39 Yams 0.39 adds an unobstrusive API to get the package, or cube, defining a part of the schema (entity type, relation type/def). As CubicWeb overrides part of the schema reading process, we have to handle this information on our side so it will be properly set.

[yams] follow yams 0.39 api change introduced by ad95fd2c46de (package attribute on etype/rtype/rdef)

Make the GROUP_CONCAT aggregate function not repeat values (closes #3223975) Work on sets instead of arrays, so if the same value appears twice it's not repeated in the concatenated output. This patch handles the postgresql and sqlite backends, mysql is left alone at this point (seems doable, but I don't have time or motivation to fix and test it).

[mail] allow to specify SMTP's MAIL FROM address to config.sendmails(). Closes #3373620 Also modify testlib.Email so we don't loose this information and may test it.

[mail] format_mail copy/paste fix: we want to test sender-name, not sender-addr

symmetric relations: replace bogus rql2sql translation by a hook The hook ensures X r Y => Y r X iff r is symmetric. The rql-no-hook data importer receives a small amendment but note that: * there exist no test for it * its actual semantics are undefined Hence we cannot prove this hunk breaks nothing, because we cannot prove anything. Closes #3259713.

[migration] fix handling of default value for boolean attributes We can't assert that the old value is 'True' or 'False', because False used to be stored as an empty string in pre-3.18 versions.

[entity/url publishing] fetch_rqlst should use 'is_instance_of' instead of 'is' We usually want to get any subclasses as well (think Training subclass of Workcase). Closes #3210365.

[rewriter] fix latent bug: arbitrary etype may be substituted when using is_instance_of type restriction Note: the test is not deterministic because the bug depends on the iteration order on a set.

[test] drop no more used test data file

[doc] Refer to RsetTableView in EntityTableView docstring instead of TableView

Fix use of vreg.config.anonymous_user() If no anonymous user is declared, anonymous_user() returns 'None, None', not 'None'.

[server] avoid race-condition in timer events With a granularity of one second, there's a good chance calling localtime twice will return the same value. We should consider the event ready immediately if that happens, not wait one second.

[server/pyro] try to shutdown the repository properly If RepositoryServer.trigger_events is not called we might miss a QuitEvent and thus never shutdown the repository and its looping tasks.

[book] remove broken links to web.views.wdoc.ChangeLog{View,Action} ChangeLogView and ChangeLogAction were removed in 3.16.0 (changeset fa044b9157d7, https://www.cubicweb.org/2423532) Closes #3369687

[book] fix Session.{set,free}_cnxset autodoc They're Session methods, not bare functions. Without this building the docs says: reading sources... [ 39%] devrepo/repo/sessions Traceback (most recent call last): File "/usr/lib/pymodules/python2.7/sphinx/ext/autodoc.py", line 326, in import_object obj = self.get_attr(obj, part) File "/usr/lib/pymodules/python2.7/sphinx/ext/autodoc.py", line 232, in get_attr return safe_getattr(obj, name, *defargs) File "/usr/lib/pymodules/python2.7/sphinx/util/inspect.py", line 70, in safe_getattr raise AttributeError(name) AttributeError: set_cnxset Traceback (most recent call last): File "/usr/lib/pymodules/python2.7/sphinx/ext/autodoc.py", line 326, in import_object obj = self.get_attr(obj, part) File "/usr/lib/pymodules/python2.7/sphinx/ext/autodoc.py", line 232, in get_attr return safe_getattr(obj, name, *defargs) File "/usr/lib/pymodules/python2.7/sphinx/util/inspect.py", line 70, in safe_getattr raise AttributeError(name) AttributeError: free_cnxset

cwetype is a *class* property, rename its argument accordingly

typo

[req] New method: RequestSessionBase.find(). This method does what find_entities and find_one_entity did, except it returns the resultset itself. In addition, it accepts 'reverse_' arguments and check that the relations actually exists on the entity before executing the query. Also, reimplement find_one_entity and find_entity based on the new function so they benefit from the more complete implementation, and deprecate them. Note: List of values in kwargs are NOT supported in this initial implementation. Closes #3361290

[rset] New method: ResultSet.one() This method will return exactly one entity while enforcing its existence and unicity. The idea is shamelessly borowed from SQLAlchemy Query.one(). Closes #3352314 [jcr: use len(self) instead of len(self.rows)]