merge 3.18.0 in 3.19 branch

Added tag cubicweb-version-3.18.0, cubicweb-debian-version-3.18.0-1, cubicweb-centos-version-3.18.0-1 for changeset db37bf35a147

[book] drop reference to removed cubicweb.predicates.implements

3.18.0

[i18n] update French and English translations

i18n update

merge stable

[hooks/security] provide attribute "add" permission As of today, the update permission on attributes is checked at entity *creation* time. This forbids using update permissions the proper way. We set it to be checked at entity update time only. We introduce a specific 'add' permission rule for attributes. For backward compatibility, its default value will be the same as the current 'update' permission. Notes: * needs a new yams version (ticket #149216) * introduces two new 'add_permissions' rdefs (attribute - group|rqlexpr) * if the update permission was () and the bw compat kicks in, the rule is not enforced, to avoid un-creatable entity types -- this restriction will be lifted when the bw compat is gone * small internal refactoring on check_entity_attributes * one small pre 3.6.1 bw compat snippet must be removed from schemaserial Closes #2965518.

[server/test] reorganize data/migratedapp/schema.py to allow comparison with data/schema.py Pure reorganization to make it possible to visually compare both schemas.

[schema] move definition of RQLExpression classes before default permission dicts

[server] remove old backwards compat for PropagateSubjectRelation*Hook Those classes were removed from server/hook.py.

[book] replace PropagateSubjectRelation*Hook references PropagateSubjectRelationHook, PropagateSubjectRelationAddHook and PropagateSubjectRelationDelHook were deprecated by PropagateRelationHook, PropagateRelationAddHook, PropagateRelationDelHook respectively in v3.9, and removed in 3.18. Closes #3420672

[doc] update 3.18 release notes

[pkg] bump logilab-common requirement Necessary since 4b3e657d17ab "[source configuration] silent lgc 0.60 deprecation warning"

update jquery to 1.10 (closes #2786674) Includes: * base jquery * jquery.migrate to help for the transition * jquery.ui The jquery.ui.tabs api has slightly changed hance the change in web/views/tabs. The autocompletion widget is slightly adapted to follow an API change and now extends the base functionality in a simpler way.

[notification] Use https url when available in notification view context Closes #3376252.

[req] Return base-url in case https-url is None in base_url() Closes #3376322.

[server/session] fix typo in keep_cnxset_mode docstring

[breadcrumbs] Small breadcrumb components refactoring * use `open_breadcrumbs` and `close_breadcrumbs` methods where applicable in the BreadCrumbEntityVComponent hierarchy * include the first separator related code in `open_breadcrumbs` to avoid code duplication Closes #3223911.

[views/dot] use an inlined svg for schema and workflow graphs These are currently served as "temporary" pngs which are actually consummed immediately. Which means they cannot be requested twice and any further attempt will yield an error in the logs and some end-user surprise. There is no known acceptable workaround for IE-8 (and previous versions). SVGWEB could be workable but it's not trivial to integrate it properly. Closes #3400448.

[release notes] Documents one() and find() addition

[js] noop is undefined in plain javascript (related to #2786674)

[css] drop shadow.gif & use a couple of rules (closes #3381435)

[css] replace old rounded corners hack by css rules Closes #3381426.

[views/templates] provide a modal main template (closes #3274672) The current NonTemplatableViewTemplate may seem to provide the desired feature if a '__notemplate' is provided in the form, unfortunately, it: * wraps the view in a spurious div * calls view.set_stream(), which has the effect of wiping any js/css resource recorded until then by the view, leading to hard-to-track missing resources symptoms

[js/widgets] fix the InOut widget with modern jQuery versions Several things are done there: * reduction in size and complexity of the code * the unused defaultsettings are removed * the initial `unlinked` list is now correctly populated from python-side * the unit test is adjusted because it tested an irrelevant implementation detail which is no longer true (but the widget of course still handles correctly the linkto information) Tested with ie7, ie9, chromium, firefox. Tested with jQuery 1.6 (cw 3.17.x) and 1.10. Closes #3154531.

[autoform] Allow overridding of permissions checks Closes #3063930

[schema] fix unique together index handling We now provide a more compact indexname, using the schema constraint entity type and the position of the columns set in the entity type unique constraints list. This avoids a nasty name truncation issue. The UniqueTogetherError object is made smarter: it computes the rtypes, abstracting the underlying backend (pg/sqlserver vs sqlite). The `user friendly` adapter is much simplified since there is no longer any truncation issue. Uses a new logilab.database version (ticket #151507) and a new yams version (ticket #189299) Closes #2514939 [jcr: disable hooks when temporarily dropping CWUniqueTogetherConstraint entities]

[pyrorql] deprecate pyrorql source (closes #2919299)

[web/facet] do NOT drop offset/limit of the base rql (closes #3344579)

[webctl] Generate static data directory on upgrade (closes #2167873) - if the folder already exists, ``upgrade`` asks for deletion, - add an option (``generate-staticdir``) to allow skipping this task. - add an option (``staticdir-path``) to specify the static data folder path. The ``gen-static-datadir`` command allows to specify the target folder but there is otherwise no way to retrieve this information during upgrade.

[cwctl] Don't check for old lgc versions Support for check_duplicated_command in CommandLine was added in 2011 for 0.55, and we already depend on 0.59.0.

[migration/3.18] add sanity checks before changing symmetric relations Delete supposedly impossible relations from the db. Related to #3259713.

[migration/3.18] disable all hooks when setting up symmetric relations This operation is semantically a no-op, so it should be invisible for the application. Related to #3259713.

[devtools] bases for instrumenting / debugging standard propagation hooks * a class that helps writing command to check propagation, detecting standard errors and picture the propagation as a graph * a class to instrument sets used to configure standard propagation hooks closes #3171980

instrument cwvreg, so we may know what's being loaded by asking the registry store

set proper PACKAGE information for yams 0.39 Yams 0.39 adds an unobstrusive API to get the package, or cube, defining a part of the schema (entity type, relation type/def). As CubicWeb overrides part of the schema reading process, we have to handle this information on our side so it will be properly set.

[yams] follow yams 0.39 api change introduced by ad95fd2c46de (package attribute on etype/rtype/rdef)

Make the GROUP_CONCAT aggregate function not repeat values (closes #3223975) Work on sets instead of arrays, so if the same value appears twice it's not repeated in the concatenated output. This patch handles the postgresql and sqlite backends, mysql is left alone at this point (seems doable, but I don't have time or motivation to fix and test it).

[mail] allow to specify SMTP's MAIL FROM address to config.sendmails(). Closes #3373620 Also modify testlib.Email so we don't loose this information and may test it.

[mail] format_mail copy/paste fix: we want to test sender-name, not sender-addr

symmetric relations: replace bogus rql2sql translation by a hook The hook ensures X r Y => Y r X iff r is symmetric. The rql-no-hook data importer receives a small amendment but note that: * there exist no test for it * its actual semantics are undefined Hence we cannot prove this hunk breaks nothing, because we cannot prove anything. Closes #3259713.

[migration] fix handling of default value for boolean attributes We can't assert that the old value is 'True' or 'False', because False used to be stored as an empty string in pre-3.18 versions.

[entity/url publishing] fetch_rqlst should use 'is_instance_of' instead of 'is' We usually want to get any subclasses as well (think Training subclass of Workcase). Closes #3210365.

[rewriter] fix latent bug: arbitrary etype may be substituted when using is_instance_of type restriction Note: the test is not deterministic because the bug depends on the iteration order on a set.

[test] drop no more used test data file

[doc] Refer to RsetTableView in EntityTableView docstring instead of TableView

Fix use of vreg.config.anonymous_user() If no anonymous user is declared, anonymous_user() returns 'None, None', not 'None'.

[server] avoid race-condition in timer events With a granularity of one second, there's a good chance calling localtime twice will return the same value. We should consider the event ready immediately if that happens, not wait one second.

[server/pyro] try to shutdown the repository properly If RepositoryServer.trigger_events is not called we might miss a QuitEvent and thus never shutdown the repository and its looping tasks.

[book] remove broken links to web.views.wdoc.ChangeLog{View,Action} ChangeLogView and ChangeLogAction were removed in 3.16.0 (changeset fa044b9157d7, https://www.cubicweb.org/2423532) Closes #3369687

[book] fix Session.{set,free}_cnxset autodoc They're Session methods, not bare functions. Without this building the docs says: reading sources... [ 39%] devrepo/repo/sessions Traceback (most recent call last): File "/usr/lib/pymodules/python2.7/sphinx/ext/autodoc.py", line 326, in import_object obj = self.get_attr(obj, part) File "/usr/lib/pymodules/python2.7/sphinx/ext/autodoc.py", line 232, in get_attr return safe_getattr(obj, name, *defargs) File "/usr/lib/pymodules/python2.7/sphinx/util/inspect.py", line 70, in safe_getattr raise AttributeError(name) AttributeError: set_cnxset Traceback (most recent call last): File "/usr/lib/pymodules/python2.7/sphinx/ext/autodoc.py", line 326, in import_object obj = self.get_attr(obj, part) File "/usr/lib/pymodules/python2.7/sphinx/ext/autodoc.py", line 232, in get_attr return safe_getattr(obj, name, *defargs) File "/usr/lib/pymodules/python2.7/sphinx/util/inspect.py", line 70, in safe_getattr raise AttributeError(name) AttributeError: free_cnxset

cwetype is a *class* property, rename its argument accordingly

typo

[req] New method: RequestSessionBase.find(). This method does what find_entities and find_one_entity did, except it returns the resultset itself. In addition, it accepts 'reverse_' arguments and check that the relations actually exists on the entity before executing the query. Also, reimplement find_one_entity and find_entity based on the new function so they benefit from the more complete implementation, and deprecate them. Note: List of values in kwargs are NOT supported in this initial implementation. Closes #3361290

[rset] New method: ResultSet.one() This method will return exactly one entity while enforcing its existence and unicity. The idea is shamelessly borowed from SQLAlchemy Query.one(). Closes #3352314 [jcr: use len(self) instead of len(self.rows)]

Added tag cubicweb-debian-version-3.17.11-2 for changeset b02e2912cad5

[debian] add lintian override for a false positive

[predicates] allow multiple transition names in on_fire_transition (closes #3013720) Also add a warning about the unimplemented from_state_name argument.

[css] add the entypo font as available ressource (closes #2930356) This provides a standard set of pictograms for use in CubicWeb and cubes. To use a pictogram, the "cubicweb.pictograms.css" must be explicitly loaded (using, e.g. view.add_css(...)).

[skeleton i18n] Add PO-Revision-Date header to make emacs po-mode happy Closes #3276389

[uicfg] give a set_fields_order method to the primary view display control Hence it is nicely symmetrical with the fields kwargs. Closes #2741963.

merge 3.17.11

Added tag cubicweb-centos-version-3.17.11-1, cubicweb-version-3.17.11, cubicweb-debian-version-3.17.11-1 for changeset 7f67db7c848e

prepare 3.17.11

[devtools] write db config through a temporary file Avoids other test processes reading partially written config and exploding in pickle.loads. There are still concurrency issues, presumably with the database itself. Something for another friday.

[server] Add missing indices for undo support (closes #3259691) The tables used by the 'undo' feature were missing some indices, which among other things triggered repeated sequential scans when deleting old transactions.

[sqlutils] fix sqlite group_concat harder (related to #3331906)

[sqlutils] avoid a crash with sqlite when using group_concat (closes #3331906)

[server/initrepo] show failed sql statements and abort if necessary (closes #3308564)

[views/table] use cubicweb.ajax.js for pagination Else we fail in js-land on a "AJAX_PREFIX_URL is not defined". Closes #3329072.

[rset] Fix regression with rset copying (closes #3344410) Don't override passed-in descriptions in ResultSet.__init__, even if they're empty.

[doc] Fix a sample request

[tableview] Don't share column renderers between all instances of a table (closes #3351872)

[tableview] Add some unique element to AbstractColumnRenderer.__str__ (related to #3351872) Otherwise different instances look the same which is confusing.

add failing test case related to #3013554

[web] make sign_text unicode aware, avoid crash with non-ascii chars. Closes #3289774

[doc] Use string debug mode in debugged docstring

[rql2sql] fix relation table scope when some variable from subquery is used. Closes #3252270 While it feels weird to drop this explicit ColumnAlias handling, hg history has no clue on its introduction (prior to rev 0) and no older test case fails without it, so let's drop it.

[cwctl] backout 5d5b3a865eb1, which breaks interactive shell usage

[ReST] Implement a rql-table reST directive. Closes #3252856 allowing to call table or derivated view specify headers / cellvids. Also, rql may be split accross several lines which greatly improve readability.

[server] drop support for antediluvian lgdb versions That API was changed in 2010, before logilab-database was moved out of logilab-common.

[doc/book/security] update description of entity update (Related to #2932033)

[test] test can_use_rest_path directly No need to go through the Entity.rest_path if what we want is to unittest that one function.

[ctl] set proper encoding for sys.stdout/sys.stderr. Closes #1669144

[web] typo in config file comment

[book] Typo and formatting fixes in devrepo/entityclasses/application-logic

[cwctl] Only start the instance after upgrade if it was running before Closes #3207676.

[book] Improve match_rtype selector docstring and "Relation modification related events" section

[book] properly escape backslash

[test] use assertIn / assertIsInstance

update pylint extension to astroid API

Fix example in cw_set method

[merge] incorporate cw 3.17.10 fixes

Added tag cubicweb-centos-version-3.17.10-1, cubicweb-version-3.17.10, cubicweb-debian-version-3.17.10-1 for changeset fe0e1863a137

[pkg] prepare 3.17.10

[web] don't rename resources in gen-static-datadir We should use the result from WebConfiguration.locate_resource to know which file to copy, not what name to copy it to. The destination file should be the name of the resource, as the client will request it. This matters in the case of cubicweb.css which locate_resource might rewrite to cubicweb.old.css. Closes #3206129

[notification] avoid leaking cnxsets (closes #3243810) When sending notifications, we get each recipient as either an email address or a CWUser. In the latter case, we create a temporary session for that user and use it to send the mail. However, if we later decided to not send the mail after all, we'd leak the session and its cnxset. Add a try block inside the loop to make sure the temporary sessions are closed properly.

[book] English fix

[serverctl] complete the schema-diff command with options that will fix & enhance it. Notably: - filter out eids (doesn't make sense for the filesystem schema) - handle permissions, trying to detect the default ones Needs yams #174042. Closes #2996547.

[server] deprecate the old multi-source system It is removed in 3.19. Closes #2919299

[doc] document changed attribute default value storage

defaultval migration for sqlite

[schema] store default attribute values in a Bytes field, allowing python objects as default values Notes: * Binary objects grow two methods * we depend on a newer yams version * the code is quite simplified as a result of the whole change * a very old and nasty bug where bool(default values) evaluating to False not being properly migrated is killed Closes #2414591.

[source/native] fix sqlserver pattern matcher for unique constraints (closes #3227593) The matcher is used to parse the low-level error message and extract meaningful information for the end users. If broken, the users see an alarming and bewildering message. A full & definitive fix is scheduled for CubicWeb 3.18.

[dbapi] Fix broken inmemory:// URLs parsing (closes #3225832) ParseResult.path is what comes after the "domain name" in standard URLs. So in the case of 'inmemory://<instance' URLs, it is always empty. ParseResult.netloc contains the "domain name" part that we really want.

[metadata hook] rename an operation for the sake of readability

[book] Typo fix and rephrasing in devrepo/migration

[cwshell] rename history file to .cwshell_history (erudi is dead, long live cw)

[entity] Entity.related(): add a targettypes argument (closes #2957313) This allows to restrict the entity types returned by the method (passed to cw_related_rql). When such an argument is given, no caching happens.

[source configuration] silent lgc 0.60 deprecation warning

[web] remove action "Got Rhythm" (closes #3093362)

[debian] Break old geocoding It wants cubicweb.interfaces.

[wsgi] remove ages old hack to set base_url Its meaning has been lost in the mist of time.

[wsgi] also provide an example using werkzeug (if available) Related to #3005509.

[wsgi] add the simplest possible wsgi (debug) server This server is able to: * serve on a given port using the stdlib SimpleHTTPServer * run looping tasks Closes #3005509.

[querier/security] instrument a bit the querier read security checks Related to #2920304

[book] Improve PostgreSQL configuration section * Do not advice for specific PostgreSQL version (point to meta-packages if available in Linux distros); * First mention packages from Linux distros, put the link to official postgres documentation at the end; Closes #2725302.

[js] add a selector string escaping function will help use id strings in jquery selector expressions, e.g: "foo.[subject]:42" -> "foo\.\[subject\]\:42" Related to #3154531.

[merge] backport stable fixes