[services] catch missing group early in register_user Better an assertion than a rql syntax error for such a programming error.

[services] rename register_user kwargs to cwuserkwargs They're attributes/relations for the created CWUser, this makes it clearer.

[services] don't hardcode 'users' group for register_user Let the caller choose which group the user should be in. Related to #3020639 Caveat: this is a possible security hole if untrusted values are allowed.

[sobjects/test] check that register_user properly sets user name and email address Related to #3020639

[server] Refactor Repository.register_user into a CubicWeb service (closes #3020639) [jcr: move commit to the caller; add item in release notes; fix NameErrors]

[migration] set a cnxset before process_script() Not pretty, but I don't have a better idea right now and this at least ensures migration scripts get run in the expected environment and can access sql stuff as before.

[server] make internal_cnx get and keep a connection to the db - internal_session worked that way - it's less surprising for users if their internal connections "just work", rather than having to deal with the cnxset when using low-level APIs Includes a backout of 3f62606c01a2 "[repo] Fix register_user" which is no longer needed. Depend on logilab-database 1.12.1 to avoid errors due to pointless check in python's sqlite3 binding.

[server/session] make commit/rollback obey ensure_cnx_set Inside the ensure_cnx_set context manager, commit and rollback aren't allowed to dispose of the cnxset.

[repo] fix deprecation messages for stats and gc_stats

[cwconfig] use "foo not in bar" instead of "not foo in bar"

[debian] Add Breaks on a few cubes that need changes for 3.19 - mercurial_server is affected by composite deletion changes - forgotpwd relied on session data to be forwarded to notification view - registration hooks into the authentication stack

[hooks] run cascading delete hook later Applications may have their own before_delete_entity hooks that rely on the composite relations still existing.

[devtools] set PGPORT/PGHOST when calling pg_ctl start Helps pg_ctl find the socket on squeeze, otherwise it looks in /var/run/postgresql.

[server/test] update to deal with lgdb sqlserver fix See https://www.logilab.org/ticket/235748

[test] Fix regressions from 96549de9dd70 "[test] use assertIn where appropriate"

[repo] Fix register_user Make sure we have a connection to the db around user creation.

[test] use assertNotIn where appropriate Automatically generated with: find -name unittest_*.py -o -name test_*.py | xargs sed -i -e 's/assertFalse(\([^,]*\) in \([^,]*\))/assertNotIn(\1, \2)/' plus revert of a couple false matches.

[test] use assertIn where appropriate Automatically generated with: find -name unittest_*.py -o -name test_*.py | xargs sed -i -e 's/assertTrue(\([^,]*\) in \([^,]*\))/assertIn(\1, \2)/'

[doc] Add removal of old multi-sources to 3.19 release notes

[pkg] Bump to 3.19.0

[web/debug] remove code broken by session refactoring session.cnx and cnx.check() no longer exist.

[web/debug] Fix check for open sessions itervalues returns an iterator, which never evaluates to False. Regression from 0bb18407c053 "[toward py3k] rewrite dict.keys() and dict.values() (part of #2711624)".

[cwconfig] Remove useless 'ui-cube' configuration option (closes #3728904) The mechanism introduced could never actually be used as it brings dependency loops to the cube graph. Just get rid of it. This reverts commit 4032499c701e.

[test] Use RepoAccess.shell() in unittest_predicates

[devtools] add 'shell' method to RepoAccess, deprecate CubicWebTC's Lets us control the connection's lifetime.

[repo] make extid2eid work with either a session or a connection The only place it should matter is for restoring pending_operations in case of failure. Which is not exercised by the test suite. Here be dragons.

[repository] extid2eid's 'commit' argument no more necessary This removes the need for creating an internal session.

[repository] complete argument is no more needed since we have no more 'true' multisource. It is always false.

[repository] 'session' argument is always given to type[_and_source]_from_eid, make it mandatory and simplify code accordingly

[repository] 'session' argument is always given to extid2eid, make it mandatory and simplify code accordingly

[web/data] Remove broken jquery-json plugin (closes #3590335) Conversion of Date() objects is somehow broken (don't know why beyond that it's due to jquery-json's monkeypatched Date.toJSON() method). Use the now-standard JSON.stringify() instead. http://caniuse.com/#feat=json says it's supported by all browsers, and even IE going back to version 8.

Add missing relation types items in schema global variables Some of these global variables are used, in particular, to control schema views (web/views/schema.py). While they are currently not properly documented (see #3360868), their definition are currently incomplete (e.g. one could reasonably expect to find ``cw_schema`` in ``SYSTEM_RTYPES``). Closes #3486114.

[web] return 403 for Unauthorized, not 401 401 is reserved to HTTP authentication. Just because it's also called "unauthorized" doesn't mean it's the same as cubicweb's Unauthorized exception. Closes #3648809.

[test] update unittest_repoapi to 3.19 api (sic) [jcr: while at it, fix typos and add missing __main__ section]

[test] update unittest_utils to 3.19 api

[test] update unittest_rset to 3.19 api [jcr: use self.admin_access instead of creating a new one each time]

[test] update unittest_rqlrewrite to 3.19 api

[test] update unittest_req to 3.19 api [jcr: use self.admin_access instead of creating a new one each time]

[test] update unittest_predicates to 3.19 api [jcr: use self.admin_access instead of creating a new one each time]

[web/test] port unittest_reledit to RepoAccess API

[web/test] port unittest_magicsearch to RepoAccess API request created in setUp makes context manager usage painful, so create a context manager to do the setup for each test class.

[web/test] use real_error_handling context manager instead of open-coding it

[web/test] port unittest_idownloadable to RepoAccess API

[web/test] remove useless calls to self.request() from CORS tests

[web/test] port unittest_formfields to RepoAccess API

[web/test] port unittest_form to RepoAccess API

[web/test] port unittest_facet to RepoAccess API

[web/test] move unittest_controller to RepoAccess test API

[web/test] port unittest_breadcrumbs to 3.19 test api

[web/test] avoid deprecation warnings from find_entities()

[web/test] port unittest_application to new 3.19 API

[test] silence warnings for unittest_schema by upgrading to new 3.19 API

[test] silence warnings for unittest_cwctl by upgrading to new 3.19 API

[test] replace some assert{True,False} with more specific method

[test] silence warnings for unittest_entity by upgrading to new 3.19 API

Fix constraint sync during migration - restore constraints lost during merge in test schema. - use constraint_by_eid in BeforeDeleteCWConstraintHook as done in 3.17.14 for BeforeDeleteConstrainedByHook. Fixes handling of multiple constraints of the same type. - make sync_schema_props_perms() delete the CWConstraint entity instead of the constrained_by relation. In 3.19, the latter doesn't automatically result in the former just because the relation is composite. Simplify the constraint migration to delete all removed constraints and recreate new ones even if they share the same type; that optimization made the code more complicated for (AFAICT) no significant reason.

merge 3.18.4 into default server/test/unittest_migractions fails due to interaction between new constraint test (from 3.18) and composite deletion change (in 3.19).

Added tag cubicweb-version-3.18.4, cubicweb-debian-version-3.18.4-1, cubicweb-centos-version-3.18.4-1 for changeset 0176da9bc752

[test/data] Add missing 'add' permission on attribute Silences a warning introduced in 3.18.

[hooks/notification] Fix indentation Follow-up for changeset 1b8552265f3b "[hooks/notification] use a cnx not a session"

[devtools] deprecate CWTC.user()

minor cleanups

some copyright updates

[devtools] properly close open access on tearDown avoiding further warning about living sessions while the repo is turned off. The doc string says: The RepoAccess need to be closed to destroy the associated Session. TestCase usually take care of this aspect for the user. but that wasn't true until now.

[devtools] make comment (a bit) more readable

[devtools] avoid an internal deprecation warning websession and session are now one and the same.

[server/session] undeprecate session.anonymous_session It's not connection-specific.

[web/sessions] use session.sessionid instead of deprecated session.id

[repo] don't set cnxset when getting the session to close it and avoid warning about that. session._cnx.rollback won't do anything if it's not set and a newer connection is now created to run the session_close hook, so there is not need to set cnxset. Notice we have to take care about not trying to rollback the underlying cnx when the session is not 'session_handled' (i.e. bw compat mode). This case only occurs due to hackish things to keep bw compat in testlib.

[repo] handle connection explicitly when calling session open/close hooks

[hooks/syncschema] work with connections not sessions

[server/sources] {before,after}_entity_insertion wants a cnx not a session

[hooks/syncsources] use a cnx not a session

[hooks/notification] use a cnx not a session

[hooks/metadata] use a cnx not a session

[hooks/bookmark] use a cnx not a session

[hooks/workflow] use a cnx not a session

[hooks/email] use a cnx not a session

[hooks/integrity] use a cnx not a session

[hooks/security] let's use a connection, not a session

[server/sources/native] deal with connections, not sessions

[server] eschema_eid needs a connection, not a session

[repository] operations get a connection instead of a session Left extid2eid unchanged because that function scares the shit out of me.

[server/hook] operations get a connection instead of a session

[repository] Use an internal connection in register_user

avoid to internally raise 3.19 warnings

[sources] ensure we have a cnxset in undoable_transactions

[server] move security/integrity hook management away from InternalSession.__init__ and to Repository.internal_{session,cnx} instead. Lets internal_cnx avoid deprecation warnings.

[migration] make 'session' object be a server-side Connection, not ClientConnection I think this better corresponds to the previous behaviour. However, that Connection does not have a cnxset, which should probably be changed (for both interactive_shell and cmd_process_script).

[sources] Skip problematic sources when starting shell instead of crashing. Closes #3670208 At least this gives an opportunity to fix the problem in using c-c shell, which is currently not possible because it crashes during initialization. Notice that since sources are copy based, it should not be a problem at all to have some disabled, beside for services such as authentication sources as ldapfeed.

[web] fix language negotiation Regression caused by commit 6fd0ac6506cb "[web-request] handle default language earlier". We would handle language negotiation at request init time, but overwrite the selected language with the site default when calling set_cnx.

[merge] bring the 3.17.14 fixes to 3.18

[test/utils] repair concat-resources test That was forgotten in 240a620b9cd3. Related to #3670503.

[pkg] bump cubicweb.spec

[dataimport] Correctly call rschema(rtype) in SqlGenObjectStore, closes #3694139

[pkg] prepare 3.18.4

[web] Disable 'concat-resources' by default (closes #3670503) It makes debugging harder and it can actually *break* working JS or CSS files. This is the first step towards complete removal.

[testlib] Drop override of assertItemsEqual method The override is not compatible with underlying (standard) method because it builds sets and rely on eid. Closes #3641111.

[rewrite] Fix crash when the main variable doesn't appear in the snippet's vargraph Also, the added test discover another bug once the first has been fixed (erroneous argument given to .rewrite recursive call). Closes #3661918

[ext/rest] Catch SystemExit raised by docutils docutils publisher may raise SystemExit which is not caught by Exception. Closes #3648763.

[entity] User-defined relation to skip for copy has precedence Otherwise permission problems can occur on rtypes not yet skipped Closes #3653459

Make EditController edit_entity method always return an eid In cases the entity was to be created or copied, eid was None hence the method returned None despite the dosctring says the method should always return an eid. Now if the eid variable is None, it is assigned to the newly created entity eid. Closes #3593029.

[migration] Improve update of in-memory schema during 3.18 CWAttribute.defaultval change We weren't updating the schema properly so a bunch of structures still had the old rdef with 'String' as object. That caused failures to add new attributes later in the migration as their default value would be a Binary object but the schema would expect a String. Closes #3683640

[migration] always rebuild infered relation This was skipped for some bad reason (see 12ad88615a12 which introduced the change). Fix for #231956 in Yams is necessary to allow this cset: during a migration, we want to always reinfer relations while allowing explicit redefinition of an infered relation later. Else, we may run the migration with missing parts of the schema (the one that should have been infered). Closes #3685463.

Backout "[web/navigation] use add_onload instead of inline javascript href" This reverts changeset 170e1437948d, for at least two reasons: the PageNavigationSelect component was not updated for the new type of navigation links, and external cubes defining View.page_navigation_url exist in the wild and also got broken navigation links. This reopens #3501626 and closes #3677945.

[test] those are rql tests, not cubicweb (and they break with rql 0.31.5). Remove them.

[source/native] allow many eid creation per .create_eid call (closes #3526594) [jcr: move migration code to bootstrapmigration_repository.py to make sure we stop using the old sequence ASAP]

[pkg] bump the dependency on logilab.database This will allow use of an extended API. Related to #3526594.

[source/native] refactor eid generation code This set of related methods clutters the native source API. It will be better served with a dedicated eid genrator object. Related to #3526594. [jcr: allocate the eid generator in __init__]

Move entity cache from web.request.ConnectionCubicWebRequestBase to repoapi.ClientConnection ResultSet._build_entity relies on an entity cache to avoid going round in circles (stack overflow due to infinite recursion) e.g. in a postcreate script. Clear the cache at commit/rollback time to avoid unexpected side effects; and explicitly clear the cache in a few tests where we mix ORM and RQL using the same connection.

[server] Fix AttributeError in extid2eid Session._tx was renamed in cb87e831c183, use _cnx instead.

[book] Update documentation for new repoapi Quite a few things change in 3.19: - repoapi instead of dbapi - ClientConnection / Connection / Session rework - web authentication process - test APIs Closes #3638793

[utils] the json module is always available It's included since python 2.6, and we don't support earlier versions.

[fti] properly close the ProgressBar Ensures cubicweb-ctl db-rebuild-fti ends its output with a newline.

[serverctl] use repoapi for db-check, add-source, rebuild-fti commands One more step towards getting rid of dbapi.

[devtools] make get_repo_and_cnx return a repoapi ClientConnection And adjust callers accordingly (they need to use it as a context manager to open/close it properly).

[repoapi] Avoid deprecation warnings from session.id Apparently it's called session.sessionid now.

[server] some s/session/cnx/ For merely accessing the database we need a connection, not a session.

Move setting session.mtime from dbapi to web session manager clean_session was broken since the switch away from dbapi on the web side, since session.mtime wasn't being set anywhere.

[web] whitespace cleanup in http_headers.py

[web] implement cross origin resource sharing (CORS) (closes #2491768) Partial implementation that is enough to get started but leaves out some of the advanced features like caching and non-simple methods and headers.

[devtools] add a 'method' argument to RepoAccess.web_request so that one can easily forge a request with any HTTP method. Also a bunch of clone cleanups.

[web] Fix handling of some http headers The generator takes as argument a list of raw string values, and is supposed to return the parsed version. Calling str on that list makes no sense.

[web/request] deprecate user_callback Storing closures in session data considered harmful. Closes #3567793

[doc] undocument user_callback Related to #3567793

Added tag cubicweb-centos-version-3.17.14-1, cubicweb-version-3.17.14, cubicweb-debian-version-3.17.14-1 for changeset fa00fc251d57

[migractions] Better handle removal of RQLConstraint in sync_schema In rdef synchronisation, filter unmodified constraints before processing the sets of old and new constraints. Add a new `constraint_by_eid` method on RelationDefinitionSchema, which eliminates the ambiguity of `constraint_by_type` usage when several constraints of the same type are involved in the same migration. Closes #3633644.

[test] when the config change, we should cleanup sys.path and sys.modules. Closes #3634164 Without this, modules imported within instance home will be messed up.

[wsgi] make sure request.content is available for consumption The wsgi.input stream is not seekable, so make a copy either to memory or disk so it's still available to the user even after we've parsed it (for POST form processing). Closes #3554996

[facet] create a RangeRQLPathFacet (closes #2852512) This facet is a mix between a RQLPathFacet and a RangeFacet, allowing to use the FacetRangeWidget on the RQLPathFacet target attribute

[request] Make sure set_message() actually displays its given message (closes #3003425) The default automatic form sets a standard submit message as session data. However in some cases, such as the cancellation of a form edition, another message is intended to be printed on-screen. This patch makes sure that set_message() really scrubs all previous messages (and ids) from the request object and from the session data.

[pkg] prepare 3.17.14

[hooks/syncschema] do not crash on double removal of a constraint Some complicated migrations may entail a double constraint removal. It is not yet known whether this is a genuine cw bug or the result of a complicated migration story, but we should nevertheless not crash. Rather, emit a CRITICAL warning and proceed. Closes #3595309.

[doc/book] Fix typo in import

[server/session] fix a couple typos in doc strings

[server/migractions] finish migration to repoapi objects Changeset 241b1232ed7f (Use repoapi instead of dbapi for cwctl shell, upgrade and db-init) only did half of the job. It left the migration handler with both a session (cubicweb.server.session.Session) and a cnx (cubicweb.repoapi.ClientConnection) attribute with different ideas of what Connection they were talking to. With this change, we: - make the caller responsible of disabling security on the Connection if it wants to - turn mih.session into an actual attribute, set on __init__ - same for cnx (the client connection) - drop the "lazy connection" logic, and establish the connection up-front - always go through the connection instead of the session when we need to talk to the db

[server] use the ClientConnection directly now that it has more methods available Gets rid of ugly self.cnx._cnx.

[repoapi] add security/hook control and system_sql Just redirect these methods to the repo-side Connection.

[predicates] fix thinko in doc strings Replace "e.g." with more appropriate "i.e.". These are not examples.

[web/basecomponents] remove EtypeRestrictionComponent Closes #3119951

[ldapparser, book] document additional error causes

[js] remove leftover dead code in loadxhtml form.callback support was removed in changeset 68cde7431c2c, but some signs of it remained.

[web] Avoid using an empty dict as default parameter of 'CubicWebRequestBase'. Also update the related docstring of the __init__ method.

[schema] fix composite deletion handling Do not delete component entities if the composite is not deleted in the same transaction. Deletion semantics are thus: if the composite is deleted, the components are deleted. Closes #3463112.

[schema] add composite handling helpers on EntitySchema (related to #3463112) These will help write predicates, hooks, etc.

[server] Handle unique constraint violations under recent sqlite The error message changed from "columns foo, bar, baz are not unique" to "UNIQUE constraint failed: table.foo, table.bar, table.baz". Closes #3564510

[i18n] update de: 1065 translated messages, 1 fuzzy translation, 259 untranslated messages. en: 656 translated messages, 669 untranslated messages. es: 1325 translated messages. fr: 1314 translated messages, 11 untranslated messages.

[i18n] es.po updated

merge 3.18.x in 3.19 branch

3.18 is stable

[schema] explicitly declare 'add' permissions for a couple attributes

Added tag cubicweb-version-3.17.13, cubicweb-debian-version-3.17.13-1, cubicweb-centos-version-3.17.13-1 for changeset 09b4ebb9b0f1

Added tag cubicweb-version-3.18.3, cubicweb-debian-version-3.18.3-1, cubicweb-centos-version-3.18.3-1 for changeset afd21fea201a

Update translations

[pkg] prepare 3.18.3

merge 3.17.13

[ldapfeed] fix encode error during initial user import Closes #3539196.

[web/data] Ignore disabled widgets in cw.utils.formContents() (closes #3544492) 17.12.1 Disabled controls : http://www.w3.org/TR/REC-html40/interact/forms.html#h-17.12.1 When set, the disabled attribute has the following effects on an element: * Disabled controls do not receive focus. * Disabled controls are skipped in tabbing navigation. * Disabled controls cannot be successful. The third one is the important one. 17.13.2 Successful controls : http://www.w3.org/TR/REC-html40/interact/forms.html#successful-controls A successful control is "valid" for submission. Every successful control has its control name paired with its current value as part of the submitted form data set. Bottom line, disable widgets should not be part of the names and values lists returned by formContents().

[etwist] Fix an empty request content after a Twisted processing (closes #3546795). The content of a POST request could be empty when the Twisted server is used.

[web/data] remove images that aren't used anywhere

[web/data] Add missing images from jquery-ui 1.10.3 Closes #3175933

[web/navigation] use add_onload instead of inline javascript href This way our javascript code isn't thrown out by the html cleaner e.g. when using the rql rest directive and a table view. To make things simpler, we now always use ajax URLs for navigation, even when we would previously have used regular links. Closes #3501626

[web/data] fix treeview regression (closes #3526466) Changeset 68cde7431c2c "[js] remove 3.9 bw compat (where apparently unused)" removed the use of form.callback from loadxhtml, which treeview relied on. Update to add a callback on the loadxhtml return value instead.

backout 8f3e963501e2, which is not ready yet

[pkg] also bump it there

[pkg] prepare 3.17.13

[navigation] use add_onload instead of inline javascript href This way our javascript code isn't thrown out by the html cleaner e.g. when using the rql rest directive and a table view. Closes #3501626

[uilib] allow canvas tags in the html cleaner Used by the iprogress cube. Closes #3524254.

[ajax] use a custom tag to handle dynamically loaded js Using <pre class="script"> makes it trivial for a malicious user to inject arbitrary javascript into a html or rest text element (because it looks innocent to the html sanitizer). Using a custom tag we can be sure that it actually comes from our code and not from untrusted user data. IE ignores custom tags, though, so we put it in its own namespace. https://extranet.logilab.fr/1530578

[dataimport] fix comment

[hooks/security, devtools/fill] silence yams 0.38.0 warnings

Drop 3.13 incomplete backward compat in edit controller. It is very old and broken (there is another non-backward-copmpatible usage of `_cw_entity_fields`), better to remove it instead of fixing. Closes #3515223.

[devtools] force locale to C for postgresql test clusters Avoids initdb failure with e.g. iso8859-X locale and utf-8 encoding.

[rql2sql] remove special behaviour of symmetric relation vs DISTINCT 0542a85fe667 replacing SQL OR by hooks for symmetric relations allows that. This involve a single test value change for a border case: when querying a symmetric relation without specifying the subject nor the object, you may get some duplicated result. IMO this is fine to let the user explicitly use DISTINCT or not and to remove the dedicated handling we had which didn't let any choice. Related to #3259713

Typo in comments and error messages

Fix typo in a setup.py comment

[dataimport, migration] more fixes in the spirit of a6c32edabc8d: [entity, metadata] huuum, use resolvable url as cwuri... And fix existing ones. Closes #3390388

[doc] Fix typo in devrepo/fti

[doc/3.19] Clarify repoapi.get_repository usage Followup for 9a62c52d167e.

[server] use a connection instead of a session for user authentication

Use repoapi instead of dbapi for cwctl shell, upgrade and db-init Hopefully nobody uses dbapi-specific API in their migration scripts. I guess we'll find out.

[server/repo] use internal_cnx in init_sources_from_database

[server/session] avoid spurious warnings from Session.close Add internal non-noisy _rollback method.

[server] rename session to cnx in querier and plan They don't actually need a session, so let's make that clear.

[repo] Use a connection instead of a session for repo.stats()

[repo] Use a connection instead of a session for repo.gc_stats()

[repo] Use a connection instead of a session for repo.get_versions()

[webconfig] get_repository moved to repoapi

[schemaserial] Replace 'cursor' with 'cnx' We don't have cursors in rql, since execute is synchronous.

[book] basic documentation for LDAPfeed

[testlib] Use session.sessionid instead of deprecated session.id

Add data/pgdb to hgignore patterns PostgresTestDataBaseHandler uses it as datadir.