[3.17 migration] when some cube is missing, add_cube raise ConfigurationError, not ImportError. Closes #2981477

[migraction] rename_entity_type simply warn if old entity type isn't in the schema. Closes #3004069

[deprecation] fix uihelper deprecation warning, use the rtag name, not the class, so it may be copy/pasted

[test] makes fti test order-agnostic (closes #3005633)

[web/views/staticcontrollers] Make ConcatFilesHandler write to a tempfile In order to be more resistant to errors such as ENOSPC, and against different threads accessing the same static data, we: - concatenate data files to a temporary file, and rename it once we're done - delete the tempfile in case of errors - protect the creation of the cache_concat file with a lock Closes #3005547

[test] make unittest_schema.py compatible with cubicweb-file 0.14

merge 3.16.6 branch in 3.17

Added tag cubicweb-centos-version-3.16.6-1, cubicweb-debian-version-3.16.6-1, cubicweb-version-3.16.6 for changeset b4ccaf13081d

Prepare 3.16.6

[devctl] properly generate i18n messsages for cubes that uses uicfg instances (closes #2811282)

[test/devctl] add a test case for i18ncube command

Protect against crash in the `relation_possible` predicate with ambiguous relations. It was still possible to get a KeyError when using relations that are ambiguous at both ends, like (Executable, version_of, Program) & (Version, version_of, Project). Closes #3010148.

[test] give a non-ambiguous order to sync_schema assertion (closes #3001959) Sorting by ordernum alone is unstable since several values have the same ordernum. ordernum + name should be stable.

Added tag cubicweb-version-3.17.3, cubicweb-debian-version-3.17.3-1, cubicweb-centos-version-3.17.3-1 for changeset 32b4d5314fd9

[test] typo

[schema,server] add a security debugging aid (closes #2920304) - Add a DGB_SEC debugging flag (to be used with set_debug/debugged). - Add a context manager (tunesecurity) to filter security assertions. Note: this does not address all read-security mecanisms.

[etwist] fix handling of multiple files per field html5 permits multiple files uploads, which can be expressed as:: <input type='file' multiple='multiple' /> This changeset avoids previous crash. Nothing is changed when a single file is uploaded (backward compat is thus preserved). When multiple files are uploaded for a single html input tag, the corresponding web request form key receives a list of tuples like [('filename-1', IStream1), ('filename-2', IStream2), ...]. closes #2847207.

[merge] start 3.18 development

[pkg] prepare 3.17.3

[pkg] Remove obsolete ubuntu hardy packaging It's been EOL for a while.

[book] fix sphinx documentation generation (closes #2991997) changeset 17994bf95d6a ([doc] update Session documentation) added cubicweb.server.session.Session autodoc to the book. This caused errors from sphinx due to a section title not being allowed in the class's docstring.

[test/ldap] fix ldap tests - make sure the url is properly updated on database setup (when the test database already exists and the ldap URI has changed) - fix the ldapuser test setup process.

[test] fix unittest_schemaserial.py A spurious add permission has been added in expected result by d988eec2d5d3

[test] make unittest_schemaserial.py runnable with python used to run fine only when launched using pytest

[facet] use facet name as input name for text widget (eg has_text) we need to ba able to distinguish between text-inputs so we can write a widget for a single facet that uses more than one text input. This fix consists in the diff in cubicweb.facts.js; modifications in facet.py result from this former.

[web doctype] don't give through reset_xmldecl to avoid double deprecation warning

[web doctype test] don't give reset_xmldecl to avoid deprecation warning

[devtool] randomise available ports search in http test This lowers the chance of parallel tests to race for the same port.

[facet] don't crash if no title specified on a facet and filtered rset is empty. Closes #2587883

[css, html] add a css_class attribute on Button, allowing to change easily default CSS class for buttons (think orbui integration)

[view] add 2 missing spaces before the previous link

[server/repository] Go through the repo to close pyro sessions Turns out session.close() doesn't DTRT.

fix migration from pre-3.13.1 versions (closes #2846978) Need to check the existence of the asource column before the first call to eid_type_source.

[constraint] more robust unicity constraint failures reporting for end-users Postgres or Sqlserver have limits on the index names (around resp. 64 and 128 characters). Because `logilab.database` encodes the `unique together` constraint rtypes in the index names, we sometimes get truncated index names, from which it is impossible to retrieve all rtypes. In the long run, the way such index are named should be changed. In the short term, we try to reduce the end-user confusion resulting from this design flaw: * in source/native, the regex filtering ``IntegrityError`` message does not impose an `_idx` suffix, which indeed may be absent (the result being an UI message that resembles a catastrophic failure), * also we avoid including a trailing " (double quote) from the error message * in entities/adapters, the well-named ``IUserFriendly`` adapter is made a bit smarter about how to handle missing rtypes. * the adapter also always produces a global message explaining the issue (and the fact that sometimes, the user is not shown all the relevant info) * i18n is updated Closes #2793789

[hooks/security] Streamline attributes default permission check. The current default permission on attributes delegates the check to the entity permission update policy. Since this is already checked it can be skipped. The equality comparison will work, even with a deserialized schema, because the default update perm is:: ('managers', ERQLExpression(Any X WHERE U has_update_permission X, X eid %(x)s, U eid %(u)s)) which will always be deserialized in this order (groups first). However this is a slight semantic change: entity type level 'update' permissions can now be effectively used to encode update-time rules if the default attribute permissions are used (before this change, the 'update' rules at entity type level were fired at creation time). Closes #2930861.

[test/schemaserial] swap got/expected to get nicer unittest2 diagnostics Prepares #2965518.

[repository] drop safe attribute on ``internal_cnx`` People that need to disable hook can do it explicitly anyway.

[dbapi] deprecated the dbapi

[repoapi] deprecate dbapi compat method

[testlib] deprecated the older api to access the repo.

[connection] deprecated free_cnset and set_cnxset

[session] deprecate all Connection related method on session

[session] drop dead _current_cnx_id Not used anymore for a few commit.

[session] privatise get_cnx and close_cnx The only user, repoapi now use ``new_cnx``

[doc] add documentation for the new API in test

[testlib] add an default testcase.adminaccess (and use it for default session) This adminaccess is the new offical way to get connection, and request on a repo.

[testlib] introduce a RepoAccess class to easily create connection and request Each RepoAccess hold a session for a user and three helper function to help create Connection, ClientConnection of WebRequest related to this session. related to #2920299

[repoapi] make ClientConnection.__enter__ return self This allow the standard idiom:: with repoapi.connect(repo, login='babar', passwork='elephant') as cnx: cnx.execute(…)

[documentation] describe repoapi and web side change. Short version explaining what object replace what and that BC existes for a few version.

[testlib] use internal_cnx instead of internal_session internal_session is deprecated.

[repository] add an ``internal_cnx`` method to replace ``internal_session`` Accessing the repo through a Session is deprecated. We need an easy replacement for ``internal_session``. This API change was a good occasion to stop disabling integrity hook by default. This is huge source of bug in user-code. related to #2503918

[connection] enforce that a connection must be open to be used The same than for ClientConnection, we ensure the connection is used inside a its context. .. note:: We may rely on that for ClientConnection and remove de dedicated code in Client Connection but I prefer the current explicite and duplicated version for now.

use standalone Connection to Client Connection

[session] add a new_cnx factory Having user-code importing cubicweb.server.session.Connection is inconvenient. We add a simple factory fonction on the session.

[connection] allow simple instantiation of standalone Connection Such connection will automatically pick a connection id. Note, They are not automatically closed on session close. But they will fails to grab new cnxset once the session is closed.

[connection] handle and explicitly life cycle on Connection Like ClientConnection, Connection object need to be explicitly started and stop. They aims to be used as context manager.

[sesion] distinction between Connection handled by the session and other. Not mixing the new and backward compat approach seems a good idea. Let's enforce it.

[session] replace _clear_thread_storage with close_cnx There is not good reason to keep two distinct method.

[session] explicitly take Connection object in close_cnx Now that ClientConnection explicitly reference and use the Connection object we do not need to use connectionid here. We can safely change this signature, ClientConnection is the only use of close_cnx for now.

[session] drop the Session._clear_cnx_storage method It is just cnx.clear() now.

[session] wrap too long line Too long line is too long. (Confucius 503 BC)

[session] gather close_cnx with get_cnx and set_cnx They do the same kind of operation and deserve to be grouped together.

[client-connection] remove the _srv_cnx usage It does not do anything special now that we use explicite Connection object with automatic cnx_set handling.

[client-connection] explicitly check that the client-connection is open The check is also perform by the _srv_cnx property. But we do not need those property anymore.

[client-connection] handle the lack of connection id while not open the connection id is computed at opening and forgotten when closing. We can't rely on it in various messages … like the "connection closed" exception.

[connection] transparent cnx_set handling Connection object while take cares of there cnxset themself (as dbapi connection does). The ``set_cnxset`` and ``free_cnxset`` operation are still available for backward compatibility purpose. The ``_auto_free_cnx_set`` is introduced to handle mixed usage. A new context manager ``connection.ensure_cnx_set`` is added for code that access ``cnx.cnxset`` directly and are not wrapped in any specific ``Connection`` method. A ``_with_cnx_set`` decorator is used on all Connection method that need a cnxset.

[client-connection] explicit the client part in __repr__ Now that we have real server side connection we need to remove ambiguity.

[ClientConnection] directly use the Connection object to access the database Now that Connection are a full featured standalone object we can directly reference and use it in the ClientConnection instead of using the session. The session object is kept around for a while to perform various utility role.

[connection] invert __init__ parameter Takes session first. At some point, the connection_id will become optional for Connection created explicitly.

[service] enforce that Service argument and return are json-serialisable The call_service API need to be able to run through RPC. So we ensure front start that it is possible serialise both input and output.

[connection] move call_service on Connection This is the last step toward standalone transaction.

[connection] move the commit method on Connection object One step closer of standalone Connection!

[connection] move the rollback method on Connection object One step closer of standalon Connection.

[connection] pass a Connection object to RQLRewriter RQLRewriter can now directly use a Connection object. No need for specific handling session side.

[connection] Connection.execute touch Session timestamp This allow to keep Session alive while using Connection object only for database access.

[connection] Connection now call _set_user to CWUser object linked to itseld Session.user is linked to the session.

[server/session] better faking of user for InternalManager

[connection] move execute All necessary method are now available on the Connection object. So the Connection can handle RQL execution itself.

[connection] mark Connection as "not a request" The ORM use this information. Having a wrong value will lead to crash.

[server/session] ensure appobject obtained from the session are linked to the session

[connection] have rql_rewriter accession on Connection Code expecting a session search the RQLRewriter on this attribute.

[connection] move local_perm_cache management on Connection It belongs here anyway.

[session] use a rich object for timestamp This allows the Transaction to keep a reference to it

[migraction] do not access session.data directly use set_shared_data and get_shared_data instead

[connection] move relation management method

[connection] give access to shared data method A reference to session data are kept

[connection] move relation cache method to Connection needed for execute (and need execute themself...)

[connection] give access to session execute from connection This allows multiple function that both requires an execute method and are required by the execute method.

[connection] move last part of undo logic in connection Now that Connection have a reference to the repo it can handle all of it itself

[connection] initialize connection.user and connection.lang A RequestSessionBase need a user and lang. For now we use the session ones. Later the Connection will have it's own.

[Connection] inherit from RequestSessionBase This contains a lot method. We need them to move more code from Session to connection.

[multi-sources] drop multi-sources related test. another branch is removing the multi-source itself. We do not want to bother fixing those test.

Use new repoapi for the web stack The publisher now link repoapi.ClientConnection to request. and explicitly control there scope. Web side, appobject._cw.cnx is now a repoapi.ClientConnection. This actually kill webonly possibility until the repoapi is able to use some RPC. The change in the authentication stack is very hasty and need cleanup

PARTIAL: Using the repoapi in test Test now use the repoapi.ClientConnection to access the repo. This is very big change. The current method to access the repo are kept for backward compatibility. The new methods will be introduced much later. The TestCase keep a ClientConnection To an admin session for the whole test. This changeset does not makes all tests pass without the next one that change set htt without the next one that change the http stack

[web/test] drop proff of concept Facebook login in test The authentication stack will change. The change in API will requires rework of user cube anyway.

[test/dbapi] do not rely on the Testcase provided cnx We plan to use the repoapi in the test. So we need the DBAPI test to explicitly create a DBAPI connection.

[dbapi] explicitly use the DBAPI version of CubicwebRequestBase As we plan to use a ConnectionCubicwebRequestBase by default we better have the dbapi explicitly asking for a DBAPI implementation.

[web/application] drop unused import unused since 3c85e734ce00

[web/test] properly reset the request connection related attribute A new request does not have those attribute to None. There have special value that we now properly reinstall.

[web/test] drop two undocumented suspicious regression test Those two test are not documented and use the API in the wrong way. They do not raise exception by chance. the current request signature is:: def request(self, rollbackfirst=False, url=None, headers={}, **kwargs): So the RQL have nothing to do here.

[webrequest] introduce an alternative implementation using the repoapi DBAPI specific parts of ``CubicWebRequestBase`` are extracted in a ``DBAPICubicWebRequestBase`` decicated class and a new ``ConnectionCubicWebRequestBase`` is introduced that provide the same services but using a repoapi.ClientConnection as data source. For now the ``DBAPICubicWebRequestBase`` is still used by default.

[repoapi] add an anonymous_cnx function This function return an anonymous ClientConnection. It aims to replace ``dbapi.anonymous_session``.

[repoapi] add a connect function This function takes a repo and authentication information to open a new Session and return a ClientConnection associated to it.

[client-connection] add an auto-close property for ClientConnection The next commit introduce a connect function that open a new Session and return an associated ClientConnection. The Session should not be used by anything else than the created ClientConnection, so we want to close it at the same time than the ClientConnection. The implementation in this changeset is simplistic. We probably want to move this notion in the session itself. This be improved once we have server side Connection

[cwuser] make CWUser callable, returning self for dbapi compatibility In the dbapi, Connection and Session have a ``user`` method to generated a user for a request In the repo api, Connection and Session have a user attribute inherited from SessionRequestBase prototype. This ugly hack allows to not break user of the user method. To ease transition from dbapi to repoapi we need the ClientConnection to be as compatible as possible with the dbapi. Adding this method goes in this direction. It'll get deprecated in the deprecation wave that will conclude the repoapi refactoring. related to #2503918

[client-connect] drop rqlst on rset returned client side The DBAPI used to drop the RQL syntax tree on rset to save bandwidth. We could stop doing it for In-memory client connection. However keeping the syntax tree leads to multiple tests failures. So we keep dropping it as the DBAPI always did. related to #2503918

[client-connection] add a repo property for dbapi compatibility To ease transition from dbapi to repoapi we need the ClientConnection to be as compatible as possible with the dbapi. Adding this method goes in this direction. It'll get deprecated in the deprecation wave that will conclude the repoapi refactoring. related to #2503918

[client-connection] add a connection property for dbapi compatibility To ease transition from dbapi to repoapi we need the ClientConnection to be as compatible as possible with the dbapi. Adding this method goes in this direction. It'll get deprecated in the deprecation wave that will conclude the repoapi refactoring. related to #2503918

[client-connection] add a sessionid property for dbapi compatibility To ease transition from dbapi to repoapi we need the ClientConnection to be as compatible as possible with the dbapi. Adding this method goes in this direction. It'll get deprecated in the deprecation wave that will conclude the repoapi refactoring. related to #2503918

[client-connection] add a cursor() method for dbapi compatibility To ease transition from dbapi to repoapi we need the ClientConnection to be as compatible as possible with the dbapi. Adding this method goes in this direction. It'll get deprecated in the deprecation wave that will conclude the repoapi refactoring. related to #2503918

[client-connection] add a request() method for dbapi compatibility To ease transition from dbapi to repoapi we need the ClientConnection to be as compatible as possible with the dbapi. Adding this method goes in this direction. It'll get deprecated in the deprecation wave that will conclude the repoapi refactoring. related to #2503918

[repoapi] introduce a basic ClientConnection class This is the new official way to access the repo from client side. It still access Session object directly as the server side connection is not up yet (and it's not up because it would have no user). Multiple follow up commit will install compatibility with the DBAPI. This will ease the migration from dbapi to repoapi. ClientConnection has no user yet but later commit will use it in the whole Web stack. related to #2503918

[req] add a _set_user method RequestSessionBase usally need to recreate a new CWuser appobject linked to themself. We add it to the base class to avoid multiple redifinition.

[server/session] add a login property session.login is a DBAPISession attribute. Having it on server side session will helps the rework of the API to access repository. The new schema drop the concept of DBAPISession and use server side session for the same purpose. related to #2503918

[server/session] allow access to session id using sessionid session.sessionid is a DBAPISession attribute. Having it on server side session will helps the rework of the API to access repository. The new schema drop the concept of DBAPISession and use server side session for the same purpose. related to #2503918

[server/session] Implement anonymous_session Now we have a simple rule to compute if a session is anonymous we can implement the property for server session too. Having it on server side session will helps the rework of the API to access repository. The new schema drop the concept of DBAPISession and use server side session for the same purpose. Related to #2953943 Related to #2503918

[server/session] do not clear session.cnx if unrelated to cnx actually cleared The cnx parameter may be different from the one actually loaded for the current thread. This will be possible in future commit when you close a client connection for example.

[dbapi] move ProgrammingError into cubicweb module A new ``repoapi`` will be introduced as a replacement for the dbapi. It will need ProgrammingError too. Related to #2503918

[repoapi] move get_repository function into a new repoapi module This new module aims to host the function of the new API replacing the old DBAPI. `get_repository` is still needed hence moved to the new module. Related to #2503918

[testlib] gather all repository access logic in one place Refactoring of the repository access API in test is imminent. We plan to move from the "old" dbapi to the new repoapi. Gathering all impacted method in one place help to understand how all those method interact and help readability for both patch and resulting code. No code change is done at all in this changeset. The refactoring will code later.