[date picker] revert #ec65ca70aac9 which breaks the date picker (closes #3182844)

[merge] backport 3.17 fixes into the future 3.18

Added tag cubicweb-centos-version-3.17.7-1, cubicweb-version-3.17.7, cubicweb-debian-version-3.17.7-1 for changeset 483181543899

[pkg] prepare 3.17.7

i18n update

fix typos in workflow constraint error messages

[migractions] rschema.final.inlined -> rschema.inlined Closes #3153086.

[server] Make internal sessions not reset 'safe'-ness on first commit Increment the ctx_count when disabling integrity hooks so the next commit/rollback doesn't reset our transaction and magically turn us into a safe session. Closes #3168027

[facets] Correctly replace old 'edit box' HTML on facet-induced page refresh (closes #3161100)

[debian] replace find | xargs rm with find -delete in cube skeleton. Closes #3161797

[debian] don't require (fake)root to run the clean target. Closes #3161797 It works just fine as a normal user.

[debian] Add ${misc:Depends} to cube packaging skeleton. Closes #3161797 Best current practice is to have that in Depends in all packages using debhelper.

[devtools] fix race when creating backupdir If somebody else creates backupdir between our stat() and mkdir() (like, say, another test running in parallel), we shouldn't crash.

[staticcontrollers] Raise Forbidden, not Unauthorized Unauthorized means "log in to get access", as it results in a HTTP 401. Here, the error is pretty much permanent, and returning 401 instead of 403 confuses things terribly. (This seems to be a pretty widespread confusion :/)

[web] allow /data/ url again (closes #2464798) This feature was broken by 65fecbeb9c3a (which fixed another one itself).

[utils] fix typos in make_uid docstring

[c-c i18ncubicweb] fix crash, closes #3096647

[rql2sql] fix bad sql generated when outer joining 'identity' relation and lhs var comes from a subquery. Closes #3099418 The generated SQL was attempting to access table.cw_eid which doesn't exist.

[web] stop using deprecated StatusResponse. Closes #3098215

[deprecation] add (approximate) version number to deprecation message and set proper stacklevel

[schema] fix spurious warning when rqlexpr/constraint mainvars specify a non predefined variable. Closes #3098165

[repository] properly use IUserFriendlyError when UniqueTogetherError is raised during entity update. Closes #3096638

[deprecation] add cw version number to the deprecation message and help user to understand the change

[datafeed] fix crash due to bad http_timeout handling. Closes #3096585

[date picker widgets] properly distinguish DOM id and input name. Closes #3096575

[date picker widgets] use bare jquery expression rather than jqNode

Use return in CnxSetTracker context manager methods Just in case.

[web/views] make PrimaryView.entity_call accept kwargs It's useful when using this view in a different context to have it not crash on random arguments.

[ldapparser] demote some logs from warning to debug (closes #2713671)

[doc] add a what's new file for 3.18

merge stable in default (3.18) branch

Added tag cubicweb-centos-version-3.17.6-1, cubicweb-version-3.17.6, cubicweb-debian-version-3.17.6-1 for changeset 5b9fedf67a29

[pkg] prepare 3.17.6

Do not use cubicweb-card in rest_path tests Card overrides the rest_path method, so it does not make sense to use it in cw tests.

[inlined form field] fix regression introduced in 3.16.4/570208f74a84. Closes #3064653 In the above changeset, we changed ordered entities to be edited according to mandatory inlined constraints, though this leads to unpredictable order in cases where there are no constraint. This leads to case where inlined relation field may think there is no value related to a mandatory relation, while this is simply because there is a value specified but to an entity that will be created later (together with the relation). To fix this, we've to use the fact that the RelationField.process_form_value return None in cases where there is some entity to be created, while an empty set when there are no linked entity. The new no_value() method allows to differentiate between those different notion of 'no-value'.

[testlib] temporary_appobjects should call __registered__ if it exists. Closes #3064075

[entities] properly escape in EmailAddress.printable_value when format is html. Closes #3064025

Added tag cubicweb-version-3.17.5, cubicweb-debian-version-3.17.5-1, cubicweb-centos-version-3.17.5-1 for changeset 15dd5b37998b

[pkg] prepare 3.17.5

[pkg] fix debian packaging (closes #3058542) - remove useless Python 2.4 reference, - simplify dh_install configuration to install files consistently instead of having half in /usr/lib/pythonXX and half in /usr/share/pyshared, - incidentally, this makes the package work on squeeze. Regression from eaa58d1c7d5f

[testing] add missing generate_bigint method to ValueGenerator. Closes #3059327

[rql / querier] fix bad interpretation of some RQL SET query when some neged relation is involved. Closes #3058527.

merge stable in default (3.18) branch

Added tag cubicweb-version-3.17.4, cubicweb-debian-version-3.17.4-1, cubicweb-centos-version-3.17.4-1 for changeset c7ba8e5d2e45

[i18n] cleanup dead code also fix small bug introduced in b0f6e8c14e7f

[i18n test] hack to make i18n tests run using a Python interpreter they used to work fine when executed via pytest only. This should be removed as soon as logilab.common.registry is fixed.

[i18ncube] fix crash due to duplicated messages in generated schema.pot Add a test case and fix regression introduced in e713c47a993d. Relates to #2811282

[i18n test] load_po: fix bug in case of multi-lines msgid and ensure there are no duplicated messages

[i18n test] simplify test cube a bit by only providing a views module, no package

[web] consider inlined relations in has_editable_relation. Closes #3049970 In some cases where the user can only add/edit inlined relations (though actually edit perms of the inlined relation doesn't seem checked), the "modify" action doesn't appear while it should.

[web test] silent 3.13 warning

[editcontrollers] Account for role in the ordering of entities (Complements #3031719) Only role='object' was correctly accounted for by 570208f74a84

[migration.sync_schema_props_perms] ensure all participants to a unique together constraint are there before adding CWUniqueConstraint. Closes #3038345

[session commit] save back exception context to avoid potentiel cluttering if some revert operation raise an exception

[pkg] python-central has been removed from Debian

[pkg] prepare 3.17.4

[pkg] merge centos 3.17.1-2 in stable (prepare 3.17.4)

Added tag cubicweb-centos-version-3.17.1-2 for changeset 965f894b63cb

[rqlrewrite] fix rqlrewrite unpredictability vs relation sharing. Closes #3036144 The relations index used to determine if relation may be shared only considered a single relation node per relation type. So when the same relation type occurs several time, dict order give unpredictable result. We shall properly consider all relations instead. Tentative test case included (but bug reproduction is by definition unpredictable...)

[rqlrewrite] rewrite doesn't need a solutions argument, always use the Select'ones Also, do a copy systematically else it may lead to empty solutions depending on the rewrite path. Test case tentative included in the following changeset.

[rql rewrite] equivalent but much simpler flow

[pkg] Prepare a new RPM release

[pkg] Add log directory to rpm

[repo] normalize ValidationError on edited entity (closes #2509729) In CubicWeb, ValidationError.entity MUST be the eid of the involved entity, not the entity iteself (as done by yams).

[doc] one must now manipulate the req.session.data dict (closes #2842345) instead of deprecated get/set/del_session_data methods.

[datafeed] add a timeout config option (closes #2745677) So a HTTP GET do not hang forever in the datafeed looping task.

[dataimport] ucsvreader should skip empty lines unless specified otherwise. Closes #3035944

[querier] Add timings to debug prints (DBG_RQL) this may ease to spot some problematic queries

[editcontrollers] Ensure entities are created in an order satisfying schema constraints. Closes #3031719 changes below are also necessary to make the whole thing works: * stop considering InlinedFormFile as eidparam field since they don't hold any value * rework 'pendingfields' handling to have separate processing of inlined fields whose subject entity is created during the edition

[editcontroller] extract RQLQuery.set_attribute/set_inlined methods from handle_field/handle_inlined_relation

[editcontroller] req=self._cw makes things easier to read

[facets,js] fix bogus checkbox icon appearing after the first interaction (closes #2790332) This is because of the hardcoding of the data/ url.

fix typos in docstring, doc and comments

[debian] remove workaround for broken sphinx/jquery combination sphinx in squeeze was updated: http://packages.qa.debian.org/s/sphinx/news/20130112T154737Z.html

[schema] mark CWDataImport as an internal type. Closes #3025536

[pkg] remove deprecated dependency on pysqlite2

Fix two crashes in db-check (closes #3024964) Also improves various debugging messages.

[rql rewrite] may_be_shared_with should consider relation's scope (closes #3024730) When a relation's scope is not the current statement, it must not be shared (instead of comparing the statement).

[rql rewrite] fix may_be_shared_with method so that it actually considers all variable infos and not only the first one

[schema/security] add __hash__ to rql expression. Closes #3013535 This is required so that when some rql expression participate to a dictionary key, only the expression is considered (consistent with comparison). This behaviour is expected in security at least, see the related bug for instance. This scramble msplanner unit tests a bit.

[rql rewrite] move some code from querier to rqlrewrite where it makes more sense. Also, make some minor cleanup/refactoring on the way and try to enhance docstrings. Relates to #3013535

Do not compute actions list in TableLayout view when display_actions attribute is unset Closes #3007281

Use the list of cubes from the filesystem when reading the schema from the filesystem Closes #3007259

[migration] when adding a cube, skip infered relations (closes #3005576) Skipped infered relations are recalculated later.

[3.17 migration] when some cube is missing, add_cube raise ConfigurationError, not ImportError. Closes #2981477

[migraction] rename_entity_type simply warn if old entity type isn't in the schema. Closes #3004069

[deprecation] fix uihelper deprecation warning, use the rtag name, not the class, so it may be copy/pasted

[test] makes fti test order-agnostic (closes #3005633)

[web/views/staticcontrollers] Make ConcatFilesHandler write to a tempfile In order to be more resistant to errors such as ENOSPC, and against different threads accessing the same static data, we: - concatenate data files to a temporary file, and rename it once we're done - delete the tempfile in case of errors - protect the creation of the cache_concat file with a lock Closes #3005547

[test] make unittest_schema.py compatible with cubicweb-file 0.14

merge 3.16.6 branch in 3.17

Added tag cubicweb-centos-version-3.16.6-1, cubicweb-debian-version-3.16.6-1, cubicweb-version-3.16.6 for changeset b4ccaf13081d

Prepare 3.16.6

[devctl] properly generate i18n messsages for cubes that uses uicfg instances (closes #2811282)

[test/devctl] add a test case for i18ncube command

Protect against crash in the `relation_possible` predicate with ambiguous relations. It was still possible to get a KeyError when using relations that are ambiguous at both ends, like (Executable, version_of, Program) & (Version, version_of, Project). Closes #3010148.

[test] give a non-ambiguous order to sync_schema assertion (closes #3001959) Sorting by ordernum alone is unstable since several values have the same ordernum. ordernum + name should be stable.

Added tag cubicweb-version-3.17.3, cubicweb-debian-version-3.17.3-1, cubicweb-centos-version-3.17.3-1 for changeset 32b4d5314fd9

[test] typo

[schema,server] add a security debugging aid (closes #2920304) - Add a DGB_SEC debugging flag (to be used with set_debug/debugged). - Add a context manager (tunesecurity) to filter security assertions. Note: this does not address all read-security mecanisms.

[etwist] fix handling of multiple files per field html5 permits multiple files uploads, which can be expressed as:: <input type='file' multiple='multiple' /> This changeset avoids previous crash. Nothing is changed when a single file is uploaded (backward compat is thus preserved). When multiple files are uploaded for a single html input tag, the corresponding web request form key receives a list of tuples like [('filename-1', IStream1), ('filename-2', IStream2), ...]. closes #2847207.

[merge] start 3.18 development

[pkg] prepare 3.17.3

[pkg] Remove obsolete ubuntu hardy packaging It's been EOL for a while.

[book] fix sphinx documentation generation (closes #2991997) changeset 17994bf95d6a ([doc] update Session documentation) added cubicweb.server.session.Session autodoc to the book. This caused errors from sphinx due to a section title not being allowed in the class's docstring.

[test/ldap] fix ldap tests - make sure the url is properly updated on database setup (when the test database already exists and the ldap URI has changed) - fix the ldapuser test setup process.

[test] fix unittest_schemaserial.py A spurious add permission has been added in expected result by d988eec2d5d3

[test] make unittest_schemaserial.py runnable with python used to run fine only when launched using pytest

[facet] use facet name as input name for text widget (eg has_text) we need to ba able to distinguish between text-inputs so we can write a widget for a single facet that uses more than one text input. This fix consists in the diff in cubicweb.facts.js; modifications in facet.py result from this former.

[web doctype] don't give through reset_xmldecl to avoid double deprecation warning

[web doctype test] don't give reset_xmldecl to avoid deprecation warning

[devtool] randomise available ports search in http test This lowers the chance of parallel tests to race for the same port.

[facet] don't crash if no title specified on a facet and filtered rset is empty. Closes #2587883

[css, html] add a css_class attribute on Button, allowing to change easily default CSS class for buttons (think orbui integration)

[view] add 2 missing spaces before the previous link

[server/repository] Go through the repo to close pyro sessions Turns out session.close() doesn't DTRT.

fix migration from pre-3.13.1 versions (closes #2846978) Need to check the existence of the asource column before the first call to eid_type_source.

[constraint] more robust unicity constraint failures reporting for end-users Postgres or Sqlserver have limits on the index names (around resp. 64 and 128 characters). Because `logilab.database` encodes the `unique together` constraint rtypes in the index names, we sometimes get truncated index names, from which it is impossible to retrieve all rtypes. In the long run, the way such index are named should be changed. In the short term, we try to reduce the end-user confusion resulting from this design flaw: * in source/native, the regex filtering ``IntegrityError`` message does not impose an `_idx` suffix, which indeed may be absent (the result being an UI message that resembles a catastrophic failure), * also we avoid including a trailing " (double quote) from the error message * in entities/adapters, the well-named ``IUserFriendly`` adapter is made a bit smarter about how to handle missing rtypes. * the adapter also always produces a global message explaining the issue (and the fact that sometimes, the user is not shown all the relevant info) * i18n is updated Closes #2793789

[hooks/security] Streamline attributes default permission check. The current default permission on attributes delegates the check to the entity permission update policy. Since this is already checked it can be skipped. The equality comparison will work, even with a deserialized schema, because the default update perm is:: ('managers', ERQLExpression(Any X WHERE U has_update_permission X, X eid %(x)s, U eid %(u)s)) which will always be deserialized in this order (groups first). However this is a slight semantic change: entity type level 'update' permissions can now be effectively used to encode update-time rules if the default attribute permissions are used (before this change, the 'update' rules at entity type level were fired at creation time). Closes #2930861.

[test/schemaserial] swap got/expected to get nicer unittest2 diagnostics Prepares #2965518.

[repository] drop safe attribute on ``internal_cnx`` People that need to disable hook can do it explicitly anyway.

[dbapi] deprecated the dbapi

[repoapi] deprecate dbapi compat method

[testlib] deprecated the older api to access the repo.

[connection] deprecated free_cnset and set_cnxset

[session] deprecate all Connection related method on session

[session] drop dead _current_cnx_id Not used anymore for a few commit.

[session] privatise get_cnx and close_cnx The only user, repoapi now use ``new_cnx``

[doc] add documentation for the new API in test

[testlib] add an default testcase.adminaccess (and use it for default session) This adminaccess is the new offical way to get connection, and request on a repo.

[testlib] introduce a RepoAccess class to easily create connection and request Each RepoAccess hold a session for a user and three helper function to help create Connection, ClientConnection of WebRequest related to this session. related to #2920299

[repoapi] make ClientConnection.__enter__ return self This allow the standard idiom:: with repoapi.connect(repo, login='babar', passwork='elephant') as cnx: cnx.execute(…)

[documentation] describe repoapi and web side change. Short version explaining what object replace what and that BC existes for a few version.

[testlib] use internal_cnx instead of internal_session internal_session is deprecated.

[repository] add an ``internal_cnx`` method to replace ``internal_session`` Accessing the repo through a Session is deprecated. We need an easy replacement for ``internal_session``. This API change was a good occasion to stop disabling integrity hook by default. This is huge source of bug in user-code. related to #2503918

[connection] enforce that a connection must be open to be used The same than for ClientConnection, we ensure the connection is used inside a its context. .. note:: We may rely on that for ClientConnection and remove de dedicated code in Client Connection but I prefer the current explicite and duplicated version for now.

use standalone Connection to Client Connection

[session] add a new_cnx factory Having user-code importing cubicweb.server.session.Connection is inconvenient. We add a simple factory fonction on the session.

[connection] allow simple instantiation of standalone Connection Such connection will automatically pick a connection id. Note, They are not automatically closed on session close. But they will fails to grab new cnxset once the session is closed.

[connection] handle and explicitly life cycle on Connection Like ClientConnection, Connection object need to be explicitly started and stop. They aims to be used as context manager.

[sesion] distinction between Connection handled by the session and other. Not mixing the new and backward compat approach seems a good idea. Let's enforce it.

[session] replace _clear_thread_storage with close_cnx There is not good reason to keep two distinct method.

[session] explicitly take Connection object in close_cnx Now that ClientConnection explicitly reference and use the Connection object we do not need to use connectionid here. We can safely change this signature, ClientConnection is the only use of close_cnx for now.

[session] drop the Session._clear_cnx_storage method It is just cnx.clear() now.

[session] wrap too long line Too long line is too long. (Confucius 503 BC)

[session] gather close_cnx with get_cnx and set_cnx They do the same kind of operation and deserve to be grouped together.

[client-connection] remove the _srv_cnx usage It does not do anything special now that we use explicite Connection object with automatic cnx_set handling.

[client-connection] explicitly check that the client-connection is open The check is also perform by the _srv_cnx property. But we do not need those property anymore.

[client-connection] handle the lack of connection id while not open the connection id is computed at opening and forgotten when closing. We can't rely on it in various messages … like the "connection closed" exception.

[connection] transparent cnx_set handling Connection object while take cares of there cnxset themself (as dbapi connection does). The ``set_cnxset`` and ``free_cnxset`` operation are still available for backward compatibility purpose. The ``_auto_free_cnx_set`` is introduced to handle mixed usage. A new context manager ``connection.ensure_cnx_set`` is added for code that access ``cnx.cnxset`` directly and are not wrapped in any specific ``Connection`` method. A ``_with_cnx_set`` decorator is used on all Connection method that need a cnxset.

[client-connection] explicit the client part in __repr__ Now that we have real server side connection we need to remove ambiguity.

[ClientConnection] directly use the Connection object to access the database Now that Connection are a full featured standalone object we can directly reference and use it in the ClientConnection instead of using the session. The session object is kept around for a while to perform various utility role.

[connection] invert __init__ parameter Takes session first. At some point, the connection_id will become optional for Connection created explicitly.

[service] enforce that Service argument and return are json-serialisable The call_service API need to be able to run through RPC. So we ensure front start that it is possible serialise both input and output.

[connection] move call_service on Connection This is the last step toward standalone transaction.

[connection] move the commit method on Connection object One step closer of standalone Connection!

[connection] move the rollback method on Connection object One step closer of standalon Connection.

[connection] pass a Connection object to RQLRewriter RQLRewriter can now directly use a Connection object. No need for specific handling session side.

[connection] Connection.execute touch Session timestamp This allow to keep Session alive while using Connection object only for database access.

[connection] Connection now call _set_user to CWUser object linked to itseld Session.user is linked to the session.

[server/session] better faking of user for InternalManager

[connection] move execute All necessary method are now available on the Connection object. So the Connection can handle RQL execution itself.

[connection] mark Connection as "not a request" The ORM use this information. Having a wrong value will lead to crash.

[server/session] ensure appobject obtained from the session are linked to the session

[connection] have rql_rewriter accession on Connection Code expecting a session search the RQLRewriter on this attribute.

[connection] move local_perm_cache management on Connection It belongs here anyway.

[session] use a rich object for timestamp This allows the Transaction to keep a reference to it

[migraction] do not access session.data directly use set_shared_data and get_shared_data instead

[connection] move relation management method

[connection] give access to shared data method A reference to session data are kept

[connection] move relation cache method to Connection needed for execute (and need execute themself...)

[connection] give access to session execute from connection This allows multiple function that both requires an execute method and are required by the execute method.

[connection] move last part of undo logic in connection Now that Connection have a reference to the repo it can handle all of it itself

[connection] initialize connection.user and connection.lang A RequestSessionBase need a user and lang. For now we use the session ones. Later the Connection will have it's own.

[Connection] inherit from RequestSessionBase This contains a lot method. We need them to move more code from Session to connection.

[multi-sources] drop multi-sources related test. another branch is removing the multi-source itself. We do not want to bother fixing those test.

Use new repoapi for the web stack The publisher now link repoapi.ClientConnection to request. and explicitly control there scope. Web side, appobject._cw.cnx is now a repoapi.ClientConnection. This actually kill webonly possibility until the repoapi is able to use some RPC. The change in the authentication stack is very hasty and need cleanup

PARTIAL: Using the repoapi in test Test now use the repoapi.ClientConnection to access the repo. This is very big change. The current method to access the repo are kept for backward compatibility. The new methods will be introduced much later. The TestCase keep a ClientConnection To an admin session for the whole test. This changeset does not makes all tests pass without the next one that change set htt without the next one that change the http stack

[web/test] drop proff of concept Facebook login in test The authentication stack will change. The change in API will requires rework of user cube anyway.

[test/dbapi] do not rely on the Testcase provided cnx We plan to use the repoapi in the test. So we need the DBAPI test to explicitly create a DBAPI connection.

[dbapi] explicitly use the DBAPI version of CubicwebRequestBase As we plan to use a ConnectionCubicwebRequestBase by default we better have the dbapi explicitly asking for a DBAPI implementation.

[web/application] drop unused import unused since 3c85e734ce00

[web/test] properly reset the request connection related attribute A new request does not have those attribute to None. There have special value that we now properly reinstall.

[web/test] drop two undocumented suspicious regression test Those two test are not documented and use the API in the wrong way. They do not raise exception by chance. the current request signature is:: def request(self, rollbackfirst=False, url=None, headers={}, **kwargs): So the RQL have nothing to do here.

[webrequest] introduce an alternative implementation using the repoapi DBAPI specific parts of ``CubicWebRequestBase`` are extracted in a ``DBAPICubicWebRequestBase`` decicated class and a new ``ConnectionCubicWebRequestBase`` is introduced that provide the same services but using a repoapi.ClientConnection as data source. For now the ``DBAPICubicWebRequestBase`` is still used by default.

[repoapi] add an anonymous_cnx function This function return an anonymous ClientConnection. It aims to replace ``dbapi.anonymous_session``.

[repoapi] add a connect function This function takes a repo and authentication information to open a new Session and return a ClientConnection associated to it.

[client-connection] add an auto-close property for ClientConnection The next commit introduce a connect function that open a new Session and return an associated ClientConnection. The Session should not be used by anything else than the created ClientConnection, so we want to close it at the same time than the ClientConnection. The implementation in this changeset is simplistic. We probably want to move this notion in the session itself. This be improved once we have server side Connection

[cwuser] make CWUser callable, returning self for dbapi compatibility In the dbapi, Connection and Session have a ``user`` method to generated a user for a request In the repo api, Connection and Session have a user attribute inherited from SessionRequestBase prototype. This ugly hack allows to not break user of the user method. To ease transition from dbapi to repoapi we need the ClientConnection to be as compatible as possible with the dbapi. Adding this method goes in this direction. It'll get deprecated in the deprecation wave that will conclude the repoapi refactoring. related to #2503918

[client-connect] drop rqlst on rset returned client side The DBAPI used to drop the RQL syntax tree on rset to save bandwidth. We could stop doing it for In-memory client connection. However keeping the syntax tree leads to multiple tests failures. So we keep dropping it as the DBAPI always did. related to #2503918

[client-connection] add a repo property for dbapi compatibility To ease transition from dbapi to repoapi we need the ClientConnection to be as compatible as possible with the dbapi. Adding this method goes in this direction. It'll get deprecated in the deprecation wave that will conclude the repoapi refactoring. related to #2503918

[client-connection] add a connection property for dbapi compatibility To ease transition from dbapi to repoapi we need the ClientConnection to be as compatible as possible with the dbapi. Adding this method goes in this direction. It'll get deprecated in the deprecation wave that will conclude the repoapi refactoring. related to #2503918

[client-connection] add a sessionid property for dbapi compatibility To ease transition from dbapi to repoapi we need the ClientConnection to be as compatible as possible with the dbapi. Adding this method goes in this direction. It'll get deprecated in the deprecation wave that will conclude the repoapi refactoring. related to #2503918

[client-connection] add a cursor() method for dbapi compatibility To ease transition from dbapi to repoapi we need the ClientConnection to be as compatible as possible with the dbapi. Adding this method goes in this direction. It'll get deprecated in the deprecation wave that will conclude the repoapi refactoring. related to #2503918

[client-connection] add a request() method for dbapi compatibility To ease transition from dbapi to repoapi we need the ClientConnection to be as compatible as possible with the dbapi. Adding this method goes in this direction. It'll get deprecated in the deprecation wave that will conclude the repoapi refactoring. related to #2503918

[repoapi] introduce a basic ClientConnection class This is the new official way to access the repo from client side. It still access Session object directly as the server side connection is not up yet (and it's not up because it would have no user). Multiple follow up commit will install compatibility with the DBAPI. This will ease the migration from dbapi to repoapi. ClientConnection has no user yet but later commit will use it in the whole Web stack. related to #2503918

[req] add a _set_user method RequestSessionBase usally need to recreate a new CWuser appobject linked to themself. We add it to the base class to avoid multiple redifinition.

[server/session] add a login property session.login is a DBAPISession attribute. Having it on server side session will helps the rework of the API to access repository. The new schema drop the concept of DBAPISession and use server side session for the same purpose. related to #2503918

[server/session] allow access to session id using sessionid session.sessionid is a DBAPISession attribute. Having it on server side session will helps the rework of the API to access repository. The new schema drop the concept of DBAPISession and use server side session for the same purpose. related to #2503918

[server/session] Implement anonymous_session Now we have a simple rule to compute if a session is anonymous we can implement the property for server session too. Having it on server side session will helps the rework of the API to access repository. The new schema drop the concept of DBAPISession and use server side session for the same purpose. Related to #2953943 Related to #2503918

[server/session] do not clear session.cnx if unrelated to cnx actually cleared The cnx parameter may be different from the one actually loaded for the current thread. This will be possible in future commit when you close a client connection for example.

[dbapi] move ProgrammingError into cubicweb module A new ``repoapi`` will be introduced as a replacement for the dbapi. It will need ProgrammingError too. Related to #2503918

[repoapi] move get_repository function into a new repoapi module This new module aims to host the function of the new API replacing the old DBAPI. `get_repository` is still needed hence moved to the new module. Related to #2503918

[testlib] gather all repository access logic in one place Refactoring of the repository access API in test is imminent. We plan to move from the "old" dbapi to the new repoapi. Gathering all impacted method in one place help to understand how all those method interact and help readability for both patch and resulting code. No code change is done at all in this changeset. The refactoring will code later.

[testlib] move repo and related attribute back on Instance instead of Class The repo and cnx was hold by the TestCase class to work as a cache and avoid recreation of the repo from scratch of each test. However since bad26a22fe29 the caching is done by the DatabaseHandler object and it is not necessary have a second layer of cache on the TestCase itself. We move the repo and cnx attribute back on the TestCase instance itself and make several class methods instance methods again. This will helps to change dbapi access in test from dbapi to repoapi. related to #2503918

[dbapi] makes anonymous_connection a computed property The current implementation is a boolean flag set manually by client code after connection creation. This led to different way to decide a anonymous_connection should be True (eg. different in the test than in the actual application code). It should not be client responsibility to set this flag. ``cnx.anonymous_connection`` is now a purely computed property. Connection with user in the "guests" group are anonymous, the other ain't. ``Session.anonymous_session`` is computed from ``cnx.anonymous_connection`` and get the updated behavior transparently. Closes #2953943

[webrequest] simplify set_session code Thanks to the previous changeset we are assured that session handed to set_session is full featured one. This allows a simpler code for this method. related to #2503918

[webrequest] set DBAPISession without cnx at initialisation time Such session are necessary for minimal use of a Request. Setting on by default allow simplification later linking with a full featured DBAPISession or equivalent. This was not possible before as all session was tracked by session manager. They are not tracked anymore since aa709bc6b6c1 and we can safely create them by default. related to #2503918

[testlib] rework request building in init_authentication The previous code was building a full authenticated request and tried to undo the authentication afterward. The new code just create a bare Request with no authenticated session linked yet. This is much closer of what the actual authentication process does.

[request] drop the user argument for set_session I see no code nor test that use this optional argument. removing it help to clean the session code. The set_session function will soon be deprecated anyway (at the same time than the dbapi) related to #2503918

[web-request] handle default language earlier We now read language negotiation header at initialisation time (previously done during ``set_session``). This simplify the set_session code for later reworking. There is no change in behavior, the user selected language continue to overwrite the one picked from the header/default when applicable. related to #2503918

[etwist] pass the repository to the root resource This is another step toward a cleaner instantiation scheme for the repo. The http test now pass an already created object to the etwist server instead of relying on the config cache. Related to #2249513

[repo] move repo.gc_stats to Service API (closes #2951068) We currently have a method on the repo that anyone can call. `call_service` gives a similar thing, modulo: * can be restricted access to manager only (thanks to selection on services), * less unrelated code on the repository class, * no need to fetch a reference to a repo object from the client side * (`req._cnx.repo` is not an API and config.repository() is on its way out). The old way to access this information (repo.gc_stats()) is deprecated.

[repo] move repo stats to Service (closes #2951067) We currently add a new method on repo that anyone can call. Call service allows to reach the same result with: * Restricted access to manager only (thanks to selection on services), * Less unrelated code on the repo. * No need to fetch a reference to a repo object from client side. `req._cnx.repo` is not an API and config.repository() is on its way out. The old way to access this information (repo.stats()) is deprecated.

[service] drop the asynchronous execution possibility Call_service was able of both sync and sync execution making the API confusing. There is not user of the async case. We drop the async argument in favor of synchronous execution only. This makes call_service the official API to call server side code from the client side. This is a remplacement for the usual monkey patching of the repo object. The zmq notification bus is a solid alternative for codes that needs to start an async execution.

[auth] pass `repo` instead of `vreg` to SessionManager and AuthenticationManager Those object actually need a repo object to connect and create new session object. They used to retrieve the repo object using ``vreg.config.repository()``. I'm trying to sanitise the initialisation stack and get ride of ``config.repository()``, a function that mix factory and cache. Retrieving the vreg from the repo is trivial (``repo.vreg``) so we now pass the repo object instead of the vreg.

web/application: instantiate the repository outside of CubicWebPublisher This improves decoupling and allows ``CubicWebPublisher`` user to choose which argument are used during ``Repository`` instantiation. In particular is allows caller to provide a ``TaskManager``. The `vreg` argument of publisher is made mandatory. See CubicWebPublisher docstring for details. Related to #2249513

remove vreg argument to CWPublisher The passing a vreg to CWPublisher was used by test to allow reused of an existing vreg. We dropped this feature two commit ago and can safely removes it. Removing this argument allows later cleanup and improvement on the CWPublisher and Repository front. closes #2944813

[devtools] vreg becomes a class ppty After multiple devtools refactoring, the vreg used for test is now *always* got from the repository. Making this explicit is a step toward simplification of the vreg handling during various initialisation phases. Related to #2944813

[devtools] drop unused `reset_schema` and `reset_vreg` class attribute They have no effect and no user... Related to #2944813

[connection] move security control logic on Connection The _security_enabled context manager now operate on connection. We have to keep a specific entry point in Session to ensure Connection object are properly cleaned up when using session to "manage" Connection live cycle. Related to #2503918

[connection] move hook control logic on Connection The _hook_control context manager now operate on connection. We have to keep a specific entry point in Session to ensure Connection object are properly cleaned up when using session to "manage" Connection live cycle. Related to #2503918

[session] update _hook_control docstring

[connection] reinstall cnx.data as cnx.transaction_data Too much code expects self._cw.data to be session data. backout 9b2f68916474 Related to #2503918 Related to #2912807

[connection] give access to is_internal_session boolean Needed if we are to use this object like we currently do with session. Related to #2503918

[req] drop from_controller on non WebRequest object (Closes #2901079) The `controller` concept is purely a web things. It does not belong to generic `req` object. The `build_url` code is slightly changed to handle that. This probably requires extended cleanup later. This is a barely used and very internal API. No deprecation period

[connection] add logging method on connection If we are to use this object like we currently do with session, we need to have the logging method as a lot of code use them.

rename server.session.transaction into server.session.connection The ongoing rework of the API to access the database include the splitting of Session in two objects: The ``Session`` object will only hold credential (user) and global session data. A new object will publicly emerge to handle a database access. This object is named ``Connection`` since that the way database accessors with the same property are named in other system. So we renamed the ``Transaction`` object into ``Connection``. The ``Transaction`` object have already grown in the direction of something directly usable by generic code, but the name ``Transaction`` is ill suited as such object can be used for multiple database transaction in a row. Related to #2503918

[application] call req.set_session in application.main_handle_request The Session handling chain is no more responsible for calling req.set_session. It just returns a valid session and lets the caller link it to the Request. This opens the way to explicitly creating and closing a connection/transaction in ``application.main_handle_request``. Related to #2503918

[session-handler] use session directly to update last usage We don't really need the WebRequest for that. Not using the WebRequest to access the cubicweb repository here will allow a delayed set_session. Related to #2503918

[application/connect] simplify connection logic ``application.connect`` now either sets a full featured ``DBAPISession`` to the ``WebRequest`` object or raises ``AuthenticationError``. The creation and usage of a fake DBAPISession is now handled by ``main_handle_request`` when needed. This means that fake DBAPISession are no longer tracked by the session manager and that user are not given anyway to retrieve them for a later request. This fake DBAPISession is still passed to ``core_handle`` because multiple cubes like registration or forgotten password need this behavior. We would like to get ride of it in the future. This clarification of the connection API greatly simplifies ``DBAPISession`` retrieval//creation process opening the way to improvements in this area. Related to #2503918

Drop hijack user (closes #2901093) Deprecated since 3.17.0. Dropping this function help the session cleanup business.

[sources] drop support for ldapuser source (closes #2936496) The ldapfeed source is a replacement for ldapuser. Use it instead.

merge stable back into default

3.17 is the new stable

Added tag cubicweb-version-3.16.5, cubicweb-debiann-version-3.16.5-1, cubicweb-centos-version-3.16.5-1 for changeset 810a05fba1a4

[pkg] prepare 3.16.5

Added tag cubicweb-version-3.17.2, cubicweb-debian-version-3.17.2-1 for changeset 195e519fe97c

[pkg] prepare 3.17.2

repository: make tests pass again The tests use mono-threaded pyro, which breaks assumptions made in e27337dfec8c. To fix that, remove sessions from the _pyro_sessions dict when they're closed, and force the test pyro client to actually disconnect when it's done.

repository: monkey patch pyro connection handling to detect clients going away When a pyro client goes away we need to close their session (to release their cnxset). Add a dict to the repository mapping pyro client threads to session object. Closes #2932058

notification: use viewargs for notif_entity_updated instead of transaction_data Now that notifications use separate sessions, they can't rely on the original transaction_data being around, so pass the data through view arguments instead, so the notification view knows what changed on updated entities. Closes #2936180

notification: properly handle cnx and lifetime of the hijacked session When we create a session for the notification rendering, we need to 1) give it a cnxset 2) commit and close it after we're done 3) restore the original session for subsequent notifications This changes the ordering of actual mail sending, since there are several different commits involved, but I don't see a way to fix that short of restoring hijack_user. Related to #2934523

notification: fix session creation (closes #2934523) Session() takes a user and repo, not repo and user.

Added tag cubicweb-version-3.17.1, cubicweb-debian-version-3.17.1-1, cubicweb-centos-version-3.17.1-1 for changeset f98d1c46ed9f

[cwctl] add configure command to cw-ctl (closes #2709702) usage: cw-ctl configure <appid> --param key=value,key2=value2 --param key3=value3

[cwctl] rename option no-post-create to no-db-create

[pkg] version 3.17.1 for rpm packages

[session] upgrade session closed error from Exception to SessionClosedError Exception is far too wide and we already have a SessionClosedError anyway. Closes #2897696

[pkg] prepare 3.17.1

[web/application] add some minimal documentation Some documentation to the Application main entry point.

[session] raise proper exception with get_tx is called on a closed session Prevent attribute error on _txs Closes #2897836

[view] typo in a docstring

[cwctl] pass missing inputlevel argument to postcreate() (closes #2901037)

[server] fix documentation of ZMQ options harder The syntax for zmq addresses is tcp://host:port, no zmqpickle involved.

merge with 3.16.x fixes

[server] fix documentation of ZMQ options

[zmq] set order for ZMQStartHook so other hooks don't need to If an application wants to subscribe to some messages on the zmq bus, that needs to happen after ZMQStartHook.

import merge_options directly from logilab.common

[server] fix error messages in write_sources_file()

[doc] a couple almost-typos

[schemaserial] mitigate critical message during migration We can't fetch extra_props because it is not yet there.

[doc] typo

[cw-ctl] fix help message for schemadiff In global help (closes #2888538) The leading space make it appears on multiple line. The help content is a bit improved in the process.

[dbapi] fix connect backwards compat harder Passing None as group and host should result in "pyro:///cubicweb.appid", not "pyro://None/None.appid". Followup for 2091d275fe5c. Closes #2882666.

[zmq] make publish address optional It should be possible to use the zmq communication bus in a read-only manner. Allow setting zmq-address-sub without zmq-address-pub. Closes #2897178.

Fix documentation for zmq pub/sub hooks and configuration Closes #2897177

[entity] improve deprecation messages

[devtools] use self._parse so AssertionError is properly raised instead of lxml error (test failure introduced in 6711f78c18be)

[testlib] fix page validator selection. Closes #2869456 * use a validator that checks XML well-formness for the html5 doctype (as a convenient side-effect, it does not choke on e.g. a canvas tag like the HTMLParser does) * use a DTD validator if there is an xml declaration * else use the HTMLValidator

[testlib] introduce a validator that check xml-well formness This validator simply check XML is well formed and accept any entities (think HTML defines much more entities than bare XML)

[view] return HTML5 doctype. Closes #2869426 We dropped xhtml support in 3.17 but we kept providing documents using the XHTML strict doctype, while our content wasn't conform to the DTD since we dropped proper namespaces definition as well as declaration of cubicweb DTD extensions.

[testlib] repreprocess content so contextual display has a chance to show the proper line. Closes #2869481 the context is calculated using position information relative to a preprocessed content, not the original source

[web request] drop no more necessary request.document_surrounding_div method This is awful and no more necessary since we dropped xhtml support

[testlib] unspaghettify Validator / PageInfo api

[testlib] update htmlparsers.VALMAP: stop using SaxOnlyValidator and add an entry for html

[testlib] SaxOnlyValidator is deprecated, use XMLValidator instead

[dataimport] minor typo in error handling

[views] attempt to clarify the inline help system documentation

[sobject] fix notification operation regression introduced in bf4003760e02 Operation use self. session. not self._cw. I'm a bit surprised that its not tested. closes ##2870077

Added tag cubicweb-debian-version-3.17.0-2 for changeset 09a0c7ea6c3c

[pkg/debian] prepare release 3.17.0-2

[pkg/debian] update yams runtime depends Only the built depends were updated to 0.37.0.

[pkg/debian] expand all depends and co on multiple line This is more readable and simplifies patching.

Added tag cubicweb-debian-version-3.17.0-1 for changeset 22be40c492e9

Added tag cubicweb-version-3.17.0 for changeset cc1a0aad580c

[test/view] exclude some Blog entity from automatic view testing Some Blog view requires the sioc cubes that we extracted in 3.17 we do not really want to test cubes view here. So we exclude CWEtype that really requires the sioc cubes.

[doc] fix: cubicweb.gmap.js has moved to the geocoding cube

[test/ldap] do not assume order Do no assume a specific order is stable when testing group members

Added tags for version 3.16.4 on changeset 041804bc48e9

[notification] ensure official API is test By using the official API in the hook we ensure it is tested.

fix typo in notification causing NameError introduced by: f5b40b66d36e

[test/notification] merge related test classe They have no reason to be distinct.

merge with stable

[server] add a looping task to free unused cnxsets capture by dead pyro When a pyro client disconnects, we're not necessarily notified so they can hang on to their session/cnxset forever. Don't let them do that. The hacky looping task is only installed when pyro is enabled. This optional activation save this very hacky patch from being rejected. closes #2825380

[pkg/debian] update changelog to 3.17.0

[devtools] Use XMLValidator instead deprecated SaxOnlyValidator The renaming was introduced in 198fdadafed6.o This prevent a meta class conflict introduced by logilab-common 0fcd2e050621.

[pkg] update changelog

[pkg] prepare 3.17.0

[pkg] prepare 3.16.4

[repo] straightforward bootstrap sequence. Closes #2841188 * kill fill_schema method which was doing to much different things, in favor of a deserialize_method which is simply reading schema from the db and returning it * set_schema doesn't bother with cubes This allows to easilly follow what's going on and so to cleanup init_cnxset_pool to get the following boostrap order: 1. load cubes (implies loading site_cubicweb) 2. load schema 3. init data sources 4. init cnx sets

[repo] minor cleanups to bootstrap sequence * fix some docstring/comments * move initialization of HooksManager to the __init__.py, this has nothing to do with pool initialization / boostraping

[schemaserial] serialize additional yams parameter for customs type Yams have a new custom type feature. We add support for extra parameter on CWAttribute in schema serialization handling. Closes #2847151

[sql] preprocess_entity uses lgdb helper's SQL converters. The preprocess_entity does not yield anymore the conversion logic for different base types, but relies on a new SQL_CONVERTERS dictionary of the db helper. This will allow later inclusion of new base types.

[repo] kill rebuildinfered feature from Repository.set_schema This clutters thing and is useless in most cases: when schema just have been readen from the database or from the file-system, infered relations have just be calculated. The only use-case where it should be rebuilt is on migration (ie schema modification by editing the meta-model using rql), so move this feature there.

[test] stop deepcopying for nothing

drop xhtml content-type support (closes #2065651) * HTMLStream does not care about xml any more * reqquest.demote_to_html and .xhtml_browser are deprecated * web config: drop force-html-content-type option * adjust tests

[htmlparser] exclude <script> tag from html source lxml are confused by them :-(

[htmlparser] store unaltered source in pageinfo.source Do not store the parser to preserve the source aspect. Store the initial input in the source argument instead. This is very useful when parsing html. In such case we need to drop ``<script>`` tag.

[htmlparser] rename SaxOnlyValidator to XMLValidator

[htmlparser] add missing deprecation message

[form] remove hard coded id html id should be unique. It is clearly not achieved here.

[server/test] use server config in migration This allows a lighter initialisation (no rtag needed). Execution of those test move from 300 secondes to 220 seconds here. A 25% speedup

[session] deprecate `hijack_user` method The semantic of this method is wicked and lead to very hard violation of sanity. (multiple transaction using the very same cursors). We deprecated the API to be able to drop it as soon as possible. The method was added long ago for some notification trick. It is not needed since we sent notification on ``postcommit_event``. (closes #2781782)

[mail] drop hijack_user usage The transaction is now commited before notification. We can create a new session directly and drop `hijack_user` usage. This prepare #2781782

[notification] merge BaseNotificationView and NotificationView They are in the same module now.

[notification] move notification view in ``sobject.notification`` It has no user outside this module. This enforce serversideness of notification and allow future cleanup. No backward compat is set up to prevent circular import. The class has no other user anyway. (closes 2845144)

remove unused import pyflakes say they are unused. Tests suite confirms.

one more merge

[notification] introduce an official `notify_on_commit` function It provides a proper notification api instead of replacing it by yet another clumsy one. Closes #2837251

[cubicweb/doc] Replace dc_type() by cw_etype dc_type() is may be translated. Leading to terrible terrible damage.

[dataimport] backout 6947201033be (related to #2788402) (and add a try: except to cache the intended error) The problem actually comes from the ``MassiveObjectStore`` in the ``dataio`` cube, so it should be corrected there. Here, we only protect it with a ``RuntimeWarning`` so that the user can see the problem. ``value`` is set to ``None`` (whence to ``NULL`` from a database standpoint), so that the data can be nevertheless inserted in the database. However, only the keys present in ``row`` are actually non-'``NULL``'. The real solution is to work out the issue in ``MassiveObjectStore`` directly. The current try/except should only be a temporary hack.

merge default heads

[c-c serverctl] import locally to speed up c-c

[facet js] minor refactoring and cleanups * extract facetCheckBoxSelect and facetCheckBoxUnselect * use $ as prefix for jQuery variable (facet -> $facet) * other minor rephrasing

[ldapfeed] Add support for LDAP groups (closes #2528116) Groups from the LDAP server are imported as CWGourp entities, and in_group relationships are created between existing CWUsers and CWGroups Unit tests have been refactored a bit, especially ti add helper methods to manage LDAP database.

[test/ldap] small improvement to ldapfeed unit tests better group testing and further testing of imported user life cycle.

[ldap] prepare import of CWGroup All CWUser specific code is put in dedicated sections.

[ldap] handle modification date We are now able to import modification date from ldap.

[ldap] refactor attributes mapping handling * use explicit attributes lists when calling ldap search * direct and reverse attributes maps contains now the same elements.

[ldapfeed] add support for multiple email addresses from ldap Multiple emails case was ignored before this change. email test

merge 3.16.x fix in 3.17.x branch

[session] fix hooks_control backward compat Session object have no ``hooks_control`` control method. it has ``allow_all_hooks_but`` and ``deny_all_hooks_but`` ones. Regression introduced in bd5b5759c9b3.

[facet js] fix reordering of facet check boxes. Closes #2732947 Before this patch, when one select an element, it's moved to the top of the select content. Fine. But when it's later deselected, it stays there instead of moving back to its original location. This patch fixes that by introducing a facetCheckBoxReorder function which properly reorder the whole facet, instead of buggy attempt to locally reorder.

[vreg] (re)build entity classes cache after registry initialization. This fix potential memory fragmentation issue. Closes #2719113

[task] allow negative intervals for add_looping_task (closes #2818280) Instead of rejecting values < 0 with a ValueError, we ignore tasks with a negative interval. This introduce a simple way to disable task from configuration.

[task] allow interval=0 for looping tasks There is valid usecase for perpetual running task.

[portable dump] skip virtual relations, they have no table associated. Closes #2841199

[test] fix skipIf import, currently failing if unittest2 is not there

[test] fix bad merge

merge with other default head

[ldap] an empty 'user-base-dn' disable the user importation process, The same semantic will be used for group. We start by using it for user for the sake of consistency.

drop commented print

[ldap] Major test refactoring Mostly fix tests (previous refactoring), but also refactor a bit the base class to have better/simpler interaction with the slapd server.

[test/ldap] have RQL2LDAPFilterTC use its own database I do not know what RQL2LDAPFilterTC is about and why it does this strange reuse of LDAPUserSourceTC pre_setup_database but it is certainly a good idea to have it use a dedicated database-cache.

[ldap] quieter tests Capture slapadd and slapd output if everything goes well.

[test/ldap source] better source naming, some cleanups (prepares #2528116) ldap uri becomes "ldap" in all cases prepare separation of CONFIGs by source type

use cw_etype instead of __regid__ We have a clean public API now.

[entity] add an official ``cw_etype`` property for entity This property holds the CWEtype id of the entity. This information was previously available through __regid__ but this is an ugly public API. So we have a new clean public API. The ``dc_type()`` method could not be used as it is translated. (closes #2793792)

[notification] use new style operation for notification This results in far less operations created. Operation creation is expensive. We keep some compatibility with the previous API because some cubes have been reported to use it.

[cleanup] drop code deprecated since 3.6 The 3.6 API have been dropped some time ago. Those three lines survived the purge. One of them is even deprecated since 3.5.

[notification] fix documentation The doc say precommit, the code says postcommit. The code is right. We also clarify the documentation of another related operation looking suspiciously wrong but actually correct.

[ldapuser] Deprecated the source We have ldapfeed for some time. maintaining both is cumbersome. This old style source will be dropped in 3.18. closes #2567712

[rql2sql] Remove an XXX by explaining why Int non-substitution is desired (in the current state of the API)

[rql2sql] Boolean values should be substitued like others. Closes #2829481 Furthermore lgdb 1.10 deprecate boolean_value method and this simplify testing.

[toward-py3k] rewrite __cmp__ (closes #2715115) In Python 3k `__cmp__` special method is no longer supported. So we split it in `__lt__` and `__eq__`. details on Python3k behavior here: http://docs.python.org/3.0/whatsnew/3.0.html#ordering-comparisons

skeleton: add RPM spec template (closes #2800884)

[clean_session] revert itervalues usage from 0bb18407c053 self.close(session) changes the self._sessions content. We need to create a full list of value before starting closing them.

merge 3.16.x fix

[ldap] Use correct API for logging message The good practice cat prefer logging to do the substitution himself.

Fix obsolete docstring for add_looping_task The docstring was telling lies since changeset cae198371548 "add_looping_task can be used any time"

[devtools] prevent Xvfb resets Seems like firefox opens more than one connection to the X server in some cases, and things get utterly confused.

[LDAP] rename unittest ldapuser to ldapsource (prepares #2528116) the tests involved are not only for ldapuser module but also for the more general ldapfeed Necessary to prepare LDAP patch for CWgroup

[devtools] protect xvfb-run.sh against clean_up being called twice

Added tag cubicweb-version-3.16.3, cubicweb-debian-version-3.16.3-1, cubicweb-centos-version-3.16.3-1 for changeset ee860c51f56b

3.16.3

[service] make sure synchronous call_service has a cnxset Fixes regression in 3.16.2 (closes #2807720)

Merge testfunc testfunc resurrection Closes #2535377

[web] move qunit test back into web and fix the hanging qunit test can not work outside web. a lot of try/finally clause have been added to prevent qunit to hang on error. A not a test a indeed broken. But we at least put in them a running state again..

[test] drop forsaken windmill test (Closes 2535376) * not run since multiple years * windmill itself is in poor shape not even packaged in Debian. The peace which other seek they find; The heaviest storms not longet last; Heaven grants even to the guiltiest mind An amnesty for what is past; When will my sentence be reversed? I only pray to know the worst; And wish as if my heart would burst.

Added tag cubicweb-debian-version-3.16.2-1 for changeset b3c1ad0cbf00

Added tag cubicweb-version-3.16.2 for changeset a2b4f245aa57

3.16.2

[server/test] remove dependency on a pyro name server (closes #2801737) Use a direct pyroloc:// url instead of going through a NS.

[packaging] add spec file to build RPMs (closes #2800850)

[predicates] keep making `yes` predicate importable from cubicweb (closes #2790319) The `yes` predicate was moved out in CubicWeb 3.15 but it should have remained importable from the cubicweb.predicates module, for bw compat reasons and also because logilab.common.registry is an implementation detail.

[views/tables|css] backout rev 895e34d9ae0d This changeset introduced a GUI behaviour change that has usability side-effects (eg. shows a non-undoable massive action as a first-class action). The idea is good, but it requires a better implementation; tabs are not the proper GUI solution to the original usability issue.

[webconfig] cleanly handles ``fckeditor_installed`` when no uiprops The previous version of the code relied on a default value with a dummy ``FCKEDITOR_PATH`` entry. The new code check if uiprops is initialized or not.

[webconfig] fallback to default configs if https ones are not defined When cubicweb serves a request with an https scheme, it tries to access ``https_uiprops`` and ``https_datadir_urls``. However if those are not defined we should fallback to unsecure url instead of crashing with a traceback. There are very valid usecases for undefined https url. For example if your ``base_url``, ``datadir_url`` and uiprops content are already https urls. Closes #2786643

[webconfig] explicit some webconfig attribute Explicit declaration of attribute are much clearer. Just do it.

merge 3.16.x fix in 3.17.x branch

merge with 3.15.x fix

Added tag cubicweb-debian-version-3.15.11-1 for changeset 09d65bc1f025

Added tag cubicweb-version-3.15.11 for changeset 38c6a3ea8252

[querier] improve 'Password selection not allowed' error message to report the involved variable name whose type is Password, it usually help understanding the problem.

[predicates] use select_or_none rather than select+try/except, expected to be more efficient

[pkg] prepare version 3.15.11 ?

merge 3.14.x fix into 3.15.x

[facet] ensure a facet DOM id is a valid jQuery identifier. Closes #2789089

[predicates] fix 'adaptable' implementation, used to return 0 on a rset with individualy adaptable entities of different types. Closes #2709663

[cmd] fix exlog documentation (closes #2715913) The table documentation was out of sync since 8a075bc94563.

Added tag cubicweb-debian-version-3.14.10-1 for changeset 197bcd087c87

Added tag cubicweb-version-3.14.10 for changeset 0ff798f80138

[pkg] prepare 3.14.10 release

[querier] fix eid relations handling in SET queries (closes #2797052) _extract_eid_consts() implementation must consider arithmetic operators when trying to associate rql variables to specific eids.

[ext/rest] add directive bookmark to rest (closes #2545595)

[c-c create] make post-create step optional (closes #2712041) In this context the "post-create" step refers to the database's creation and initialization. We want to be able to create a new instance but skip the database creation (`db-create` and `db-init` commands) in automatic setup.

[cwconfig] Add optional configuration param to load a UI cube before other cubes Needed later on to load UI cubes such as legacyui, and one day squareui.

[web/views] Move massmailing to its own cube (closes #2788086)

merge with stable fix

[req] explicit `user` attribute The `RequestSessionBase` object needs a `user` attribute. It is initialized by sub class, but having it explicitly declared is much cleaner.

[test] use the right order in some assertEqual The unittest2 function use ``(expected, got)`` order.

[session/transaction] move resource accessors on Transaction

[session/transaction] move multiple utility on Transaction

[session/transaction] handle cnxset repository logic in transaction Interacting with the repo is now handled by the Transaction code directly.

[transaction] keep a reference to the repo object

[session] make security_enabled API private The doc already says "internal API". The old entry point is now officially deprecated and the context manager have been made private.

[session] make hook_control API private The doc already says "internal API". The old entry point is now officially deprecated and the context manager have been made private.

[service] split session retrieval and service execution Internal session are not tracker by the repository and can not be retrieved using _getsession. Moreover their session id is not unique… Closes #2735700

[service] handle cnxset in the async case only The service may be called by a session, we really do not want the call to service to mess with the transaction state. This does not apply to async were thread magic ensure a dedicated transaction.

[service] extract session retrieval from "task" closure The session was retrieved there because `set_cnxset` must be called in the same thread as the code using it. We now retrieve the session beforehand and explicitly call `set_cnxset`.

[cubicweb/doc] Add tutorial on data import in CubicWeb. This involves creating the "tutorials/dataimport" directory structure under "cubicweb/doc" and, inside the "dataimport" directory, putting several files: - a ResT file containing the tutorial *per se*; this tutorial addresses the following issues: * creating a CubicWeb schema for representing a given data set (here, the Diseasome RDF data, for illustration purposes); * parsing the data; * importing the data, by using several stores: + the ``RQLObjectStore``, ``NoHookRQLObjectStore`` and ``SQLGenObjectStore`` from the ``dataimport`` module in CubicWeb; + the ``MassiveObjectStore`` from the ``dataimport`` module in the ``dataio`` cube. The tutorial also provides timing benchmarks of the various stores. - a set of Python files illustrating the data import, in the context of Diseasome RDF data parsing: * a Diseasome RDF data parse module, * a Diseasome data import module, * a CubicWeb schema for representing Diseasome data.

[dataimport] Slight message modification in exception handling code.

[dataimport] Handle various data formats when creating buffers from data. :note: On the long run, this should be addressed before, when these data are created.

[dataimport] Add a ``flush`` method for all stores. For ``ObjectStore``, ``RQLObjectStore`` and ``NoHookRQLObjectStore`` the method tries to call the ``commit`` method. For ``SQLGenObjectStore``, the ``flush`` method is basically unchanged with respect to previous changesets.

[dataimport] Uniformize the API across the different stores. This is achieved by modifying the ``relate`` method so that it takes an extra ``**kwargs``. More specifically, ``SQLGenObjectStore``'s ``relate`` method needs the the type of the subject entity which is passed through ``**kwargs`` as the ``subjtype`` keyword argument. Actually, it is the ``add_relation`` method of the ``SQLGenObjectStore`` who needs this argument. However, as this method is not called directly (but via the ``relate`` method), the ``subjtype`` argument is passed to ``add_relation`` via ``relate``. The other stores' ``relate`` methods do not need this extra argument, hence for the other stores ``**kwargs`` is empty. In this manner, the API is unified across the different stores.

[data] Add a CSS for undo UI rendering The nesting of ordered list and unordered list was badly handled. This has gone unoticed becaused it was not used. But the undo feature use this markup to render the changesets. That's why this minor change to ths CSS is introduced.

[web/views] extract cube geocoding (closes #353000)

[c-c list] add an optional argument to the list command (closes #2709703) This optional argument allows to restrict listing to only a given type of items (cubes, instances, configurations).

default base-url now uses fdqn instead of hostname only (closes #2542815)

[editcontroller] a small debugging help (closes #2518980)

[doc] small doc improvement

[doc] fix Pyro url

[doc] fix Narval project url

[tutorial] minor documentation fixes

merge 3.16.x fix in 3.17.x

merge 3.15.x into 3.16.x

server/source/native: fix wrong usage of .lstrip that produce garbled error messages (closes #2777641) Fixed an .rstrip(...) for the general case (untested). Fixed an .lstrip(...) for the sqlite path (tested).

[doc] update 3.17 release note

[migration] fix bug in `CWAttributeAddOp.revertprecommit_event` The `rdefdef` attribute is set during the operation execution. In some the operation crash before this assignment. In such case `revertprecommit` raise an attribute error crashing the whole process and shadowing the original error. This changeset detect and Avoid this situation.

[session] security context directly use transaction object when applicable Most of the security logic have been moved on Transaction anyway.

[session] move context manager logic in the security context manager The `init_security` and `reset_security` method have no other user that the context manager itself. Having dedicated public function is not useful.

[session] hook control context directly use transaction object when applicable Most of the hooks control logic have been moved on Transaction anyway.

[session] move context manager logic in the hook controls context manager The `init_hooks_mode_categories` and `reset_hooks_mode_categories` have no other user that the context manager itself. Having dedicated public function is not useful.

[session] drop the useless set_hooks_mode method It has not external user and trivial code

[session/transaction] move most undo support into transaction A small part that need repo access has not been moved into transaction yet.

[session] simplify mode property We can delegate very thing to transaction utility

[session/transaction] pass the session object to the Transaction __init__ This simplify access to variables. As we are moving more and more logic and states in the Transaction it will greatly helps in future commit.

[session] use a dedicated class to track cnxset We introduce a new CnxSetTracker to track `cnxset` used by Transaction and allows to wait for them. This new class does not use Thread ID not thread joining to work. This allows to use multiple transaction per thread and a transaction in multiple thread. The class itself is totally threadsafe by the Transaction is still not thread safe. The old _threads_in_transaction attribute is dropped in favor of a new logic based on this object. The registration of cnxset used is not done by the Transaction itself. tx.cnset is a property handling the Consistency of its value with the CnxSetTracker instance. Note: The CnxSetTracker instance only track transaction id, not transaction itself, So not reference cycle are created.

[session] cleanup Transaction.cnxset The attribute should always be there. It is None when not cnxset exists.

[session] drop useless getter and setter for security Those function have very few user and does not anything fancy. Internal user now access tx.read/write_security directly. External user have been blessed with a writable property. They are only interested in the write part of they did not used the return of the removed function. They access it through the session since I prefer the external world to not be aware of Transaction yet.

[session/transaction] move read_security subtlety in transaction This is transaction data, itshould be handled by the transaction itself.

[transaction] initialize dbapi_request in __init__ There is not reason to have it implicitly declared. The attribute is renamed to match its public name.

[whatsnew] notify drop of deprecated code

[web/views] extract cube embed (closes #1916015)

[skeleton] Update debian packaging to support only python 2.6 and up

Drop iprogress code (closes #2777628)

[web/views] extract cube sioc (closes #1916018)

[doc] Add 3.17 release notes

[migration] mark all version prior 3.7 as non-migrable API possible used by such old migration script are not supported in recent Cubiweb. Closes #2772958

[dataimport] drop checkpoint backward compat API Deprecated since 3.7. See #2772958 for details.

[dataimport] drop index support deprecated since 3.7 Dropping of pre-3.7 API decided in #2772958

[utils] drop pre-3.7 compat in util Dropping of API deprecated since 3.7 decided in #2772958

[migration] drop pre-3.7 utility Dropping of API deprecated since 3.7 decided in #2772958

[entity] drop pre-3.7 api on session Dropping of API deprecated since 3.7 decided in #2772958

drop dubious comment

[req] drop pre-3.7 api on request Dropping of API deprecated since 3.7 decided in #2772958

[session] drop pre-3.7 api on session Dropping of API deprecated since 3.7 decided in #2772958

[transaction] use set operation in the hook control code We use direct operator on set instead of looping by hand, This simplify the codes.

[session/transaction] move hook management in transaction itself This is transaction level code.

[session] allow writable tx_attr and use it for commit_state This keep clarifying the code.

[session/transaction] move add operation in session code

[session/transaction] move entity life utility on transaction This access transaction related data and therefor should live in transaction.

[transaction] small simplification in ecache code A single assignation is simpler.

[session/transaction] move entity cache management on session A new utility method is added for that. It is similar to the one used for attribute access. code is copy pasted as his. Function documentation is added.

[session] have a nice helper function to forward access to session This allows a lighter and clearer code. This will also ease the addition of deprecation warning in the future.

[session] drop dead code session._tx is never None. New transaction are created if necessary.

[session] use tx.data directly in session code.

[transaction] rename transaction_data to data Transaction.transaction_data is redundant. A transaction_data property is left for backward compatibility. Deprecation mechanism will come in later changeset

[session] document Session._tx

[session] split session creation from default session assignation This is the first step for more independence for Transaction.

[session] make session lock reentrant We are going to use it for other session related business. It is renamed from _closed_lock to _lock in the same move.

[session] document set_tx

[session] rename txstore variable to tx This is a more accurate name

[session] rename `_threaddata` to `_tx` The returned object is a Transaction object.

[session] rename self.__threaddata.txdata to self.__threaddata.tx This is Transaction object now.

[session] rename _tx_data into _txs The object inside are now Transaction object. Not TransactionData object.

[transaction] relocate method building transaction context manager It is in the middle of methods going to be refactored.

[transaction] move RQLRewriter in Transaction This remove the last ``AttributeError`` magic. The ``RQLRewriter`` needs a Session. Its reset is handled in ``Session`` code to handle that

[transaction] initialize security control attribute in Transaction There is no reason to not initialize them in __init__ time. This simplify the Session code.

[session] move security constant out of the class There is no reason for it to not be global. This will ease it use by ``Transaction object``.

[transaction] initialize hook control attribute in Transaction There is no reason to not initialize them in __init__ time. this simplify the transaction code.

[session] Move hook control constants out of the class There is no reason for them to not be global. This will ease they use by Transaction object.

[transaction] initialize transaction data and state related attribute. There is not reason to not explicitly declare and initialize attribute in Transaction.__init__. This change requires the introduction of a Transaction.clear() method to get ride of some wicked ``AttributeError`` based logic.

[transaction] handle ``mode`` default value in Transaction The transaction mode is now explicitly passed at creation time and always read from the Transaction object. Note that there is a slight behavior change. The transaction mode is now set at the creation of the transaction. Changes made to the default value have no longer any effect on existing transaction.

[transaction] explicit Transaction cnxset attribute The Transaction object have a lot more attribute that those previously declared in __init__. They are just dynamically assigned in Session code. This changeset explicit the ``cnxset`` and ``mode`` attribute declaration.

[server] rename TransactionData to Transaction Transaction will become a full featured object.

[doc] update Session documentation A major refactoring of Session/Transaction handling is commit. This is a good occasion to improves the documention.

merge 3.16 fix in default

[devtools] fix a couple issues with xvfb-run xvfb-run didn't quite clean up correctly after itself, because: - dash doesn't run EXIT traps on signals - if we don't run the command in the background, a TERM handler doesn't run until the command exits

[devtools/qunit] don't open-code subprocess.Popen.terminate

[devtools/qunit] get rid of unused variable

[server] *init_repository* lookup the database instead of the schema to drop tables (closes #810743) So, tables are dropped even if the schema was changed. :note: if mssql, drop views before tables. :note: because of table dependencies we try a second drop sequence

[book] remove XXX to an old discussion about appengine

[web publish] in case of error, ensure proper http status is set and Content-disposition header is reseted. Closes #2553066

Add CubicWebRequestBase.content (closes #2742453) This is the body of the HTTP request (stream)

add the match_http_method predicate (closes #2559932) This aim at creating views which are selected on other HTTP methods (PUT, DELETE, ...)

[pkg] the previous commit requires new feature in yams.

add a command to compare db schema and file system schema (closes #464991)

drop typed_eid() in favour of int() (closes #2742462) typed_eid was introduced to abstract the eid type when running on Google AppEngine. It is not used anymore and can be removed. Let's use int() instead.

merge with stable

3.15 is the new old stable

[web] don't link to None in author box (when author has been deleted) (closes #2409855)

dbapi: try to restore compatibility of the connect api with 3.15 A few issues here: - urlparse('foo').scheme is '', not None - cnxprops is an optional argument Regression from 62213a34726e ("[db-api/configuration] simplify db-api and configuration so that all the connection information is in the repository url, closes #2521848") Closes #2754322

3.16 is the new stable After discussion with David Douard I'm merging 3.16.x branches in stable and starting 3.17 feature on default.

Added tag cubicweb-debian-version-3.15.10-1 for changeset feca12e4a618

Added tag cubicweb-version-3.15.10 for changeset 89bdb5444cd2

prepare 3.15.10

Added tag cubicweb-debian-version-3.16.1-1 for changeset 84fbcdc8021c

Added tag cubicweb-version-3.16.1 for changeset d95cbb7349f0

prepare 3.16.1

[views/primary] some inner sections should use the `limit` by default to avoid a denial of service (closes #2719110) Today, it is possible to call .related and get a huge unlimited database-dos-inducing resultset that will be nevertheless limited a bit further in pure python in the `autolimited` view. While we cannot completely avoid potential denial of services such as these we mitigate the problem with the default ui settings: if the inner vid is `autolimited`, then the relation result sets is computed using the user-defined limit. This change respects the semantics of the `autolimited` view and shouldn't break anything.

[entity] ensure the .related(entities=False) parameter is honored all the way down (closes #2755994) As of today, such a call will always fill the relation cache by calling .entities() on every single related rset entry. As a consequence, the `limit` parameter handling also had to be fixed. It was bogus in the following ways: * not used in the related_rql, hence potentially huge database requests, but also actually * foolishly used in the .entities()-calling cache routine we now bypass (this changeset ticket's main topic) Now: * we set a limit on the rql expression, and * forbid caching if given a non-None limit (as we don't want to make the cache handling code more complicated than it is already) With this, entity.unrelated gets a better limit implementation (so the code in related/unrelated is nice and symmetric) Risk: * _cw_relation_cache disappears completely, which is good, but this is Python, so you never know ...

[test/web] fix invisibly bogus test (prepares #2755994) The test was wrong but that was cancelled out by a cache effect and fuzzy naming. Wiping the entity caches restores sanity: the choices list are the same before and after the SET. Also field.choices uses entity.unrelated but always returns related + unrelated elements. Hence `choice` replaces `unrelated` where it makes sense. AssertIn is used in place of AssertTrue.

[cwconfig] Fix exception handling when building the cube dependency graph UnorderableGraph exceptions do not have a 'cycles' property. A simple cast to str does the job.

[merge] backport stable fixes

[test/sobject] fix test regression Was complaining: - * updated comment #EID (#EID) ? ^^^^ + * updated comment #EID (duh?) ? ^^^^ in sobjects/test/unittest_supervising.

[web/request] Prune extraneous 'pageid' from generated ajax URL parameters (closes #2758130) If 'pageid' is given through extraparams, it is sent twice to the browser. On the JS side, the final URL loadxhtml() will end up using will have 'pageid' set twice which CubicWeb will readily accept as a list. Pruning this parameter makes sure it is exactly once.

[web/component] Use global variable to point to ajax controller (part of #2758254)

[web] Use the new '/ajax' URL path to access the AjaxController (closes #2758254) 5a81fa526b30 took care of all the JS code, this patch cleans up the remaining python bits.

[devtools/httptest] fix syntax error introduced by ce5ae7b80d2c

[component] Fix URL generation for navigation component (closes #2464832)

merge with another default heads

Merge stable into default (NON-TRIVIAL) - dropping __future__ statement added by d2472948da9c - added migration from 407acc41beb5 to migration file for 3.16.1

maintain python2.5 compatibility

[schema/workflow] one more typo fix Also add proper migration.

[devtools] add http_publish to CubicWebTC (closes #2565882) Fakes a http request, without overriding the normal error handling. Returns the request object so the caller can check for errors.

[testlib] url_publish should give url to the request and the rset returned by the path evaluator to ctrl_publish. Closes #2557468 Also, url_publish now accepts a `data` argument to simulate POST of values.

[towards py3k] import reduce() from functools (part of #2711624)

merge stable fix into default

[doc] document dbapi.Connection

[c-c shell] fix command description (-P option is gone long ago)

[hooks/syncschema] do not crash when adding a new entity type (closes #2741643) The previous version assumed META_RTYPES provide relation definitions for all entity types.

[views/debug] Do not show all web sessions without CNX, just count them (closes #2602161)

[facets js] use $ instead of jQuery and other small cleanups

[devtools/httptest] allow sending other types of requests besides GET

merge with stable

[ext/rest] fix docstring of rql_role()

[web application] Fix missing self. in error_handler

[ldapfeed] all options of a source should be in the same group, else you get AssertionError on c-c add-source. Closes #2538398

[sources] fix classes that inherit from AbstractSource (closes #2718669)

[login] don't redirect to http url if we allow anonymous on https Closes #2583913

Correct typo in example

[testlib] make test_view load the json data Closes #2721472

[etwist] Properly escape traceback output (closes #2712042) Stop using StringIO in the process as well.

[dbapi] Stop shadowing exceptions in get_repository() (closes #2710515) Exceptions are already handled higher up in the stack so there's no reason to handle them here. Furthermore, non-ConnectionError exceptions would get hidden, making debugging harder.

[cwvreg] complete cw.web.uicfg cleanup callback (closes #2718217) 1dd655788ece introduced insufficient backward compat support When cubes directly import uicfg from cubicweb.web (or uihelper) this also creates a reference in the parent cubicweb.web namespace. This must be cleaned up as reloaded cubes will take back this reference and the old versions of its appobjects.

[book] fix typo. remove spurious `i`

[towards py3k] fix bug introduced by 0bb18407c053

[doc] Book entry for the undo functionnality.

[toward py3k] rewrite has_key() (part of #2711624) The `has_key` method is not in Python3K but dict supports __contains__ check for ages.

[toward py3k] rewrite dict.keys() and dict.values() (part of #2711624) Python 3K will only expose iterators. http://docs.python.org/3.0/whatsnew/3.0.html#views-and-iterators-instead-of-lists

[toward-py3k] rewrite to "except AnException as exc:" (part of #2711624) The older form is not necessary since we dropped python 2.5 compability. This will help Python 3.3 experimentation.

[toward-py3k] remove import with_statement (part of #2711624) They are unneeded since we dropped python 2.5 compatibility. This will help Python 3.3 experimentation.

pkg: drop python 2.5 compatibility (closes #2711624) 2.5 is not common anymore. Dropping compatibility will help moving to a code base compatible with Python 3.3

devtools: improve qunit timeout message Make the message clearer, shorter and properly wrapped.

merge default into stable

[devtools] fix broken windmill test case

Added tag cubicweb-debian-version-3.15.9-1 for changeset 29fbc632a696

Added tag cubicweb-version-3.15.9 for changeset 8bfc0753f1da

Added tag cubicweb-debian-version-3.16.0-1 for changeset 853237d1daf6

Added tag cubicweb-version-3.16.0 for changeset 6c7c2a02c9a0

3.15.9

[ldap test] fix bad merge

[merge] backport stable fixes

[cwctl] fix cubicweb-ctl shell command (closes #2583919) * add an uri parsing utility that documents a bit the uri cubicweb supports out of the box * cwctl: use this utility and refactor a bit for clarity (the novelty here being real support for old style 'myapp', or: * 'inmemory://myapp' * 'pyro://pyro-ns-host:pyro-ns-port/myapp' (pyro access through a pyro concentrator) * 'pyroloc://host:port/pyro-instance-id' (direct access to a pyro repository) * 'zmqpickle-tcp://host:port' * dbapi: refactor to sort out some of the complexity and restore functionality lost in 62213a34726e

[ldap test] move the slapd database directory to a tempdir (closes #2583993) This is required to be sure we are not hurt by NFS or such...

[ldap auth] make sure imported passwords from LDAP are encrypted (closes #2583994) When a user is imported from an LDAP source (in ldapfeed or when converting a user from an ldapsource to system one), we may import a clear-text password, which we do not want to do in our system database.

[ldapfeed] don't crash if one specify an unexisting group in the configuration. Closes #2538399

[doc/ldap] update the ldap chapter wrt the ldapfeed source type (closes #2551863)

prepare 3.16

[doc/3.16] complete description of what changed

[rql sources] uses an URL as remote host address for pyro/zmq rql sources, instead of extra configuration parameters. Follows modification of dbapi.connect api. Closes #2539822

[repo sources] move handling of source's url into abstract source as this becomes shared by most sources prepare #2539822

[db-api] rename repo_connect into _repo_connect to mark it private. Closes #2521848

[repository sources] copy source configuration: source's __init__ modify it while config.sources() return a cached dictionary which shouldn't be modified This causes pb since f8aa87a773b2 (though it's unclear why it hasn't been a problem before). Closes #2521848

cleanup repository connect doc: cnxtype is now deprecated / base_url has never been given that way. Follow #2521848

prefix "tcp://" zmq uris with "zmqpickle" (closes #2574114) * it turns "tcp://" into "zmqpickle-tcp://" * will leave room for future non-pickling communications using zmq

[db-api/configuration] simplify db-api and configuration so that all the connection information is in the repository url, closes #2521848 eg no more specific option of pyro ns host, group, etc. This also fixes broken ZMQ sources. Changes: * dropped pyro-ns-host, pyro-instance-id, pyro-ns-group from client side config, in favor of repository-uri. No migration done, supposing there is **no web-only config** in the wild. Also stop discovering the connection method through the repo_method class attribute of the configuration, varying according to the configuration class. This is a first step on the way to a simpler configuration handling. Notice those pyro options are still available for repository only / all-in-one configurations as they are needed to configure the pyro server. * stop telling connection method using ConnectionProperties, this is so boring. Also, drop _cnxtype from Connection and cnxtype from Session. The former is replaced by a is_repo_in_memory property and the later is totaly useless. * deprecate in_memory_cnx which becomes useless, use _repo_connect instead

[ldap test] ensure slapdd launch went fine and move ldapfeed setUp to parent class, as all ldap sources deal with url

[web/uihelper,uicfg] transform uihelper functions into uicfg objects methods (closes #2543949)

make ui configurations selectable (closes #2406609) * introduce a new 'uicfg' registry (storing instances) * use the relevant new APIs from lgc.registry to manage the new registrable uicfg objects * cw event manager useage is gone; instead thze standard registry reloading mechanism is used * ensure i18n commands still work (devctl) * introduce dynamic uicfgs use whenever possible (various views), even though sometimes the classic 'static' usage remains

[uicfg] uicfg.py moves from web/ to web/views/ (prepares #2406609) We are about to make uicfg selectable. Registrable / selectable views and helpers live in web/views. Hence the move. Backward compat is put in place. However CubicWeb imports of uicfg are updated to the new location.

[appobject] introduce RegistrableObject base class (prepares #2406609)

[cwvreg] introduce lgc version 0.59 and remove unneeded method redefinition (prepares #2406609) The new logilab.common.registry makes this moot.

[BFSS] mark fs_imported value as uncachable This will finally fix the `test_bfss_fs_importing_transparency` pass again. Regression was introduced by 4ba11607d84a

[entity] add a "repo_side" parameter to `_cw_dont_cache_attribute` This parameter (defaulting to `False`) is used to inform cubicweb that we really really do not want to cache a value during creation or update. This will be used by a storage that may do very specific processing on attribute values that result in a stored value different than the provided one (e.g. BFSS `fs_importing` mode).

[merge] backport stable fixes

[entities] fix test failures introduced by fixes in workflow related message strings (completes 9bb93efa1952)

[doc/book/] Corrected typo in en/tutorials/base/customizing-the-application.rst, for the add_cubes function call

[entity,views/json] backout 353bbd17a8b6 (reopens #2559931) Calling .complete() unconditionnally from the json encoder is unsafe since on entity creation validation: * an eid may have been drawn (hence even .has_eid() would not help) while processing form data in the edit controller * a ValidationError may have been raised and the entity-creating transaction rollbacked This leads to a crash on the return path from the validation to the browser, where the json_dumps((status, args, entity)) call will stumble upon the .complete() call which will fail because the entity does not (any more) exist in the database.

[schemas] fix typo

merge with stable

[appobject] cleanup module (prepares #2406609) * remove unused imports * break long lines

[uicfg] remove pre 3.6 bw compat structure (prepares #2406609) This should have been done in a673d1d9a738.

[testlib] test_view now parse json data, closes #2557467

Improve entity json export (closes #2559931) Make output from __json_encode__ predictable; use it in the ejsonexport view.

[web/application/test] drop useless code This is already done in CubicWebTC

[test] precheck initial condition

[packaging] Fix search in squeeze cubicweb-documentation package (closes #2558496) Work around broken combination of jquery 1.4 and sphinx 0.6 in squeeze by patching up doctools.js.

[packaging] don't compress txt files (closes #2558496)

[wf/test] fix tests broken by changes from another patch which have slipped there

[merge] backport stable

[web/test] fix stale reference to the changelog view (closes #2423532)

Added tag cubicweb-debian-version-3.15.8-1 for changeset 4ef457479337

Added tag cubicweb-version-3.15.8 for changeset 459d0c48dfaf

[test] fix possible views test (closes #2544358)

prepare 3.15.8

[ldapfeed] if a deactivated user becomes available again in its source, reactivate it (closes #2542776) * cwusers that see their ldap source counterpart disappear are deactivated * until now they were not activated again on reappearance * note this behaviour in the doc

[ldaputils,ldapparser] correctly compute "deleted" users (closes #2542083) * in ldaputils.py, removal of an unneeded method (checking existence of an object in the ldap, but not using the configuration filters) * in the ldapparser, memoize the (filtered) search so as to serve both .process and .is_deleted * add a non regression test for the 'user-filter' source config entry + deletion

[dataimport] new ignore_errors argument to ucsvreader, default to False. Closes #2547200

[ldapsource] keep python 2.5 compatibility (closes #2541602)

[ldapuser2ldapfeed] .values -> .iteritems (closes #2542872)

[ldap tests] cleanup in preparation of #2542083

[ldapparser] demote warning level (actually important user info can leak there) (prepares #2542083)

Remove changelog view (closes #2423532)

[dataimport] Allow to replace escape char in the create_copyfrom_buffer

[doc] fix function name FTIRANK

[json] enable jsonexport selection on empty rsets (closes #2544358)

[test] fix req default value handling in CubicWebTC.view (closes #2544376) It's no more than the standard ``if rset`` vs. ``if rset is None`` error.

[ajax] fix attribute lookup (closes #2491038)

[server] add debugging for Hooks & Operations (closes #2470048)

[dataimport] Add SQL Store for faster import - works ONLY with Postgres for now, as it requires "copy from" command - closes #2410822 This store will use: - copy from for massive insertions. - execute from for update. The API of this store is similar to the other stores.

[test] swap order in assert of `test_users_and_groups_non_readable_by_guests` unittest2 order is (expected, got), not the opposite. This help test report readbility.

[test] fix server/querier/build_descr1 test. There is no self.transaction of TestCase. I assumed the test writer meant a session. This seems to make the test pass.

[server] make `test_bfss_fs_importing_transparency` less verbose on failure Dumping the whole content of a file on failure is a bit annoying. We only print the bad version here (it's usually the file path).

[entity] drop redundant cache update This ``cw_attr_cache`` update is redundant, ``_cw_update_attr_cache`` already do this step.

[cw-ctl] silence msgcat and msgfmt (closes #2527594)

Added tag cubicweb-debian-version-3.15.7-1 for changeset c5400558f370

Added tag cubicweb-version-3.15.7 for changeset d8916cee7b70

prepare 3.15.7

[migration] fix crash when adding a new meta relation type. Closes #2532483

[ldapfeed] fix connection leak (closes #2532528)

[zmq] add an introductive docstring on ZMQComm

[server config] on quick start, system source is still activated. Closes #2535714

[web] use `inline` `Content-Disposition` by default (closes #2535734) since known time we have been serving static file with a `filename` parameter on the `Content-Disposition` header. But since d74addac92bb we explicitly serve file as attachment if a filename is provided. However this is valid to have `inline` disposition and `filename` parameter. This changeset revert this part of d74addac92bb, going back to `inlined` by default. `IDownloadable` code explicitly request `attachment` content to preserve expected behavior.

[web] allow configuration of the Content-disposition value The `set_content_type` function now takes and optional `disposition` parameter to control the value of this HTTP header. Use of `Content-disposition: inline` with a filename parameter are valid, so the presence of filename does not allows to choose between `attachment` and `inline`

[downloadable] fix filename HTTP header for simple name with space (closes #2535715) Since d74addac92bb, we export simple ascii filename without any encoding in the `filename` parameter of the `Content-Disposition` header. If this name contains space this will fails, the parameter value will be truncated at the space position. (eg. `filename=jungle babar.txt` read as `jungle`) We need to quote the filename to prevent this (eg. `filename="jungle babar.txt"`). Then literal quote and backslash needs to be escaped too. The new escaping is correct according this extensive test case data base: http://greenbytes.de/tech/tc2231/

merge fixes from stable

Added tag cubicweb-debian-version-3.15.6-1 for changeset b05e156b8fe7

Added tag cubicweb-version-3.15.6 for changeset 0163bd9f4880

prepare 3.15.6

[web] add a digital signature to error form (closes #2522526) Simple (and quite weak) implementation of a digital signature of the content to be submited by email in the error report view generated by ErrorView. The signature is a simple hmac hash computed using a secret key (generated at repository startup) and the "secret" form content to be included in the notification email. The controller can then check this content has not been modified or forged by a malicious user.

[web/views] bugfix: the mime type is text/plain, not text/txt (closes #2526345)

[doc] fix of personnal etc directory in book

[web] add a Forbidden exception This is similar to the Unauthorized exception, but generates a 403 error instead of a 401 (Unauthorized)

[web] add a ``anonymize-jsonp-queries`` option in file configuration (closes #2465388) This option controls connection anonymizing before executing any query for CSRF / safety reason.

[downloadable] fix filename in HTTP header (closes #2522325, #2522324) Before this changeset we use the `filename` header with utf8 encoded filename all the time. However RFC6266 says: The parameters "filename" and "filename*" differ only in that "filename*" uses the encoding defined in [RFC5987], allowing the use of characters not present in the ISO-8859-1 character set ([ISO-8859-1]). Therefore, we alter the code to: 1. Use `filename` and `ascii` encoding whenever possible, 2. use `filename*` with `utf8` encoding otherwise (with a filename fallback for old browser) We also switch the `content-disposition` value to attachement if filename is specified, this will result as a mandatory download according to RFC6266. This mandatory download is the expected behavior. We changes the filename encoding to RFC5987 which is simpler, supported by all and modern browser (including IE from version 6) and does not suffer from the continuation issue. (see ticket #2522324 for details)

docstring typo

[facet] make BitFieldFacet allow special 0 value. Closes #2522697

[doc] ubuntu LTS is now Precise Pangolin

backport stable

[server] implement base_url with secure=True (closes #2508638)

[validation api] properly use yams 0.36 validation error api and update message catalog. Follows bbe0d6985e59 the creation of the `translate` method in the 23a10f049447 yams commit.

[wsgi] saner use of `self.config` instead of `config`

[server] fix repository initialisation Changeset d753d6a6798f was breaking database creation. Dropping the `config.creating` clause lead to trying to load the schema from database even in creation mode. Conditional are a bit altered and reordered to prevent this to happen.

[web test] make unittest_viewselectors work if rdflib isn't available

[js/ajax] documentation of 'reload' is missing an argument

[ldapfeed] move docstring to the class instead of the module

[hook] fix hook base class so access to __registries__ doesn't call check_event, only call it in registered callback. Closes #2517748

[startup hook/looping tasks] separated hook for each looping task to ease modification from cubes. Closes #2517096

[ldaputils] should use entity.eid instead of entity on raising ValidationError. closes #2517095

[dbapi] provide get_option_value over DBAPIRequest (closes #2515522)

Added tag cubicweb-debian-version-3.15.5-1 for changeset 19e115ae5442

Added tag cubicweb-version-3.15.5 for changeset b0e086f451b7

prepare 3.15.5

[entity attr cache] mark attribute as uncacheable in the underlying function else we may miss some changes. Also rewrite a storage test currently failing because cache of the entity created by the test transaction, distinct from the entity created internally and given to hooks and all, has its attributes cache not updated. As this doesn't seems a proper usage, rewrite it as expected. Much probably closes #2423719 definitly.

[repository] move modification of appobject_path to repository initialization code so we can restore it later to avoid side effect on the config. Fix regression introduced in d32ab8570e5d

[req / session] drop is_internal_session (buggy) compat on base request by implementing necessary methods on internal manager

[check integrity] use session consistently

[test] use session commit/rollback to be consistent with the test

[merge] reintegrate that black sheep

[ldapparser] utf-8 uri + unicode emailaddr will crash if the later is not properly encoded (closes #2508515)

backporting

backport stable

fix no more running zmq repository test (closes #2500153)

[dbapi] load_appobjects must attempt to load available cubicweb configurations to avoid error when some object use a persistent propery (CWProperty) defined there. Closes #2497697

dummy merge

[ajax] reload function should set 'processing' cursor. Closes #2503899

[misc/scripts] a slightly experimental script to help repair LDAPUser cwusers suffering from split-brainite (closes #2497108)

[views/boxes] re-establish the proper selector (closes #2496294)

[ldapparser] raise specific error if the configuration is wrong (closes #2498164)

[skeleton] add pypi classifiers in __pkginfo__ and setup.py (closes #2502156)

[c-c shell -H] add verbosity=0 so we are not asked to confirm everything, as when not using -H

mock ``CWUser.prefered_language()`` on InternalManager objects. The recent split of patches for session refactoring broke this.

[session] fix arguments default value and promote usage of security_enabled as session method. Closes #2481820 One should use session method rather than direct usage of the context manager of the same name. Fix default argument values for consistency with the context manager: when one omit an argument, meaning is "keep the current value", not "disable security".

[session] promote usage of [deny|all]_all_hooks_but session methods rather than hooks_control context manager directly

[web app] move set of status_out into validation_error_handler to ease readability

[workflow test] don't use session.user, subject to internal changes

[db creation test] more testing of db initialization: call build_db_cache and ensure admin user exists

[source synchronization hooks] Fix/enhance system source hooks They are broken if launched during repository initialization (this was not the case yet, but will be soon...). Add additional checks to ensure one doesn't try to store system source config in the database, as it will be ignored in favor of the "sources" file.

[validation error] refactor validation error handling so translation is done on the web side Users should now use cubicweb.validation_error helper function that will activate the feature with other handy behaviours. Also test testing for message in errors should call exception.tr(unicode) before comparing. Using bare ValidationError keep backward compat.

[repo cleanup] drop code moved to querier by 7e264ce34cd4

[repo cleanup] drop code moved to querier by 7e264ce34cd4