[statsd] add a statsd_timethis context manager This context manager allows to report to statsd the execution time of any piece of code, without the need for it to be a function or a method, like:: def some_function(): do_stuff() with statsd_timethis('some context message'): do_that() ensure_this()

[test/rset] add tests for ResultSet.first() and .last() methods

[test/rset] flake8

[rset] update copyright

[rset] add convenient first() and last() methods on ResultSet These make life easier for the heavy user of the CW shell. Also aliases entities() with all(); so we end with a pretty consistant set: - .one() - .first() - .last() - .all()

[server] Fixed issue with the adapters cache of a cloned EditedEntity The adapters' cache of the entity within an EditedEntity used to be copied along the entity itself during a cloning operation of the EditedEntity. This raises an issue when the adapters' cache is not empty and contains cached values linking to transaction-specific data (or the transaction itself). This prevents the clone EditedEntity being used in a transaction outside the one that created it. This change resets the adapters' cache of the new copy so that no cached value remain. The new copy can then be used in a new transaction afterwards.

[rql2sql] fix handling of the ORDERBY clause when the aggregation function is "embedded" ie. a situation like: Any X GROUPBY X ORDERBY COUNT(Y) WHERE ... works fine since the aggregation function (COUNT) if the first one, whereas a situation like the proposed test: Any X GROUPBY X ORDERBY ABS(COUNT(Y)) WHERE ... failed due to the fact we only checked the top aggratation function (ABS). Closes #17108671

Merge 3.26

[pkg] Set version to 3.27.0.dev0

Added tag 3.26.3, debian/3.26.3-1 for changeset f7067be5f69c

[pkg] Version 3.26.3

[utils/statsd_logger] flake8

[source/native] do NOT do anything with fti if do_fti is False ensure no fti related process is done when fti is disabled on a db source.

[test/web] flake8 unittest_propertysheet.py

[web] fix css file generation permission bug (closes #17143773) PropertySheet.process_resource() uses mkstemp which enforce a 0600 mode on the output file, which is not the file permission we want for css files.

[test/web] add a test for css generation file permission problem (related to #17143773)

[doc] include the 3.26 changelog in changes's index

[server] set PGPASSWORD when running pg commands for backup and restore For db-dump (implied by upgrade) and db-restore with postgres with non-trusted authentication, the db password is asked by pg tools (pg_dump, pg_restore, createdb, dropdb). Avoid this behavior by setting the PGPASSWORD environment variable when database driver is postgres and db-password is set in sources file.

Added tag 3.26.2, centos/3.26.2-1, debian/3.26.2-1 for changeset 9bee3134d304

[pkg] Version 3.26.2

Flake8 cubicweb/web/webctl.py Having the file flake8 clean makes it easier to catch basic programming errors (e.g. name errors).

[web] Add missing import of _DATA_DIR in webctl Followup on d84bc85f7f70 which introduced usage of this name in _datadirs method of GenStaticDataDirMixIn but resulted into a NameError at runtime and makes migrations crash.

[server] unicode → text_type in utils

[views] unicode → six.text_type in views/authentication.py

[pyramid] Recreate CWSession entity when eid no longer exists As noted in comment, it might occur that we receive a "sessioneid" from a valid cookie while respective CWSession got dropped (typical case is db being recreated while users got preserved). In such case, we just recreate the CWSession entity as if it did not exist.

Added tag 3.26.1, debian/3.26.1-1, centos/3.26.1-1 for changeset 027676243aaa

[pkg] Drop non-existant files from MANIFEST.in

[pkg] Version 3.26.1

[test] Drop an unused function in cwconfig tests

[cwconfig] Always use "sys.prefix" to determine installation prefix We drop all legacy logic implemented in _find_prefix() which now happens to be buggy since we cleaned up setup.py in 3.26 (noticeably because virtualenv is no longer properly detected). Closes #17132990.

[cwconfig] Look for migration scripts directory relatively to code Thus getting rid of the last piece of data in "shared" directory. As a consequence, we can drop data_files from __pkginfo__.py. Related to #17132990.

[pkg] Get rid of "pydir" in __pkginfo__.py Nothing uses this.

[pkg] Get rid of "shared" data These are not used anymore per previous changesets. Related to #17132990.

[cwconfig] Drop shared_dir method There's no more users of this method. Related to #17132990.

[config] Avoid relying on shared_dir method when possible From 3.26, some "data" directories (like web/data and i18n or wdoc) are installed along with CubicWeb in site packages. So we can retrieve them relatively to the module that needs them (namely "cubicweb" for "i18n" and "cubicweb.web" for "web/data" and "wdoc"). Reason for doing this is that "share_dir()" is not working when cubicweb is installed in "develop" mode and we are about to drop this method. Related to #17132990.

[test] Add a test for WebConfiguration.locate_all_files() We are going to change the implementation, make sure there's no regression.

[cwconfig] Get rid of _forced_mode global And compute CubicWebNoAppConfiguration.mode within the class namespace. Related to #17132990.

[cwconfig] Get rid of CWDEV This is in preparation of a fix for #17132990 to cleanup legacy (and probably no longer working) behaviors.

[doc] Set release date of 3.26

[web] Fix UnboundLocalError about "exc" in ajax controller Introduced in 159dce89a145.

Link to "3.26" docs in README

Added tag 3.26.0, debian/3.26.0-1, centos/3.26.0-1 for changeset 199851fcddd4

[pkg] Version 3.26.0

[pkg] Version 3.26.0rc0

[schema] Define a decorator to handle methods override There is a number of external classes (from yams/rql) methods overridden "by hand" in cubicweb/schema.py. Define a single decorator to factor out the pattern. It handles specifying the method name (to avoid conflict with the local namespace) and passing the original method to the new definition when needed.

[pkg] Use setuptools.find_packages() We drop the custom get_packages() function in setup.py that used to look for test files (not discovered automatically since test directories are not packages) and replace it by explicit entries in MANIFEST.in.

[pkg] Drop SandboxViolation verification cracking Dunno what this is for. Let's see if this work without.

[pkg] Drop custom "install_data" command Not sure what this is for, the comment mentions "egg" directory, but this is quite likely an old hack that should be unnecessary when using a recent setuptools.

[pkg] Replace custom "scripts" handling by a setuptools entry points http://setuptools.readthedocs.io/en/latest/setuptools.html#automatic-script-creation

Set a default value for arguments of cubicweb.cwctl:run() This will help making a console script entry point from this to get rid of the custom installation of scripts in setup.py. Accordingly, update all calls of this function.

[pkg] Drop custom install_lib command and use include_package_data instead The custom install_lib command appeared to be meant to install package data, as declared in __pkginfo__.py. Let's rely on setuptools' include_package_data[1] option to handle this. [1]: http://setuptools.readthedocs.io/en/latest/setuptools.html#including-data-files

[py3] Replace a unicode() occurrence in workflow views

[py3] Drop a couple of unicode() in devtools.testlib

[py3] Alias long to int in pgstore tests The previous version of handling PY2/PY3 made flake8 complain.

Merge with 3.25

Added tag 3.25.4, centos/3.25.4-1, debian/3.25.4-1 for changeset b8567725c473

[pkg] Version 3.25.4

[pyramid] Let logging be configured through .ini file We do not initialize logging in CubicWebPyramidConfiguration thus preventing logging to be started with cubicweb's configuration. On the other hand, any logging configuration defined in ``development.ini`` file will be loaded by whatever start the instance using this file (i.e. pserve, gunicorn, etc.). The benefit is that logging can now be configured per "qualified name" in this file (i.e. one can easily set the DEBUG level for the application cube while keeping all dependencies' level to WARNING). In the development.ini template, we add logger configurations for "logilab" and "cubicweb" qualified names (along with those of the application cube).

Disable pytest's logs reporting CubicWeb already prints captured logs in stderr upon test failure, so we get them twice. Until we fix our logging management, disabling pytest capture and restore brings the output level back to sanity.

[pyramid] Do not issue security warnings in test mode When some session or authtk secret is missing in Pyramid settings, scary "!! SECURITY WARNING !!" are issued. This is arguably pointless in tests. So disable them in this case.

[ssplanner] Prevent execution of write queries involving computed relations Previously, setting a computed relation upon entity creation or modification (using the ORM or an RQL query) would usually fail with an operational error in the backend ("no such table"). However, under some mysterious circumstances (like passing a string as value in cw_set for a computed relation), the RQL to SQL transformation would simply drop the clause. To prevent this to happen, we add a check for computed relation before adding a relation to an execution plan. This check raises a QueryError. It happens in several places: * in querier.InsertPlan.add_relation_def() (called from several places in ssplanner steps) for INSERT queries, * in ssplanner.UpdateStep.execute() for SET queries and, * in ssplanner.SSplanner.build_delete_plan() for DELETE queries. Tests added to unittest_querier.py because unittest_sslplanner.py looked inappropriate (it has only unit tests) and the former already had a NonRegressionTC class.

[querier] Expand 'rdef' argument in InsertPlan.add_relation_def() Expand the 'rdef' tuple to improve readability. The 'rtype' value is not used at the moment, but will in the next changeset.

[server] Use dict.setdefault() instead of if/else in repo.glob_add_relations()

[cleanup] Fix some flake8 errors

[test] Stop using lgc.testlib in unittest_migractions

[test] Drop no more used "maxeid" based deletion in BaseQuerierTC and derived tests This is probably only necessary for QuerierTC itself, move it there and drop incantation from other derived classes to deactivate this feature.

[rql2sql/test] Remove dead setUpModule tearDownModule they are overwritten a few lines below , hence this code is dead and not necessary for test success, drop it.

[rql2sql/test] Move base class from repotest to unittest_rql2sql It's only used there, so move it closer to its usage. Along the way, stop using lgc.testlib.

[rql2sql/test] Set maxDiff to None to see full diffs this is usually useful to compare SQL strings

[rql2sql] Use string_type instead of unicode for py3k support

[gcdebug] Only ignore weakref.WeakKeyDictionary class on Python 2 From Python 3.5, this class fails isinstance check with the following error: :: cls = <class 'weakref.WeakKeyDictionary'>, instance = <functools._lru_list_elem object at 0x7f4331859a48> def __instancecheck__(cls, instance): """Override for isinstance(instance, cls).""" # Inline the cache checking > subclass = instance.__class__ E AssertionError: [<class 'AttributeError'> in gc] 'functools._lru_list_elem' object has no attribute '__class__' /usr/lib/python3.5/abc.py:181: AssertionError I have no clue why this happens, but it makes cubicweb.web.test.test_views.AutomaticWebTest.test_startup_views fail on Python 3.5. So only consider this class for Python 2.

[wsgi] Convert "code" passed to WSGIResponse as integer WSGIResponse is passed a Request.status_out in CubicWebWSGIApplication._render(), which, starting from Python 3.5, is an instance of HTTPStatus. However, webtest still expects a 3-digits value as status string. Fortunately, calling int() on an HTTPStatus works.

[test] Add a "version" to cubicweb_mycube/__pkginfo__.py Starting from Python 3.5, unittest_cwctl.py::CubicWebCtlTC.test_list() would fail without this.

[tox] Use py3 environment instead of py34 The former would select any Python 3 interpreter.

[querier] Allow usage of computed relation into WHERE clause of write queries This used to crash because of an attempt to access unexisting table. Closes #17113286

[cleanup] Fix some flake8 errors

[tox] Bump flake8 version to 3.5 Previous patches fixed new errors raised in this version.

[pyramid] Get rid of a couple of bare "except"

[pyramid] Get rid of a bare "except" in UpdateLoginTimeAuthenticationPolicy Also refactor a bit the method to restrict the try/except block to the code for which we actually want to catch an exception (i.e. the cnx.execute() + cnx.commit()).

[web] Avoid bare "except" in ConcatFilesHandler.concat_cached_filepath()

[test] Use ignore_errors=True in shutil.rmtree instead of try/except/pass

[server] Get rid of a bare except in sqlutils

[server] Do not use logilab.common.date.strptime in sqlutils This function is aliased to datetime.datetime.strptime() and exists only for compatibility with Python <= 2.5 which we no longer support.

[dataimport/test] Ignore flake8 error E741 in test_pgstore.py This error is about "l" being an ambiguous variable name. Here, it's arguably acceptable.

[test] Avoid ambiguous variable name "l" in unittest_req.py This is an error (E741) from flake8 3.5.

[tox] Bound flake8 version to >=3.4, <3.5 The code base is not clean with 3.5, so keep using 3.4 in *stable* branches.

[pyramid] set waitress url_scheme to https when base-url scheme is https See https://docs.pylonsproject.org/projects/waitress/en/latest/#using-behind-a-reverse-proxy Since we do not configure trusted_proxy waitress ignore X-Forwarded-Proto header and may generate http urls (pyramid request.url) when the instance is behind a https reverse proxy. This cause cubicweb-signredrequest to not work with since it rely on the url (including scheme). Set url_scheme to 'https' when CubicWeb base-url scheme is https as a workaround.

Added tag 3.25.3, centos/3.25.3-1, debian/3.25.3-1 for changeset d238badfc268

[pkg] version 3.25.3

[uicfg] Fix autoform_section rtags wrt derivation cw 3.25 introduced the 'derived rtags' feature, but missed that `autoform_section` implementation was overriding several methods where this was implemented in the base class. The following problems are tested and fixed: * during init, we should not attempt to call `_initfunc_step2` if there is some parent, * but we should handle expansion of wildcard ('*' used as subject or object of a tag) since this is handled during init step for this class, unlike others which handle this in `get` method (i.e. at lookup time), * fix overrided `get` method to consider parent rtag if any. Closes #17107020

[rtags] Enhance repr of rtags * we don't care of the class, the registry id is enough (never seen a case of sub-classing) * add the module from which the rtag is coming as well as parent rtag if any.

[web/test] Use bare unittest main instead of lgc.main

[cleanup] Fix some flake8 errors and drop upper bound of copyright along the way. autoform module is not added to flake8-ok-files since there are some remaining errors to be handled, but that's still a start.

[doc] Update the advanced tutorial to follow the new style layout of cubes among others.

[views] Respond with 400 Bad Request in case of validation error in ajax request Previously, we raised a plain RemoteCallFailed without a status code, in which case it defaults to "500 Internal Server Error". Now, for validation errors, we issue a 400 Bad Request since these are clearly client errors.

[web] fix session /cnx mix in anonymized_request

[test] Pin some test dependencies We currently have CI failures because cubes used as test dependencies have been updated to new-style cube layout. To avoid this, pin them to previous released. Those dependencies should be removed but in the mean time this should be enough (and backported in all active branches).

[test] Pin some test dependencies We currently have CI failures because cubes used as test dependencies have been updated to new-style cube layout. To avoid this, pin them to previous released. Those dependencies should be removed but in the mean time this should be enough (and backported in all active branches).

[web/form] Relax condition for non-translating field names we should test true condition instead of None, so if one raise a ValidationError using an empty string as 'no-field' marker instead of None, we don't get the PO file header that gettext likes to return when one attempt to translate an empty string.

Merge with 3.25 branch

[entities] Restore dc_ methods instead of __getattr__ proxy In 8de62610cea2, a __getattr__ proxy was introduced to handle compatibility with dc_* methods being called on entity instances instead of through the IDublinCore adapter. Unfortunately, __getattr__ does not play well with super() and since the latter is often used by, we here restore all dc_ methods with an explicit proxy to the adapter. More verbose but should work. Closes #17103999.

[pkg] add missing unittest2 dependency to spec file

Added tag 3.25.2, debian/3.25.2-1, centos/3.25.2-1 for changeset 5010381099f1

[pkg] Prepare version 3.25.2

[cubicweb/pyramid] take into account two ignored CORS configuration items from all-in-one.conf

[cwctl] upgrade: do delete the static data dir if verbosity is 0 (closes #17069749)

[doc] Fix option_*() migration commands in doc The function signature in the doc is not the same as in the code. Also option_removed should probably be removed from the doc since the function in migration.py is empty

[massive store] ensure the cwmassive_initialized table is deleted if needed This may be necessary if a previous import did badly crached, leaving the cwmassive_initialized behind.

[server] fix deserialize_schema to handle properly pg schema (closes 17076486) Postgres behaviour is to look first for a table in the configured schema then fall back to 'public' if no table is found. This must be implemented in deserialize_schema when listing existing tables to check for computed relations (cw_cwcomputedrtype) otherwise computed relations may not work when using non-default pg schema.

[server/utils] update passlib API to 1.7 (closes #17054805) while keeping bw-compat with 1.6 - 2.7 is not available for all distributions, e.g. jessie. The dependancy has been bumped in 6186cd15a46d because of a change in bc9d901cb9e6. We may easily use the appropriate method depending on the version installed.

Merge with 3.25 branch

Added tag 3.25.1, debian/3.25.1-1, centos/3.25.1-1 for changeset 5fe62978801a

[pkg] Version 3.25.1

[facets] Fix disappearance of navtop component on facet filtering which is because facets are replacing the whole #pageContent div, while this one contains other stuff than the view: * a type selector component that should be dropped for a while, * a computed title, * the page navigation. Then the view content itself is in a #contentmain div. The thing is that the navigation should be rebuilded on filtering (this is not the case for other bullets in the list above). This is currently handled specifically in the ajaxcontroller (except for the type selector which will disappear... who said it should be dropped at once?). So to fix this we: * put the page navigation into the "contentmain" div * don't replace anymore "pageContent" but "contentmain" After that we can even remove from the ajax controller the code that reimplements title handling similarly to the main template. Notice the part that changes the main template has to be ported to squareui. Closes #17074195

[web] Set response status to 400 when appropriate in ajax controller When this is clearly a client error, set status to 400. Otherwise, keep the default value for RemoteCallFailed (status=500).

[web] Only log exceptions in debug mode in Ajax controllers When we raise a RemoteCallFailed error (erroneously turned into a '500 Internal Server Error' response, but that's another business). In production environment, we do not want to log the exception in most cases where it's actually a client error. So only log the exception in debug mode.

[pyramid] Only expose 'cubicweb.bwcompat' setting for "all-in-one" configuration type The "pyramid" instance configuration does not work with "cubiwceb.bwcompat" mode (on purpose). Yet, having the setting exposed in development.ini file (generate by `cubicweb-ctl create --config pyramid <cube> <instance>` command) is misleading and we want to remove it. Thus, we only query this setting when cubicweb configuration is "all-in-one" and drop the setting line from templated development.ini file. If the option is found and True for any other configuration type, we issue a user warning (and ignore the option).

[migration] Only drop moved table entities if it does not exists For very old instances (namely docaster), this table has never been created.

[web/views] delete: show composite entities in predictible order Iterate over relations type in alphabetical order, so the order is predictible dans does not depend on PYTHONHASHSEED.

[req] fix find() generating non-rewritable rql on non final relations When filtering on a relation, find() was generating rql like 'Any X WHERE X is ETYPE, X relation EID' which work without being rewritten (it should probably not), but when applying some rewrite (eg. permissions) it raise in rqlrewrite code. def _use_orig_term(self, snippet_varname, term): ... > self.rewritten[key] = term.name E AttributeError: 'Constant' object has no attribute 'name' Generate valid rql instead 'Any X WHERE X is ETYPE, X relation Y, Y eid EID'.