Add a migration function in order to relocate bfss Related to https://www.cubicweb.org/ticket/1903304

[pyramid/debug] on every request display request path and selected controller This will display lines like: 2019-06-12 16:26:57 - (cubicweb.pyramid.bwcompat) INFO: REQUEST [view] '/wiki/334037' selected controller <cubicweb.web.views.basecontrollers.ViewController object at 0x7f133802ae90> at /root/.virtualenvs/abreton/local/lib/python2.7/site-packages/cubicweb/web/views/basecontrollers.py:120 2019-06-12 16:26:59 - (cubicweb.pyramid.bwcompat) INFO: REQUEST [data] '/data/ea5eb3b6f89fba0ff4b671093c88d3dd/images/cartouche-mur.jpg' selected controller <cubicweb.web.views.staticcontrollers.DataController object at 0x7f1332a84d50> at /root/.virtualenvs/abreton/local/lib/python2.7/site-packages/cubicweb/web/views/staticcontrollers.py:179 The make debugging easier as this isn't done already. Most other web frameworks do something similar for the developper to be able to understand what is happening right now. Closes #17219741

[logging/debug] move transaction operations logs from self.debug to DBG_OPS There were a mixing between displaying transaction operations log on logger.debug and only when DBG_OPS was on, uniformize everything behind DBG_OPS. Closes #17229126

[debug] syntax highlight SQL and RQL debug output This prevent DBG_RQL and DBG_SQL output to looks like a gigantic blob of white text. Do this only if we detect that we are in a tty and that pygments is installed. Closes #17222885

[logging] send logs by default to stdout Following the move to have commands only working on foreground sending logs to a file by default doesn't make much more sens anymore so send them to stdout by default and add a new option to send them to a file if needed

[cubicweb-ctl] backport --dbglevel option of pyramid to all instance commands Closes #17219653

[cubicweb-ctl] backport --loglevel option of pyramid to all instance commands

[cubicweb-ctl/fix] correctly get exception traceback_ for pdb.post_mortem In python 3 the behavior of sys.exc_info had a very subtle change: - in python 2 you can call if whenever you want after a try/except statement and you'll get information about this last raise - ipython 3, once you get out of try/except, sys.exc_info is cleaned and you'll get (None, None, None) Hardened the test to avoid this error from happening again.

Document mercurial configuration for contributing * how to use master/review repositories * how to send the "Available at" command in patchbomb emails * how to add the "cubicweb" flag in patchbomb emails * how to use our "jenkins" mercurial extension Closes #17243538

Add documentation on workflow permission

Remove undefined function in documentation

Merge 3.26

[pyramid/ctl] pyramid command will always run in foreground from now on Daemonization used to make sens in the past, but today "pyramid" command is only used for dev and should never be used for deployement (you should use a wsgi server for that instead) so daemonization doesn't fit any use case anymore. Closes #17232923

[clean] remove pyramid warning about looping task This warning isn't useful anymore.

[doc] add instruction to re-enable ckeditor

[fix] reference to underclared and unused variable kwargs

Avoid to update inlined relation column to NULL when deleting entities When deleting entities, cubicweb run a rql DELETE on all relations to trigger hooks. For an inlined relation this also mean set the column to NULL. This operation may fail if there's additional constraints on the column. Also this is a weird and useless behavior since deleting the entity row will by definition delete the relation. We still doesn't handle the case where both subject and object are going to be deleted because rows need to be deleted in a particular order that cubicweb doesn't handle. Add a test checking UPDATE does not occur but hooks are correctly called. Closes #17236690

Drop useless ConstraintInsertionTC This test does nothing and is here from the begining of repo history. Now it raise an error "TypeError: Skipped expected string as 'msg' parameter, got 'SkipTest' instead. Perhaps you meant to use a mark?" Just drop it.

Backed out changeset 741dc2590f90 It was for cube s3storage, but this piece of information was apparently not in the email (used for review), so this patch was published in the wrong repository. :-/

[.gitlab-ci] initial version (tox targets)

Merge with 3.26

Added tag debian/3.26.14-1, 3.26.14 for changeset 172f683a84f6

[pkg] version 3.26.14

[py3] Pass bytes as "msg" to smtplib.SMTP.sendmail() When passing a unicode string to smtplib.SMTP.sendmail() as "msg" argument, there is an implicit bytes encoding using "ascii" encoding in python3. Of course this does not work if the string contains non-ASCII characters. In fact, config's sendmails method intent to pass bytes to smtplib.SMTP.sendmail() as it uses msg.as_string() method. Unfortunately, in python3, this method returns a unicode string whereas it returns a bytes string in python2; we thus fix this by calling as_bytes() method on python3. As there is no "as_bytes" method in python2, we need to handle python2 compatibility by hand and either call as_string() or as_bytes(). In testlib, where we mock smtplib.SMTP, we need to keep the "msg" argument of Email class (defined in testlib as well) a unicode string. Otherwise, it fails to be parsed by email.message_from_string() (from stdlib) if it is bytes on python3.

[debian] Cleanup d/source/options from now gone symlinks

[pkg] Version 3.27.0.a2

[debug/clean] remove unused DBG_MS flag

Avoid using gone hooks_control() in migration This deprecated function got dropped in a8c1ea390400. Remove unused import in 3.10.9_Any.py migration; replace usage in bootstrapmigration_repository.py by non-deprecated form.

[tox] Remove --ignore in check-manifest check All symlinks have been dropped in previous changesets.

[doc] link to 3.27 (and missing 3.26) pages

[pkg] Version 3.27.0a1

[test] Replace symlinks by real copies This solves a "python setup.py build" issue.

[pkg] Version 3.27.0a0

Merge with 3.26

Added tag debian/3.26.13-1, 3.26.13 for changeset 1fc684253207

[pkg] Version 3.26.13

[test] Replace symlinks in legacy_cubes by real copies This solves a "python setup.py build" issue on my machine. Copying these files should not be a big deal because they have been dropped in the "default" branch and we would not touch them in 3.26 branch.

[pkg] Prune packaging files from sdist

[pkg] Fix spelling mistake in doc-base abstract

[pkg] Fix doc-base section As pointed by Lintian, the “Apps” section is only used in menu, not in doc-base. We can just remove it.

[pkg] Move lintian-overrides file to debian/source directory Just use the now preferred location.

[pkg] Tidy substvars usage in control file The documentation package was missing a Built-Using field to record the provenance of the templates. Empty packages or meta packages do not require Depends on a specific Python version, so we can remove the relevant substvars.

[pkg] Specify priority “optional” instead of the obsolete “extra” Quoting Lintian, “Since Debian Policy version 4.0.1, the priority extra has been deprecated.” So let's use “optional” instead.

[pkg] Switch to Debian source format 3.0 (quilt) This forces us to base the Debian package on the source of the Python package (as produced by running `python3 setup.py sdist`). While it might seem cumbersome, this greatly reduces likelyhood of mismatch between an installation via `pip` and one using the Debian package. `dpkg-source` will also document for us what is currently in the source repository but not in the Python package. Currently the missing file are: PKG-INFO cubicweb/sobjects/test/data/cubicweb_comment/schema.py cubicweb/sobjects/test/data/cubicweb_comment/__init__.py cubicweb/sobjects/test/data/cubicweb_comment/__pkginfo__.py cubicweb/sobjects/test/data/cubicweb_card/schema.py cubicweb/sobjects/test/data/cubicweb_card/entities.py cubicweb/sobjects/test/data/cubicweb_card/__init__.py cubicweb/sobjects/test/data/cubicweb_card/__pkginfo__.py cubicweb/server/test/data-migractions/cubicweb_comment/schema.py cubicweb/server/test/data-migractions/cubicweb_comment/__init__.py cubicweb/server/test/data-migractions/cubicweb_comment/__pkginfo__.py cubicweb/server/test/data-migractions/cubicweb_localperms/schema.py cubicweb/server/test/data-migractions/cubicweb_localperms/__init__.py cubicweb/server/test/data-migractions/cubicweb_localperms/__pkginfo__.py cubicweb/server/test/data-migractions/cubicweb_basket/schema.py cubicweb/server/test/data-migractions/cubicweb_basket/__init__.py cubicweb/server/test/data-migractions/cubicweb_basket/__pkginfo__.py cubicweb/server/test/data-migractions/migratedapp/cubicweb_comment/schema.py cubicweb/server/test/data-migractions/migratedapp/cubicweb_comment/__init__.py cubicweb/server/test/data-migractions/migratedapp/cubicweb_comment/__pkginfo__.py cubicweb/server/test/data-migractions/migratedapp/cubicweb_localperms/schema.py cubicweb/server/test/data-migractions/migratedapp/cubicweb_localperms/__init__.py cubicweb/server/test/data-migractions/migratedapp/cubicweb_localperms/__pkginfo__.py cubicweb/server/test/data-migractions/migratedapp/cubicweb_basket/schema.py cubicweb/server/test/data-migractions/migratedapp/cubicweb_basket/__init__.py cubicweb/server/test/data-migractions/migratedapp/cubicweb_basket/__pkginfo__.py cubicweb/server/test/data-migractions/migratedapp/cubicweb_tag/schema.py cubicweb/server/test/data-migractions/migratedapp/cubicweb_tag/__init__.py cubicweb/server/test/data-migractions/migratedapp/cubicweb_tag/__pkginfo__.py cubicweb/server/test/data-migractions/migratedapp/cubicweb_card/schema.py cubicweb/server/test/data-migractions/migratedapp/cubicweb_card/__init__.py cubicweb/server/test/data-migractions/migratedapp/cubicweb_card/__pkginfo__.py cubicweb/server/test/data-migractions/migratedapp/cubicweb_file/schema.py cubicweb/server/test/data-migractions/migratedapp/cubicweb_file/__init__.py cubicweb/server/test/data-migractions/migratedapp/cubicweb_file/__pkginfo__.py cubicweb/server/test/data-migractions/cubicweb_tag/schema.py cubicweb/server/test/data-migractions/cubicweb_tag/__init__.py cubicweb/server/test/data-migractions/cubicweb_tag/__pkginfo__.py cubicweb/server/test/data-migractions/cubicweb_card/schema.py cubicweb/server/test/data-migractions/cubicweb_card/__init__.py cubicweb/server/test/data-migractions/cubicweb_card/__pkginfo__.py cubicweb/server/test/data-migractions/cubicweb_file/schema.py cubicweb/server/test/data-migractions/cubicweb_file/__init__.py cubicweb/server/test/data-migractions/cubicweb_file/__pkginfo__.py cubicweb/test/data-rewrite/cubicweb_localperms/schema.py cubicweb/test/data-rewrite/cubicweb_localperms/__init__.py cubicweb/test/data-rewrite/cubicweb_localperms/__pkginfo__.py cubicweb/test/data-rewrite/cubicweb_card/schema.py cubicweb/test/data-rewrite/cubicweb_card/entities.py cubicweb/test/data-rewrite/cubicweb_card/__init__.py cubicweb/test/data-rewrite/cubicweb_card/__pkginfo__.py The current manifest will duplicate several files that are stored as symlinks in the Mercurial repository, hence the long list of ignored files in `extend-diff-ignore`.

[debian] Update description of cubicweb-ctl* packages init scripts are no longer shipped. Avoid mentionning "start" and "stop" command which should not be used in production environments.

[debian] Introduce a cubicweb-ctl3 binary package This packages conflicts with cubicweb-ctl binary which uses the python2 library. It is meant to ease upgrade of existing installation during the migration from python2 to python3: in particular, by installing cubicweb-ctl3, scripts calling "cubicweb-ctl" command should still work as before.

[cubicweb-ctl] respect sys.exit status code when aborting a command When exploring the stack of all calls to a cubicweb-ctl command, it has been discovered than on a KeyboardInterrupt and on a SystemExit exception the base class InstanceCommand (for commands that works on one instance) will always set the return code of cubicweb-ctl to 8: this mean that if another command do a `sys.exit(some_code)` the exit code will be ignored and overwritten by '8'. This behavior is not intuitive, apparently not documented and doesn't seems to have any justification. It also prevent commands from exciting with different return codes which could be a desired behavior in the situation of scripting.

[test] Add an "actionverb" attribute to test commands

[test] use self.assertRaises() in cwctl tests

[test] ensure mocking is stopped in cwctl tests Either use a decorator or ensure that cleanup is performed. This is so as to prevent global state modification.

[cubicweb-ctl] add '--pdb' global option to launch (i)pdb on exception

[test] Rename TestCommand to avoid PytestCollectionWarning

[pkg] Add new autopkgtest for skeleton packaging When running `cubicweb-ctl newcube`, the generated template provides debian packaging. So let's add automated tests for that using autopkgtest. The new test will generated a new cube, build a source tarball, build the package, try to install it, see if the Python 2 and 3 modules are available and even run autopkgtest against the newly created packages. Along the way it will also print information about the generated package: buildinfo, package content, and lintian report.

[skeleton] Update Debian packaging template Here are multiple updates to the Debian packaging template provided when running `cubicweb-ctl newcube`: * Switch to source format 3.0 (quilt). * Switch to debhelper 9. * Switch to dh-python. * Add Python 3 packages. * Name the binary packages `python-cubicweb-CUBENAME` and `python3-cubicweb-CUBENAME` instead of just `cubicweb-CUBENAME` (which is still the source package). * Populate Depends using dh_python{2,3} support for Python requirements. * Run test suite at build time using pytest. * Add autopkgtest to run test suite against the installed package. * Bump Standards-Version to 4.3.0.

[test] Use --short-description when testing newcube command

[devtools] Allow to specify short desc on `newcube` command line In order to implement automated testing of `cubicweb-ctl newcube`, we need to avoid any user interaction. So we add a `-s` switch to specify a short description on the command-line instead of requiring an input.