author Philippe Pepiot <>
Tue, 31 Mar 2020 18:22:05 +0200
changeset 12966 6cd938c29ca3
parent 12851 e434d8f4734c
permissions -rw-r--r--
[server] Make connection pooler configurable and set better default values Drop the configuration connections-pool-size and add new configurations options: * connections-pool-min-size. Set to 0 by default so we open connections only when needed. This avoid opening min-size*processes connections at startup, which is, it think, a good default. * connections-pool-max-size. Set to 0 (unlimited) by default, so we move the bottleneck to postgresql. * connections-idle-timeout. Set to 10 minutes. I don't have arguments about this except that this is the default in pgbouncer.


New features

* Tests can now be run concurrently across multiple processes. You can use
  `pytest-xdist`_ for that. For tests using `PostgresApptestConfiguration` you
  should be aware that `startpgcluster()` can't run concurrently. Workaround is
  to call pytest with ``--dist=loadfile`` to use a single test process per test
  module or use an existing database cluster and set ``db-host`` and
  ``db-port`` of ``devtools.DEFAULT_PSQL_SOURCES['system']`` accordingly.

.. _pytest-xdist:

* on `cubicweb-ctl create` and `cubicweb-ctl pyramid`, if it doesn't already
  exist in the instance directory, the `pyramid.ini` file will be generated
  with the needed secrets.

* add a --pdb flag to all cubicweb-ctl command to launch (i)pdb if an exception
  occurs during a command execution.

* the --loglevel and --dbglevel flags are available for all cubicweb-ctl
  instance commands (and not only the ``pyramid`` one)

* following "only in foreground" behavior all commands logs to stdout by
  default from now on. To still log to a file pass ``log_to_file=True`` to

* add a new migration function `update_bfss_path(old_path, new_path)` to update
  the path in Bytes File-System Storage (bfss).

* on every request display request path and selected controller in CLI

* migration interactive mode improvements:

  * when an exception occurs, display the full traceback instead of only the exception

  * on migration p(db) choice, launch ipdb if it's installed

  * on migration p(db) choice, give the traceback to pdb if it's available,
    this mean that the (i)pdb interactive session will be on the stack of
    the exception instead of being on the stack where pdb is launched which
    will allow the user to access all the relevant context of the exception
    which otherwise is lost

* on DBG_SQL and/or DBG_RQL, if pygments is installed, syntax highlight sql/rql
  debug output

* allow to specify the instance id for any instance command using the
  CW_INSTANCE global variable instead of or giving it as a cli argument

* when debugmode is activated ('-D/--debug' on the pyramid command for
  example), the HTML generated by CW will contains new tags that will indicate
  by which object in the code it has been generated and in which line of which
  source code.
  For example:

.. code-block:: html

            unset title

While this hasn't been done yet, this feature is an open path for building dynamic tools that can help inspect the page.

* a new debug channels mechanism has been added, you can subscribe to one of
  those channels in your python code to build debug tools for example (the
  pyramid custom panels are built using that) and you will receive a
  datastructure (a dict) containing related information. The available channels
  are: controller, rql, sql, vreg, registry_decisions

* add a new '-t/--toolbar' option the pyramid command to activate the pyramid debugtoolbar

* a series of pyramid debugtoolbar panels specifically made for CW, see bellow

Pyramid debugtoolbar and custom panel

The pyramid debugtoolbar is now integrated into CubicWeb during the development
phase when you use the 'pyramid' command. To activate it you need to pass the
'-t/--toolbar' argument to the 'pyramid' command.

In addition, a series of custom panels specifically done for CW are now
available, they display useful information for the development and the
debugging of each page. The available panels are:

  * a general panel which contains the selected controller, the current
    settings and useful links `screenshot <../../_images/debugtoolbar_general_panel.png>`_
  * a panel listing all decisions taken in registry for building this page `screenshot <../../_images/debugtoolbar_registry_decisions_panel.png>`_
  * a panel listing the content of the vreg registries `screenshot <../../_images/debugtoolbar_registry_content_panel.png>`_
  * a panel listing all the RQL queries made during a request `screenshot <../../_images/debugtoolbar_rql_panel.png>`_
  * a panel listing all the SQL queries made during a request `screenshot <../../_images/debugtoolbar_sql_panel.png>`_

Furthermore, in all those panels, next to each object/class/function/method a
link to display its source code is available (shown as '[source]' `screenshot
<../../_images/debugtoolbar_show_source_link.png>`_) and also every file path shown
is a traceback is also a link to display the corresponding file (`screenshot
<../../_images/debugtoolbar_traceback_source_link.png>`_). For example: `screenshot

Backwards incompatible changes

* Standardization on the way to launch a cubicweb instance, from now on the
  only way to do that will be the used the ``pyramid`` command. Therefore:

   * ``cubicweb-ctl`` commands "start", "stop", "restart", "reload" and "status"
     have been removed because they relied on the Twisted web server backend that
     is no longer maintained nor working with Python 3.

   * Twisted web server support has been removed.

   * ``cubicweb-ctl wsgi`` has also been removed.

* Support for legacy cubes (in the 'cubes' python namespace) has been dropped.
  Use of environment variables CW_CUBES_PATH and CUBES_DIR is removed.

* Python 2 support has been dropped.

* Exceptions in notification hooks aren't catched-all anymore during tests so
  one can expect tests that seem to pass (but were actually silently failing)
  to fail now.

* All "cubicweb-ctl" command only accept one instance argument from now one
  (instead of 0 to n)

* 'pyramid' command will always run in the foreground now, by consequence the
  option ``--no-daemon`` has been removed.

* DBG_MS flag has been removed since it is not used anymore

* transactions db logs where displayed using the logging
  (debug/info/warning...) mechanism, now it is only displayed if the
  corresponding DBG_OPS flag is used

Deprecated code drops

Most code deprecated until version 3.25 has been dropped.