Project Structure Related to #4291173

[tox] Have less test environments This is to reduce the load on CI server because of parallel clones of the repository in Docker containers. Only keep "server", and "web" environments separated. All other prior environments are in "misc". To avoid duplicate entries in requirements files, move them all in a "requirements" directory (this appears to be a "common" practice, see e.g. Celery). Adjustments in tests: * Rename cubicweb/hooks/test/unittest_notification.py so that it does not conflict with cubicweb/sobjects/test/unittest_notification.py during test discovery as they would have the same module name but different __file__ attribute. * Add "comment" cube to the list of expected cubes in unittest_cwconfig.py as this cube is pulled by requirements/test-mist.txt. Closes #15440662.

[repo] Don't crash on start when fs schema is missing some db schema entities This occurs usually while developping and we don't want systematically to rebuild the database to start the instance or run e.g. i18ninstance.

Let configuration option be overridden by environment variables Related to #13889793.

[tox] Add a dummy test-results.xml file in check-manifest and flake8 environments This is to work around Jenkins Junit plugin that does not apparently account for the absence of this file (though it accepts it to be empty).

[tox] Basic flake8 config

Use pkgutil.ImpLoader instead of a custom loader for cubes import redirection It just works fine (same implementation of load_module method as _CubesLoader introduced in d404fd8499dd) and is complete w.r.t. PEP 302 (all methods implemented). Related to #13001466.

[config] Make available_cubes aware of cubes installed as packages For this add a "cubicweb.cubes" entry points group which will be scanned through to find out installed cubes. Entries in this group are expected to expose the module name of a cube (i.e. `cubicweb_foo`). Note that CubicWebConfiguration's available_cubes method will return the module name of cubes as packages (cubicweb_foo), so we had to add a special "key" sorting function to keep cubes sorted as before, despite possible different distribution schemes. This makes it possible to handle loading of CTL plugins in an almost similar manner as before (just tweaking the package name from cube name in load_cwctl_plugins method). I had to tweak (again?) the test_cubes_path method in unittest_cwconfig.py but did not find out why. Apart from unforeseen bugs and pending documentation, this finishes the work on porting cubes to standard Python packages. Closes #13001466.

[skeleton] Use distname directly in setup.py distname is part of required metadata and modname cannot actually be used in place of it.

[skeleton] Fix modname in __pkginfo__

[test] Use plain unittest in unittest_cwctl.py

[devctl] Remove commented LiveServerCommand The code got commented in 2010 (changeset 5ab3f63b06ad)...

[pkg] Inline call to setup() in setup.py The point of having this "install" function, called in "main" mode is not clear. Better stick to standard practices.

[pkg] Drop old checks in setup.py

[tox] Exclude symlinks from check-manifest call These are not handled by check-manifest, see https://github.com/mgedmin/check-manifest/issues/69.

Drop the only reference to an '__insert' form key

Drop bits from the past (no more dbapi isolated installation)

[devtools] CWTC.request is gone, don't attempt to use it in CWTC.view So one will have to provide at least one of `rset` or `req` argument of the `view` method.

Merge 3.22 into 3.23 Some fixes there are necessary for client work depending on cubicweb 3.23.

[devtools] Handle i18ncube command for "cubes as packages" And add a test so that both layouts are tested. Move the test cube into a package. Related to #13001466.

Handle cubes as packages in cwconfig Rely on importlib.import_module in several places in cwconfig.py, so this would not work with python 2.6. Method available_cubes will not list cubes installed as package for now. I'm not sure it's worth implementing this method (used in `cubicweb-ctl list cubes` command) for new cubes layout as the same result can basically be obtained with `pip freeze | grep cubicweb_`. In unittest_cwconfig.py, duplicate CubicWebConfigurationTC test case to test both the "cubes as packages" layout and the "legacy cubes" layout. The former having a custom sys.path set (pointing to datapath('libpython') where all cubes' packages live) and the latter having the previous config attribute setup. All test data cubes are moved to packages in libpython directory and symlinks are introduced in the cubes directory. Related to #13001466.

[cwconfig] Reorder imports by alphabetic order

Add an import redirect hook from "cubes.<name>" to "cubicweb_<name>" The hook consists of a finder and a loader implemented following PEP-302; it is responsible for loading cubes distributed as packages (i.e. installed as ``cubicweb_<name>`` in site-packages) but imported (in client code) as ``from cubes.<name> import ...``. So this is a transitional mechanism allowing cubes following the new layout to be used by old-style cubes/applications. The importer is installed upon calling CubicWebConfiguration's cls_adjust_sys_path method (also called in cubicweb.devtools.__init__.py, which is a prerequisite for importing any "legacy" cube. The loading of old-style cubes is still handled by the CubicWeb configuration, based on adjustment of sys.path etc. Related to #13001466.

[devtools] Update skeleton's setup.py to install cube as a package Most of the prior logic of skeleton's setup.py gets dropped as installing a cube as a "classic package" (i.e. in site-packages) is just the default behavior of distutils. Also add a test checking installation of new cube. Related to #13001466.