[test] Use stdlib doctest module instead of logilab-common class The former just works fine now but the later does not play well with py.test.

[test] Re-implement generative tests with subtests

[devtools] Re-implement generative tests using subtests Generative tests as implemented in logilab.common.testib are not compatible with tests runner other than lgc.pytest and this implementation differs from the standard library, which has support for subtests_ since Python 3.4. Use unittest2 to bridge the gap. Maybe it'd be good to implement this on logilab.common.testlib side at some point. Let's see how this gets received here first. .. _subtests: https://docs.python.org/3/library/unittest.html#subtests

cleanups

[pkg] python-tz does not exist in centos should be pytz (closes #10290056)

[test] Skip LDAP tests if any of the underlying programs is not found Some programs are in ldap-utils or ldapscripts, which may not be installed despite slapd may. Follow-up on 282880f81311.

Handle virtualenv development mode Looking for VIRTUAL_ENV environment variable as a sign of virtualenv activation. This at least works for developing CubicWeb itself (not sure yet about cubes). And it makes `tox --develop` work as well. Add a test for cwconfig._find_prefix within a virtualenv. In tox.ini, use `develop option`_ accordingly. This makes tox running significantly faster since it avoids building and installing the sdist. Aside, the "develop" install is now performed *after* all test requirements are processed, which means that the cubicweb package should always be the one from source -- instead of a released version pulled by a cube through test dependencies, which may occur in "recreate" step. .. _`develop option`: <http://tox.readthedocs.org/en/latest/config.html#confval-usedevelop=BOOL

[skeleton] Use os.walk in setup.py for Python3 compatibility

[skeleton] Use unittest.main directly Instead of lgc.testlib.unittest_main as they are identical.

[devtools/testlib] Disable pause trace in setUp Closes #9856028.

[test] Remove dependency on person cube in devtools package

[pkg] add missing python-tz dependency

[edit controller] Fix composite entity reparenting when inlined This implies handling the reparenting case sooner, otherwise the child entity modifications are cancelled (see 113e9da47afc). Add a new test to ensure non-regression in all previous tests. Closes #10291977.

[forms] Do not raise errors on composite required relation being deleted The entity included in the composite will be deleted in such a case, so the field is not required even when the cardinality is '1' or '+'. Related to #8529868

[edit controller] Cancel RQL queries to be performed on entities to be deleted Composite relation removal via the EditController trigger an entity removal. Changes on such a to-be-removed entity must be cancelled as they may trigger integrity errors before the deletion occurs. Closes #8529868.

[edit controller] fix handling of removal of subentities from an edit form In CubicWeb 3.18 the semantics of dropping a composite relation were changed. It had before the effect of deleting the subentities. Now the edit controller must handle this by itself (it was not adapted then by mistake). So if it detects the removal of a composite relation (on the browser side, the user clicked on the "remove" action), the subentity is now explicitly deleted. It was chosen not to change the form generation but the EditController: - to ensure pre-3.18 forms backward compat - to avoid the introduction of a specific entity-deletion API (RQL is probably our best API for complex DB operations) Related to #8529868.

remove unused import

[editcontroller] a slight code reorganisation, in preparation for a fix * `entity` defined at method entry * a combined `continue` and a dedent of the next big block * some vertical white space Related to #8529868.

[sources/native] ask permission to continue restoring even if metadata do not match (closes #7771864) Moreover we try to clarify the message (Expected/Found was never too clear for users).

[views] fix ProcessInformationView: SESSION_MANAGER can be None (closes #5753280) eg. with pyramid. (grafted from 6a07c2dc3d85f4de0cf48221607e02c4473e11e4)

[web] Move "display all"/ "back to pagination" link generation into the navigation component This makes the html generation customizable. Closes #9219057

Reorganize source tree to have a "cubicweb" top-level package Basically: mkdir cubicweb hg mv *.py -X setup.py cubicweb hg mv dataimport devtools entities etwist ext hooks i18n misc schemas server skeleton sobjects test web wsgi cubicweb Other changes: * adjust path to cubicweb-ctl in devtools tests * update setup.py to avoid importing __pkginfo__ (exec it instead), replace os.path.walk by os.walk and prepend `modname` here and there * update tox.ini to account for new test locations * update doc/conf.py so that it still finds __pkginfo__.py and CWDIR in doc/Makefile

clean some remaining dust from goa time

[datafeed] io.BytesIO requires a buffer, not a unicode (closes #9783743) The URLLibResponseAdapter should probably be refactored (or killed?)

[pkg] rql 0.34 and logilab-database 1.15 break cubicweb 3.21

[pkg] rql 0.34 and logilab-database 1.15 break cubicweb 3.21

Added tag 3.22.0, debian/3.22.0-1, centos/3.22.0-1 for changeset de472896fc0a

[pkg] 3.22.0

merge

[formwidgets] a bit of pep8

[views] improve JQueryDatePicker so that a date field can be defined as the min/max of an other one closes #5169039

[entity] don't look at fetch order recursively If fetch_attrs includes relations, stop including the entities on the other side in our sorting. Fixes regression from 73ea636a5562 where we would sort on the target entity before the attribute we wanted if the relation's name sorted before the attribute's. This showed up in the forge/tracker cubes with Version entities being sorted by their State's name in preference to their version number.

[pkg] fix paths in debian doc-base control file

[ldapfeed] extid and cwuri aren't the same thing Historically, we used a ldap://... URI as cwuri attribute, not just the dn. The extid field, OTOH, was always the dn. So when initializing our dict of known entities from an ldap source, look at existing extids, not cwuris, to avoid importing already-existing users (and violated unicity constraints).

[entities] add eid to ISerializableAdapter Life without an eid is made of sadness.

[doc] 3.22 release notes update

[datafeed] use tz-aware datetime objects Avoid TypeError when comparing tz-naive and tz-aware objects in fresh() method.

[pkg] fix syntax error in debian/control

[pkg] remove no-longer-used addresses from debian control file

[schema] rebuild computed attributes on schema reload If we have new computed attributes we need to set their formula_select attribute or things get sad.

[hooks] fix extra_props handling on python3 The json module expects str, not bytes, so we need to do the decoding.

[py3k/ldapparser] avoid confusion between unicode and bytes - ldap3 dicts contain unicode strings - extid must be bytes

[pkg] use correct lgdb version for tz rework Related to #4848923.

pep8 bits

[server] improve TZDatetime support by depending on logilab-database 1.15.0 including https://www.logilab.org/ticket/1485893, we can now keep the tzinfo attribute on datetime objects for TZDatetime attributes, so one knows that this is a tz-aware datetime. To easily make it work with backends that have no tz support, we keep converting tz-aware datetime objects into utc naive datetime objects before sending them to the database.

[schema] Use TZDatetime for creation_date and modification_date No work has been done for form fields/widgets since creation_date / modification_date are by default not editable through the default UI. One may want to add such feature at some point. Time will be displayed as UTC, let the end-application decide otherwise by customizing the tzdatetime printer if desired. Closes #4848923

[server] fix SQLite patch to ensure TZDatetime are properly typed NB: at this point, the TZ info is completely ignored. It will be properly handled in later commits.

[server] fix name and add docstring to connection initialization functions

[server] always monkeypatch QuerierHelper to handle dates and times on SQLite The patch prevents problems due to a SQLite bug causing some values to be returned as strings instead of properly typed objects. This should probably be improved to use SQLite connection hooks. But the major improvement is that the monkeypatch is now done all the time, and not just for tests.

[testlib] put postgres test database in test/data/database directory with other (sqlite) test databases. Closes #9240765 Simplify .hgignore in passing.

[dataimport] make eids_seq_range as massive store instance attribute again I changed it to a class attribute recently, but on further thought (and prompting from Adrien) it seems to make more sense as an instance attribute.

[dataimport] stop supporting None as eids_seq_range It couldn't work as-is because entities_id_seq is not a sequence object.

[dataimport] remove eids_seq_start attribute from massive store Instead provide a utility function to reset the eid sequence.

[dataimport] massive store in slave mode shouldn't flush metadata The metadata insertion is to be done only by the master-mode store.

[dataimport] check that MassiveObjectStore restores the db schema properly Ideally we'd use "pg_dump --schema-only" to compare before/after, but when restoring a constraint such as: cstr48f9ceae537d68859b62c601681c8d3e CHECK (((cw_type)::text = ANY ((ARRAY['normal'::character varying, 'auto'::character varying])::text[]))) postgres will convert the text cast on array to a cast on each cell of array cstr48f9ceae537d68859b62c601681c8d3e CHECK (cw_type::text = ANY (ARRAY['normal'::character varying::text, 'auto'::character varying::text]))

[dataimport] split out listing indices/constraints from getting the corresponding query

[dataimport] massive_store: do not drop constraints multiple times It is now done in __init__.

[dataimport] massive_store: do not drop constraints with `cascade` since it breaks things We want to be able to restore constraints at the end of the import. If they are silently dropped due to a CASCADE, we can't do that.

[dataimport] massive_store: drop all constraints in __init__ and restore all in finish This should avoid dependency issues when dropping specific constraints (e.g. when dropping a primary constraint referenced by a foreign key constraint). We only do this in master mode, slave mode stores are supposed to work with an initialized db already.

[dataimport] massive_store: ignore primary key when requesting indexes on a table Primary keys show up in both lists of constraints and indexes. Pick a side and stick to it.

[dataimport] massive_store: reformat sql query

[devtools] Use super() in TestServerConfiguration.__init__() While cleaning up the use of init_config() in WSGIAppTC, I wondered why the method was trying to set the 'https_uiprops' and 'https_datadir_url' attributes on the config, when WebConfiguration's __init__ should be taking care of setting them. WSGIAppTC uses the default configuration - one that derives from ApptestConfiguration (noted as G below). The issue is that the class hierarchy of ApptestConfiguration includes TestServerConfiguration (C in the graph below) which calls ServerConfiguration's __init__ directly (B below). As B does not have an __init__ method, CubicWebConfiguration's __init__ is called directly (A below). This has the unintended consequence of preventing WebConfiguration's __init__ from being called. A / \ B D | | C E \ / F | G | ... A: cwconfig.CubicWebConfiguration B: server.serverconfig.ServerConfiguration C: devtools.TestServerConfiguration D: web.webconfig.WebConfiguration E: etwist.twconfig.WebConfigurationBase F: devtools.BaseApptestConfiguration G: devtools.ApptestConfiguration NB: unittest_wsgi wasn't really broken, as it's the only test in wsgi/test/, but I figured I might as well clean it up. Related to #9297611.

[devtools] Fix database creation issues for CubicWebTestTC tests To reproduce (with a purged repo): * python web/test/test_jscript.py * python web/test/unittest_views_xmlrss.py The '__default_empty_db__' will contain the wrong 'base_url' configuration, thus breaking URLs in the xmlrss tests. Closes #9297611.

[devtools] Fix database creation issues for CubicWebServerTC tests To reproduce (with a purged repo): * python web/test/unittest_web.py MiscOptionsTC * python web/test/unittest_views_xmlrss.py The core issue is that CubicWebServerConfig.default_base_url() is called inside init_config(), which is called during the equivalent of 'db-create'. The '__default_empty_db__' will contain the wrong 'base_url' configuration, thus breaking URLs in the xmlrss tests. Related to #9297611.