[hooks] Skip modification_date attribute in "notifentityupdated" hook Depending on the call order of hooks, "modification_date" may come in entity.cw_edited or not (from "metaattrsupdate" hook usually). It should however not be considered in the notification context. Closes #8842064.

[test] Gather tests about mail module From sobjects/test/unittest_notification.py to test/unittest_mail.py where other tests for mail.py are already. Some style fixes along the way.

[web/component] guard against invalid page_size form parameter Same as dff28956f37e, except the almost same code exists in two places (yay us).

[predicates] fix paginated_rset for components that change page_size_property The predicate and component had better agree on the page size.

[devtools] fix get_bounds Say an attribute has two IntervalBoundConstraints, one with minvalue=0 and the other with maxvalue=10 (not literally, or that could be a single constraint, but e.g. with references to other attributes). We don't want to override the minvalue of the former with None just because the latter happens last in the constraints list. This doesn't fix the case where two constraints both have a min or a max value. Meh.

[devtools] "fix" boundary condition for autofill maxvalue in an IntervalBoundConstraint is inclusive. If the min and max values happened to coincide, we would call random.choice on an empty list, which it didn't like. This is still broken in the case of BoundaryConstraint('>', minvalue), as we assume the minvalue is in the range, but that's something for another time.

[test] Use stdlib doctest module instead of logilab-common class The former just works fine now but the later does not play well with py.test.

[test] Re-implement generative tests with subtests

[devtools] Re-implement generative tests using subtests Generative tests as implemented in logilab.common.testib are not compatible with tests runner other than lgc.pytest and this implementation differs from the standard library, which has support for subtests_ since Python 3.4. Use unittest2 to bridge the gap. Maybe it'd be good to implement this on logilab.common.testlib side at some point. Let's see how this gets received here first. .. _subtests: https://docs.python.org/3/library/unittest.html#subtests

cleanups

[pkg] python-tz does not exist in centos should be pytz (closes #10290056)

[test] Skip LDAP tests if any of the underlying programs is not found Some programs are in ldap-utils or ldapscripts, which may not be installed despite slapd may. Follow-up on 282880f81311.

Handle virtualenv development mode Looking for VIRTUAL_ENV environment variable as a sign of virtualenv activation. This at least works for developing CubicWeb itself (not sure yet about cubes). And it makes `tox --develop` work as well. Add a test for cwconfig._find_prefix within a virtualenv. In tox.ini, use `develop option`_ accordingly. This makes tox running significantly faster since it avoids building and installing the sdist. Aside, the "develop" install is now performed *after* all test requirements are processed, which means that the cubicweb package should always be the one from source -- instead of a released version pulled by a cube through test dependencies, which may occur in "recreate" step. .. _`develop option`: <http://tox.readthedocs.org/en/latest/config.html#confval-usedevelop=BOOL

[skeleton] Use os.walk in setup.py for Python3 compatibility

[skeleton] Use unittest.main directly Instead of lgc.testlib.unittest_main as they are identical.

[devtools/testlib] Disable pause trace in setUp Closes #9856028.

[test] Remove dependency on person cube in devtools package

[pkg] add missing python-tz dependency

[edit controller] Fix composite entity reparenting when inlined This implies handling the reparenting case sooner, otherwise the child entity modifications are cancelled (see 113e9da47afc). Add a new test to ensure non-regression in all previous tests. Closes #10291977.

[forms] Do not raise errors on composite required relation being deleted The entity included in the composite will be deleted in such a case, so the field is not required even when the cardinality is '1' or '+'. Related to #8529868

[edit controller] Cancel RQL queries to be performed on entities to be deleted Composite relation removal via the EditController trigger an entity removal. Changes on such a to-be-removed entity must be cancelled as they may trigger integrity errors before the deletion occurs. Closes #8529868.

[edit controller] fix handling of removal of subentities from an edit form In CubicWeb 3.18 the semantics of dropping a composite relation were changed. It had before the effect of deleting the subentities. Now the edit controller must handle this by itself (it was not adapted then by mistake). So if it detects the removal of a composite relation (on the browser side, the user clicked on the "remove" action), the subentity is now explicitly deleted. It was chosen not to change the form generation but the EditController: - to ensure pre-3.18 forms backward compat - to avoid the introduction of a specific entity-deletion API (RQL is probably our best API for complex DB operations) Related to #8529868.

remove unused import

[editcontroller] a slight code reorganisation, in preparation for a fix * `entity` defined at method entry * a combined `continue` and a dedent of the next big block * some vertical white space Related to #8529868.

[sources/native] ask permission to continue restoring even if metadata do not match (closes #7771864) Moreover we try to clarify the message (Expected/Found was never too clear for users).

[views] fix ProcessInformationView: SESSION_MANAGER can be None (closes #5753280) eg. with pyramid. (grafted from 6a07c2dc3d85f4de0cf48221607e02c4473e11e4)

[web] Move "display all"/ "back to pagination" link generation into the navigation component This makes the html generation customizable. Closes #9219057

Reorganize source tree to have a "cubicweb" top-level package Basically: mkdir cubicweb hg mv *.py -X setup.py cubicweb hg mv dataimport devtools entities etwist ext hooks i18n misc schemas server skeleton sobjects test web wsgi cubicweb Other changes: * adjust path to cubicweb-ctl in devtools tests * update setup.py to avoid importing __pkginfo__ (exec it instead), replace os.path.walk by os.walk and prepend `modname` here and there * update tox.ini to account for new test locations * update doc/conf.py so that it still finds __pkginfo__.py and CWDIR in doc/Makefile

clean some remaining dust from goa time

[datafeed] io.BytesIO requires a buffer, not a unicode (closes #9783743) The URLLibResponseAdapter should probably be refactored (or killed?)

[pkg] rql 0.34 and logilab-database 1.15 break cubicweb 3.21

[pkg] rql 0.34 and logilab-database 1.15 break cubicweb 3.21

Added tag 3.22.0, debian/3.22.0-1, centos/3.22.0-1 for changeset de472896fc0a

[pkg] 3.22.0

merge

[formwidgets] a bit of pep8

[views] improve JQueryDatePicker so that a date field can be defined as the min/max of an other one closes #5169039

[entity] don't look at fetch order recursively If fetch_attrs includes relations, stop including the entities on the other side in our sorting. Fixes regression from 73ea636a5562 where we would sort on the target entity before the attribute we wanted if the relation's name sorted before the attribute's. This showed up in the forge/tracker cubes with Version entities being sorted by their State's name in preference to their version number.

[pkg] fix paths in debian doc-base control file

[ldapfeed] extid and cwuri aren't the same thing Historically, we used a ldap://... URI as cwuri attribute, not just the dn. The extid field, OTOH, was always the dn. So when initializing our dict of known entities from an ldap source, look at existing extids, not cwuris, to avoid importing already-existing users (and violated unicity constraints).

[entities] add eid to ISerializableAdapter Life without an eid is made of sadness.

[doc] 3.22 release notes update

[datafeed] use tz-aware datetime objects Avoid TypeError when comparing tz-naive and tz-aware objects in fresh() method.

[pkg] fix syntax error in debian/control

[pkg] remove no-longer-used addresses from debian control file

[schema] rebuild computed attributes on schema reload If we have new computed attributes we need to set their formula_select attribute or things get sad.

[hooks] fix extra_props handling on python3 The json module expects str, not bytes, so we need to do the decoding.

[py3k/ldapparser] avoid confusion between unicode and bytes - ldap3 dicts contain unicode strings - extid must be bytes

[pkg] use correct lgdb version for tz rework Related to #4848923.

pep8 bits

[server] improve TZDatetime support by depending on logilab-database 1.15.0 including https://www.logilab.org/ticket/1485893, we can now keep the tzinfo attribute on datetime objects for TZDatetime attributes, so one knows that this is a tz-aware datetime. To easily make it work with backends that have no tz support, we keep converting tz-aware datetime objects into utc naive datetime objects before sending them to the database.

[schema] Use TZDatetime for creation_date and modification_date No work has been done for form fields/widgets since creation_date / modification_date are by default not editable through the default UI. One may want to add such feature at some point. Time will be displayed as UTC, let the end-application decide otherwise by customizing the tzdatetime printer if desired. Closes #4848923

[server] fix SQLite patch to ensure TZDatetime are properly typed NB: at this point, the TZ info is completely ignored. It will be properly handled in later commits.

[server] fix name and add docstring to connection initialization functions

[server] always monkeypatch QuerierHelper to handle dates and times on SQLite The patch prevents problems due to a SQLite bug causing some values to be returned as strings instead of properly typed objects. This should probably be improved to use SQLite connection hooks. But the major improvement is that the monkeypatch is now done all the time, and not just for tests.

[testlib] put postgres test database in test/data/database directory with other (sqlite) test databases. Closes #9240765 Simplify .hgignore in passing.

[dataimport] make eids_seq_range as massive store instance attribute again I changed it to a class attribute recently, but on further thought (and prompting from Adrien) it seems to make more sense as an instance attribute.

[dataimport] stop supporting None as eids_seq_range It couldn't work as-is because entities_id_seq is not a sequence object.

[dataimport] remove eids_seq_start attribute from massive store Instead provide a utility function to reset the eid sequence.

[dataimport] massive store in slave mode shouldn't flush metadata The metadata insertion is to be done only by the master-mode store.

[dataimport] check that MassiveObjectStore restores the db schema properly Ideally we'd use "pg_dump --schema-only" to compare before/after, but when restoring a constraint such as: cstr48f9ceae537d68859b62c601681c8d3e CHECK (((cw_type)::text = ANY ((ARRAY['normal'::character varying, 'auto'::character varying])::text[]))) postgres will convert the text cast on array to a cast on each cell of array cstr48f9ceae537d68859b62c601681c8d3e CHECK (cw_type::text = ANY (ARRAY['normal'::character varying::text, 'auto'::character varying::text]))

[dataimport] split out listing indices/constraints from getting the corresponding query

[dataimport] massive_store: do not drop constraints multiple times It is now done in __init__.

[dataimport] massive_store: do not drop constraints with `cascade` since it breaks things We want to be able to restore constraints at the end of the import. If they are silently dropped due to a CASCADE, we can't do that.

[dataimport] massive_store: drop all constraints in __init__ and restore all in finish This should avoid dependency issues when dropping specific constraints (e.g. when dropping a primary constraint referenced by a foreign key constraint). We only do this in master mode, slave mode stores are supposed to work with an initialized db already.

[dataimport] massive_store: ignore primary key when requesting indexes on a table Primary keys show up in both lists of constraints and indexes. Pick a side and stick to it.

[dataimport] massive_store: reformat sql query

[devtools] Use super() in TestServerConfiguration.__init__() While cleaning up the use of init_config() in WSGIAppTC, I wondered why the method was trying to set the 'https_uiprops' and 'https_datadir_url' attributes on the config, when WebConfiguration's __init__ should be taking care of setting them. WSGIAppTC uses the default configuration - one that derives from ApptestConfiguration (noted as G below). The issue is that the class hierarchy of ApptestConfiguration includes TestServerConfiguration (C in the graph below) which calls ServerConfiguration's __init__ directly (B below). As B does not have an __init__ method, CubicWebConfiguration's __init__ is called directly (A below). This has the unintended consequence of preventing WebConfiguration's __init__ from being called. A / \ B D | | C E \ / F | G | ... A: cwconfig.CubicWebConfiguration B: server.serverconfig.ServerConfiguration C: devtools.TestServerConfiguration D: web.webconfig.WebConfiguration E: etwist.twconfig.WebConfigurationBase F: devtools.BaseApptestConfiguration G: devtools.ApptestConfiguration NB: unittest_wsgi wasn't really broken, as it's the only test in wsgi/test/, but I figured I might as well clean it up. Related to #9297611.

[devtools] Fix database creation issues for CubicWebTestTC tests To reproduce (with a purged repo): * python web/test/test_jscript.py * python web/test/unittest_views_xmlrss.py The '__default_empty_db__' will contain the wrong 'base_url' configuration, thus breaking URLs in the xmlrss tests. Closes #9297611.

[devtools] Fix database creation issues for CubicWebServerTC tests To reproduce (with a purged repo): * python web/test/unittest_web.py MiscOptionsTC * python web/test/unittest_views_xmlrss.py The core issue is that CubicWebServerConfig.default_base_url() is called inside init_config(), which is called during the equivalent of 'db-create'. The '__default_empty_db__' will contain the wrong 'base_url' configuration, thus breaking URLs in the xmlrss tests. Related to #9297611.

[devtools] Move attribute docstring to correct location

[web] Make tests pass if tal isn't available

[py3k] fix cubicweb.mail.parse_message_id base64.b64decode returns bytes. Fixes sobjects/test/unittest_notification.py.

[test/py3k] fix test_recipients_finder AssertionError: <zip object at 0x7f1dfb0230c8> != [('abcd@logilab.fr', 'en'), ('efgh@logilab.fr', 'en')]

[dataimport] fix SQLGenObjectStore's add_info for python3

[dataimport] py3k doesn't like + on dict_items objects

[repository] remove unused check_session method and its test Related to #1381328.

[migration/test] avoid test interdependency by removing fakecustomtype cube There is still a failing test afterward but it doesn't seem related to changes in this series.

[migration/test] use numeric(precision, scale) instead of geometry The latter requires the postgis extension which is overkill for this test.

[server, hooks] allow callable in dbh.TYPE_MAPPING This is necessary for cubes introducing new custom final types whose generated sql may depend on some extra configuration, avoiding ugly-buggy hack as done currently by the postgis cube. Closes #7569998

[migration] consider extra_props in rdef created during attribute creation hook else we may miss important data such as type parametrization (see postgis.Geometry for instance). Related to #7569998

[migration] only define variable if needed

[migration] handle non-existing attribute type when adding an entity type The final type needs to be added to the schema first. Related to #7569998.

[migration] handle the case where new final entity type is not yet in the schema This may occur on new cube addition. Related to #7569998.

[migration] config.add_cubes should ensure site_cubicweb of new cubes is called Related to #7569998

[migration] test addition of a cube providing a custom final type This does not work yet. Related to #7569998

[server] add test demonstrating cache update pb since cw > 3.21 Fixed by the previous changeset.

Bring back the separate web-side entity cache Prior to changeset 635cfac73d28 "[repoapi] fold ClientConnection into Connection", we had two entity caches: one on the client/dbapi/web side, and one on the server/repo side. This is a waste, but it is actually needed as long as we have the magic _cw attribute on entities which must sometimes be a request and sometimes a cnx. Removing the duplication caused weird problems with entity._cw alternating between both types of objects, which is unexpected by both the repo and the web sides. We add an entity cache on ConnectionCubicWebRequestBase, separate from the Connection's, and bring back the _cw_update_attr_cache/dont-cache-attrs mechanism, to let the server/edition code let caches know which attributes have been modified Entity.as_rset can be cached again, as ResultSet no longer modifies the entity when fetching it from a cache. Contrary to the pre-3.21 code, _cw_update_attr_cache now handles web requests and connections in the same way (otherwise the cache ends up with wrong values if a hook modifies attributes), but dont-cache-attrs is never set for (inlined) relations. Closes #6863543

Backout "[entity,storage] remove hackish code from the bfss path The merge of the repo-side and web-side entity caches doesn't actually work, because entities have a _cw attribute which is sometimes a connection and sometimes a web request, and whose nature changes not-really-predictably whenever ResultSet.get_entity() is called and brings it out of the cache. This reverts commit 59a79300f213.

merge with 3.21.5-2

[doc] add documentation about drop_cube

Added tag debian/3.21.5-2 for changeset 228b6d2777e4

[pkg] don't install dataimport/test in both cubicweb-server and cubicweb-dev Closes #9239572

merge with 3.21.5

Added tag 3.21.5, debian/3.21.5-1, centos/3.21.5-1 for changeset e0572a786e6b

[pkg] 3.21.5

[js] Fix a ReferenceError in cubicweb.preferences.js Rename function using proper English along the way. Closes #8864731.

[dataimport] add tests for SQLGenObjectStore (closes #9177565) Stolen from the massive import store, with the irrelevant bits removed. Remove redundant tests from test_pgstore.py.

[dataimport] separate entities table from other metadata in SQLGenObjectStore 'entities' needs to be updated first as every other table has references to its rows. Remove use of thread-local storage in the process. related to #9177565.

[dataimport] remove threading support from SQLGenObjectStore It's untested and does not actually work at the moment with nb_threads > 1. related to #9177565.

[dataimport] prepare_insert_entity should not crash with SQLGenObjectStore grafted from 1079d68130e1e780791ceb2050e8a8b7a910f48d (3.22) related to #9177565.

[dataimport] Fix method signature Since d4bd28d5fca8, _storage_handler expect 4 args. grafted from b089858c49a2806b6ea8a7ee66773436701e947c (3.22) related to #9177565.

[dataimport] add missing import 'eschema_eid' grafted from e3ec46b5c710067a0cecca53cc593a0dd4bd927e (3.22) related to #9177565

merge with 3.21.4

Added tag 3.21.4, debian/3.21.4-1, centos/3.21.4-1 for changeset d3b92d3a7db0

[pkg] 3.21.4

merge with 3.20.12

Added tag 3.20.12, debian/3.20.12-1, centos/3.20.12-1 for changeset 03e8fc9f79a6

[tox] Ensure pytest gets run with virtualenv's python

[tox] Use generating environments Drop the "env" option, which is unknown, and set a generative_ "envlist". Then use conditional dependencies based on factors_ to avoid repeating environment sections. The only singular one is the "cubicweb" environment because of its "commands". At some point, it'd be nice to find a way to have python version prefixes (py27-hooks or py34-web) for environment names but I could not find a way to extract the package name from such names. .. _generative: http://tox.readthedocs.org/en/latest/config.html#generative-envlist .. _factors: http://tox.readthedocs.org/en/latest/config.html#factors-and-factor-conditional-settings

a bit of pep8

[adapters] Use an adapter for serializing entities to a bare python structure which may be used for e.g. JSON export or other and allow to have different serializations for each entity types. Closes #6378035

[pkg] 3.20.12

[dataimport] Fallback from COPY to executemany (closes #9135780) If we can't build a suitable COPY command (e.g. because data has unsupported type), use executemany instead. Regression introduced in 3.20.0.

[web/views] unlink svg temp file after we're done Otherwise we fill up the tempdir. Closes #9133268.

Use cubicweb._

[web/views] Controllers should always return bytes Previous code breaks only on py3k.

[devtools] Fix 'newcube' on py3k

[toolsutils] Small fixes to fill_templated_file() * Use context managers to close _both_ files * Use io.open to enforce text mode on both python 2 and 3

[skeleton] Fix broken spec file template Broken in commit 73c1c9cf6bda.

[devtools] Fix fd leak in tests

[devtools] stop using datetime.now() in cwctl newcube and i18ncube

[hooks] fix timestamp confusion in DataImportsCleanupStartupHook CWDataImport.start_timestamp is inserted as datetime.utcnow(), in server/sources/datafeed.py:DataFeedSourceinit_import_log. Don't compare it to datetime.now().

[hooks] fix timestamp confusion in TransactionsCleanupStartupHook transactions.tx_time is inserted as datetime.utcnow(), in server/sources/native.py:NativeSQLSource.start_undoable_transaction. Don't compare it to datetime.now().

merge with 3.21.3 The change in unittest_serverctl.py is needed because of daef7ce08fea (from 3.20.11) and 3914388b2d0f (from the 3.22 branch). Due to both changes, CubicWebTC.config.repository no longer creates a new repository (and thus, a new connection). Since both DBDumpCommand and CubicWebTC.tearDown try to shutdown the repo, tearDown breaks apart. The solution is to temporarily disable ServerConfiguration's config cache. By forcing DBDumpCommand to get a new configuration object, it then gets its own Repo object, allowing tearDown and DBDumpCommand to work independently.

Added tag 3.21.3, debian/3.21.3-1, centos/3.21.3-1 for changeset 9edfe9429209

[pkg] 3.21.3

merge with 3.20.11

update repo sources on update of source's entities .url attribute

Added tag 3.20.11, debian/3.20.11-1, centos/3.20.11-1 for changeset c44930ac9579

[pkg] 3.20.11

[rest] a bit of pep8

[rest] avoid field on two lines if name is above some size limit. Closes #8961835

action, view and service so managers can start source synchronization from the web ui Closes #5474286

[test] Skip ldap source tests if slapd is not available

[doc] Minor changes to the blog tutorial

[skel] remove (what looks like) a spurious copy/paste

Merge default heads published "concurrently"

[qunit] move from CubicWebServerTC to CubicWebTestTC This test now uses wsgi instead of twisted, and thus passes under python3.

[qunit] no need to enable CORS anymore We serve everything needed for the tests through cubicweb.

[web/test] use GET, not POST, to load html files POST comes with a body. There isn't one.

[dataimport] prepare_insert_entity should not crash with SQLGenObjectStore

[dataimport] Fix method signature Since d4bd28d5fca8, _storage_handler expect 4 args.

[dataimport] add missing import 'eschema_eid'

[test] Remove unused import in unittest_qunit.py

[dataimport] Format strings with % instead of .format() For consistency with CubicWeb code base and to avoid Python 2.6 compatibility traps.

[test] Use datapath() in massive store tests

[devtools] Disable anonymous login right before executing tests CubicWebTC.init_config is called many times, especially when creating the default database. This database is then reused by all tests in the same directory. The issue is that, if a test has 'anonymous_allowed' set to False, the anonymous user won't be created by postcreate.py in the default database. All other tests will fail miserably if they expected anonymous login to work. By actually disabling anonymous login after _init_repo, the default database (and configuration) will be initialized with a working anonymous user. Closes #7485127.

[hooks] build EnsureSymmetricRelations{Add,Delete} dynamically Instead of setting the __select__ attribute at registration time, which confuses the filterable_selectors cache when running different tests (with different schemas) in the same process, just make entirely new classes. Closes #8785708.

[qunit] stop dealing with filesystem paths qunit tests need a few things served by cubicweb: - qunit itself, which is now handled by CWDevtoolsStaticController (serving files in cubicweb/devtools/data) - standard cubicweb or cubes data files, handled by the DataController - the tests themselves and their dependencies. These can live in <apphome>/data or <apphome>/static and be served by one of the STATIC_CONTROLLERS This avoids having to guess in CWSoftwareRootStaticController where to serve things from (some files may be installed, others are in the source tree), and should hopefully make it possible to have these tests pass when using tox, and to write qunit tests for cubes, outside of cubicweb itself. This requires modifying the tests to only declare URL paths instead of filesystem paths, and moving support files below test/data/static.

[devtools] fix QUnitResultController It needs to return something.

[utils] Drop now-unused SizeConstrainedList It's last usage in CubicWeb got removed from changeset '[views] remove breadcrumbs usage to retrieve last visited page' Related to #5456850.

[views] remove breadcrumbs usage to retrieve last visited page Use JavaScript history.back to go to the last visited page. Rename _return_to_lastpage to _redirect as it becomes misnamed. Drop `test_redirect_default` test in `unittest_views_basecontrollers` which essentially tested this (now removed) breadcrumbs "feature". Closes #5456850.

[cwconfig] Explicitly close generated config file

[doc] Add 3.21 release date

[dataimport] fix db-namespace default value handling In a fresh new instance, the "db-namespace" option is set as empty by default in the "sources" file. This seems correct but empty string is not a valid pg_schema and actually means that we should use "public"

[server] unbreak db-restore

[doc] add 3.22 changelog

[rset] remove stray (commented-out) print Left from debugging in af266f27c4d5

Un-hide jshintrc dot-files belong in ~, not anywhere else.

[views] Use 'pyvaltable' instead of hand-crafted <table> tags

[web] Stop using config.repository() Since commit 3914388b2d0f, this method creates a new Repository object and thus reloads all the cubes, taking for ever. Closes #8634417.

[web] Stop using webconfig.vc_config() The main issue is that this creates an entirely new Repository object (since commit 3914388b2d0f) and thus reloads all the cubes, taking for ever. Related to #8634417.

[migration] don't handle data deletion anymore on schema changes In most cases when we want to drop some entity/relation type, we don't care whether hooks are called on their deletion. There is even low chances that some hooks still exists, based on an old version of the schema. Last but not least, this is horribly inefficient. So this should be clearly documented and handled by application's programmer if desired. This patch removes unnecessary deletion (because table or column will be later dropped) and reimplements the case of partial deletion (only one relation definition among several, hence the database structure isn't modified) using sql. Only one test regarding deletion of inlined relation def is added as other cases seem to be covered by existing tests. Closes #7023315

[hooks/syncschema] drop unused import

[hooks/syncschema] indentation fix

[server/test] cosmetic whitespace fix

[hooks/syncschema] don't set NOT NULL on inlined relation columns The NOT NULL constraint was being added on inlined relation cardinality change while it shouldn't. We don't want this, because this forbids to add an entity without the relation set at once (it may be added later in the transaction). This patch fix synchronization hook to stop doing this. Closes #6211101

[hooks/syncschema] Turn DropColumnOp into a data operation While writing a test for a migration issue where "NOT NULL" was added to an inlined relation column, I ran into a case where the same column was deleted twice, which of course results in an error. This is because the "ecrit_par" relation used by the test is ambiguous. I've hence turned the DropColumnOp into a data operation to ensure we'll delete the column only once in such case. Related to #6211101

[server/sources] fix name error in eid_type_source_pre_131 i.e. the eid_type_source implementation used when some pre 3.13.1 database is detected. Regression from 5fbdbbe51867.

[repository] deprecate the extid2eid based multi-sources API This API is cumbersome and lead to obfuscated code because of the callback mecanism implied when some entity has to be created. Since we dropped the "true" multi-source, this mecanism is not needed anymore and one should prefer to use the dataimport API inside its parser instead. Notice the cwxml parser will trigger the deprecation warning, it should not be used anymore in favor of ad-hoc parsers.

[ldapfeed] use source's extid2eid, the repo's one will be deprecated in a near future.

[ldapfeed / dataimport] port ldapfeed parser to dataimport API This makes the code easier to understand and will allow to deprecate the old multi-sources api, based on creation callback through repo.extid2eid. Currently with this patch, modification dates are not checked, hence entities will be systematically updated. We run the import with no hooks, because RQLObjectStore can only add entities to the system source.

[server/test] document expected packages to run ldap tests

[ldapfeed] simplify ldap2attrs * tdict argument is never given * separate value retrieval from setting output (will ease a latter cset)

[ldapfeed] simplify deactivation of unencountered users there is no need for the by_etype intermediary dictionnary since we only consider CWUsers.

[server/test] display rset content on failure

Remove trailing empty lines Makes pep8 happier

[ldapparser] simplify extid splitting

[test/ldap] a bit of pep8

[js] add configuration file for jshint (closes #5593987) copied from https://gist.github.com/haschek/2595796

spelling: *aly → *ally

[web] Add a <label> around choices on CheckBox and Radio form widgets Allows clicking directly on the text rather than only on the widget. Closes #7672774.

[web] xml_escape CheckBox and Radio labels (closes #7672764)

[test/tox] Enforce installation of modified yapps 580a6821713d is not enough for tox apparently... The thing is that it will first install dependencies (the `deps` option from tox.ini), actually pulling yapps from Logilab github but then, it will install the project (that is, CubicWeb) from the sdist *along with its dependencies*, which will pull yapps again and override the previously installed version for some mysterious reason. So add a `pip install` command in tox.ini, so that the modified yapps is the last one installed.

[web] delete unused PropertySheet._sourcefile_mtime

[etwist] remove no-longer-necessary uicache cleaning It was done before we were actually up, which means it could be too eager when combined with #6893185. Closes #6893247.

[web] remove PropertySheet._cache Instead of remembering that a css file has been processed, we re-process it at every request. This prevents our cache getting out of date with the file system. Closes #6893185 While we're at it, write the file atomically so that we don't risk serving partial data.

[test] Add missing u'' Test actually works in py3k, but not in python 2... That has to be a first!

[data] Use correct syntax to unset CSS attributes (closes #8602446) 'default' is actually a recognized CSS cursor name (the regular pointer). Using this value doesn't revert to whatever cursor DOM elements have by default ('pointer' on links, 'text' on text, etc). The proper way to unset a CSS attribute on DOM elements is to set it to an empty string. http://stackoverflow.com/questions/2027935/how-to-remove-css-property-using-javascript

[server] install custom sql scripts before creating tables for the schema A cube might need something like 'CREATE EXTENSION postgis' before creating tables for its entities, to add new final types.

[hooks] update internal URL in doc string

[doc] Fix restructured tables + improve english a bit

[debian] Install dataimport package Closes #8173104.

[web/test] Remove use of old test API to silence warning

[web/views] Fix sqlite unicode warning

[devtools] Fix sqlite unicode warning

[tests] Don't import QUnitTestCase into tests' global namespace Unittest's test loader will try to execute QUnitTestCase's test_javascripts method. However this class is intended to be subclassed, not to be executed directly. This change has the following consequences: * slightly faster test execution * almost all rtags warnings are gone

[devtools/qunit] Fail quickly if firefox is not installed

[devtools/qunit] Remove unused code VerboseCalledProcessError is unused since 3905682ece11.

[py3k] import range using six.moves Also, use create_entity instead of hand-crafting RQL queries. Fixes regression introduced in commit 768d332de17b.

[dataimport] remove size constraint handling We do not want to silently truncate data. If that is wanted, the application-specific importer should handle that itself. [v2: Keep the get_size_constraints function so applications don't have to reimplement it if they need to keep the truncate behaviour]

[dataimport] fix doc string to stop referring to the old API

[dataimport] drop duplicate init of MassiveObjectStore.sql

[dataimport] remove implicit commits from MassiveObjectStore Let the user decide when to commit the transaction.

[dataimport] drop extra indirection through MassiveObjectStore._initialized dict We already have self.__dict__, no need to add another one for a static set of keys.

[dataimport] remove remaining references to dataio from MassiveObjectStore In particular, rename metadata tables to live in the ^cw.* namespace.

[dataimport] rename logger from dataio.massiveimport to dataimport.massive_store

[dataimport] remove commit_at_flush parameter from MassiveObjectStore WARNING: the default value was True, but flush() now never commits.

[dataimport] remove autoflush_metadata from MassiveObjectStore parameters Hopefully not needed.

[dataimport] remove uri_param_name from MassiveObjectStore parameters It doesn't belong here.

[dataimport] turn iid_maxsize into a class attribute

[dataimport] turn eids_seq_{start,range} into class attributes Only the master-mode store will actually use eids_seq_start to reset the eid sequence.

[dataimport] remove replace_sep parameter from massive store It is unused since changeset efbbf1e93a04 "[dataimport] Properly escape strings sent to COPY FROM"

[dataimport] remove pg_schema parameter from massive store It can be recovered from the system source's configuration.

[dataimport] remove drop_index parameter from massive store "drop_index=False" also implied not dropping any constraints, in particular foreign keys, which meant any attempt to import entities would fail, because we only add metadata (the entities table) after the entity's insertion.

[dataimport/test] update call to ucsvreader Silences warning: [3.20] 'separator' kwarg is deprecated, use 'delimiter' instead

[dataimport] remove deprecated methods from massive store This code is new in cubicweb, no reason for it to carry deprecated stuff from the start.

[dataimport/test] use store.finish instead of deprecated store.cleanup

[dataimport] drop no more used parameter on massive store

[dataimport] methods that modify in-place shouldn't return value Also mark those methods private and drop some useless comments in passing. Related to #5414760

[dataimport] implement new store API on massive store and deprecate the old one. Related to #5414760

[dataimport] .finish method was missing from (NoHook)RQLObjectStore

[massive store] identity_relation should not be considered, is_relation seems missing The "identity_relation" table should not even exist.