[dataimport] remove remaining references to dataio from MassiveObjectStore In particular, rename metadata tables to live in the ^cw.* namespace.

[dataimport] rename logger from dataio.massiveimport to dataimport.massive_store

[dataimport] remove commit_at_flush parameter from MassiveObjectStore WARNING: the default value was True, but flush() now never commits.

[dataimport] remove autoflush_metadata from MassiveObjectStore parameters Hopefully not needed.

[dataimport] remove uri_param_name from MassiveObjectStore parameters It doesn't belong here.

[dataimport] turn iid_maxsize into a class attribute

[dataimport] turn eids_seq_{start,range} into class attributes Only the master-mode store will actually use eids_seq_start to reset the eid sequence.

[dataimport] remove replace_sep parameter from massive store It is unused since changeset efbbf1e93a04 "[dataimport] Properly escape strings sent to COPY FROM"

[dataimport] remove pg_schema parameter from massive store It can be recovered from the system source's configuration.

[dataimport] remove drop_index parameter from massive store "drop_index=False" also implied not dropping any constraints, in particular foreign keys, which meant any attempt to import entities would fail, because we only add metadata (the entities table) after the entity's insertion.

[dataimport/test] update call to ucsvreader Silences warning: [3.20] 'separator' kwarg is deprecated, use 'delimiter' instead

[dataimport] remove deprecated methods from massive store This code is new in cubicweb, no reason for it to carry deprecated stuff from the start.

[dataimport/test] use store.finish instead of deprecated store.cleanup

[dataimport] drop no more used parameter on massive store

[dataimport] methods that modify in-place shouldn't return value Also mark those methods private and drop some useless comments in passing. Related to #5414760

[dataimport] implement new store API on massive store and deprecate the old one. Related to #5414760

[dataimport] .finish method was missing from (NoHook)RQLObjectStore

[massive store] identity_relation should not be considered, is_relation seems missing The "identity_relation" table should not even exist.

[dataimport/test] use the right assert methods instead of assertTrue with a comparison

[dataimport] import range from six.moves

[dataimport/test] feed binary data to ucsvreader

[dataimport/test] fix broken comparison We want to compare two ints, not an int and a db row

[dataimport] dict.iteritems() → dict.items()

[dataimport] use next builtin instead of next method on iterators

[dataimport] import StringIO from io

[dataimport] backport massive store from dataio cube Some tweaks have been made to work with 3.21 foreign key constraints. Also test are not executed in our own pg cluster instead of polluting the system one. Closes #5414760

[web] set proper Cache-Control header for static files - set Cache-Control max-age to match Expires - use the max_age method (defaults to 1 week) instead of hardcoded 6 months

[web] use a redirect for /data/foo requests that don't use the instance md5 hash The actual file contents can be cached for a long time. If the request doesn't use the hash, that's wrong. Using a redirect lets keep the caching for the actual file contents while not breaking requests missing the hash.

[web] fix typo in doc string

[web] stop adding an Expires header with all responses As per RFC 1945 (HTTP 1.0): If the date given is equal to or earlier than the value of the Date header, the recipient must not cache the enclosed entity. As per RFC 7234 (HTTP 1.1 Caching): If a response includes a Cache-Control field with the max-age directive (Section 5.2.2.8), a recipient MUST ignore the Expires field. Bottom line, the Expires header is now handled wherever the Cache-Control is handled: inside cache managers.

[web] stop looking at a request's Cache-Control header As per RFC 7234 (HTTP 1.1 Caching): The "Cache-Control" header field is used to specify directives for caches along the request/response chain. CubicWeb is not a cache.

[fti] add cw_fti_index_rql_limit method Improves on and deprecate cw_fti_index_rql_queries: execute the rql directly, so that we don't need to compute the set of eids up-front, but can just keep track of the last seen entity. Use the new method in reindex_entities. Keep calling cw_fti_index_rql_queries if it has been redefined (e.g in cubes). Related to #3621392.