Wed, 05 Oct 2016 09:38:05 +0200 [massive store] Use a slave specific table for relation insertion in the massive store
Sylvain Thénault <sylvain.thenault@logilab.fr> [Wed, 05 Oct 2016 09:38:05 +0200] rev 11784
[massive store] Use a slave specific table for relation insertion in the massive store * attribute a random id to the store * add a column containing this id in the cwmassive_initialized table * separate generic `master_init` which create the cwmassive_initialized table from rtype specific initialization which must be done in each slave now * delay removal of table indexes to `finish` * get back relations from each table on `finish` Similar work still has to be done for entities insertion - which will allow more cleanup to the constraints handling which is still rough for now. Related to #15538303
Tue, 04 Oct 2016 13:14:35 +0200 [massive store] docstring and __init__ cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr> [Tue, 04 Oct 2016 13:14:35 +0200] rev 11783
[massive store] docstring and __init__ cleanup
Tue, 04 Oct 2016 13:14:18 +0200 [massive store] Drop dead code
Sylvain Thénault <sylvain.thenault@logilab.fr> [Tue, 04 Oct 2016 13:14:18 +0200] rev 11782
[massive store] Drop dead code This function is never used
Fri, 30 Sep 2016 17:34:59 +0200 [massive store] Reintroduce methods that are necessary to properly handle master/slave configuration
Sylvain Thénault <sylvain.thenault@logilab.fr> [Fri, 30 Sep 2016 17:34:59 +0200] rev 11781
[massive store] Reintroduce methods that are necessary to properly handle master/slave configuration Related to #15538303
Wed, 28 Sep 2016 09:02:14 +0200 [massive store] Follow configuration of the metadata generator
Sylvain Thénault <sylvain.thenault@logilab.fr> [Wed, 28 Sep 2016 09:02:14 +0200] rev 11780
[massive store] Follow configuration of the metadata generator Don't drop constraints and indexes for tables that are ignored by the metadata generator given to the store. One may now easily disable insertion of e.g. created_by / owned_by by removing them from the MetadataGenerator.META_RELATIONS set, in which case indexes for associated table won't be removed by the massive store.
Mon, 17 Oct 2016 16:53:28 +0200 [dataimport] make MetadataGenerator.META_RELATIONS customizable
Adrien Di Mascio <Adrien.DiMascio@logilab.fr> [Mon, 17 Oct 2016 16:53:28 +0200] rev 11779
[dataimport] make MetadataGenerator.META_RELATIONS customizable This should be done on the instance rather than on the class
Wed, 28 Sep 2016 08:57:48 +0200 [massive store] Rework constraint/index handling
Sylvain Thénault <sylvain.thenault@logilab.fr> [Wed, 28 Sep 2016 08:57:48 +0200] rev 11778
[massive store] Rework constraint/index handling The basic idea is to keep the primary constraint on entities.eid since it's heavily used in metadata insertions. Other option would have been to drop / recreate but its very costly on big database, and the index is used for insertion into the entities table itself, so it's not worth droping it at a first glance. Also, keeping it avoids to systematically drop all constraints which depends on it. We may thus now lazily drop constraints, only on insertion of some etype/rtype for the related table. Related to #15538359
Tue, 27 Sep 2016 12:02:07 +0200 [massive store] Lazy removal of constraints and metadata indexes
Sylvain Thénault <sylvain.thenault@logilab.fr> [Tue, 27 Sep 2016 12:02:07 +0200] rev 11777
[massive store] Lazy removal of constraints and metadata indexes They should not be removed during store's init, because we may want to query the database with its index between store creation and call to prepare_insert* (e.g. to build the extid2eid map). Along the way: * rename drop_metadata_constraints into drop_metadata_indexes, because that's what it does * rework a bit impacted tests Closes #15538359
Mon, 17 Oct 2016 14:50:47 +0200 [repository] Drop the index on entities.type
Sylvain Thénault <sylvain.thenault@logilab.fr> [Mon, 17 Oct 2016 14:50:47 +0200] rev 11776
[repository] Drop the index on entities.type It should be now only used in maintenance queries in checkintegrity. Those are not worth maintaining it. Closes #15538317
Thu, 06 Oct 2016 12:11:23 +0200 [source] Drop source mapping handling
Sylvain Thénault <sylvain.thenault@logilab.fr> [Thu, 06 Oct 2016 12:11:23 +0200] rev 11775
[source] Drop source mapping handling It was only used by the cwxmlparser which has been deleted. This is too complex for litle benefit, we don't want to maintain that in cubicweb.
Wed, 05 Oct 2016 15:30:10 +0200 [repository] Drop the entities.extid column and associated cache
Sylvain Thénault <sylvain.thenault@logilab.fr> [Wed, 05 Oct 2016 15:30:10 +0200] rev 11774
[repository] Drop the entities.extid column and associated cache This was not necessary anymore with promoted usage of the new data import API. Turn repository's _type_extid_cache to _type_cache with only the entity's type as key. This introduces an backward incompatible change: entity_metas dict doesn't contains anymore the extid key, but it doesn't seem used at all anywhere, so this sounds acceptable. Closes #15538317
Fri, 30 Sep 2016 17:38:12 +0200 [repository] Drop the entities.asource column
Sylvain Thénault <sylvain.thenault@logilab.fr> [Fri, 30 Sep 2016 17:38:12 +0200] rev 11773
[repository] Drop the entities.asource column It not used anymore thanks to previous patches. Closes #15538288
Thu, 29 Sep 2016 22:54:26 +0200 [integrity check] Stop using entities.asource column in integrity checks
Sylvain Thénault <sylvain.thenault@logilab.fr> [Thu, 29 Sep 2016 22:54:26 +0200] rev 11772
[integrity check] Stop using entities.asource column in integrity checks Simply check if entities have a cw_source, and link it to the system source when not found. Related to #15538288
Mon, 17 Oct 2016 13:42:30 +0200 [rql2sql/test] Ensure test results predictability
Sylvain Thénault <sylvain.thenault@logilab.fr> [Mon, 17 Oct 2016 13:42:30 +0200] rev 11771
[rql2sql/test] Ensure test results predictability Without this, test were randomly failing within tox due to different order of solutions.
Thu, 06 Oct 2016 21:17:01 +0200 [rql2sql] Stop generating SQL query from RQL using entities.type
Sylvain Thénault <sylvain.thenault@logilab.fr> [Thu, 06 Oct 2016 21:17:01 +0200] rev 11770
[rql2sql] Stop generating SQL query from RQL using entities.type This was necessary in the "true multi-sources" time, it's not anymore, while maintaining this index is costly. Related to #15538317
Tue, 11 Oct 2016 10:30:59 +0200 [test] Remove some usages of lgc.testlib
Sylvain Thénault <sylvain.thenault@logilab.fr> [Tue, 11 Oct 2016 10:30:59 +0200] rev 11769
[test] Remove some usages of lgc.testlib
Thu, 06 Oct 2016 21:14:49 +0200 [test] Avoid pytest discovery warnings
Sylvain Thénault <sylvain.thenault@logilab.fr> [Thu, 06 Oct 2016 21:14:49 +0200] rev 11768
[test] Avoid pytest discovery warnings Having TestServerConfiguration and alike in the test module namespace causes pytest discovery errors, because it thinks it's a test class.
Fri, 30 Sep 2016 18:25:08 +0200 flake8 and all
Sylvain Thénault <sylvain.thenault@logilab.fr> [Fri, 30 Sep 2016 18:25:08 +0200] rev 11767
flake8 and all * update some copyright * drop most __docformat__ declaration * fix some flake8 warnings / errors
Wed, 05 Oct 2016 15:28:43 +0200 [schema2sql] Drop deprecated comment
Sylvain Thénault <sylvain.thenault@logilab.fr> [Wed, 05 Oct 2016 15:28:43 +0200] rev 11766
[schema2sql] Drop deprecated comment
Sat, 01 Oct 2016 11:56:27 +0200 [cnx] Use entity_type instead of entity_metas()['type']
Sylvain Thénault <sylvain.thenault@logilab.fr> [Sat, 01 Oct 2016 11:56:27 +0200] rev 11765
[cnx] Use entity_type instead of entity_metas()['type'] The latter is deprecated in favor of the former which has been introduced on cnx to replace it.
Thu, 06 Oct 2016 12:15:50 +0200 [entity] Reimplement and deprecate entity.cw_metainformation
Sylvain Thénault <sylvain.thenault@logilab.fr> [Thu, 06 Oct 2016 12:15:50 +0200] rev 11764
[entity] Reimplement and deprecate entity.cw_metainformation It shouldn't rely anymore on cnx.entity_metas which is itself deprecated, and the cw_metainformation API is similarly not needed anymore, access directly to cwuri / cw_source if needed. Related to #15538288
Fri, 30 Sep 2016 17:39:17 +0200 [repository] Drop type_and_source_from_eid and rename related cache
Sylvain Thénault <sylvain.thenault@logilab.fr> [Fri, 30 Sep 2016 17:39:17 +0200] rev 11763
[repository] Drop type_and_source_from_eid and rename related cache We don't want to handle anymore the entities.asource column so we shouldn't use those anymore. Also rename repository's _type_source_cache into _type_extid_cache as this is what it's containing now. Do similar renaming to the system source API. Related to #15538288
Fri, 30 Sep 2016 17:34:11 +0200 [entity] Stop returning source information in cw_metainformation()
Sylvain Thénault <sylvain.thenault@logilab.fr> [Fri, 30 Sep 2016 17:34:11 +0200] rev 11762
[entity] Stop returning source information in cw_metainformation() We should not need that. It has been introduced to handle things we should not do and probably don't do anymore (e.g. attempting to grasp recursive import of several sites). Related to #15538288.
Fri, 30 Sep 2016 17:04:42 +0200 [json] Stop serializing cw_source into default json representation of an entity
Sylvain Thénault <sylvain.thenault@logilab.fr> [Fri, 30 Sep 2016 17:04:42 +0200] rev 11761
[json] Stop serializing cw_source into default json representation of an entity Related to #15538288
(0) -10000 -3000 -1000 -300 -100 -50 -24 +24 +50 +100 +300 +1000 tip