Sylvain Thénault <sylvain.thenault@logilab.fr> [Thu, 28 Jan 2016 15:13:39 +0100] rev 11319
[dataimport] test for a value is in a set and insertion in a set should live together
also inline some methods on the way, in the hope to make the public api clearer
at some point, and only put valuable values in those sets so we don't have to
endlessly test for consistency.
Sylvain Thénault <sylvain.thenault@logilab.fr> [Thu, 28 Jan 2016 15:08:16 +0100] rev 11318
[dataimport] use a single _initialized set instead of _entities / _rtypes
and rework a bit methods using them so that 'in' test and set insertion leave in
the same method.
Sylvain Thénault <sylvain.thenault@logilab.fr> [Thu, 28 Jan 2016 14:34:47 +0100] rev 11317
[dataimport] drop massive store etype to eid index
and fix potential problem with is_instance_of relation insertion in case where
the class of inserted entity has some (yams) parent.
Sylvain Thénault <sylvain.thenault@logilab.fr> [Thu, 28 Jan 2016 14:00:50 +0100] rev 11316
[dataimport] add a .schema shortcut attribute on the massive store
also rephrase a bit a method along the way
Sylvain Thénault <sylvain.thenault@logilab.fr> [Thu, 28 Jan 2016 13:41:39 +0100] rev 11315
[dataimport] rework massive store's __init__
* _cnx is set by the parent class
* count_cwuri is not used
* group related things together
Sylvain Thénault <sylvain.thenault@logilab.fr> [Thu, 28 Jan 2016 18:18:20 +0100] rev 11314
[dataimport] rework PGHelper class
Sylvain Thénault <sylvain.thenault@logilab.fr> [Thu, 28 Jan 2016 18:18:44 +0100] rev 11313
[dataimport] flake8
Sylvain Thénault <sylvain.thenault@logilab.fr> [Thu, 28 Jan 2016 14:02:31 +0100] rev 11312
[dataimport] enhance a bit sql queries readability
Sylvain Thénault <sylvain.thenault@logilab.fr> [Thu, 28 Jan 2016 14:14:57 +0100] rev 11311
[dataimport] use IF EXISTS when possible
it's better than try/except and avoid potential concurrency problems.
Sylvain Thénault <sylvain.thenault@logilab.fr> [Thu, 28 Jan 2016 11:32:12 +0100] rev 11310
[dataimport] PGHelper should be responsible to retrieve the database schema
Sylvain Thénault <sylvain.thenault@logilab.fr> [Mon, 01 Feb 2016 17:31:35 +0100] rev 11309
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
with a similar-but-different API that will allow its usage by the massive store
(no more entity object manipulation), while making it a bit more flexible for
customization by subclasses (experience taken from client code).
Related to #10338621
Sylvain Thénault <sylvain.thenault@logilab.fr> [Tue, 21 Jun 2016 16:37:17 +0200] rev 11308
[dataimport] Prefer now(pytz.utc) to utcnow
Sylvain Thénault <sylvain.thenault@logilab.fr> [Fri, 29 Jan 2016 15:08:13 +0100] rev 11307
[dataimport] rename nohook store .source attribute
so it's clearly not the samething as metagenerator.source. Also add a bit of doc
along the way.
Sylvain Thénault <sylvain.thenault@logilab.fr> [Thu, 28 Jan 2016 11:17:31 +0100] rev 11306
[dataimport] make rql store test easier to read and test a few more thing, then reuse it to test nohook start which had no tests at all