misc/migration/3.16.0_Any.py
author Vladimir Popescu <vladimir.popescu@logilab.fr>
Tue, 12 Mar 2013 18:31:15 +0100
changeset 8836 8a57802d40d3
parent 8675 b60329e40e26
permissions -rw-r--r--
[cubicweb/doc] Add tutorial on data import in CubicWeb. This involves creating the "tutorials/dataimport" directory structure under "cubicweb/doc" and, inside the "dataimport" directory, putting several files: - a ResT file containing the tutorial *per se*; this tutorial addresses the following issues: * creating a CubicWeb schema for representing a given data set (here, the Diseasome RDF data, for illustration purposes); * parsing the data; * importing the data, by using several stores: + the ``RQLObjectStore``, ``NoHookRQLObjectStore`` and ``SQLGenObjectStore`` from the ``dataimport`` module in CubicWeb; + the ``MassiveObjectStore`` from the ``dataimport`` module in the ``dataio`` cube. The tutorial also provides timing benchmarks of the various stores. - a set of Python files illustrating the data import, in the context of Diseasome RDF data parsing: * a Diseasome RDF data parse module, * a Diseasome data import module, * a CubicWeb schema for representing Diseasome data.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
8675
b60329e40e26 [rql sources] uses an URL as remote host address for pyro/zmq rql sources,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     1
sync_schema_props_perms('EmailAddress')
b60329e40e26 [rql sources] uses an URL as remote host address for pyro/zmq rql sources,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     2
b60329e40e26 [rql sources] uses an URL as remote host address for pyro/zmq rql sources,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     3
for source in rql('CWSource X WHERE X type "pyrorql"').entities():
b60329e40e26 [rql sources] uses an URL as remote host address for pyro/zmq rql sources,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     4
    sconfig = source.dictconfig
b60329e40e26 [rql sources] uses an URL as remote host address for pyro/zmq rql sources,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     5
    nsid = sconfig.pop('pyro-ns-id', config.appid)
b60329e40e26 [rql sources] uses an URL as remote host address for pyro/zmq rql sources,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     6
    nshost = sconfig.pop('pyro-ns-host', '')
b60329e40e26 [rql sources] uses an URL as remote host address for pyro/zmq rql sources,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     7
    nsgroup = sconfig.pop('pyro-ns-group', ':cubicweb')
b60329e40e26 [rql sources] uses an URL as remote host address for pyro/zmq rql sources,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     8
    if nsgroup:
b60329e40e26 [rql sources] uses an URL as remote host address for pyro/zmq rql sources,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     9
        nsgroup += '.'
b60329e40e26 [rql sources] uses an URL as remote host address for pyro/zmq rql sources,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    10
    source.cw_set(url=u'pyro://%s/%s%s' % (nshost, nsgroup, nsid))
b60329e40e26 [rql sources] uses an URL as remote host address for pyro/zmq rql sources,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    11
    source.update_config(skip_unknown=True, **sconfig)
b60329e40e26 [rql sources] uses an URL as remote host address for pyro/zmq rql sources,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    12
b60329e40e26 [rql sources] uses an URL as remote host address for pyro/zmq rql sources,
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    13
commit()