10 speed/security tradeoffs. Those keeping all the *CubicWeb* hooks and security will be slower but the |
10 speed/security tradeoffs. Those keeping all the *CubicWeb* hooks and security will be slower but the |
11 possible errors in insertion (bad data types, integrity error, ...) will be raised. |
11 possible errors in insertion (bad data types, integrity error, ...) will be raised. |
12 |
12 |
13 These data import utilities are provided in the package `cubicweb.dataimport`. |
13 These data import utilities are provided in the package `cubicweb.dataimport`. |
14 |
14 |
15 All the stores have the following API:: |
15 The API is built on top of the following concepts: |
|
16 |
|
17 * `Store`, class responsible for inserting values in the backend database |
|
18 |
|
19 * `ExtEntity`, some intermediate representation of data to import, using external identifier but no |
|
20 eid, and usually with slightly different representation than the associated entity's schema |
|
21 |
|
22 * `Generator`, class or functions that will yield `ExtEntity` from some data source (eg RDF, CSV) |
|
23 |
|
24 * `Importer`, class responsible for turning `ExtEntity`'s extid to eid, doing creation or update |
|
25 accordingly and may be controlling the insertion order of entities before feeding them to a |
|
26 `Store` |
|
27 |
|
28 Stores |
|
29 ~~~~~~ |
|
30 |
|
31 Stores are responsible to insert properly formatted entities and relations into the database. They |
|
32 have the following API:: |
16 |
33 |
17 >>> user_eid = store.prepare_insert_entity('CWUser', login=u'johndoe') |
34 >>> user_eid = store.prepare_insert_entity('CWUser', login=u'johndoe') |
18 >>> group_eid = store.prepare_insert_entity('CWUser', name=u'unknown') |
35 >>> group_eid = store.prepare_insert_entity('CWUser', name=u'unknown') |
19 >>> store.relate(user_eid, 'in_group', group_eid) |
36 >>> store.relate(user_eid, 'in_group', group_eid) |
20 >>> store.flush() |
37 >>> store.flush() |
71 ----------------- |
88 ----------------- |
72 |
89 |
73 This store relies on *COPY FROM*/execute many sql commands to directly push data using SQL commands |
90 This store relies on *COPY FROM*/execute many sql commands to directly push data using SQL commands |
74 rather than using the whole *CubicWeb* API. For now, **it only works with PostgresSQL** as it requires |
91 rather than using the whole *CubicWeb* API. For now, **it only works with PostgresSQL** as it requires |
75 the *COPY FROM* command. |
92 the *COPY FROM* command. |
|
93 |
|
94 ExtEntity and Importer |
|
95 ~~~~~~~~~~~~~~~~~~~~~~ |
|
96 |
|
97 .. automodule:: cubicweb.dataimport.importer |