doc/api/dataimport.rst
author Sylvain Thénault <sylvain.thenault@logilab.fr>
Thu, 06 Oct 2016 12:12:04 +0200
changeset 11789 71df2811b422
parent 10495 5bd914ebf3ae
child 12814 75e0377ed3c2
permissions -rw-r--r--
[massive store] Store entities in temporary table as well * on some entity type is encountered by a slave, create a dedicated table for insertion of entities of this type by this slave, similarly to what is done for relation - this should lower changes of conflicts in master/slaves mode ; * delay drop of constraints and indexes to `finish` method, where copy from temporary tables to regular table is done ; * insertion of metadata is done by scanning temporary tables, which may be way shorter than theier associated regular table ; * drop drop_metadata_constraints with its constraint_dropped friend attribute, there are no more necessary since this is done once in the `finish`. Related to #15538303
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
10494
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
     1
.. _dataimport_module:
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
     2
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
     3
:mod:`cubicweb.dataimport`
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
     4
==========================
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
     5
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
     6
.. automodule:: cubicweb.dataimport
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
     7
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
     8
    Utilities
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
     9
    ---------
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
    10
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
    11
    .. autofunction:: count_lines
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
    12
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
    13
    .. autofunction:: ucsvreader_pb
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
    14
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
    15
    .. autofunction:: ucsvreader
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
    16
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
    17
    .. autofunction:: callfunc_every
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
    18
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
    19
    .. autofunction:: lazytable
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
    20
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
    21
    .. autofunction:: lazydbtable
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
    22
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
    23
    .. autofunction:: mk_entity
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
    24
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
    25
    Sanitizing/coercing functions
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
    26
    -----------------------------
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
    27
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
    28
    .. autofunction:: optional
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
    29
    .. autofunction:: required
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
    30
    .. autofunction:: todatetime
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
    31
    .. autofunction:: call_transform_method
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
    32
    .. autofunction:: call_check_method
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
    33
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
    34
    Integrity functions
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
    35
    -------------------
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
    36
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
    37
    .. autofunction:: check_doubles
10495
5bd914ebf3ae [doc] fix warnings/errors in doc build
Julien Cristau <julien.cristau@logilab.fr>
parents: 10494
diff changeset
    38
    .. autofunction:: check_doubles_not_none
10494
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
    39
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
    40
    Object Stores
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
    41
    -------------
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
    42
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
    43
    .. autoclass:: ObjectStore
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
    44
        :members:
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
    45
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
    46
    .. autoclass:: RQLObjectStore
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
    47
        :show-inheritance:
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
    48
        :members:
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
    49
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
    50
    .. autoclass:: NoHookRQLObjectStore
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
    51
        :show-inheritance:
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
    52
        :members:
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
    53
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
    54
    .. autoclass:: SQLGenObjectStore
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
    55
        :show-inheritance:
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
    56
        :members:
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
    57
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
    58
    Import Controller
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
    59
    -----------------
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
    60
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
    61
    .. autoclass:: CWImportController
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
    62
        :show-inheritance:
b6f00889355c [doc] Start documenting the API
Christophe de Vienne <christophe@unlish.com>
parents:
diff changeset
    63
        :members: