1 .. -*- coding: utf-8 -*-

     2 

     3 .. _migration:

     4 

     5 Migration

     6 =========

     7 

     8 One of the main design goals of *CubicWeb* was to support iterative and agile

     9 development. For this purpose, multiple actions are provided to facilitate the

    10 improvement of an instance, and in particular to handle the changes to be

    11 applied to the data model, without loosing existing data.

    12 

    13 The current version of a cube (and of cubicweb itself) is provided in the file

    14 `__pkginfo__.py` as a tuple of 3 integers.

    15 

    16 Migration scripts management

    17 ----------------------------

    18 

    19 Migration scripts has to be located in the directory `migration` of your

    20 cube and named accordingly:

    21 

    22 ::

    23 

    24   <version n° X.Y.Z>[_<description>]_<mode>.py

    25 

    26 in which :

    27 

    28 * X.Y.Z is the model version number to which the script enables to migrate.

    29 

    30 * *mode* (between the last "_" and the extension ".py") is used for

    31   distributed installation. It indicates to which part

    32   of the application (RQL server, web server) the script applies.

    33   Its value could be :

    34 

    35   * `common`, applies to the RQL server as well as the web server and updates

    36     files on the hard drive (configuration files migration for example).

    37 

    38   * `web`, applies only to the web server and updates files on the hard drive.

    39 

    40   * `repository`, applies only to the RQL server and updates files on the

    41     hard drive.

    42 

    43   * `Any`, applies only to the RQL server and updates data in the database

    44     (schema and data migration for example).

    45 

    46 Again in the directory `migration`, the file `depends.map` allows to indicate

    47 that for the migration to a particular model version, you always have to first

    48 migrate to a particular *CubicWeb* version. This file can contain comments (lines

    49 starting with `#`) and a dependency is listed as follows: ::

    50 

    51   <model version n° X.Y.Z> : <cubicweb version n° X.Y.Z>

    52 

    53 For example: ::

    54 

    55   0.12.0: 2.26.0

    56   0.13.0: 2.27.0

    57   # 0.14 works with 2.27 <= cubicweb <= 2.28 at least

    58   0.15.0: 2.28.0

    59 

    60 Base context

    61 ------------

    62 

    63 The following identifiers are pre-defined in migration scripts:

    64 

    65 * `config`, instance configuration

    66 

    67 * `interactive_mode`, boolean indicating that the script is executed in

    68   an interactive mode or not

    69 

    70 * `versions_map`, dictionary of migrated versions  (key are cubes

    71   names, including 'cubicweb', values are (from version, to version)

    72 

    73 * `confirm(question)`, function asking the user and returning true

    74   if the user answers yes, false otherwise (always returns true in

    75   non-interactive mode)

    76 

    77 * `_()` is equivalent to `unicode` allowing to flag the strings to

    78   internationalize in the migration scripts.

    79 

    80 In the `repository` scripts, the following identifiers are also defined:

    81 

    82 * `commit(ask_confirm=True)`, request confirming and executing a "commit"

    83 

    84 * `schema`, instance schema (readen from the database)

    85 

    86 * `fsschema`, installed schema on the file system (e.g. schema of

    87   the updated model and cubicweb)

    88 

    89 * `repo`, repository object

    90 

    91 * `session`, repository session object

    92 

    93 

    94 New cube dependencies

    95 ---------------------

    96 

    97 If your code depends on some new cubes, you have to add them in a migration

    98 script by using:

    99 

   100 * `add_cube(cube, update_database=True)`, add a cube.

   101 * `add_cubes(cubes, update_database=True)`, add a list of cubes.

   102 

   103 The `update_database` parameter is telling if the database schema

   104 should be updated or if only the relevant persistent property should be

   105 inserted (for the case where a new cube has been extracted from an

   106 existing one, so the new cube schema is actually already in there).

   107 

   108 If some of the added cubes are already used by an instance, they'll simply be

   109 silently skipped.

   110 

   111 

   112 Schema migration

   113 ----------------

   114 The following functions for schema migration are available in `repository`

   115 scripts:

   116 

   117 * `add_attribute(etype, attrname, attrtype=None, commit=True)`, adds a new

   118   attribute to an existing entity type. If the attribute type is not specified,

   119   then it is extracted from the updated schema.

   120 

   121 * `drop_attribute(etype, attrname, commit=True)`, removes an attribute from an

   122   existing entity type.

   123 

   124 * `rename_attribute(etype, oldname, newname, commit=True)`, renames an attribute

   125 

   126 * `add_entity_type(etype, auto=True, commit=True)`, adds a new entity type.

   127   If `auto` is True, all the relations using this entity type and having a known

   128   entity type on the other hand will automatically be added.

   129 

   130 * `drop_entity_type(etype, commit=True)`, removes an entity type and all the

   131   relations using it.

   132 

   133 * `rename_entity_type(oldname, newname, commit=True)`, renames an entity type

   134 

   135 * `add_relation_type(rtype, addrdef=True, commit=True)`, adds a new relation

   136   type. If `addrdef` is True, all the relations definitions of this type will

   137   be added.

   138 

   139 * `drop_relation_type(rtype, commit=True)`, removes a relation type and all the

   140   definitions of this type.

   141 

   142 * `rename_relation_type(oldname, newname, commit=True)`, renames a relation type.

   143 

   144 * `add_relation_definition(subjtype, rtype, objtype, commit=True)`, adds a new

   145   relation definition.

   146 

   147 * `drop_relation_definition(subjtype, rtype, objtype, commit=True)`, removes

   148   a relation definition.

   149 

   150 * `sync_schema_props_perms(ertype=None, syncperms=True, syncprops=True, syncrdefs=True, commit=True)`,

   151   synchronizes properties and/or permissions on:

   152   - the whole schema if ertype is None

   153   - an entity or relation type schema if ertype is a string

   154   - a relation definition  if ertype is a 3-uple (subject, relation, object)

   155 

   156 * `change_relation_props(subjtype, rtype, objtype, commit=True, **kwargs)`, changes

   157   properties of a relation definition by using the named parameters of the properties

   158   to change.

   159 

   160 * `set_widget(etype, rtype, widget, commit=True)`, changes the widget used for the

   161   relation <rtype> of entity type <etype>.

   162 

   163 * `set_size_constraint(etype, rtype, size, commit=True)`, changes the size constraints

   164   for the relation <rtype> of entity type <etype>.

   165 

   166 Data migration

   167 --------------

   168 The following functions for data migration are available in `repository` scripts:

   169 

   170 * `rql(rql, kwargs=None, cachekey=None, ask_confirm=True)`, executes an arbitrary RQL

   171   query, either to interrogate or update. A result set object is returned.

   172 

   173 * `add_entity(etype, *args, **kwargs)`, adds a new entity of the given type.

   174   The attribute and relation values are specified as named positional

   175   arguments.

   176 

   177 Workflow creation

   178 -----------------

   179 

   180 The following functions for workflow creation are available in `repository`

   181 scripts:

   182 

   183 * `add_workflow(label, workflowof, initial=False, commit=False, **kwargs)`, adds a new workflow

   184   for a given type(s)

   185 

   186 You can find more details about workflows in the chapter :ref:`Workflow` .

   187 

   188 Configuration migration

   189 -----------------------

   190 

   191 The following functions for configuration migration are available in all

   192 scripts:

   193 

   194 * `option_renamed(oldname, newname)`, indicates that an option has been renamed

   195 

   196 * `option_group_change(option, oldgroup, newgroup)`, indicates that an option does not

   197   belong anymore to the same group.

   198 

   199 * `option_added(oldname, newname)`, indicates that an option has been added.

   200 

   201 * `option_removed(oldname, newname)`, indicates that an option has been deleted.

   202 

   203 The `config` variable is an object which can be used to access the

   204 configuration values, for reading and updating, with a dictionary-like

   205 syntax. 

   206 

   207 Example 1: migration script changing the variable 'sender-addr' in

   208 all-in-one.conf. The script also checks that in that the instance is

   209 configured with a known value for that variable, and only updates the

   210 value in that case.

   211 

   212 .. sourcecode:: python

   213 

   214  wrong_addr = 'cubicweb@loiglab.fr' # known wrong address

   215  fixed_addr = 'cubicweb@logilab.fr'

   216  configured_addr = config.get('sender-addr')

   217  # check that the address has not been hand fixed by a sysadmin

   218  if configured_addr == wrong_addr: 

   219      config['sender-addr'] = fixed-addr

   220      config.save()

   221 

   222 Example 2: checking the value of the database backend driver, which

   223 can be useful in case you need to issue backend-dependent raw SQL

   224 queries in a migration script.

   225 

   226 .. sourcecode:: python

   227 

   228  dbdriver  = config.sources()['system']['db-driver']

   229  if dbdriver == "sqlserver2005":

   230      # this is now correctly handled by CW :-)

   231      sql('ALTER TABLE cw_Xxxx ALTER COLUMN cw_name varchar(64) NOT NULL;')

   232      commit()

   233  else: # postgresql

   234      sync_schema_props_perms(ertype=('Xxxx', 'name', 'String'),

   235      syncperms=False)

   236 

   237 

   238 Others migration functions

   239 --------------------------

   240 Those functions are only used for low level operations that could not be

   241 accomplished otherwise or to repair damaged databases during interactive

   242 session. They are available in `repository` scripts:

   243 

   244 * `sql(sql, args=None, ask_confirm=True)`, executes an arbitrary SQL query on the system source

   245 * `add_entity_type_table(etype, commit=True)`

   246 * `add_relation_type_table(rtype, commit=True)`

   247 * `uninline_relation(rtype, commit=True)`

   248 

   249 

   250 [FIXME] Add explanation on how to use cubicweb-ctl shell