1  .. -*- coding: utf-8 -*-

     2 

     3 Yams *schema*

     4 -------------

     5 

     6 The **schema** is the core piece of a *CubicWeb* instance as it

     7 defines and handles the data model. It is based on entity types that

     8 are either already defined in `Yams`_ and the *CubicWeb* standard

     9 library; or more specific types defined in cubes. The schema for a

    10 cube is defined in a `schema` python module or package.

    11 

    12 .. _`Yams`: http://www.logilab.org/project/yams

    13 

    14 Overview

    15 ~~~~~~~~

    16 

    17 The core idea of the yams schema is not far from the classical

    18 `Entity-relationship`_ model. But while an E/R model (or `logical

    19 model`) traditionally has to be manually translated to a lower-level

    20 data description language (such as the SQL `create table`

    21 sublanguage), also often described as the `physical model`, no such

    22 step is required with |yams| and |cubicweb|.

    23 

    24 .. _`Entity-relationship`: http://en.wikipedia.org/wiki/Entity-relationship_model

    25 

    26 This is because in addition to high-level, logical |yams| models, one

    27 uses the |rql| data manipulation language to query, insert, update and

    28 delete data. |rql| abstracts as much of the underlying SQL database as

    29 a |yams| schema abstracts from the physical layout. The vagaries of

    30 SQL are avoided.

    31 

    32 As a bonus point, such abstraction make it quite comfortable to build

    33 or use different backends to which |rql| queries apply.

    34 

    35 So, as in the E/R formalism, the building blocks are ``entities``

    36 (:ref:`EntityType`), ``relationships`` (:ref:`RelationType`,

    37 :ref:`RelationDefinition`) and ``attributes`` (handled like relation

    38 with |yams|).

    39 

    40 Let us detail a little the divergences between E/R and |yams|:

    41 

    42 * all relationship are binary which means that to represent a

    43   non-binary relationship, one has to use an entity,

    44 * relationships do not support attributes (yet, see:

    45   http://www.cubicweb.org/ticket/341318), hence the need to reify it

    46   as an entity if need arises,

    47 * all entities have an `eid` attribute (an integer) that is its

    48   primary key (but it is possible to declare uniqueness on other

    49   attributes)

    50 

    51 Also |yams| supports the notions of:

    52 

    53 * entity inheritance,

    54 * relation type: that is, relationships can be established over a set

    55   of couple of entity types (henre the distinction made between

    56   `RelationType` and `RelationDefinition` below)

    57 

    58 Finally |yams| has a few concepts of its own:

    59 

    60 * relationships being oriented and binary, we call the left hand

    61   entity type the `subject` and the right hand entity type the

    62   `object`

    63 

    64 .. note::

    65 

    66    The |yams| schema is available at run time through the .schema

    67    attribute of the `vregistry`.  It's an instance of

    68    :class:`cubicweb.schema.Schema`, which extends

    69    :class:`yams.schema.Schema`.

    70 

    71 .. _EntityType:

    72 

    73 Entity type

    74 ~~~~~~~~~~~

    75 

    76 An entity type is an instance of :class:`yams.schema.EntitySchema`. Each entity type has

    77 a set of attributes and relations, and some permissions which define who can add, read,

    78 update or delete entities of this type.

    79 

    80 The following built-in types are available: ``String``, ``Int``,

    81 ``Float``, ``Decimal``, ``Boolean``, ``Date``, ``Datetime``, ``Time``,

    82 ``Interval``, ``Byte`` and ``Password``. They can only be used as

    83 attributes of an other entity type.

    84 

    85 You can find more base entity types in

    86 :ref:`pre_defined_entity_types`.

    87 

    88 .. XXX yams inheritance

    89 

    90 .. _RelationType:

    91 

    92 Relation type

    93 ~~~~~~~~~~~~~

    94 

    95 A relation type is an instance of

    96 :class:`yams.schema.RelationSchema`. A relation type is simply a

    97 semantic definition of a kind of relationship that may occur in an

    98 application.

    99 

   100 It may be referenced by zero, one or more relation definitions.

   101 

   102 It is important to choose a good name, at least to avoid conflicts

   103 with some semantically different relation defined in other cubes

   104 (since there's only a shared name space for these names).

   105 

   106 A relation type holds the following properties (which are hence shared

   107 between all relation definitions of that type):

   108 

   109 * `inlined`: boolean handling the physical optimization for archiving

   110   the relation in the subject entity table, instead of creating a specific

   111   table for the relation. This applies to relations where cardinality

   112   of subject->relation->object is 0..1 (`?`) or 1..1 (`1`) for *all* its relation

   113   definitions.

   114 

   115 * `symmetric`: boolean indicating that the relation is symmetrical, which

   116   means that `X relation Y` implies `Y relation X`.

   117 

   118 .. _RelationDefinition:

   119 

   120 Relation definition

   121 ~~~~~~~~~~~~~~~~~~~

   122 

   123 A relation definition is an instance of

   124 :class:`yams.schema.RelationDefinition`. It is a complete triplet

   125 "<subject entity type> <relation type> <object entity type>".

   126 

   127 When creating a new instance of that class, the corresponding

   128 :class:`RelationType` instance is created on the fly if necessary.

   129 

   130 Properties

   131 ``````````

   132 

   133 The available properties for relation definitions are enumerated

   134 here. There are several kind of properties, as some relation

   135 definitions are actually attribute definitions, and other are not.

   136 

   137 Some properties may be completely optional, other may have a default

   138 value.

   139 

   140 Common properties for attributes and relations:

   141 

   142 * `description`: an unicode string describing an attribute or a

   143   relation. By default this string will be used in the editing form of

   144   the entity, which means that it is supposed to help the end-user and

   145   should be flagged by the function `_` to be properly

   146   internationalized.

   147 

   148 * `constraints`: a list of conditions/constraints that the relation has to

   149   satisfy (c.f. `Constraints`_)

   150 

   151 * `cardinality`: a two character string specifying the cardinality of

   152   the relation. The first character defines the cardinality of the

   153   relation on the subject, and the second on the object. When a

   154   relation can have multiple subjects or objects, the cardinality

   155   applies to all, not on a one-to-one basis (so it must be

   156   consistent...). Default value is '**'. The possible values are

   157   inspired from regular expression syntax:

   158 

   159     * `1`: 1..1

   160     * `?`: 0..1

   161     * `+`: 1..n

   162     * `*`: 0..n

   163 

   164 Attributes properties:

   165 

   166 * `unique`: boolean indicating if the value of the attribute has to be

   167   unique or not within all entities of the same type (false by

   168   default)

   169 

   170 * `indexed`: boolean indicating if an index needs to be created for

   171   this attribute in the database (false by default). This is useful

   172   only if you know that you will have to run numerous searches on the

   173   value of this attribute.

   174 

   175 * `default`: default value of the attribute. In case of date types, the values

   176   which could be used correspond to the RQL keywords `TODAY` and `NOW`.

   177 

   178 Properties for `String` attributes:

   179 

   180 * `fulltextindexed`: boolean indicating if the attribute is part of

   181   the full text index (false by default) (*applicable on the type

   182   `Byte` as well*)

   183 

   184 * `internationalizable`: boolean indicating if the value of the

   185   attribute is internationalizable (false by default)

   186 

   187 Relation properties:

   188 

   189 * `composite`: string indicating that the subject (composite ==

   190   'subject') is composed of the objects of the relations. For the

   191   opposite case (when the object is composed of the subjects of the

   192   relation), we just set 'object' as value. The composition implies

   193   that when the relation is deleted (so when the composite is deleted,

   194   at least), the composed are also deleted.

   195 

   196 * `fulltext_container`: string indicating if the value if the full

   197   text indexation of the entity on one end of the relation should be

   198   used to find the entity on the other end. The possible values are

   199   'subject' or 'object'. For instance the use_email relation has that

   200   property set to 'subject', since when performing a full text search

   201   people want to find the entity using an email address, and not the

   202   entity representing the email address.

   203 

   204 Constraints

   205 ```````````

   206 

   207 By default, the available constraint types are:

   208 

   209 General Constraints

   210 ......................

   211 

   212 * `SizeConstraint`: allows to specify a minimum and/or maximum size on

   213   string (generic case of `maxsize`)

   214 

   215 * `BoundConstraint`: allows to specify a minimum and/or maximum value

   216   on numeric types and date

   217 

   218 .. sourcecode:: python

   219 

   220    from yams.constraints import BoundConstraint, TODAY

   221    BoundConstraint('<=', TODAY())

   222 

   223 * `IntervalBoundConstraint`: allows to specify an interval with

   224   included values

   225 

   226 .. sourcecode:: python

   227 

   228      class Node(EntityType):

   229          latitude = Float(constraints=[IntervalBoundConstraint(-90, +90)])

   230 

   231 * `UniqueConstraint`: identical to "unique=True"

   232 

   233 * `StaticVocabularyConstraint`: identical to "vocabulary=(...)"

   234 

   235 .. XXX Attribute, NOW

   236 

   237 RQL Based Constraints

   238 ......................

   239 

   240 RQL based constraints may take three arguments. The first one is the ``WHERE``

   241 clause of a RQL query used by the constraint. The second argument ``mainvars``

   242 is the ``Any`` clause of the query. By default this include `S` reserved for the

   243 subject of the relation and `O` for the object. Additional variables could be

   244 specified using ``mainvars``. The argument expects a single string with all

   245 variable's name separated by spaces. The last one, ``msg``, is the error message

   246 displayed when the constraint fails. As RQLVocabularyConstraint never fails the

   247 third argument is not available.

   248 

   249 * `RQLConstraint`: allows to specify a RQL query that has to be satisfied

   250   by the subject and/or the object of relation. In this query the variables

   251   `S` and `O` are reserved for the relation subject and object entities.

   252 

   253 * `RQLVocabularyConstraint`: similar to the previous type of constraint except

   254   that it does not express a "strong" constraint, which means it is only used to

   255   restrict the values listed in the drop-down menu of editing form, but it does

   256   not prevent another entity to be selected.

   257 

   258 * `RQLUniqueConstraint`: allows to the specify a RQL query that ensure that an

   259   attribute is unique in a specific context. The Query must **never** return more

   260   than a single result to be satisfied. In this query the variables `S` is

   261   reserved for the relation subject entity. The other variables should be

   262   specified with the second constructor argument (mainvars). This constraints

   263   should be used when UniqueConstraint doesn't fit. Here is a simple example.

   264 

   265 .. sourcecode:: python

   266 

   267     # Check that in the same Workflow each state's name is unique.  Using

   268     # UniqueConstraint (or unique=True) here would prevent states in different

   269     # workflows to have the same name.

   270 

   271     # With: State S, Workflow W, String N ; S state_of W, S name N

   272 

   273     RQLUniqueConstraint('S name N, S state_of WF, Y state_of WF, Y name N',

   274                         mainvars='Y',

   275                         msg=_('workflow already has a state of that name'))

   276 

   277 .. XXX note about how to add new constraint

   278 

   279 .. _securitymodel:

   280 

   281 The security model

   282 ~~~~~~~~~~~~~~~~~~

   283 

   284 The security model of `CubicWeb` is based on `Access Control List`.

   285 The main principles are:

   286 

   287 * users and groups of users

   288 * a user belongs to at least one group of user

   289 * permissions (read, update, create, delete)

   290 * permissions are assigned to groups (and not to users)

   291 

   292 For *CubicWeb* in particular:

   293 

   294 * we associate rights at the entities/relations schema level

   295 * for each entity, we distinguish four kinds of permissions: `read`,

   296   `add`, `update` and `delete`

   297 * for each relation, we distinguish three kinds of permissions: `read`,

   298   `add` and `delete` (it is not possible to `modify` a relation)

   299 * the default groups are: `administrators`, `users` and `guests`

   300 * by default, users belong to the `users` group

   301 * there is a virtual group called `owners` to which we

   302   can associate only `delete` and `update` permissions

   303 

   304   * we can not add users to the `Owners` group, they are

   305     implicitly added to it according to the context of the objects

   306     they own

   307   * the permissions of this group are only checked on `update`/`delete`

   308     actions if all the other groups the user belongs to do not provide

   309     those permissions

   310 

   311 Setting permissions is done with the attribute `__permissions__` of entities and

   312 relation types. The value of this attribute is a dictionary where the keys are the access types

   313 (action), and the values are the authorized groups or expressions.

   314 

   315 For an entity type, the possible actions are `read`, `add`, `update` and

   316 `delete`.

   317 

   318 For a relation type, the possible actions are `read`, `add`, and `delete`.

   319 

   320 For each access type, a tuple indicates the name of the authorized groups and/or

   321 one or multiple RQL expressions to satisfy to grant access. The access is

   322 provided if the user is in one of the listed groups or if one of the RQL condition

   323 is satisfied.

   324 

   325 The standard user groups

   326 ````````````````````````

   327 

   328 * `guests`

   329 

   330 * `users`

   331 

   332 * `managers`

   333 

   334 * `owners`: virtual group corresponding to the entity's owner.

   335   This can only be used for the actions `update` and `delete` of an entity

   336   type.

   337 

   338 It is also possible to use specific groups if they are defined in the

   339 precreate script of the cube (``migration/precreate.py``). Defining groups in

   340 postcreate script or later makes them unavailable for security

   341 purposes (in this case, an `sync_schema_props_perms` command has to

   342 be issued in a CubicWeb shell).

   343 

   344 

   345 Use of RQL expression for write permissions

   346 ```````````````````````````````````````````

   347 It is possible to define RQL expression to provide update permission

   348 (`add`, `delete` and `update`) on relation and entity types.

   349 

   350 RQL expression for entity type permission:

   351 

   352 * you have to use the class `ERQLExpression`

   353 

   354 * the used expression corresponds to the WHERE statement of an RQL query

   355 

   356 * in this expression, the variables `X` and `U` are pre-defined references

   357   respectively on the current entity (on which the action is verified) and

   358   on the user who send the request

   359 

   360 * it is possible to use, in this expression, a special relation

   361   "has_<ACTION>_permission" where the subject is the user and the

   362   object is any variable, meaning that the user needs to have

   363   permission to execute the action <ACTION> on the entities related

   364   to this variable

   365 

   366 For RQL expressions on a relation type, the principles are the same except

   367 for the following:

   368 

   369 * you have to use the class `RRQLExpression` in the case of a non-final relation

   370 

   371 * in the expression, the variables `S`, `O` and `U` are pre-defined references

   372   to respectively the subject and the object of the current relation (on

   373   which the action is being verified) and the user who executed the query

   374 

   375 * we can also define rights over attributes of an entity (non-final relation),

   376   knowing that:

   377 

   378   - to define RQL expression, we have to use the class `ERQLExpression`

   379     in which `X` represents the entity the attribute belongs to

   380 

   381   - the permissions `add` and `delete` are equivalent. Only `add`/`read`

   382     are actually taken in consideration.

   383 

   384 .. note::

   385 

   386   Potentially, the `use of an RQL expression to add an entity or a

   387   relation` can cause problems for the user interface, because if the

   388   expression uses the entity or the relation to create, then we are

   389   not able to verify the permissions before we actually add the entity

   390   (please note that this is not a problem for the RQL server at all,

   391   because the permissions checks are done after the creation). In such

   392   case, the permission check methods (CubicWebEntitySchema.check_perm

   393   and has_perm) can indicate that the user is not allowed to create

   394   this entity but can obtain the permission.  To compensate this

   395   problem, it is usually necessary, for such case, to use an action

   396   that reflects the schema permissions but which enables to check

   397   properly the permissions so that it would show up if necessary.

   398 

   399 

   400 Use of RQL expression for reading rights

   401 ````````````````````````````````````````

   402 

   403 The principles are the same but with the following restrictions:

   404 

   405 * we can not use `RRQLExpression` on relation types for reading

   406 

   407 * special relations "has_<ACTION>_permission" can not be used

   408 

   409 

   410 

   411 

   412 Defining your schema using yams

   413 -------------------------------

   414 

   415 Entity type definition

   416 ~~~~~~~~~~~~~~~~~~~~~~

   417 

   418 An entity type is defined by a Python class which inherits from

   419 :class:`yams.buildobjs.EntityType`.  The class definition contains the

   420 description of attributes and relations for the defined entity type.

   421 The class name corresponds to the entity type name. It is expected to

   422 be defined in the module ``mycube.schema``.

   423 

   424 :Note on schema definition:

   425 

   426  The code in ``mycube.schema`` is not meant to be executed. The class

   427  EntityType mentioned above is different from the EntitySchema class

   428  described in the previous chapter. EntityType is a helper class to

   429  make Entity definition easier. Yams will process EntityType classes

   430  and create EntitySchema instances from these class definitions. Similar

   431  manipulation happen for relations.

   432 

   433 When defining a schema using python files, you may use the following shortcuts:

   434 

   435 - `required`: boolean indicating if the attribute is required, ed subject cardinality is '1'

   436 

   437 - `vocabulary`: specify static possible values of an attribute

   438 

   439 - `maxsize`: integer providing the maximum size of a string (no limit by default)

   440 

   441 For example:

   442 

   443 .. sourcecode:: python

   444 

   445   class Person(EntityType):

   446     """A person with the properties and the relations necessary for my

   447     application"""

   448 

   449     last_name = String(required=True, fulltextindexed=True)

   450     first_name = String(required=True, fulltextindexed=True)

   451     title = String(vocabulary=('Mr', 'Mrs', 'Miss'))

   452     date_of_birth = Date()

   453     works_for = SubjectRelation('Company', cardinality='?*')

   454 

   455 

   456 The entity described above defines three attributes of type String,

   457 last_name, first_name and title, an attribute of type Date for the date of

   458 birth and a relation that connects a `Person` to another entity of type

   459 `Company` through the semantic `works_for`.

   460 

   461 :Naming convention:

   462 

   463  Entity class names must start with an uppercase letter. The common

   464  usage is to use ``CamelCase`` names.

   465 

   466  Attribute and relation names must start with a lowercase letter. The

   467  common usage is to use ``underscore_separated_words``. Attribute and

   468  relation names starting with a single underscore are permitted, to

   469  denote a somewhat "protected" or "private" attribute.

   470 

   471  In any case, identifiers starting with "CW" or "cw" are reserved for

   472  internal use by the framework.

   473 

   474 

   475 The name of the Python attribute corresponds to the name of the attribute

   476 or the relation in *CubicWeb* application.

   477 

   478 An attribute is defined in the schema as follows::

   479 

   480     attr_name = attr_type(properties)

   481 

   482 where `attr_type` is one of the type listed above and `properties` is

   483 a list of the attribute needs to satisfy (see `Properties`_

   484 for more details).

   485 

   486 * it is possible to use the attribute `meta` to flag an entity type as a `meta`

   487   (e.g. used to describe/categorize other entities)

   488 

   489 .. XXX the paragraph below needs clarification and / or moving out in

   490 .. another place

   491 

   492 *Note*: if you end up with an `if` in the definition of your entity, this probably

   493 means that you need two separate entities that implement the `ITree` interface and

   494 get the result from `.children()` which ever entity is concerned.

   495 

   496 Inheritance

   497 ```````````

   498 XXX feed me

   499 

   500 

   501 Definition of relations

   502 ~~~~~~~~~~~~~~~~~~~~~~~

   503 

   504 XXX add note about defining relation type / definition

   505 

   506 A relation is defined by a Python class heriting `RelationType`. The name

   507 of the class corresponds to the name of the type. The class then contains

   508 a description of the properties of this type of relation, and could as well

   509 contain a string for the subject and a string for the object. This allows to create

   510 new definition of associated relations, (so that the class can have the

   511 definition properties from the relation) for example ::

   512 

   513   class locked_by(RelationType):

   514     """relation on all entities indicating that they are locked"""

   515     inlined = True

   516     cardinality = '?*'

   517     subject = '*'

   518     object = 'CWUser'

   519 

   520 If provided, the `subject` and `object` attributes denote the subject

   521 and object of the various relation definitions related to the relation

   522 type. Allowed values for these attributes are:

   523 

   524 * a string corresponding to an entity type

   525 * a tuple of string corresponding to multiple entity types

   526 * special string such as follows:

   527 

   528   - "**": all types of entities

   529   - "*": all types of non-meta entities

   530   - "@": all types of meta entities but not system entities (e.g. used for

   531     the basic schema description)

   532 

   533 When a relation is not inlined and not symmetrical, and it does not require

   534 specific permissions, it can be defined using a `SubjectRelation`

   535 attribute in the EntityType class. The first argument of `SubjectRelation` gives

   536 the entity type for the object of the relation.

   537 

   538 :Naming convention:

   539 

   540  Although this way of defining relations uses a Python class, the

   541  naming convention defined earlier prevails over the PEP8 conventions

   542  used in the framework: relation type class names use

   543  ``underscore_separated_words``.

   544 

   545 :Historical note:

   546 

   547    It has been historically possible to use `ObjectRelation` which

   548    defines a relation in the opposite direction. This feature is soon to be

   549    deprecated and therefore should not be used in newly written code.

   550 

   551 :Future deprecation note:

   552 

   553   In an even more remote future, it is quite possible that the

   554   SubjectRelation shortcut will become deprecated, in favor of the

   555   RelationType declaration which offers some advantages in the context

   556   of reusable cubes.

   557 

   558 Definition of permissions

   559 ~~~~~~~~~~~~~~~~~~~~~~~~~~

   560 The entity type `CWPermission` from the standard library

   561 allows to build very complex and dynamic security architectures. The schema of

   562 this entity type is as follow:

   563 

   564 .. sourcecode:: python

   565 

   566     class CWPermission(EntityType):

   567         """entity type that may be used to construct some advanced security configuration

   568         """

   569         name = String(required=True, indexed=True, internationalizable=True, maxsize=100)

   570         require_group = SubjectRelation('CWGroup', cardinality='+*',

   571                                         description=_('groups to which the permission is granted'))

   572         require_state = SubjectRelation('State',

   573                                         description=_("entity's state in which the permission is applicable"))

   574         # can be used on any entity

   575         require_permission = ObjectRelation('**', cardinality='*1', composite='subject',

   576                                             description=_("link a permission to the entity. This "

   577                                                           "permission should be used in the security "

   578                                                           "definition of the entity's type to be useful."))

   579 

   580 

   581 Example of configuration:

   582 

   583 .. sourcecode:: python

   584 

   585     class Version(EntityType):

   586         """a version is defining the content of a particular project's release"""

   587 

   588         __permissions__ = {'read':   ('managers', 'users', 'guests',),

   589                            'update': ('managers', 'logilab', 'owners',),

   590                            'delete': ('managers', ),

   591                            'add':    ('managers', 'logilab',

   592                                        ERQLExpression('X version_of PROJ, U in_group G,'

   593                                                  'PROJ require_permission P, P name "add_version",'

   594                                                  'P require_group G'),)}

   595 

   596 

   597     class version_of(RelationType):

   598         """link a version to its project. A version is necessarily linked to one and only one project.

   599         """

   600         __permissions__ = {'read':   ('managers', 'users', 'guests',),

   601                            'delete': ('managers', ),

   602                            'add':    ('managers', 'logilab',

   603                                   RRQLExpression('O require_permission P, P name "add_version",'

   604                                                  'U in_group G, P require_group G'),)

   605                        }

   606         inlined = True

   607 

   608 

   609 This configuration indicates that an entity `CWPermission` named

   610 "add_version" can be associated to a project and provides rights to create

   611 new versions on this project to specific groups. It is important to notice that:

   612 

   613 * in such case, we have to protect both the entity type "Version" and the relation

   614   associating a version to a project ("version_of")

   615 

   616 * because of the genericity of the entity type `CWPermission`, we have to execute

   617   a unification with the groups and/or the states if necessary in the expression

   618   ("U in_group G, P require_group G" in the above example)

   619 

   620 

   621 

   622 Handling schema changes

   623 ~~~~~~~~~~~~~~~~~~~~~~~

   624 

   625 Also, it should be clear that to properly handle data migration, an

   626 instance's schema is stored in the database, so the python schema file

   627 used to defined it is only read when the instance is created or

   628 upgraded.

   629 

   630 .. XXX complete me