1 .. -*- coding: utf-8 -*-

     2 

     3 Concepts

     4 --------

     5 

     6 This section aims to provide you the keys of success with `CubicWeb`

     7 by clarifying the terms specific to our framework.

     8 

     9 Global architecture

    10 ~~~~~~~~~~~~~~~~~~~

    11 .. image:: images/archi_globale.en.png

    12 

    13 

    14 `CubicWeb` framework is a server/client application framework. Those two

    15 parts communicate through RQL (`CubicWeb` query language implementation)

    16 and ResultSet (which will be explained in :ref:`TermsVocabulary`).

    17 

    18 The server manages all interactions with sources.

    19 

    20 

    21 .. note::

    22   Usually, the client and server sides are integrated in the same

    23   process and interact directly, without the need for distant

    24   calls using Pyro. But, it is important to note that those two

    25   sides, client/server, are disjointed and it is possible to execute

    26   a couple of calls in distinct processes to balance the load of

    27   your web site on one or more machines.

    28 

    29 .. _TermsVocabulary:

    30 

    31 Terms and vocabulary

    32 ~~~~~~~~~~~~~~~~~~~~~

    33 

    34 `CubicWeb` defines its own terminology. To make sure there is no confusion

    35 while reading this book, we strongly recommand you take time to go through

    36 the following definitions that are the basics to understand while

    37 developing with `CubicWeb`.

    38 

    39 *schema*

    40   The schema defines the data model of an application based on entities

    41   and relations, modeled with a comprehensive language made of Python

    42   classes based on `yams`_ library. This is the core piece

    43   of an application. It is initially defined in the file system and is

    44   stored in the database at the time an instance is created. `CubicWeb`

    45   provides a certain number of system entities included automatically    

    46   (necessary for the core of `CubicWeb`) and a library of

    47   cubes (which defined application entities) that can be explicitely

    48   included if necessary.

    49 

    50 *entity type*

    51   An entity type is a set of attributes; the essential attribute of

    52   an entity is its key, named eid.

    53 

    54 *relation type*

    55   Entities are linked to each others by relations. In `CubicWeb`

    56   relations are binary: by convention we name the first item of

    57   a relation the `subject` and the second the `object`.

    58 

    59 *final entity type*

    60   Final types correspond to the basic types such as string of characters,

    61   integers... Those types have a main property which is that they can

    62   only be used as `object` of a relation. The attributes of an entity

    63   (non final) are entities (finals).

    64 

    65 *final relation type*

    66   A relation is said final if its `object` is a final type. This is equivalent

    67   to an entity attribute.

    68 

    69 *relation definition*

    70   A relation definition is a 3-uple (subject entity type, relation type, object

    71   entity type), with an associated set of property such as cardinality, constraints...

    72 

    73 *repository*

    74   This is the RQL server side of `CubicWeb`. Be carefull not to get

    75   confused with a Mercurial repository or a debian repository.

    76 

    77 *source*

    78   A data source is a container of data (SGBD, LDAP directory, `Google

    79   App Engine`'s datastore ...) integrated in the

    80   `CubicWeb` repository. This repository has at least one source, `system` which

    81   contains the schema of the application, plain-text index and other

    82   vital informations for the system.

    83 

    84 *configuration*

    85   It is possible to create different configurations for an instance:

    86 

    87   - ``repository`` : repository only, accessible for clients using Pyro

    88   - ``twisted`` : web interface only, access the repository using Pyro

    89   - ``all-in-one`` : web interface and repository in a single process.

    90      The repository could be or not accessible using Pyro.

    91 

    92 *cube*

    93   A cube is a model grouping one or multiple data types and/or views

    94   to provide a specific functionality or a complete `CubicWeb` application

    95   potentially using other cubes. The available cubes are located in the file

    96   system at `/path/to/forest/cubicweb/cubes` for a Mercurial forest installation.

    97   For a debian packages installation they will be located in

    98   `/usr/share/cubicweb/cubes`.

    99   Larger applications can be built quite fast by importing cubes,

   100   adding entities and relationships, overriding the

   101   *views* that display the cubes or by editing informations not provided by

   102   the cubes.

   103 

   104 *instance*

   105   An instance is a specific installation of one or multiple cubes. All the required

   106   configuration files necessary for the well being of your web application

   107   are grouped in an instance. This will refer to the cube(s) your application

   108   is based on.

   109   For example logilab.org and our intranet are two instances of a single

   110   cube "jpl", developped internally.

   111   The instances are defined in the directory `/etc/cubicweb.d`.

   112 

   113 *application*

   114   The term application is sometimes used to talk about an instance

   115   and sometimes to talk of a cube depending on the context.

   116   So we would like to avoid using this term and try to use *cube* and

   117   *instance* instead.

   118 

   119 *result set*

   120   This object contains the results of an RQL query sent to the source

   121   and informations on the query.

   122 

   123 *Pyro*

   124   `Python Remote Object`_, distributed objects system similar to Java's RMI

   125   (Remote Method Invocation), which can be used for the dialog between the web

   126   side of the framework and the RQL repository.

   127 

   128 *query language*

   129   A full-blown query language named RQL is used to formulate requests

   130   to the database or any sources such as LDAP or `Google App Engine`'s

   131   datastore.

   132 

   133 *views*

   134   A view is applied to a `result set` to present it as HTML, XML,

   135   JSON, CSV, etc. Views are implemented as Python classes. There is no

   136   templating language.

   137 

   138 *generated user interface*

   139   A user interface is generated on-the-fly from the schema definition:

   140   entities can be created, displayed, updated and deleted. As display

   141   views are not very fancy, it is usually necessary to develop your

   142   own. Any generated view can be overridden by defining a new one with

   143   the same identifier.

   144 

   145 *rql*

   146  Relation Query Language in order to emphasize the way of browsing relations.

   147  This query language is inspired by SQL but is on a higher level;

   148  its implementation generates SQL.

   149 

   150 

   151 .. _`Python Remote Object`: http://pyro.sourceforge.net/

   152 .. _`yams`: http://www.logilab.org/project/yams/

   153 

   154 

   155 `CubicWeb` engine

   156 ~~~~~~~~~~~~~~~~~

   157 

   158 The engine in `CubicWeb` is a set of classes managing a set of objects loaded

   159 dynamically at the startup of `CubicWeb` (*appobjects*). Those dynamic objects,

   160 based on the schema or the library, are building the final application.

   161 The different dynamic components are for example:

   162 

   163 * client and server side

   164 

   165   - entities definition, containing the logic which enables application data manipulation

   166 

   167 * client side

   168 

   169   - *views*, or more specifically

   170 

   171     - boxes

   172     - header and footer

   173     - forms

   174     - page templates

   175 

   176   - *actions*

   177   - *controllers*

   178 

   179 * server side

   180 

   181   - notification hooks

   182   - notification views

   183 

   184 The components of the engine are:

   185 

   186 * a frontal web (only twisted is available so far), transparent for dynamic objects

   187 * an object that encapsulates the configuration

   188 * a `registry` (`cubicweb.cwvreg`) containing the dynamic objects loaded automatically

   189 

   190 Every *appobject* may access to the instance configuration using its *config* attribute

   191 and to the registry using its *vreg* attribute.

   192 

   193 API Python/RQL

   194 ~~~~~~~~~~~~~~

   195 

   196 The Python API developped to interface with RQL is inspired from the standard db-api,

   197 with a Connection object having the methods cursor, rollback and commit essentially.

   198 The most important method is the `execute` method of a cursor :

   199 

   200 `execute(rqlstring, args=None, eid_key=None, build_descr=True)`

   201 

   202 :rqlstring: the RQL query to execute (unicode)

   203 :args: if the query contains substitutions, a dictionary containing the values to use

   204 :eid_key:

   205    an implementation detail of the RQL cache implies that if a substitution

   206    is used to introduce an eid *susceptible to raise the ambiguities in the query

   207    type resolution*, then we have to specify the corresponding key in the dictionary

   208    through this argument

   209 

   210 

   211 The `Connection` object owns the methods `commit` and `rollback`. You *should

   212 never need to use them* during the development of the web interface based on

   213 the `CubicWeb` framework as it determines the end of the transaction depending

   214 on the query execution success.

   215 

   216 .. note::

   217   While executing update queries (SET, INSERT, DELETE), if a query generates

   218   an error related to security, a rollback is automatically done on the current

   219   transaction.

   220 

   221 

   222 The `Request` class (`cubicweb.web`)

   223 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

   224 

   225 A request instance is created when an HTTP request is sent to the web server.

   226 It contains informations such as form parameters, user authenticated, etc.

   227 

   228 **Globally, a request represents a user query, either through HTTP or not

   229 (we also talk about RQL queries on the server side for example).**

   230 

   231 An instance of `Request` has the following attributes:

   232 

   233 * `user`, instance of `cubicweb.common.utils.User` corresponding to the authenticated

   234   user

   235 * `form`, dictionary containing the values of a web form

   236 * `encoding`, character encoding to use in the response

   237 

   238 But also:

   239 

   240 :Session data handling:

   241   * `session_data()`, returns a dictionary containing all the session data

   242   * `get_session_data(key, default=None)`, returns a value associated to the given

   243     key or the value `default` if the key is not defined

   244   * `set_session_data(key, value)`, assign a value to a key

   245   * `del_session_data(key)`,  suppress the value associated to a key

   246 

   247 

   248 :Cookies handling:

   249   * `get_cookie()`, returns a dictionary containing the value of the header

   250     HTTP 'Cookie'

   251   * `set_cookie(cookie, key, maxage=300)`, adds a header HTTP `Set-Cookie`,

   252     with a minimal 5 minutes length of duration by default (`maxage` = None

   253     returns a *session* cookie which will expire when the user closes the browser

   254     window)

   255   * `remove_cookie(cookie, key)`, forces a value to expire

   256 

   257 :URL handling:

   258   * `url()`, returns the full URL of the HTTP request

   259   * `base_url()`, returns the root URL of the web application

   260   * `relative_path()`, returns the relative path of the request

   261 

   262 :And more...:

   263   * `set_content_type(content_type, filename=None)`, adds the header HTTP

   264     'Content-Type'

   265   * `get_header(header)`, returns the value associated to an arbitrary header

   266     of the HTTP request

   267   * `set_header(header, value)`, adds an arbitrary header in the response

   268   * `cursor()` returns a RQL cursor on the session

   269   * `execute(*args, **kwargs)`, shortcut to ``.cursor().execute()``

   270   * `property_value(key)`, properties management (`EProperty`)

   271   * dictionary `data` to store data to share informations between components

   272     *while a request is executed*

   273 

   274 Please note that this class is abstract and that a concrete implementation

   275 will be provided by the *frontend* web used (in particular *twisted* as of

   276 today). For the views or others that are executed on the server side,

   277 most of the interface of `Request` is defined in the session associated

   278 to the client.

   279 

   280 The `AppObject` class

   281 ~~~~~~~~~~~~~~~~~~~~~

   282 

   283 In general:

   284 

   285 * we do not inherit directly from this class but from a more specific

   286   class such as `AnyEntity`, `EntityView`, `AnyRsetView`,

   287   `Action`...

   288 

   289 * to be recordable, a subclass has to define its own register (attribute

   290   `__registry__`) and its identifier (attribute `id`). Usually we do not have

   291   to take care of the register, only the identifier `id`.

   292 

   293 We can find a certain number of attributes and methods defined in this class

   294 and common to all the application objects.

   295 

   296 At the recording, the following attributes are dynamically added to

   297 the *subclasses*:

   298 

   299 * `vreg`, the `vregistry` of the application

   300 * `schema`, the application schema

   301 * `config`, the application configuration

   302 

   303 We also find on instances, the following attributes:

   304 

   305 * `req`, `Request` instance

   306 * `rset`, the *result set* associated to the object if necessary

   307 * `cursor`, rql cursor on the session

   308 

   309 

   310 :URL handling:

   311   * `build_url(method=None, **kwargs)`, returns an absolute URL based on

   312     the given arguments. The *controller* supposed to handle the response,

   313     can be specified through the special parameter `method` (the connection

   314     is theoretically done automatically :).

   315 

   316   * `datadir_url()`, returns the directory of the application data

   317     (contains static files such as images, css, js...)

   318 

   319   * `base_url()`, shortcut to `req.base_url()`

   320 

   321   * `url_quote(value)`, version *unicode safe* of the function `urllib.quote`

   322 

   323 :Data manipulation:

   324 

   325   * `etype_rset(etype, size=1)`, shortcut to `vreg.etype_rset()`

   326 

   327   * `eid_rset(eid, rql=None, descr=True)`, returns a *result set* object for

   328     the given eid

   329   * `entity(row, col=0)`, returns the entity corresponding to the data position

   330     in the *result set* associated to the object

   331 

   332   * `complete_entity(row, col=0, skip_bytes=True)`, is equivalent to `entity` but

   333     also call the method `complete()` on the entity before returning it

   334 

   335 :Data formatting:

   336   * `format_date(date, date_format=None, time=False)` returns a string for a

   337     mx date time according to application's configuration

   338   * `format_time(time)` returns a string for a mx date time according to

   339     application's configuration

   340 

   341 :And more...:

   342 

   343   * `external_resource(rid, default=_MARKER)`, access to a value defined in the

   344     configuration file `external_resource`

   345 

   346   * `tal_render(template, variables)`, renders a precompiled page template with

   347     variables in the given dictionary as context

   348 

   349 .. note::

   350   When we inherit from `AppObject` (even not directly), you *always* have to use

   351   **super()** to get the methods and attributes of the superclasses, and not

   352   use the class identifier.

   353   For example, instead of writting: ::

   354 

   355       class Truc(PrimaryView):

   356           def f(self, arg1):

   357               PrimaryView.f(self, arg1)

   358 

   359   You'd better write: ::

   360 

   361       class Truc(PrimaryView):

   362           def f(self, arg1):

   363               super(Truc, self).f(arg1)

   364 

   365 .. _cubesConcepts:

   366 

   367 Cubes

   368 ~~~~~

   369 

   370 What is a cube ?

   371 ````````````````

   372 

   373 A cube is a model grouping one or more entity types and/or views associated

   374 in order to provide a specific feature or even a complete application using

   375 other cubes.

   376 

   377 You can decide to write your own set of cubes if you wish to re-use the

   378 entity types you develop. Lots of cubes are available from the `CubicWeb

   379 Forge`_ under a free software license.

   380 

   381 .. _`CubicWeb Forge`: http://www.cubicweb.org/project/

   382 

   383 .. _foundationsCube:

   384 

   385 Standard structure for a cube

   386 `````````````````````````````

   387 

   388 A cube is structured as follows:

   389 

   390 ::

   391 

   392   mycube/

   393   |

   394   |-- data/

   395   |   |-- cubes.mycube.css

   396   |   |-- cubes.mycube.js

   397   |   `-- external_resources

   398   |

   399   |-- debian/

   400   |   |-- changelog

   401   |   |-- compat

   402   |   |-- control

   403   |   |-- copyright

   404   |   |-- cubicweb-mycube.prerm

   405   |   `-- rules

   406   |

   407   |-- entities.py

   408   |

   409   |-- i18n/

   410   |   |-- en.po

   411   |   `-- fr.po

   412   |

   413   |-- __init__.py

   414   |

   415   |-- MANIFEST.in

   416   |

   417   |-- migration/

   418   |   |-- postcreate.py

   419   |   `-- precreate.py

   420   |

   421   |-- __pkginfo__.py

   422   |

   423   |-- schema.py

   424   |

   425   |-- setup.py

   426   |

   427   |-- site_cubicweb.py

   428   |

   429   |-- hooks.py

   430   |

   431   |-- test/

   432   |   |-- data/

   433   |   |   `-- bootstrap_cubes

   434   |   |-- pytestconf.py

   435   |   |-- realdb_test_mycube.py

   436   |   `-- test_mycube.py

   437   |

   438   `-- views.py

   439 

   440 

   441 We can use subpackages instead of python modules for ``views.py``, ``entities.py``,

   442 ``schema.py`` or ``hooks.py``. For example, we could have:

   443 

   444 ::

   445 

   446   mycube/

   447   |

   448   |-- entities.py

   449   |-- hooks.py

   450   `-- views/

   451       |-- forms.py

   452       |-- primary.py

   453       `-- widgets.py

   454 

   455 

   456 where :

   457 

   458 * ``schema`` contains the schema definition (server side only)

   459 * ``entities`` contains the entities definition (server side and web interface)

   460 * ``sobjects`` contains hooks and/or views notifications (server side only)

   461 * ``views`` contains the web interface components (web interface only)

   462 * ``test`` contains tests related to the application (not installed)

   463 * ``i18n`` contains message catalogs for supported languages (server side and

   464   web interface)

   465 * ``data`` contains data files for static content (images, css, javascripts)

   466   ...(web interface only)

   467 * ``migration`` contains initialization file for new instances (``postcreate.py``)

   468   and a file containing dependencies of the component depending on the version

   469   (``depends.map``)

   470 * ``debian`` contains all the files managing debian packaging (you will find

   471   the usual files ``control``, ``rules``, ``changelog``... not installed)

   472 * file ``__pkginfo__.py`` provides component meta-data, especially the distribution

   473   and the current version (server side and web interface) or sub-cubes used by

   474   the cube.

   475 

   476 

   477 At least you should have:

   478 

   479 * the file ``__pkginfo__.py``

   480 * the schema definition

   481   XXX false, we may want to have cubes which are only adding a service,

   482   no persistent data (eg embedding for instance)

   483 

   484 

   485 Standard library

   486 ````````````````

   487 

   488 A library of standard cubes are available from `CubicWeb Forge`_

   489 Cubes provide entities and views.

   490 

   491 The available application entities are:

   492 

   493 * addressbook_: PhoneNumber and PostalAddress

   494 

   495 * basket_: Basket (like a shopping cart)

   496 

   497 * blog_: Blog (a *very* basic blog)

   498 

   499 * comment_: Comment (to attach comment threads to entities)

   500 

   501 * event_: Event (define events, display them in calendars)

   502 

   503 * file_: File (to allow users to upload and store binary or text files)

   504 

   505 * folder_: Folder (to organize things but grouping them in folders)

   506 

   507 * keyword_: Keyword (to define classification schemes)

   508 

   509 * link_: Link (to collect links to web resources)

   510 

   511 * mailinglist_: MailingList (to reference a mailing-list and the URLs

   512   for its archives and its admin interface)

   513 

   514 * person_: Person (easily mixed with addressbook)

   515 

   516 * tag_: Tag (to tag anything)

   517 

   518 * task_: Task (something to be done between start and stop date)

   519 

   520 * zone_: Zone (to define places within larger places, for example a

   521   city in a state in a country)

   522 

   523 .. _addressbook: http://www.cubicweb.org/project/cubicweb-addressbook

   524 .. _basket: http://www.cubicweb.org/project/cubicweb-basket

   525 .. _blog: http://www.cubicweb.org/project/cubicweb-blog

   526 .. _comment: http://www.cubicweb.org/project/cubicweb-comment

   527 .. _event: http://www.cubicweb.org/project/cubicweb-event

   528 .. _file: http://www.cubicweb.org/project/cubicweb-file

   529 .. _folder: http://www.cubicweb.org/project/cubicweb-folder

   530 .. _keyword: http://www.cubicweb.org/project/cubicweb-keyword

   531 .. _link: http://www.cubicweb.org/project/cubicweb-link

   532 .. _mailinglist: http://www.cubicweb.org/project/cubicweb-mailinglist

   533 .. _person: http://www.cubicweb.org/project/cubicweb-person

   534 .. _tag: http://www.cubicweb.org/project/cubicweb-tag

   535 .. _task: http://www.cubicweb.org/project/cubicweb-task

   536 .. _zone: http://www.cubicweb.org/project/cubicweb-zone