doc/book/en/devrepo/testing.rst
changeset 9880 9adf36ce805e
parent 9878 f3936f64bd98
equal deleted inserted replaced
9879:21278eb03bbf 9880:9adf36ce805e
    23 Most unit tests need a live database to work against. This is achieved
    23 Most unit tests need a live database to work against. This is achieved
    24 by CubicWeb using automatically sqlite (bundled with Python, see
    24 by CubicWeb using automatically sqlite (bundled with Python, see
    25 http://docs.python.org/library/sqlite3.html) as a backend.
    25 http://docs.python.org/library/sqlite3.html) as a backend.
    26 
    26 
    27 The database is stored in the mycube/test/tmpdb,
    27 The database is stored in the mycube/test/tmpdb,
    28 mycube/test/tmpdb-template files. If it does not (yet) exists, it will
    28 mycube/test/tmpdb-template files. If it does not (yet) exist, it will
    29 be built automatically when the test suit starts.
    29 be built automatically when the test suite starts.
    30 
    30 
    31 .. warning::
    31 .. warning::
    32 
    32 
    33   Whenever the schema changes (new entities, attributes, relations)
    33   Whenever the schema changes (new entities, attributes, relations)
    34   one must delete these two files. Changes concerned only with entity
    34   one must delete these two files. Changes concerned only with entity
    35   or relation type properties (constraints, cardinalities,
    35   or relation type properties (constraints, cardinalities,
    36   permissions) and generally dealt with using the
    36   permissions) and generally dealt with using the
    37   `sync_schema_props_perms()` fonction of the migration environment
    37   `sync_schema_props_perms()` function of the migration environment do
    38   need not a database regeneration step.
    38   not need a database regeneration step.
    39 
    39 
    40 .. _hook_test:
    40 .. _hook_test:
    41 
    41 
    42 Unit test by example
    42 Unit test by example
    43 ````````````````````
    43 ````````````````````
    83 
    83 
    84 The test case itself checks that an Operation does its job of
    84 The test case itself checks that an Operation does its job of
    85 preventing cycles amongst Keyword entities.
    85 preventing cycles amongst Keyword entities.
    86 
    86 
    87 The `create_entity` method of connection (or request) objects allows
    87 The `create_entity` method of connection (or request) objects allows
    88 to create an entity. You can link this entity to others entities, by
    88 to create an entity. You can link this entity to other entities, by
    89 specifying as argument, the relation name, and the entity to link, as
    89 specifying as argument, the relation name, and the entity to link, as
    90 value. In the above example, the `Classification` entity is linked to
    90 value. In the above example, the `Classification` entity is linked to
    91 a `CWEtype` via the relation `classifies`. Conversely, if you are
    91 a `CWEtype` via the relation `classifies`. Conversely, if you are
    92 creating a `CWEtype` entity, you can link it to a `Classification`
    92 creating a `CWEtype` entity, you can link it to a `Classification`
    93 entity, by adding `reverse_classifies` as argument.
    93 entity, by adding `reverse_classifies` as argument.
    94 
    94 
    95 .. note::
    95 .. note::
    96 
    96 
    97    the :meth:`commit` method is not called automatically. You have to
    97    the :meth:`commit` method is not called automatically. You have to
    98    call it explicitely if needed (notably to test operations). It is a
    98    call it explicitly if needed (notably to test operations). It is a
    99    good practice to regenerate entities with :meth:`entity_from_eid`
    99    good practice to regenerate entities with :meth:`entity_from_eid`
   100    after a commit to avoid request cache effects.
   100    after a commit to avoid request cache effects.
   101 
   101 
   102 You can see an example of security tests in the
   102 You can see an example of security tests in the
   103 :ref:`adv_tuto_security`.
   103 :ref:`adv_tuto_security`.
   104 
   104 
   105 It is possible to have these tests run continuously using `apycot`_.
   105 It is possible to have these tests run continuously using `apycot`_.
   106 
   106 
   107 .. _apycot: http://www.logilab.org/project/apycot
   107 .. _apycot: http://www.cubicweb.org/project/apycot
   108 
   108 
   109 .. _securitytest:
   109 .. _securitytest:
   110 
   110 
   111 Managing connections or users
   111 Managing connections or users
   112 +++++++++++++++++++++++++++++
   112 +++++++++++++++++++++++++++++
   113 
   113 
   114 Since unit tests are done with the SQLITE backend and this does not
   114 Since unit tests are done with the SQLITE backend and this does not
   115 support multiple connections at a time, you must be careful when
   115 support multiple connections at a time, you must be careful when
   116 simulating security, changing users.
   116 simulating security, changing users.
   117 
   117 
   118 By default, tests run with a user with admin privileges. This
   118 By default, tests run with a user with admin privileges. Connections
   119 user/connection must never be closed. It is accessible through the
   119 using these credentials are accessible through the `admin_access` object
   120 `admin_access` object of the test classes.
   120 of the test classes.
   121 
   121 
   122 The `repo_cnx()` method returns a connection object that can be used as a
   122 The `repo_cnx()` method returns a connection object that can be used as a
   123 context manager:
   123 context manager:
   124 
   124 
   125 .. sourcecode:: python
   125 .. sourcecode:: python
   136    with user1access.web_request() as req:
   136    with user1access.web_request() as req:
   137        req.execute(...)
   137        req.execute(...)
   138        req.cnx.commit()
   138        req.cnx.commit()
   139 
   139 
   140 On exit of the context manager, a rollback is issued, which releases
   140 On exit of the context manager, a rollback is issued, which releases
   141 the connection. Don't forget to issue the `cnx.commit()` calls !
   141 the connection. Don't forget to issue the `cnx.commit()` calls!
   142 
   142 
   143 .. warning::
   143 .. warning::
   144 
   144 
   145    Do not use the references kept to the entities created with a
   145    Do not use references kept to the entities created with a
   146    connection from another !
   146    connection from another one!
   147 
   147 
   148 Email notifications tests
   148 Email notifications tests
   149 `````````````````````````
   149 `````````````````````````
   150 
   150 
   151 When running tests potentially generated e-mails are not really sent
   151 When running tests, potentially generated e-mails are not really sent
   152 but is found in the list `MAILBOX` of module
   152 but are found in the list `MAILBOX` of module
   153 :mod:`cubicweb.devtools.testlib`.
   153 :mod:`cubicweb.devtools.testlib`.
   154 
   154 
   155 You can test your notifications by analyzing the contents of this list, which
   155 You can test your notifications by analyzing the contents of this list, which
   156 contains objects with two attributes:
   156 contains objects with two attributes:
   157 
   157 
   158 * `recipients`, the list of recipients
   158 * `recipients`, the list of recipients
   159 * `msg`, object email.Message
   159 * `msg`, email.Message object
   160 
   160 
   161 Let us look at simple example from the ``blog`` cube.
   161 Let us look at a simple example from the ``blog`` cube.
   162 
   162 
   163 .. sourcecode:: python
   163 .. sourcecode:: python
   164 
   164 
   165     from cubicweb.devtools.testlib import CubicWebTC, MAILBOX
   165     from cubicweb.devtools.testlib import CubicWebTC, MAILBOX
   166 
   166 
   187 
   187 
   188 Visible actions tests
   188 Visible actions tests
   189 `````````````````````
   189 `````````````````````
   190 
   190 
   191 It is easy to write unit tests to test actions which are visible to
   191 It is easy to write unit tests to test actions which are visible to
   192 user or to a category of users. Let's take an example in the
   192 a user or to a category of users. Let's take an example in the
   193 `conference cube`_.
   193 `conference cube`_.
   194 
   194 
   195 .. _`conference cube`: http://www.cubicweb.org/project/cubicweb-conference
   195 .. _`conference cube`: http://www.cubicweb.org/project/cubicweb-conference
   196 .. sourcecode:: python
   196 .. sourcecode:: python
   197 
   197 
   198     class ConferenceActionsTC(CubicWebTC):
   198     class ConferenceActionsTC(CubicWebTC):
   199 
   199 
   200         def setup_database(self):
   200         def setup_database(self):
   201             with self.admin_access.repo_cnx() as cnx:
   201             with self.admin_access.repo_cnx() as cnx:
   202                 self.conf = cnx.create_entity('Conference',
   202                 self.confeid = cnx.create_entity('Conference',
   203                                               title=u'my conf',
   203                                                  title=u'my conf',
   204                                               url_id=u'conf',
   204                                                  url_id=u'conf',
   205                                               start_on=date(2010, 1, 27),
   205                                                  start_on=date(2010, 1, 27),
   206                                               end_on = date(2010, 1, 29),
   206                                                  end_on = date(2010, 1, 29),
   207                                               call_open=True,
   207                                                  call_open=True,
   208                                               reverse_is_chair_at=chair,
   208                                                  reverse_is_chair_at=chair,
   209                                               reverse_is_reviewer_at=reviewer)
   209                                                  reverse_is_reviewer_at=reviewer).eid
   210 
   210 
   211         def test_admin(self):
   211         def test_admin(self):
   212             with self.admin_access.web_request() as req:
   212             with self.admin_access.web_request() as req:
   213                 rset = req.find('Conference').one()
   213                 rset = req.find('Conference').one()
   214                 self.assertListEqual(self.pactions(req, rset),
   214                 self.assertListEqual(self.pactions(req, rset),
   223                 self.assertListEqual(self.action_submenu(req, rset, 'addrelated'),
   223                 self.assertListEqual(self.action_submenu(req, rset, 'addrelated'),
   224                                       [(u'add Track in_conf Conference object',
   224                                       [(u'add Track in_conf Conference object',
   225                                         u'http://testing.fr/cubicweb/add/Track'
   225                                         u'http://testing.fr/cubicweb/add/Track'
   226                                         u'?__linkto=in_conf%%3A%(conf)s%%3Asubject&'
   226                                         u'?__linkto=in_conf%%3A%(conf)s%%3Asubject&'
   227                                         u'__redirectpath=conference%%2Fconf&'
   227                                         u'__redirectpath=conference%%2Fconf&'
   228                                         u'__redirectvid=' % {'conf': self.conf.eid}),
   228                                         u'__redirectvid=' % {'conf': self.confeid}),
   229                                        ])
   229                                        ])
   230 
   230 
   231 You just have to execute a rql query corresponding to the view you want to test,
   231 You just have to execute a rql query corresponding to the view you want to test,
   232 and to compare the result of
   232 and to compare the result of
   233 :meth:`~cubicweb.devtools.testlib.CubicWebTC.pactions` with the list of actions
   233 :meth:`~cubicweb.devtools.testlib.CubicWebTC.pactions` with the list of actions
   234 that must be visible in the interface. This is a list of tuples. The first
   234 that must be visible in the interface. This is a list of tuples. The first
   235 element is the action's `__regid__`, the second the action's class.
   235 element is the action's `__regid__`, the second the action's class.
   236 
   236 
   237 To test actions in submenu, you just have to test the result of
   237 To test actions in a submenu, you just have to test the result of
   238 :meth:`~cubicweb.devtools.testlib.CubicWebTC.action_submenu` method. The last
   238 :meth:`~cubicweb.devtools.testlib.CubicWebTC.action_submenu` method. The last
   239 parameter of the method is the action's category. The result is a list of
   239 parameter of the method is the action's category. The result is a list of
   240 tuples. The first element is the action's title, and the second element the
   240 tuples. The first element is the action's title, and the second element the
   241 action's url.
   241 action's url.
   242 
   242 
   275   namespace, else both your subclass *and* this parent class will be run.
   275   namespace, else both your subclass *and* this parent class will be run.
   276 
   276 
   277 Cache heavy database setup
   277 Cache heavy database setup
   278 -------------------------------
   278 -------------------------------
   279 
   279 
   280 Some tests suite require a complex setup of the database that takes
   280 Some test suites require a complex setup of the database that takes
   281 seconds (or event minutes) to complete. Doing the whole setup for all
   281 seconds (or even minutes) to complete. Doing the whole setup for each
   282 individual tests make the whole run very slow. The ``CubicWebTC``
   282 individual test makes the whole run very slow. The ``CubicWebTC``
   283 class offer a simple way to prepare specific database once for
   283 class offer a simple way to prepare a specific database once for
   284 multiple tests. The `test_db_id` class attribute of your
   284 multiple tests. The `test_db_id` class attribute of your
   285 ``CubicWebTC`` must be set a unique identifier and the
   285 ``CubicWebTC`` subclass must be set to a unique identifier and the
   286 :meth:`pre_setup_database` class method build the cached content. As
   286 :meth:`pre_setup_database` class method must build the cached content. As
   287 the :meth:`pre_setup_database` method is not garantee to be called
   287 the :meth:`pre_setup_database` method is not garanteed to be called
   288 every time a test method is run, you must not set any class attribute
   288 every time a test method is run, you must not set any class attribute
   289 to be used during test *there*. Databases for each `test_db_id` are
   289 to be used during test *there*. Databases for each `test_db_id` are
   290 automatically created if not already in cache. Clearing the cache is
   290 automatically created if not already in cache. Clearing the cache is
   291 up to the user. Cache files are found in the :file:`data/database`
   291 up to the user. Cache files are found in the :file:`data/database`
   292 subdirectory of your test directory.
   292 subdirectory of your test directory.
   293 
   293 
   294 .. warning::
   294 .. warning::
   295 
   295 
   296   Take care to always have the same :meth:`pre_setup_database`
   296   Take care to always have the same :meth:`pre_setup_database`
   297   function for all calls with a given `test_db_id` otherwise your test
   297   function for all classes with a given `test_db_id` otherwise your
   298   will have unpredictable results depending on the first encountered one.
   298   tests will have unpredictable results depending on the first
       
   299   encountered one.
   299 
   300 
   300 
   301 
   301 Testing on a real-life database
   302 Testing on a real-life database
   302 -------------------------------
   303 -------------------------------
   303 
   304 
   339  card, file, preview
   340  card, file, preview
   340 
   341 
   341 The format is:
   342 The format is:
   342 
   343 
   343 * possibly several empy lines or lines starting with ``#`` (comment lines)
   344 * possibly several empy lines or lines starting with ``#`` (comment lines)
   344 * one line containing a coma separated list of cube names.
   345 * one line containing a comma-separated list of cube names.
   345 
   346 
   346 It is also possible to add a ``schema.py`` file in
   347 It is also possible to add a ``schema.py`` file in
   347 ``mycube/test/data``, which will be used by the testing framework,
   348 ``mycube/test/data``, which will be used by the testing framework,
   348 therefore making new entity types and relations available to the
   349 therefore making new entity types and relations available to the
   349 tests. 
   350 tests. 
   350 
   351 
   351 Literate programming
   352 Literate programming
   352 --------------------
   353 --------------------
   353 
   354 
   354 CubicWeb provides some literate programming capabilities. The :ref:`cubicweb-ctl`
   355 CubicWeb provides some literate programming capabilities. The :ref:`cubicweb-ctl`
   355 `shell` command accepts differents format files. If your file ends with `.txt`
   356 `shell` command accepts different file formats. If your file ends with `.txt`
   356 or `.rst`, the file will be parsed by :mod:`doctest.testfile` with CubicWeb
   357 or `.rst`, the file will be parsed by :mod:`doctest.testfile` with CubicWeb's
   357 :ref:`migration` API enabled in it.
   358 :ref:`migration` API enabled in it.
   358 
   359 
   359 Create a `scenario.txt` file into `test/` directory and fill with some content.
   360 Create a `scenario.txt` file in the `test/` directory and fill with some content.
   360 Please refer the :mod:`doctest.testfile` `documentation`_.
   361 Refer to the :mod:`doctest.testfile` `documentation`_.
   361 
   362 
   362 .. _documentation: http://docs.python.org/library/doctest.html
   363 .. _documentation: http://docs.python.org/library/doctest.html
   363 
   364 
   364 Then, you can run it directly by::
   365 Then, you can run it directly by::
   365 
   366 
   392     >>> if condition_not_met:
   393     >>> if condition_not_met:
   393     ...     raise KeyboardInterrupt('please, check your fixture.')
   394     ...     raise KeyboardInterrupt('please, check your fixture.')
   394 
   395 
   395 Passing paramaters
   396 Passing paramaters
   396 ``````````````````
   397 ``````````````````
   397 Using extra arguments to parametrize your scenario is possible by prepend them
   398 Using extra arguments to parametrize your scenario is possible by prepending them
   398 by double dashes.
   399 by double dashes.
   399 
   400 
   400 Please refer to the `cubicweb-ctl shell --help` usage.
   401 Please refer to the `cubicweb-ctl shell --help` usage.
   401 
   402 
   402 .. important::
   403 .. important::
   419 
   420 
   420 * just launch `pytest` in your cube to execute all tests (it will
   421 * just launch `pytest` in your cube to execute all tests (it will
   421   discover them automatically)
   422   discover them automatically)
   422 * launch `pytest unittest_foo.py` to execute one test file
   423 * launch `pytest unittest_foo.py` to execute one test file
   423 * launch `pytest unittest_foo.py bar` to execute all test methods and
   424 * launch `pytest unittest_foo.py bar` to execute all test methods and
   424   all test cases whose name contain `bar`
   425   all test cases whose name contains `bar`
   425 
   426 
   426 Additionally, the `-x` option tells pytest to exit at the first error
   427 Additionally, the `-x` option tells pytest to exit at the first error
   427 or failure. The `-i` option tells pytest to drop into pdb whenever an
   428 or failure. The `-i` option tells pytest to drop into pdb whenever an
   428 exception occurs in a test.
   429 exception occurs in a test.
   429 
   430 
   448 
   449 
   449 
   450 
   450 What you need to know about request and session
   451 What you need to know about request and session
   451 -----------------------------------------------
   452 -----------------------------------------------
   452 
   453 
   453 
       
   454 .. image:: ../images/request_session.png
   454 .. image:: ../images/request_session.png
   455 
   455 
   456 First, remember to think that some code run on a client side, some
   456 First, remember to think that some code run on a client side, some
   457 other on the repository side. More precisely:
   457 other on the repository side. More precisely:
   458 
   458 
   464 The client interacts with the repository through a repoapi connection.
   464 The client interacts with the repository through a repoapi connection.
   465 
   465 
   466 
   466 
   467 .. note::
   467 .. note::
   468 
   468 
   469    These distinction are going to disappear in cubicweb 3.21 (if not
   469    These distinctions are going to disappear in cubicweb 3.21 (if not
   470    before).
   470    before).
   471 
   471 
   472 A repoapi connection is tied to a session in the repository. The connection and
   472 A repoapi connection is tied to a session in the repository. The connection and
   473 request objects are unaccessible from repository code / the session object is
   473 request objects are inaccessible from repository code / the session object is
   474 unaccessible from client code (theoretically at least).
   474 inaccessible from client code (theoretically at least).
   475 
   475 
   476 The web interface provides a request class.  That `request` object provides
   476 The web interface provides a request class.  That `request` object provides
   477 access to all cubicweb resources, eg:
   477 access to all cubicweb resources, eg:
   478 
   478 
   479 * the registry (which itself provides access to the schema and the
   479 * the registry (which itself provides access to the schema and the
   485 * other specific resources depending on the client type (url generation according
   485 * other specific resources depending on the client type (url generation according
   486   to base url, form parameters, etc.).
   486   to base url, form parameters, etc.).
   487 
   487 
   488 
   488 
   489 A `session` provides an api similar to a request regarding RQL execution and
   489 A `session` provides an api similar to a request regarding RQL execution and
   490 access to global resources (registry and all), but also have the following
   490 access to global resources (registry and all), but also has the following
   491 responsibilities:
   491 responsibilities:
   492 
   492 
   493 * handle transaction data, that will live during the time of a single
   493 * handle transaction data, that will live during the time of a single
   494   transaction. This includes the database connections that will be used to
   494   transaction. This includes the database connections that will be used to
   495   execute RQL queries.
   495   execute RQL queries.
   535 2. the web stack opens an authenticated database connection for the
   535 2. the web stack opens an authenticated database connection for the
   536    request, which is associated to a user session
   536    request, which is associated to a user session
   537 
   537 
   538 3. the query is executed (through the repository connection)
   538 3. the query is executed (through the repository connection)
   539 
   539 
   540 4. this query may trigger hooks. Hooks and operation may execute some rql queries
   540 4. this query may trigger hooks. Hooks and operations may execute some rql queries
   541    through `cnx.execute`.
   541    through `cnx.execute`.
   542 
   542 
   543 5. the repository gets the result of the query in 1. If it was a RQL read query,
   543 5. the repository gets the result of the query in 1. If it was a RQL read query,
   544    the database connection is released. If it was a write query, the connection
   544    the database connection is released. If it was a write query, the connection
   545    is then tied to the session until the transaction is commited or rollbacked.
   545    is then tied to the session until the transaction is commited or rolled back.
   546 
   546 
   547 6. results are sent back to the client
   547 6. results are sent back to the client
   548 
   548 
   549 This implies several things:
   549 This implies several things:
   550 
   550 
   552   connection handling is totally transparent
   552   connection handling is totally transparent
   553 
   553 
   554 * however, take care when writing tests: you are usually faking /
   554 * however, take care when writing tests: you are usually faking /
   555   testing both the server and the client side, so you have to decide
   555   testing both the server and the client side, so you have to decide
   556   when to use RepoAccess.client_cnx or RepoAccess.repo_cnx. Ask
   556   when to use RepoAccess.client_cnx or RepoAccess.repo_cnx. Ask
   557   yourself "where the code I want to test will be running, client or
   557   yourself "where will the code I want to test be running, client or
   558   repository side ?". The response is usually: use a repo (since the
   558   repository side?". The response is usually: use a repo (since the
   559   "client connection" concept is going away in a couple of releases).
   559   "client connection" concept is going away in a couple of releases).