+
−.. -*- coding: utf-8 -*-
+
−
+
−Tests
+
−=====
+
−
+
−Unit tests
+
−----------
+
−
+
−The *CubicWeb* framework provides the
+
−:class:`cubicweb.devtools.testlib.CubicWebTC` test base class .
+
−
+
−Tests shall be put into the mycube/test directory. Additional test
+
−data shall go into mycube/test/data.
+
−
+
−It is much advised to write tests concerning entities methods,
+
−actions, hooks and operations, security. The
+
−:class:`~cubicweb.devtools.testlib.CubicWebTC` base class has
+
−convenience methods to help test all of this.
+
−
+
−In the realm of views, automatic tests check that views are valid
+
−XHTML. See :ref:`automatic_views_tests` for details.
+
−
+
−Most unit tests need a live database to work against. This is achieved
+
−by CubicWeb using automatically sqlite (bundled with Python, see
+
−http://docs.python.org/library/sqlite3.html) as a backend.
+
−
+
−The database is stored in the mycube/test/tmpdb,
+
−mycube/test/tmpdb-template files. If it does not (yet) exist, it will
+
−be built automatically when the test suite starts.
+
−
+
−.. warning::
+
−
+
−  Whenever the schema changes (new entities, attributes, relations)
+
−  one must delete these two files. Changes concerned only with entity
+
−  or relation type properties (constraints, cardinalities,
+
−  permissions) and generally dealt with using the
+
−  `sync_schema_props_perms()` function of the migration environment do
+
−  not need a database regeneration step.
+
−
+
−.. _hook_test:
+
−
+
−Unit test by example
+
−````````````````````
+
−
+
−We start with an example extracted from the keyword cube (available
+
−from http://www.cubicweb.org/project/cubicweb-keyword).
+
−
+
−.. sourcecode:: python
+
−
+
−    from cubicweb.devtools.testlib import CubicWebTC
+
−    from cubicweb import ValidationError
+
−
+
−    class ClassificationHooksTC(CubicWebTC):
+
−
+
−        def setup_database(self):
+
−            with self.admin_access.repo_cnx() as cnx:
+
−                group_etype = cnx.find('CWEType', name='CWGroup').one()
+
−                c1 = cnx.create_entity('Classification', name=u'classif1',
+
−                                       classifies=group_etype)
+
−                user_etype = cnx.find('CWEType', name='CWUser').one()
+
−                c2 = cnx.create_entity('Classification', name=u'classif2',
+
−                                       classifies=user_etype)
+
−                self.kw1eid = cnx.create_entity('Keyword', name=u'kwgroup', included_in=c1).eid
+
−                cnx.commit()
+
−
+
−        def test_cannot_create_cycles(self):
+
−            with self.admin_access.repo_cnx() as cnx:
+
−                kw1 = cnx.entity_from_eid(self.kw1eid)
+
−                # direct obvious cycle
+
−                with self.assertRaises(ValidationError):
+
−                    kw1.cw_set(subkeyword_of=kw1)
+
−                cnx.rollback()
+
−                # testing indirect cycles
+
−                kw3 = cnx.execute('INSERT Keyword SK: SK name "kwgroup2", SK included_in C, '
+
−                                  'SK subkeyword_of K WHERE C name "classif1", K eid %(k)s'
+
−                                  {'k': kw1}).get_entity(0,0)
+
−                kw3.cw_set(reverse_subkeyword_of=kw1)
+
−                self.assertRaises(ValidationError, cnx.commit)
+
−
+
−The test class defines a :meth:`setup_database` method which populates the
+
−database with initial data. Each test of the class runs with this
+
−pre-populated database.
+
−
+
−The test case itself checks that an Operation does its job of
+
−preventing cycles amongst Keyword entities.
+
−
+
−The `create_entity` method of connection (or request) objects allows
+
−to create an entity. You can link this entity to other entities, by
+
−specifying as argument, the relation name, and the entity to link, as
+
−value. In the above example, the `Classification` entity is linked to
+
−a `CWEtype` via the relation `classifies`. Conversely, if you are
+
−creating a `CWEtype` entity, you can link it to a `Classification`
+
−entity, by adding `reverse_classifies` as argument.
+
−
+
−.. note::
+
−
+
−   the :meth:`commit` method is not called automatically. You have to
+
−   call it explicitly if needed (notably to test operations). It is a
+
−   good practice to regenerate entities with :meth:`entity_from_eid`
+
−   after a commit to avoid request cache effects.
+
−
+
−You can see an example of security tests in the
+
−:ref:`adv_tuto_security`.
+
−
+
−It is possible to have these tests run continuously using `apycot`_.
+
−
+
−.. _apycot: http://www.cubicweb.org/project/apycot
+
−
+
−.. _securitytest:
+
−
+
−Managing connections or users
+
−+++++++++++++++++++++++++++++
+
−
+
−Since unit tests are done with the SQLITE backend and this does not
+
−support multiple connections at a time, you must be careful when
+
−simulating security, changing users.
+
−
+
−By default, tests run with a user with admin privileges. Connections
+
−using these credentials are accessible through the `admin_access` object
+
−of the test classes.
+
−
+
−The `repo_cnx()` method returns a connection object that can be used as a
+
−context manager:
+
−
+
−.. sourcecode:: python
+
−
+
−   # admin_access is a pre-cooked session wrapping object
+
−   # it is built with:
+
−   # self.admin_access = self.new_access('admin')
+
−   with self.admin_access.repo_cnx() as cnx:
+
−       cnx.execute(...)
+
−       self.create_user(cnx, login='user1')
+
−       cnx.commit()
+
−
+
−   user1access = self.new_access('user1')
+
−   with user1access.web_request() as req:
+
−       req.execute(...)
+
−       req.cnx.commit()
+
−
+
−On exit of the context manager, a rollback is issued, which releases
+
−the connection. Don't forget to issue the `cnx.commit()` calls!
+
−
+
−.. warning::
+
−
+
−   Do not use references kept to the entities created with a
+
−   connection from another one!
+
−
+
−Email notifications tests
+
−`````````````````````````
+
−
+
−When running tests, potentially generated e-mails are not really sent
+
−but are found in the list `MAILBOX` of module
+
−:mod:`cubicweb.devtools.testlib`.
+
−
+
−You can test your notifications by analyzing the contents of this list, which
+
−contains objects with two attributes:
+
−
+
−* `recipients`, the list of recipients
+
−* `msg`, email.Message object
+
−
+
−Let us look at a simple example from the ``blog`` cube.
+
−
+
−.. sourcecode:: python
+
−
+
−    from cubicweb.devtools.testlib import CubicWebTC, MAILBOX
+
−
+
−    class BlogTestsCubicWebTC(CubicWebTC):
+
−        """test blog specific behaviours"""
+
−
+
−        def test_notifications(self):
+
−            with self.admin_access.web_request() as req:
+
−                cubicweb_blog = req.create_entity('Blog', title=u'cubicweb',
+
−                                    description=u'cubicweb is beautiful')
+
−                blog_entry_1 = req.create_entity('BlogEntry', title=u'hop',
+
−                                                 content=u'cubicweb hop')
+
−                blog_entry_1.cw_set(entry_of=cubicweb_blog)
+
−                blog_entry_2 = req.create_entity('BlogEntry', title=u'yes',
+
−                                                 content=u'cubicweb yes')
+
−                blog_entry_2.cw_set(entry_of=cubicweb_blog)
+
−                self.assertEqual(len(MAILBOX), 0)
+
−                req.cnx.commit()
+
−                self.assertEqual(len(MAILBOX), 2)
+
−                mail = MAILBOX[0]
+
−                self.assertEqual(mail.subject, '[data] hop')
+
−                mail = MAILBOX[1]
+
−                self.assertEqual(mail.subject, '[data] yes')
+
−
+
−Visible actions tests
+
−`````````````````````
+
−
+
−It is easy to write unit tests to test actions which are visible to
+
−a user or to a category of users. Let's take an example in the
+
−`conference cube`_.
+
−
+
−.. _`conference cube`: http://www.cubicweb.org/project/cubicweb-conference
+
−.. sourcecode:: python
+
−
+
−    class ConferenceActionsTC(CubicWebTC):
+
−
+
−        def setup_database(self):
+
−            with self.admin_access.repo_cnx() as cnx:
+
−                self.confeid = cnx.create_entity('Conference',
+
−                                                 title=u'my conf',
+
−                                                 url_id=u'conf',
+
−                                                 start_on=date(2010, 1, 27),
+
−                                                 end_on = date(2010, 1, 29),
+
−                                                 call_open=True,
+
−                                                 reverse_is_chair_at=chair,
+
−                                                 reverse_is_reviewer_at=reviewer).eid
+
−
+
−        def test_admin(self):
+
−            with self.admin_access.web_request() as req:
+
−                rset = req.find('Conference').one()
+
−                self.assertListEqual(self.pactions(req, rset),
+
−                                      [('workflow', workflow.WorkflowActions),
+
−                                       ('edit', confactions.ModifyAction),
+
−                                       ('managepermission', actions.ManagePermissionsAction),
+
−                                       ('addrelated', actions.AddRelatedActions),
+
−                                       ('delete', actions.DeleteAction),
+
−                                       ('generate_badge_action', badges.GenerateBadgeAction),
+
−                                       ('addtalkinconf', confactions.AddTalkInConferenceAction)
+
−                                       ])
+
−                self.assertListEqual(self.action_submenu(req, rset, 'addrelated'),
+
−                                      [(u'add Track in_conf Conference object',
+
−                                        u'http://testing.fr/cubicweb/add/Track'
+
−                                        u'?__linkto=in_conf%%3A%(conf)s%%3Asubject&'
+
−                                        u'__redirectpath=conference%%2Fconf&'
+
−                                        u'__redirectvid=' % {'conf': self.confeid}),
+
−                                       ])
+
−
+
−You just have to execute a rql query corresponding to the view you want to test,
+
−and to compare the result of
+
−:meth:`~cubicweb.devtools.testlib.CubicWebTC.pactions` with the list of actions
+
−that must be visible in the interface. This is a list of tuples. The first
+
−element is the action's `__regid__`, the second the action's class.
+
−
+
−To test actions in a submenu, you just have to test the result of
+
−:meth:`~cubicweb.devtools.testlib.CubicWebTC.action_submenu` method. The last
+
−parameter of the method is the action's category. The result is a list of
+
−tuples. The first element is the action's title, and the second element the
+
−action's url.
+
−
+
−
+
−.. _automatic_views_tests:
+
−
+
−Automatic views testing
+
−-----------------------
+
−
+
−This is done automatically with the :class:`cubicweb.devtools.testlib.AutomaticWebTest`
+
−class. At cube creation time, the mycube/test/test_mycube.py file
+
−contains such a test. The code here has to be uncommented to be
+
−usable, without further modification.
+
−
+
−The ``auto_populate`` method uses a smart algorithm to create
+
−pseudo-random data in the database, thus enabling the views to be
+
−invoked and tested.
+
−
+
−Depending on the schema, hooks and operations constraints, it is not
+
−always possible for the automatic auto_populate to proceed.
+
−
+
−It is possible of course to completely redefine auto_populate. A
+
−lighter solution is to give hints (fill some class attributes) about
+
−what entities and relations have to be skipped by the auto_populate
+
−mechanism. These are:
+
−
+
−* `no_auto_populate`, may contain a list of entity types to skip
+
−* `ignored_relations`, may contain a list of relation types to skip
+
−* `application_rql`, may contain a list of rql expressions that
+
−  auto_populate cannot guess by itself; these must yield resultsets
+
−  against which views may be selected.
+
−
+
−.. warning::
+
−
+
−  Take care to not let the imported `AutomaticWebTest` in your test module
+
−  namespace, else both your subclass *and* this parent class will be run.
+
−
+
−Cache heavy database setup
+
−-------------------------------
+
−
+
−Some test suites require a complex setup of the database that takes
+
−seconds (or even minutes) to complete. Doing the whole setup for each
+
−individual test makes the whole run very slow. The ``CubicWebTC``
+
−class offer a simple way to prepare a specific database once for
+
−multiple tests. The `test_db_id` class attribute of your
+
−``CubicWebTC`` subclass must be set to a unique identifier and the
+
−:meth:`pre_setup_database` class method must build the cached content. As
+
−the :meth:`pre_setup_database` method is not garanteed to be called
+
−every time a test method is run, you must not set any class attribute
+
−to be used during test *there*. Databases for each `test_db_id` are
+
−automatically created if not already in cache. Clearing the cache is
+
−up to the user. Cache files are found in the :file:`data/database`
+
−subdirectory of your test directory.
+
−
+
−.. warning::
+
−
+
−  Take care to always have the same :meth:`pre_setup_database`
+
−  function for all classes with a given `test_db_id` otherwise your
+
−  tests will have unpredictable results depending on the first
+
−  encountered one.
+
−
+
−
+
−Testing on a real-life database
+
−-------------------------------
+
−
+
−The ``CubicWebTC`` class uses the `cubicweb.devtools.ApptestConfiguration`
+
−configuration class to setup its testing environment (database driver,
+
−user password, application home, and so on). The `cubicweb.devtools`
+
−module also provides a `RealDatabaseConfiguration`
+
−class that will read a regular cubicweb sources file to fetch all
+
−this information but will also prevent the database to be initalized
+
−and reset between tests.
+
−
+
−For a test class to use a specific configuration, you have to set
+
−the `_config` class attribute on the class as in:
+
−
+
−.. sourcecode:: python
+
−
+
−    from cubicweb.devtools import RealDatabaseConfiguration
+
−    from cubicweb.devtools.testlib import CubicWebTC
+
−
+
−    class BlogRealDatabaseTC(CubicWebTC):
+
−        _config = RealDatabaseConfiguration('blog',
+
−                                            sourcefile='/path/to/realdb_sources')
+
−
+
−        def test_blog_rss(self):
+
−            with self.admin_access.web_request() as req:
+
−                rset = req.execute('Any B ORDERBY D DESC WHERE B is BlogEntry, '
+
−                                   'B created_by U, U login "logilab", B creation_date D')
+
−                self.view('rss', rset, req=req)
+
−
+
−
+
−Testing with other cubes
+
−------------------------
+
−
+
−Sometimes a small component cannot be tested all by itself, so one
+
−needs to specify other cubes to be used as part of the the unit test
+
−suite. This is handled by the ``bootstrap_cubes`` file located under
+
−``mycube/test/data``. One example from the `preview` cube::
+
−
+
− card, file, preview
+
−
+
−The format is:
+
−
+
−* possibly several empy lines or lines starting with ``#`` (comment lines)
+
−* one line containing a comma-separated list of cube names.
+
−
+
−It is also possible to add a ``schema.py`` file in
+
−``mycube/test/data``, which will be used by the testing framework,
+
−therefore making new entity types and relations available to the
+
−tests. 
+
−
+
−Literate programming
+
−--------------------
+
−
+
−CubicWeb provides some literate programming capabilities. The :ref:`cubicweb-ctl`
+
−`shell` command accepts different file formats. If your file ends with `.txt`
+
−or `.rst`, the file will be parsed by :mod:`doctest.testfile` with CubicWeb's
+
−:ref:`migration` API enabled in it.
+
−
+
−Create a `scenario.txt` file in the `test/` directory and fill with some content.
+
−Refer to the :mod:`doctest.testfile` `documentation`_.
+
−
+
−.. _documentation: http://docs.python.org/library/doctest.html
+
−
+
−Then, you can run it directly by::
+
−
+
−    $ cubicweb-ctl shell <cube_instance> test/scenario.txt
+
−
+
−When your scenario file is ready, put it in a new test case to be able to run
+
−it automatically.
+
−
+
−.. sourcecode:: python
+
−
+
−      from os.path import dirname, join
+
−      from logilab.common.testlib import unittest_main
+
−      from cubicweb.devtools.testlib import CubicWebTC
+
−
+
−      class AcceptanceTC(CubicWebTC):
+
−
+
−              def test_scenario(self):
+
−                      self.assertDocTestFile(join(dirname(__file__), 'scenario.txt'))
+
−
+
−      if __name__ == '__main__':
+
−              unittest_main()
+
−
+
−Skipping a scenario
+
−```````````````````
+
−
+
−If you want to set up initial conditions that you can't put in your unit test
+
−case, you have to use a :exc:`KeyboardInterrupt` exception only because of the
+
−way :mod:`doctest` module will catch all the exceptions internally.
+
−
+
−    >>> if condition_not_met:
+
−    ...     raise KeyboardInterrupt('please, check your fixture.')
+
−
+
−Passing paramaters
+
−``````````````````
+
−Using extra arguments to parametrize your scenario is possible by prepending them
+
−by double dashes.
+
−
+
−Please refer to the `cubicweb-ctl shell --help` usage.
+
−
+
−.. important::
+
−    Your scenario file must be utf-8 encoded.
+
−
+
−Test APIS
+
−---------
+
−
+
−Using Pytest
+
−````````````
+
−
+
−The `pytest` utility (shipping with `logilab-common`_, which is a
+
−mandatory dependency of CubicWeb) extends the Python unittest
+
−functionality and is the preferred way to run the CubicWeb test
+
−suites. Bare unittests also work the usual way.
+
−
+
−.. _logilab-common: http://www.logilab.org/project/logilab-common
+
−
+
−To use it, you may:
+
−
+
−* just launch `pytest` in your cube to execute all tests (it will
+
−  discover them automatically)
+
−* launch `pytest unittest_foo.py` to execute one test file
+
−* launch `pytest unittest_foo.py bar` to execute all test methods and
+
−  all test cases whose name contains `bar`
+
−
+
−Additionally, the `-x` option tells pytest to exit at the first error
+
−or failure. The `-i` option tells pytest to drop into pdb whenever an
+
−exception occurs in a test.
+
−
+
−When the `-x` option has been used and the run stopped on a test, it
+
−is possible, after having fixed the test, to relaunch pytest with the
+
−`-R` option to tell it to start testing again from where it previously
+
−failed.
+
−
+
−Using the `TestCase` base class
+
−```````````````````````````````
+
−
+
−The base class of CubicWebTC is logilab.common.testlib.TestCase, which
+
−provides a lot of convenient assertion methods.
+
−
+
−.. autoclass:: logilab.common.testlib.TestCase
+
−   :members:
+
−
+
−CubicWebTC API
+
−``````````````
+
−.. autoclass:: cubicweb.devtools.testlib.CubicWebTC
+
−   :members:
+
−
+
−
+
−What you need to know about request and session
+
−-----------------------------------------------
+
−
+
−.. image:: ../../images/request_session.png
+
−
+
−First, remember to think that some code run on a client side, some
+
−other on the repository side. More precisely:
+
−
+
−* client side: web interface, raw repoapi connection (cubicweb-ctl shell for
+
−  instance);
+
−
+
−* repository side: RQL query execution, that may trigger hooks and operation.
+
−
+
−The client interacts with the repository through a repoapi connection.
+
−
+
−
+
−.. note::
+
−
+
−   These distinctions are going to disappear in cubicweb 3.21 (if not
+
−   before).
+
−
+
−A repoapi connection is tied to a session in the repository. The connection and
+
−request objects are inaccessible from repository code / the session object is
+
−inaccessible from client code (theoretically at least).
+
−
+
−The web interface provides a request class.  That `request` object provides
+
−access to all cubicweb resources, eg:
+
−
+
−* the registry (which itself provides access to the schema and the
+
−  configuration);
+
−
+
−* an underlying repoapi connection (when using req.execute, you actually call the
+
−  repoapi);
+
−
+
−* other specific resources depending on the client type (url generation according
+
−  to base url, form parameters, etc.).
+
−
+
−
+
−A `session` provides an api similar to a request regarding RQL execution and
+
−access to global resources (registry and all), but also has the following
+
−responsibilities:
+
−
+
−* handle transaction data, that will live during the time of a single
+
−  transaction. This includes the database connections that will be used to
+
−  execute RQL queries.
+
−
+
−* handle persistent data that may be used across different (web) requests
+
−
+
−* security and hooks control (not possible through a request)
+
−
+
−
+
−The `_cw` attribute
+
−```````````````````
+
−The `_cw` attribute available on every application object provides access to all
+
−cubicweb resources, i.e.:
+
−
+
−- For code running on the client side (eg web interface view), `_cw` is a request
+
−  instance.
+
−
+
−- For code running on the repository side (hooks and operation), `_cw` is a
+
−  Connection or Session instance.
+
−
+
−
+
−Beware some views may be called with a session (e.g. notifications) or with a
+
−request.
+
−
+
−
+
−Request, session and transaction
+
−````````````````````````````````
+
−
+
−In the web interface, an HTTP request is handled by a single request, which will
+
−be thrown away once the response is sent.
+
−
+
−The web publisher handles the transaction:
+
−
+
−* commit / rollback is done automatically
+
−
+
−* you should not commit / rollback explicitly, except if you really
+
−  need it
+
−
+
−Let's detail the process:
+
−
+
−1. an incoming RQL query comes from a client to the web stack
+
−
+
−2. the web stack opens an authenticated database connection for the
+
−   request, which is associated to a user session
+
−
+
−3. the query is executed (through the repository connection)
+
−
+
−4. this query may trigger hooks. Hooks and operations may execute some rql queries
+
−   through `cnx.execute`.
+
−
+
−5. the repository gets the result of the query in 1. If it was a RQL read query,
+
−   the database connection is released. If it was a write query, the connection
+
−   is then tied to the session until the transaction is commited or rolled back.
+
−
+
−6. results are sent back to the client
+
−
+
−This implies several things:
+
−
+
−* when using a request, or code executed in hooks, this database
+
−  connection handling is totally transparent
+
−
+
−* however, take care when writing tests: you are usually faking /
+
−  testing both the server and the client side, so you have to decide
+
−  when to use RepoAccess.client_cnx or RepoAccess.repo_cnx. Ask
+
−  yourself "where will the code I want to test be running, client or
+
−  repository side?". The response is usually: use a repo (since the
+
−  "client connection" concept is going away in a couple of releases).