--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/doc/book/en/devrepo/testing.rst Fri Apr 23 17:31:46 2010 +0200
@@ -0,0 +1,258 @@
+.. -*- coding: utf-8 -*-
+
+Tests
+=====
+
+Unit tests
+----------
+
+The *CubicWeb* framework provides the `CubicWebTC` test base class in
+the module `cubicweb.devtools.testlib`.
+
+Tests shall be put into the mycube/test directory. Additional test
+data shall go into mycube/test/data.
+
+It is much advised to write tests concerning entities methods, hooks
+and operations, security. The CubicWebTC base class has convenience
+methods to help test all of this.
+
+.. note::
+
+ In the realm of views, there is not much to do but check that the
+ views are valid XHTML. See :ref:`automatic_views_tests` for
+ details. Integration of CubicWeb tests with UI testing tools such as
+ `selenium`_ are currently under invesitgation.
+
+.. _selenium: http://seleniumhq.org/projects/ide/
+
+Most unit tests need a live database to work against. This is achieved
+by CubicWeb using automatically sqlite (bundled with Python, see
+http://docs.python.org/library/sqlite3.html) as a backend.
+
+The database is stored in the mycube/test/tmpdb,
+mycube/test/tmpdb-template files. If it does not (yet) exists, it will
+be built automatically when the test suit starts.
+
+.. warning::
+
+ Whenever the schema changes (new entities, attributes, relations)
+ one must delete these two files. Changes concerned only with entity
+ or relation type properties (constraints, cardinalities,
+ permissions) and generally dealt with using the
+ `sync_schema_props_perms()` fonction of the migration environment
+ need not a database regeneration step.
+
+Unit test by example
+````````````````````
+
+We start with an example extracted from the keyword cube (available
+from http://www.cubicweb.org/project/cubicweb-keyword).
+
+.. sourcecode:: python
+
+ from cubicweb.devtools.testlib import CubicWebTC
+ from cubicweb import ValidationError
+
+ class ClassificationHooksTC(CubicWebTC):
+
+ def setup_database(self):
+ req = self.request()
+ group_etype = req.execute('Any X WHERE X name "CWGroup"').get_entity(0,0)
+ c1 = req.create_entity('Classification', name=u'classif1',
+ classifies=group_etype)
+ user_etype = req.execute('Any X WHERE X name "CWUser"').get_entity(0,0)
+ c2 = req.create_entity('Classification', name=u'classif2',
+ classifies=user_etype)
+ self.kw1 = req.create_entity('Keyword', name=u'kwgroup', included_in=c1)
+ self.kw2 = req.create_entity('Keyword', name=u'kwuser', included_in=c2)
+
+ def test_cannot_create_cycles(self):
+ # direct obvious cycle
+ self.assertRaises(ValidationError, self.kw1.set_relations,
+ subkeyword_of=self.kw1)
+ # testing indirect cycles
+ kw3 = self.execute('INSERT Keyword SK: SK name "kwgroup2", SK included_in C, '
+ 'SK subkeyword_of K WHERE C name "classif1", K eid %s'
+ % self.kw1.eid).get_entity(0,0)
+ self.kw1.set_relations(subkeyword_of=kw3)
+ self.assertRaises(ValidationError, self.commit)
+
+The test class defines a `setup_database` method which populates the
+database with initial data. Each test of the class runs with this
+pre-populated database.
+
+The test case itself checks that an Operation does it job of
+preventing cycles amongst Keyword entities.
+
+You can see an example of security tests in the
+:ref:`adv_tuto_security`.
+
+It is possible to have these tests run continuously using `apycot`_.
+
+.. _apycot: http://www.logilab.org/project/apycot
+
+Managing connections or users
++++++++++++++++++++++++++++++
+
+Since unit tests are done with the SQLITE backend and this does not
+support multiple connections at a time, you must be careful when
+simulating security, changing users.
+
+By default, tests run with a user with admin privileges. This
+user/connection must never be closed.
+
+Before a self.login, one has to release the connection pool in use
+with a self.commit, self.rollback or self.close.
+
+The `login` method returns a connection object that can be used as a
+context manager:
+
+.. sourcecode:: python
+
+ with self.login('user1') as user:
+ req = user.req
+ req.execute(...)
+
+On exit of the context manager, either a commit or rollback is issued,
+which releases the connection.
+
+When one is logged in as a normal user and wants to switch back to the
+admin user without committing, one has to use
+self.restore_connection().
+
+Usage with restore_connection:
+
+.. sourcecode:: python
+
+ # execute using default admin connection
+ self.execute(...)
+ # I want to login with another user, ensure to free admin connection pool
+ # (could have used rollback but not close here
+ # we should never close defaut admin connection)
+ self.commit()
+ cnx = self.login('user')
+ # execute using user connection
+ self.execute(...)
+ # I want to login with another user or with admin user
+ self.commit(); cnx.close()
+ # restore admin connection, never use cnx = self.login('admin'), it will return
+ # the default admin connection and one may be tempted to close it
+ self.restore_connection()
+
+.. warning::
+
+ Do not use the references kept to the entities created with a
+ connection from another !
+
+Email notifications tests
+-------------------------
+
+When running tests potentially generated e-mails are not really sent
+but is found in the list `MAILBOX` of module
+`cubicweb.devtools.testlib`.
+
+You can test your notifications by analyzing the contents of this list, which
+contains objects with two attributes:
+
+* `recipients`, the list of recipients
+* `msg`, object email.Message
+
+Let us look at simple example from the ``blog`` cube.
+
+.. sourcecode:: python
+
+ from cubicweb.devtools.testlib import CubicWebTC, MAILBOX
+
+ class BlogTestsCubicWebTC(CubicWebTC):
+ """test blog specific behaviours"""
+
+ def test_notifications(self):
+ req = self.request()
+ cubicweb_blog = req.create_entity('Blog', title=u'cubicweb',
+ description=u'cubicweb is beautiful')
+ blog_entry_1 = req.create_entity('BlogEntry', title=u'hop',
+ content=u'cubicweb hop')
+ blog_entry_1.set_relations(entry_of=cubicweb_blog)
+ blog_entry_2 = req.create_entity('BlogEntry', title=u'yes',
+ content=u'cubicweb yes')
+ blog_entry_2.set_relations(entry_of=cubicweb_blog)
+ self.assertEquals(len(MAILBOX), 0)
+ self.commit()
+ self.assertEquals(len(MAILBOX), 2)
+ mail = MAILBOX[0]
+ self.assertEquals(mail.subject, '[data] hop')
+ mail = MAILBOX[1]
+ self.assertEquals(mail.subject, '[data] yes')
+
+.. _automatic_views_tests:
+
+Automatic views testing
+-----------------------
+
+This is done automatically with the AutomaticWebTest class. At cube
+creation time, the mycube/test/test_mycube.py file contains such a
+test. The code here has to be uncommented to be usable, without
+further modification.
+
+The ``auto_populate`` method uses a smart algorithm to create
+pseudo-random data in the database, thus enabling the views to be
+invoked and tested.
+
+Depending on the schema, hooks and operations constraints, it is not
+always possible for the automatic auto_populate to proceed.
+
+It is possible of course to completely redefine auto_populate. A
+lighter solution is to give hints (fill some class attributes) about
+what entities and relations have to be skipped by the auto_populate
+mechanism. These are:
+
+* `no_auto_populate`, may contain a list of entity types to skip
+* `ignored_relations`, may contain a list of relation types to skip
+* `application_rql`, may contain a list of rql expressions that
+ auto_populate cannot guess by itself; these must yield resultsets
+ against which views may be selected.
+
+
+Test APIS
+---------
+
+Using Pytest
+````````````
+
+The `pytest` utility (shipping with `logilab-common`_, which is a
+mandatory dependency of CubicWeb) extends the Python unittest
+functionality and is the preferred way to run the CubicWeb test
+suites. Bare unittests also work the usual way.
+
+.. _logilab-common: http://www.logilab.org/project/logilab-common
+
+To use it, you may:
+
+* just launch `pytest` in your cube to execute all tests (it will
+ discover them automatically)
+* launch `pytest unittest_foo.py` to execute one test file
+* launch `pytest unittest_foo.py bar` to execute all test methods and
+ all test cases whose name contain `bar`
+
+Additionally, the `-x` option tells pytest to exit at the first error
+or failure. The `-i` option tells pytest to drop into pdb whenever an
+exception occurs in a test.
+
+When the `-x` option has been used and the run stopped on a test, it
+is possible, after having fixed the test, to relaunch pytest with the
+`-R` option to tell it to start testing again from where it previously
+failed.
+
+Using the `TestCase` base class
+```````````````````````````````
+
+The base class of CubicWebTC is logilab.common.testlib.TestCase, which
+provides a lot of convenient assertion methods.
+
+.. autoclass:: logilab.common.testlib.TestCase
+ :members:
+
+CubicWebTC API
+``````````````
+.. autoclass:: cubicweb.devtools.testlib.CubicWebTC
+ :members: