# HG changeset patch # User Sylvain Thénault # Date 1270195000 -7200 # Node ID f15286235aef833900695577003c3c348faec1c5 # Parent 910e021131d15cf68a03a92bf1188eb9bfc50fea [doc] enhanced setup section: fix sections'level, more postgres tips, merge environment doc with cwconfig.py module docstring using custom autodocstring directive diff -r 910e021131d1 -r f15286235aef cwconfig.py --- a/cwconfig.py Fri Apr 02 09:55:30 2010 +0200 +++ b/cwconfig.py Fri Apr 02 09:56:40 2010 +0200 @@ -1,46 +1,42 @@ # -*- coding: utf-8 -*- -"""common configuration utilities for cubicweb +#:organization: Logilab +#:copyright: 2001-2010 LOGILAB S.A. (Paris, FRANCE), license is LGPL v2. +#:contact: http://www.logilab.fr/ -- mailto:contact@logilab.fr +#:license: GNU Lesser General Public License, v2.1 - http://www.gnu.org/licenses + +# docstring included in doc/book/en/admin/setup.rst +""" +.. _ResourceMode: + +Resource mode +------------- -:organization: Logilab -:copyright: 2001-2010 LOGILAB S.A. (Paris, FRANCE), license is LGPL v2. -:contact: http://www.logilab.fr/ -- mailto:contact@logilab.fr +A resource *mode* is a predifined set of settings for various resources +directories, such as cubes, instances, etc. to ease development with the +framework. There are two running modes with |cubicweb|: + +* 'user', resources are searched / created in the user home directory: + + - instances are stored in :file:`~/etc/cubicweb.d` + - temporary files (such as pid file) in :file:`/tmp` + +* 'system', resources are searched / created in the system directories (eg + usually requiring root access): + + - instances are stored in :file:`/etc/cubicweb.d` + - temporary files (such as pid file) in :file:`/var/run/cubicweb` + + where `` is the detected installation prefix ('/usr/local' for + instance). -If cubicweb is a mercurial checkout (eg `CWDEV` is true), located in -`CW_SOFTWARE_ROOT`: - - * main cubes directory is `/../cubes`. You can specify - another one with `CW_INSTANCES_DIR` environment variable or simply add some - other directories by using `CW_CUBES_PATH`. - - * cubicweb migration files are by default searched in - `/misc/migration` instead of - `/usr/share/cubicweb/migration/`(unless another emplacement is specified - using `CW_MIGRATION_DIR`. - - * Cubicweb will start in 'user' mode (see below) - - -On startup, Cubicweb is using a specific *mode*. A mode corresponds to some -default setting for various resource directories. There are currently 2 main -modes : 'system', for system wide installation, and 'user', fur user local -installation (e.g. no root privileges). - -'user' mode is activated automatically when cubicweb is a mercurial checkout -(e.g. has a .hg directory). You can also force mode by using the `CW_MODE` -environment variable, to: - -* use system wide installation but user specific instances and all, without root - privileges on the system (`export CW_MODE=user`) - -* use local checkout of cubicweb on system wide instances (requires root - privileges on the system (`export CW_MODE=system`) - - Here is the default resource directories settings according to mode: +Notice that each resource path may be explicitly set using an environment +variable if the default doesn't suit your needs. Here are the default resource +directories that are affected according to mode: * 'system': :: - CW_INSTANCES_DIR = /etc/cubicweb.d/ + CW_INSTANCES_DIR = /etc/cubicweb.d/ CW_INSTANCES_DATA_DIR = /var/lib/cubicweb/instances/ CW_RUNTIME_DIR = /var/run/cubicweb/ @@ -50,27 +46,82 @@ CW_INSTANCES_DATA_DIR = ~/etc/cubicweb.d/ CW_RUNTIME_DIR = /tmp +Cubes search path is also affected, see the :ref:Cube section. + +By default, the mode automatically set to 'user' if a :file:`.hg` directory is found +in the cubicweb package, else it's set to 'system'. You can force this by setting +the :envvar:`CW_MODE` environment variable to either 'user' or 'system' so you can +easily: + +* use system wide installation but user specific instances and all, without root + privileges on the system (`export CW_MODE=user`) + +* use local checkout of cubicweb on system wide instances (requires root + privileges on the system (`export CW_MODE=system`) + +If you've a doubt about the mode you're currently running, check the first line +outputed by the :command:`cubicweb-ctl list` command. + +Also, if cubicweb is a mercurial checkout located in ``: + +* main cubes directory is `/../cubes`. You can specify + another one with :envvar:`CW_INSTANCES_DIR` environment variable or simply + add some other directories by using :envvar:`CW_CUBES_PATH` + +* cubicweb migration files are searched in `/misc/migration` + instead of `/share/cubicweb/migration/`. + + +.. _ConfigurationEnv: + +Environment configuration +------------------------- + +Python +`````` + +If you installed |cubicweb| by cloning the Mercurial forest or from source +distribution, then you will need to update the environment variable PYTHONPATH by +adding the path to the forest `cubicweb`: + +Add the following lines to either :file:`.bashrc` or :file:`.bash_profile` to +configure your development environment :: + + export PYTHONPATH=/full/path/to/cubicweb-forest + +If you installed |cubicweb| with packages, no configuration is required and your +new cubes will be placed in `/usr/share/cubicweb/cubes` and your instances will +be placed in `/etc/cubicweb.d`. + + +CubicWeb +```````` + +Here are all environment variables that may be used to configure |cubicweb|: .. envvar:: CW_MODE - Resource mode: user or system + + Resource mode: user or system, as explained in :ref:ResourceMode. .. envvar:: CW_CUBES_PATH - Augments the default search path for cubes + + Augments the default search path for cubes. You may specify several + directories using ':' as separator (';' under windows environment). .. envvar:: CW_INSTANCES_DIR - Directory where cubicweb instances will be found + + Directory where cubicweb instances will be found. .. envvar:: CW_INSTANCES_DATA_DIR - Directory where cubicweb instances data will be written + + Directory where cubicweb instances data will be written (backup file...) .. envvar:: CW_RUNTIME_DIR + Directory where pid files will be written -.. envvar:: CW_MIGRATION_DIR - Directory where cubicweb migration files will be found - -:license: GNU Lesser General Public License, v2.1 - http://www.gnu.org/licenses +.. |cubicweb| replace:: *CubicWeb* """ __docformat__ = "restructuredtext en" _ = unicode diff -r 910e021131d1 -r f15286235aef doc/book/en/admin/setup.rst --- a/doc/book/en/admin/setup.rst Fri Apr 02 09:55:30 2010 +0200 +++ b/doc/book/en/admin/setup.rst Fri Apr 02 09:56:40 2010 +0200 @@ -8,16 +8,17 @@ Installation of `Cubicweb` and its dependencies ----------------------------------------------- -*CubicWeb* is packaged for Debian and Ubuntu, but can be installed from source +|cubicweb| is packaged for Debian and Ubuntu, but can be installed from source using a tarball or the Mercurial version control system. + .. _DebianInstallation: Debian and Ubuntu packages ``````````````````````````` -Depending on the distribution you are using, add the appropriate line to your list -of sources (for example by editing ``/etc/apt/sources.list``). +Depending on the distribution you are using, add the appropriate line to your +list of sources (for example by editing ``/etc/apt/sources.list``). For Debian Lenny:: @@ -37,21 +38,26 @@ apt-get update apt-get install cubicweb cubicweb-dev -`cubicweb` installs the framework itself, allowing you to create -new instances. + +`cubicweb` installs the framework itself, allowing you to create new instances. + +`cubicweb-dev` installs the development environment allowing you to develop new +cubes. -`cubicweb-dev` installs the development environment allowing you to -develop new cubes. +There is also a wide variety of cubes listed on the `CubicWeb.org Forge`_ +available as debian packages and tarball. -There is also a wide variety of cubes listed on http://www.cubicweb.org/Project available as debian packages and tarball. +The repositories are signed with `Logilab's gnupg key`_. To avoid warning on +"apt-get update": -The repositories are signed with `Logilab's gnupg key`_. To avoid warning on "apt-get update": 1. become root using sudo 2. download http://ftp.logilab.org/dists/logilab-dists-key.asc using e.g. wget 3. run "apt-key add logilab-dists-key.asc" 4. re-run apt-get update (manually or through the package manager, whichever you prefer) .. _`Logilab's gnupg key`: http://ftp.logilab.org/dists/logilab-dists-key.asc +.. _`CubicWeb.org Forge`: http://www.cubicweb.org/project/ + .. _SourceInstallation: @@ -66,6 +72,11 @@ Make sure you have installed the dependencies (see appendixes for the list). +|cubicweb| should soon be pip_ installable, stay tuned (expected in 3.8). + +.. _pip: http://pypi.python.org/pypi/pip + + Install from version control system ``````````````````````````````````` @@ -85,32 +96,31 @@ Make sure you have installed the dependencies (see appendixes for the list). + .. _WindowsInstallation: Windows installation ```````````````````` Base elements -_____________ +~~~~~~~~~~~~~ -Setting up a windows development environment is not too complicated -but requires a series of small steps. What is proposed there is only -an example of what can be done. We assume everything goes into C:\ in -this document. Adjusting the installation drive should be -straightforward. +Setting up a windows development environment is not too complicated but requires +a series of small steps. What is proposed there is only an example of what can be +done. We assume everything goes into `C:\\` in this document. Adjusting the +installation drive should be straightforward. -You should start by downloading and installing the Python(x,y) -distribution. It contains python 2.5 plus numerous useful third-party -modules and applications:: +You should start by downloading and installing the Python(x,y) distribution. It +contains python 2.5 plus numerous useful third-party modules and applications:: http://www.pythonxy.com/download_fr.php -At the time of this writting, one gets version 2.1.15. Among the many -things provided, one finds Eclipse + pydev (an arguably good IDE for -python under windows). +At the time of this writting, one gets version 2.1.15. Among the many things +provided, one finds Eclipse + pydev (an arguably good IDE for python under +windows). -Then you must grab Twisted. There is a windows installer directly -available from this page:: +Then you must grab Twisted. There is a windows installer directly available from +this page:: http://twistedmatrix.com/trac/ @@ -129,11 +139,9 @@ http://www.stickpeople.com/projects/python/win-psycopg/#Version2 -Please be careful to select the right python (2.5) and postgres (8.4) -versions. +Please be careful to select the right python (2.5) and postgres (8.4) versions. -Pyro enables remote access to cubicweb repository instances. Get it -there:: +Pyro enables remote access to cubicweb repository instances. Get it there:: http://sourceforge.net/projects/pyro/files/ @@ -144,26 +152,26 @@ Check out the latest release. -Having graphviz will allow schema drawings, which is quite recommended -(albeit not mandatory). You should get an msi installer there:: +Having graphviz will allow schema drawings, which is quite recommended (albeit +not mandatory). You should get an msi installer there:: http://www.graphviz.org/Download_windows.php -Simplejson will be provided within the forest, but a win32 compiled -version will run much faster:: +Simplejson will be provided within the forest, but a win32 compiled version will +run much faster:: http://www.osuch.org/python-simplejson%3Awin32 Tools -_____ +~~~~~ -Get mercurial + its standard windows GUI (TortoiseHG) there (the -latest is the greatest):: +Get mercurial + its standard windows GUI (TortoiseHG) there (the latest is the +greatest):: http://bitbucket.org/tortoisehg/stable/wiki/download -If you need to peruse mercurial over ssh, it can be helpful to get an -ssh client like Putty:: +If you need to peruse mercurial over ssh, it can be helpful to get an ssh client +like Putty:: http://www.putty.org/ @@ -173,10 +181,9 @@ http://www.vectrace.com/mercurialeclipse/ Setting up the sources -______________________ +~~~~~~~~~~~~~~~~~~~~~~ -You need to enable the mercurial forest extension. To do this, edit -the file:: +You need to enable the mercurial forest extension. To do this, edit the file:: C:\Program Files\TortoiseHg\Mercurial.ini @@ -185,8 +192,8 @@ forest=C:\Program Files\TortoiseHg\ext\forest\forest.py Now, you need to clone the cubicweb repository. We assume that you use -Eclipse. From the IDE, choose File -> Import. In the box, select -`Mercurial/Clone repository using MercurialEclipse`. +Eclipse. From the IDE, choose File -> Import. In the box, select `Mercurial/Clone +repository using MercurialEclipse`. In the import main panel you just have to: @@ -194,28 +201,26 @@ * check the 'Repository is a forest' box. -Then, click on 'Finish'. It might take some time to get it all. Note -that the `cubicwin32` forest contains additional python packages such -as yapps, vobject, simplejson and twisted-web2 which are not provided -with Python(x,y). This is provided for convenience, as we do not -ensure the up-to-dateness of these packages, especially with respect -to security fixes. +Then, click on 'Finish'. It might take some time to get it all. Note that the +`cubicwin32` forest contains additional python packages such as yapps, vobject, +simplejson and twisted-web2 which are not provided with Python(x,y). This is +provided for convenience, as we do not ensure the up-to-dateness of these +packages, especially with respect to security fixes. Environment variables -_____________________ +~~~~~~~~~~~~~~~~~~~~~ -You will need some convenience environment variables once all is set -up. These variables are settable through the GUI by getting at the -'System properties' window (by righ-clicking on 'My Computer' -> -properties). +You will need some convenience environment variables once all is set up. These +variables are settable through the GUI by getting at the 'System properties' +window (by righ-clicking on 'My Computer' -> properties). -In the 'advanced' tab, there is an 'Environment variables' -button. Click on it. That opens a small window allowing edition of -user-related and system-wide variables. +In the 'advanced' tab, there is an 'Environment variables' button. Click on +it. That opens a small window allowing edition of user-related and system-wide +variables. -We will consider only user variables. First, the PATH variable. You -should ensure it contains, separated by semi-colons, and assuming you -are logged in as user Jane:: +We will consider only user variables. First, the PATH variable. You should ensure +it contains, separated by semi-colons, and assuming you are logged in as user +Jane:: C:\Documents and Settings\Jane\My Documents\Python\cubicweb\cubicweb\bin C:\Program Files\Graphviz2.24\bin @@ -231,13 +236,13 @@ ... and get a meaningful output. Running an instance as a service --------------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -This currently assumes that the instances configurations is located -at C:\\etc\\cubicweb.d. +This currently assumes that the instances configurations is located at +C:\\etc\\cubicweb.d. -For a cube 'my_cube', you will then find C:\\etc\\cubicweb.d\\my_cube\\win32svc.py -that has to be used thusly:: +For a cube 'my_cube', you will then find +C:\\etc\\cubicweb.d\\my_cube\\win32svc.py that has to be used thusly:: win32svc install @@ -248,17 +253,6 @@ should start the service. -PostgreSQL installation -``````````````````````` - -Please refer to the `PostgreSQL project online documentation`_. - -.. _`PostgreSQL project online documentation`: http://www.postgresql.org/ - -You need to install the three following packages: `postgresql-8.3`, -`postgresql-contrib-8.3` and `postgresql-plpython-8.3`. - - Other dependencies `````````````````` @@ -271,103 +265,115 @@ * `python-ldap` if you plan to use a LDAP source on the server -.. _ConfigurationEnv: -Environment configuration -------------------------- - -If you installed *CubicWeb* by cloning the Mercurial forest, then you -will need to update the environment variable PYTHONPATH by adding -the path to the forest ``cubicweb``: - -Add the following lines to either `.bashrc` or `.bash_profile` to configure -your development environment :: - - export PYTHONPATH=/full/path/to/cubicweb-forest - -If you installed *CubicWeb* with packages, no configuration is required and your -new cubes will be placed in `/usr/share/cubicweb/cubes` and your instances -will be placed in `/etc/cubicweb.d`. - -You may run a system-wide install of *CubicWeb* in "user mode" and use it for -development by setting the following environment variable:: - - export CW_MODE=user - export CW_CUBES_PATH=~/lib/cubes - export CW_INSTANCES_DIR=~/etc/cubicweb.d/ - export CW_INSTANCES_DATA_DIR=$CW_INSTANCES_DIR - export CW_RUNTIME_DIR=/tmp - -.. note:: - The values given above are our suggestions but of course - can be different. - +.. _DatabaseInstallation: Databases configuration ----------------------- -.. _ConfigurationPostgresql: +Whatever the backend used, database connection information are stored in the +instance's :file:`sources` file. Currently cubicweb has been tested using +Postgresql (recommanded), MySQL, SQLServer and SQLite. + +.. _PostgresqlConfiguration: PostgreSQL configuration ```````````````````````` -.. note:: - If you already have an existing cluster and PostgreSQL server - running, you do not need to execute the initilization step - of your PostgreSQL database. +For installation, please refer to the `PostgreSQL project online documentation`_. + +.. _`PostgreSQL project online documentation`: http://www.postgresql.org/ + +You need to install the three following packages: `postgresql-8.X`, +`postgresql-client-8.X`, and `postgresql-plpython-8.X`. If you run postgres +version prior to 8.3, you'll also need the `postgresql-contrib-8.X` package for +full-text search extension. -* First, initialize the database PostgreSQL with the command ``initdb``. +If you run postgres on another host than the |cubicweb| repository, you should +install the `postgresql-client` package on the |cubicweb| host, and others on the +database host. + +.. Note:: + + If you already have an existing cluster and PostgreSQL server running, you do + not need to execute the initilization step of your PostgreSQL database unless + you want a specific cluster for |cubicweb| databases or if your existing + cluster doesn't use the UTF8 encoding (see note below). + +* First, initialize a PostgreSQL cluster with the command ``initdb``. :: - $ initdb -D /path/to/pgsql + $ initdb -E UTF8 -D /path/to/pgsql + + Notice the encoding specification. This is necessary since |cubicweb| usually + want UTF8 encoded database. If you use a cluster with the wrong encoding, you'll + get error like:: - Once initialized, start the database server PostgreSQL - with the command:: + new encoding (UTF8) is incompatible with the encoding of the template database (SQL_ASCII) + HINT: Use the same encoding as in the template database, or use template0 as template. + + + Once initialized, start the database server PostgreSQL with the command:: $ postgres -D /path/to/psql - If you cannot execute this command due to permission issues, please - make sure that your username has write access on the database. - :: + If you cannot execute this command due to permission issues, please make sure + that your username has write access on the database. :: $ chown username /path/to/pgsql -* The database authentication can be either set to `ident sameuser` - or `md5`. - If set to `md5`, make sure to use an existing user - of your database. - If set to `ident sameuser`, make sure that your - client's operating system user name has a matching user in - the database. If not, please do as follow to create a user:: +* The database authentication can be either set to `ident sameuser` or `md5`. If + set to `md5`, make sure to use an existing user of your database. If set to + `ident sameuser`, make sure that your client's operating system user name has a + matching user in the database. If not, please do as follow to create a user:: $ su $ su - postgres $ createuser -s -P username - The option `-P` (for password prompt), will encrypt the password with - the method set in the configuration file ``pg_hba.conf``. - If you do not use this option `-P`, then the default value will be null - and you will need to set it with:: + The option `-P` (for password prompt), will encrypt the password with the + method set in the configuration file :file:`pg_hba.conf`. If you do not use this + option `-P`, then the default value will be null and you will need to set it + with:: $ su postgres -c "echo ALTER USER username WITH PASSWORD 'userpasswd' | psql" - This login/password will be requested when you will create an - instance with `cubicweb-ctl create` to initialize the database of - your instance. - -.. note:: - The authentication method can be configured in ``pg_hba.conf``. +.. Note:: + The authentication method can be configured in file:`pg_hba.conf`. -.. FIXME Are these steps really necessary? It seemed to work without. +The above login/password will be requested when you will create an instance with +`cubicweb-ctl create` to initialize the database of your instance. -* Installation of plain-text index extension :: +Notice that the `cubicweb-ctl db-create` does database initialization that +may requires a postgres superuser. That's why a login/password is explicitly asked +at this step, so you can use there a superuser without using this user when running +the instance. Things that require special privileges at this step: + +* database creation, require the 'create database' permission +* install the plpython extension language (require superuser) +* install the tsearch extension for postgres version prior to 8.3 (require superuser) - cat /usr/share/postgresql/8.3/contrib/tsearch2.sql | psql -U username template1 +To avoid using a super user each time you create an install, a nice trick is to +install plpython (and tsearch when needed) on the special `template1` database, +so they will be installed automatically when cubicweb databases are created +without even with needs for special access rights. To do so, run :: + + # Installation of plpythonu language by default :: + $ createlang -U pgadmin plpythonu template1 + $ psql -U pgadmin template1 + template1=# update pg_language set lanpltrusted=TRUE where lanname='plpythonu'; -* Installation of plpythonu language by default :: +Where `pgadmin` is a postgres superuser. The last command is necessary since by +default plpython is an 'untrusted' language and as such can't be used by non +superuser. This update fix that problem by making it trusted. - createlang -U pgadmin plpythonu template1 +To install the tsearch plain-text index extension on postgres prior to 8.3, run:: + + cat /usr/share/postgresql/8.X/contrib/tsearch2.sql | psql -U username template1 + + +.. _MySqlConfiguration: MySql configuration ``````````````````` @@ -378,19 +384,22 @@ default-character-set=utf8 max_allowed_packet = 128M -.. note:: +.. Note:: It is unclear whether mysql supports indexed string of arbitrary lenght or not. + +.. _SQLServerConfiguration: + SQLServer configuration ------------------------ +``````````````````````` -As of this writing, sqlserver support is in progress. You should be -able to connect, create a database and go quite far, but some of the -generated SQL is still currently not accepted by the backend. +As of this writing, sqlserver support is in progress. You should be able to +connect, create a database and go quite far, but some of the generated SQL is +still currently not accepted by the backend. -The `source` configuration file may look like this (specific parts -only are shown):: +The `source` configuration file may look like this (specific parts only are +shown):: [system] db-driver=sqlserver2005 @@ -402,17 +411,43 @@ db-encoding=utf8 + +.. _SQLiteConfiguration: + +SQLite configuration +```````````````````` +SQLite has the great advantage of requiring almost no configuration. Simply +use 'sqlite' as db-driver, and set path to the dabase as db-name. Don't specify +anything for db-user and db-password, they will be ignore anyway. + +.. Note:: + SQLite is great for testing and to play with cubicweb but is not suited for + production environments. + + +.. _PyroConfiguration: + Pyro configuration ------------------ -If you use Pyro, it is required to have a name server Pyro running on your -network (by default it is detected by a broadcast request). +If you want to use Pyro to access your instance remotly, or to have multi-source +or distributed configuration, it is required to have a name server Pyro running +on your network. By by default it is detected by a broadcast request, but you can +specify a location in the instance's configuration file. To do so, you need to : -* launch the server manually before starting cubicweb as a server with - `pyro-nsd start` +* launch the server manually before starting cubicweb as a server with `pyro-nsd + start` + +* under debian, edit the file :file:`/etc/default/pyro-nsd` so that the name + server pyro will be launched automatically when the machine fire up + -* edit the file ``/etc/default/pyro-nsd`` so that the name server pyro - will be launched automatically when the machine fire up +Cubicweb resources configuration +-------------------------------- +.. autodocstring:: cubicweb.cwconfig + + +.. |cubicweb| replace:: *CubicWeb* diff -r 910e021131d1 -r f15286235aef doc/book/en/conf.py --- a/doc/book/en/conf.py Fri Apr 02 09:55:30 2010 +0200 +++ b/doc/book/en/conf.py Fri Apr 02 09:56:40 2010 +0200 @@ -19,6 +19,7 @@ # serve to show the default value. import sys, os + from cubicweb import __pkginfo__ as cw # If your extensions are in another directory, add it here. If the directory @@ -31,7 +32,7 @@ # Add any Sphinx extension module names here, as strings. They can be extensions # coming with Sphinx (named 'sphinx.ext.*') or your custom ones. -extensions = ['sphinx.ext.autodoc'] +extensions = ['sphinx.ext.autodoc', 'logilab.common.sphinx_ext'] autoclass_content = 'both' # Add any paths that contain templates here, relative to this directory. templates_path = ['.templates']