# HG changeset patch # User Sylvain Thénault # Date 1272448453 -7200 # Node ID 0865e1e9067483dc9eca5bcbff2bb44072dc4356 # Parent 02b52bf9f5f8e0f2aedeaeee3593cc107288f48f# Parent 8167de96c52376ab3593fa4c8b8d770de616b5f8 backport stable into oldstable: oldstable is now cw 3.7 diff -r 02b52bf9f5f8 -r 0865e1e90674 .hgtags --- a/.hgtags Wed Mar 24 10:23:31 2010 +0100 +++ b/.hgtags Wed Apr 28 11:54:13 2010 +0200 @@ -107,5 +107,15 @@ 0a16f07112b90fb61d2e905855fece77e5a7e39c cubicweb-debian-version-3.6.1-2 bfebe3d14d5390492925fc294dfdafad890a7104 cubicweb-version-3.6.2 f3b4bb9121a0e7ee5961310ff79e61c890948a77 cubicweb-debian-version-3.6.2-1 +270aba1e6fa21dac6b070e7815e6d1291f9c87cd cubicweb-version-3.7.0 +0c9ff7e496ce344b7e6bf5c9dd2847daf9034e5e cubicweb-debian-version-3.7.0-1 +6b0832bbd1daf27c2ce445af5b5222e1e522fb90 cubicweb-version-3.7.1 +9194740f070e64da5a89f6a9a31050a8401ebf0c cubicweb-debian-version-3.7.1-1 9c342fa4f1b73e06917d7dc675949baff442108b cubicweb-version-3.6.3 f9fce56d6a0c2bc6c4b497b66039a8bbbbdc8074 cubicweb-debian-version-3.6.3-1 +d010f749c21d55cd85c5feb442b9cf816282953c cubicweb-version-3.7.2 +8fda29a6c2191ba3cc59242c17b28b34127c75fa cubicweb-debian-version-3.7.2-1 +768beb8e15f15e079f8ee6cfc35125e12b19e140 cubicweb-version-3.7.3 +44c7bf90df71dd562e5a7be5ced3019da603d24f cubicweb-debian-version-3.7.3-1 +ec23f3ebcd34a92b9898b312f44d56cca748d0d6 cubicweb-version-3.7.4 +fefeda65bb83dcc2d775255fe69fdee0e793d135 cubicweb-debian-version-3.7.4-1 diff -r 02b52bf9f5f8 -r 0865e1e90674 COPYING --- a/COPYING Wed Mar 24 10:23:31 2010 +0100 +++ b/COPYING Wed Apr 28 11:54:13 2010 +0200 @@ -1,165 +1,339 @@ - GNU LESSER GENERAL PUBLIC LICENSE - Version 3, 29 June 2007 + GNU GENERAL PUBLIC LICENSE + Version 2, June 1991 - Copyright (C) 2007 Free Software Foundation, Inc. + Copyright (C) 1989, 1991 Free Software Foundation, Inc., + 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed. + Preamble - This version of the GNU Lesser General Public License incorporates -the terms and conditions of version 3 of the GNU General Public -License, supplemented by the additional permissions listed below. - - 0. Additional Definitions. + The licenses for most software are designed to take away your +freedom to share and change it. By contrast, the GNU General Public +License is intended to guarantee your freedom to share and change free +software--to make sure the software is free for all its users. This +General Public License applies to most of the Free Software +Foundation's software and to any other program whose authors commit to +using it. (Some other Free Software Foundation software is covered by +the GNU Lesser General Public License instead.) You can apply it to +your programs, too. - As used herein, "this License" refers to version 3 of the GNU Lesser -General Public License, and the "GNU GPL" refers to version 3 of the GNU -General Public License. + When we speak of free software, we are referring to freedom, not +price. Our General Public Licenses are designed to make sure that you +have the freedom to distribute copies of free software (and charge for +this service if you wish), that you receive source code or can get it +if you want it, that you can change the software or use pieces of it +in new free programs; and that you know you can do these things. - "The Library" refers to a covered work governed by this License, -other than an Application or a Combined Work as defined below. + To protect your rights, we need to make restrictions that forbid +anyone to deny you these rights or to ask you to surrender the rights. +These restrictions translate to certain responsibilities for you if you +distribute copies of the software, or if you modify it. - An "Application" is any work that makes use of an interface provided -by the Library, but which is not otherwise based on the Library. -Defining a subclass of a class defined by the Library is deemed a mode -of using an interface provided by the Library. + For example, if you distribute copies of such a program, whether +gratis or for a fee, you must give the recipients all the rights that +you have. You must make sure that they, too, receive or can get the +source code. And you must show them these terms so they know their +rights. + + We protect your rights with two steps: (1) copyright the software, and +(2) offer you this license which gives you legal permission to copy, +distribute and/or modify the software. - A "Combined Work" is a work produced by combining or linking an -Application with the Library. The particular version of the Library -with which the Combined Work was made is also called the "Linked -Version". + Also, for each author's protection and ours, we want to make certain +that everyone understands that there is no warranty for this free +software. If the software is modified by someone else and passed on, we +want its recipients to know that what they have is not the original, so +that any problems introduced by others will not reflect on the original +authors' reputations. - The "Minimal Corresponding Source" for a Combined Work means the -Corresponding Source for the Combined Work, excluding any source code -for portions of the Combined Work that, considered in isolation, are -based on the Application, and not on the Linked Version. + Finally, any free program is threatened constantly by software +patents. We wish to avoid the danger that redistributors of a free +program will individually obtain patent licenses, in effect making the +program proprietary. To prevent this, we have made it clear that any +patent must be licensed for everyone's free use or not licensed at all. + + The precise terms and conditions for copying, distribution and +modification follow. + + GNU GENERAL PUBLIC LICENSE + TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION - The "Corresponding Application Code" for a Combined Work means the -object code and/or source code for the Application, including any data -and utility programs needed for reproducing the Combined Work from the -Application, but excluding the System Libraries of the Combined Work. + 0. This License applies to any program or other work which contains +a notice placed by the copyright holder saying it may be distributed +under the terms of this General Public License. The "Program", below, +refers to any such program or work, and a "work based on the Program" +means either the Program or any derivative work under copyright law: +that is to say, a work containing the Program or a portion of it, +either verbatim or with modifications and/or translated into another +language. (Hereinafter, translation is included without limitation in +the term "modification".) Each licensee is addressed as "you". - 1. Exception to Section 3 of the GNU GPL. +Activities other than copying, distribution and modification are not +covered by this License; they are outside its scope. The act of +running the Program is not restricted, and the output from the Program +is covered only if its contents constitute a work based on the +Program (independent of having been made by running the Program). +Whether that is true depends on what the Program does. - You may convey a covered work under sections 3 and 4 of this License -without being bound by section 3 of the GNU GPL. + 1. You may copy and distribute verbatim copies of the Program's +source code as you receive it, in any medium, provided that you +conspicuously and appropriately publish on each copy an appropriate +copyright notice and disclaimer of warranty; keep intact all the +notices that refer to this License and to the absence of any warranty; +and give any other recipients of the Program a copy of this License +along with the Program. + +You may charge a fee for the physical act of transferring a copy, and +you may at your option offer warranty protection in exchange for a fee. - 2. Conveying Modified Versions. + 2. You may modify your copy or copies of the Program or any portion +of it, thus forming a work based on the Program, and copy and +distribute such modifications or work under the terms of Section 1 +above, provided that you also meet all of these conditions: - If you modify a copy of the Library, and, in your modifications, a -facility refers to a function or data to be supplied by an Application -that uses the facility (other than as an argument passed when the -facility is invoked), then you may convey a copy of the modified -version: + a) You must cause the modified files to carry prominent notices + stating that you changed the files and the date of any change. + + b) You must cause any work that you distribute or publish, that in + whole or in part contains or is derived from the Program or any + part thereof, to be licensed as a whole at no charge to all third + parties under the terms of this License. - a) under this License, provided that you make a good faith effort to - ensure that, in the event an Application does not supply the - function or data, the facility still operates, and performs - whatever part of its purpose remains meaningful, or + c) If the modified program normally reads commands interactively + when run, you must cause it, when started running for such + interactive use in the most ordinary way, to print or display an + announcement including an appropriate copyright notice and a + notice that there is no warranty (or else, saying that you provide + a warranty) and that users may redistribute the program under + these conditions, and telling the user how to view a copy of this + License. (Exception: if the Program itself is interactive but + does not normally print such an announcement, your work based on + the Program is not required to print an announcement.) - b) under the GNU GPL, with none of the additional permissions of - this License applicable to that copy. +These requirements apply to the modified work as a whole. If +identifiable sections of that work are not derived from the Program, +and can be reasonably considered independent and separate works in +themselves, then this License, and its terms, do not apply to those +sections when you distribute them as separate works. But when you +distribute the same sections as part of a whole which is a work based +on the Program, the distribution of the whole must be on the terms of +this License, whose permissions for other licensees extend to the +entire whole, and thus to each and every part regardless of who wrote it. - 3. Object Code Incorporating Material from Library Header Files. +Thus, it is not the intent of this section to claim rights or contest +your rights to work written entirely by you; rather, the intent is to +exercise the right to control the distribution of derivative or +collective works based on the Program. - The object code form of an Application may incorporate material from -a header file that is part of the Library. You may convey such object -code under terms of your choice, provided that, if the incorporated -material is not limited to numerical parameters, data structure -layouts and accessors, or small macros, inline functions and templates -(ten or fewer lines in length), you do both of the following: +In addition, mere aggregation of another work not based on the Program +with the Program (or with a work based on the Program) on a volume of +a storage or distribution medium does not bring the other work under +the scope of this License. + + 3. You may copy and distribute the Program (or a work based on it, +under Section 2) in object code or executable form under the terms of +Sections 1 and 2 above provided that you also do one of the following: - a) Give prominent notice with each copy of the object code that the - Library is used in it and that the Library and its use are - covered by this License. + a) Accompany it with the complete corresponding machine-readable + source code, which must be distributed under the terms of Sections + 1 and 2 above on a medium customarily used for software interchange; or, + + b) Accompany it with a written offer, valid for at least three + years, to give any third party, for a charge no more than your + cost of physically performing source distribution, a complete + machine-readable copy of the corresponding source code, to be + distributed under the terms of Sections 1 and 2 above on a medium + customarily used for software interchange; or, - b) Accompany the object code with a copy of the GNU GPL and this license - document. - - 4. Combined Works. + c) Accompany it with the information you received as to the offer + to distribute corresponding source code. (This alternative is + allowed only for noncommercial distribution and only if you + received the program in object code or executable form with such + an offer, in accord with Subsection b above.) - You may convey a Combined Work under terms of your choice that, -taken together, effectively do not restrict modification of the -portions of the Library contained in the Combined Work and reverse -engineering for debugging such modifications, if you also do each of -the following: +The source code for a work means the preferred form of the work for +making modifications to it. For an executable work, complete source +code means all the source code for all modules it contains, plus any +associated interface definition files, plus the scripts used to +control compilation and installation of the executable. However, as a +special exception, the source code distributed need not include +anything that is normally distributed (in either source or binary +form) with the major components (compiler, kernel, and so on) of the +operating system on which the executable runs, unless that component +itself accompanies the executable. + +If distribution of executable or object code is made by offering +access to copy from a designated place, then offering equivalent +access to copy the source code from the same place counts as +distribution of the source code, even though third parties are not +compelled to copy the source along with the object code. - a) Give prominent notice with each copy of the Combined Work that - the Library is used in it and that the Library and its use are - covered by this License. + 4. You may not copy, modify, sublicense, or distribute the Program +except as expressly provided under this License. Any attempt +otherwise to copy, modify, sublicense or distribute the Program is +void, and will automatically terminate your rights under this License. +However, parties who have received copies, or rights, from you under +this License will not have their licenses terminated so long as such +parties remain in full compliance. - b) Accompany the Combined Work with a copy of the GNU GPL and this license - document. + 5. You are not required to accept this License, since you have not +signed it. However, nothing else grants you permission to modify or +distribute the Program or its derivative works. These actions are +prohibited by law if you do not accept this License. Therefore, by +modifying or distributing the Program (or any work based on the +Program), you indicate your acceptance of this License to do so, and +all its terms and conditions for copying, distributing or modifying +the Program or works based on it. - c) For a Combined Work that displays copyright notices during - execution, include the copyright notice for the Library among - these notices, as well as a reference directing the user to the - copies of the GNU GPL and this license document. + 6. Each time you redistribute the Program (or any work based on the +Program), the recipient automatically receives a license from the +original licensor to copy, distribute or modify the Program subject to +these terms and conditions. You may not impose any further +restrictions on the recipients' exercise of the rights granted herein. +You are not responsible for enforcing compliance by third parties to +this License. - d) Do one of the following: + 7. If, as a consequence of a court judgment or allegation of patent +infringement or for any other reason (not limited to patent issues), +conditions are imposed on you (whether by court order, agreement or +otherwise) that contradict the conditions of this License, they do not +excuse you from the conditions of this License. If you cannot +distribute so as to satisfy simultaneously your obligations under this +License and any other pertinent obligations, then as a consequence you +may not distribute the Program at all. For example, if a patent +license would not permit royalty-free redistribution of the Program by +all those who receive copies directly or indirectly through you, then +the only way you could satisfy both it and this License would be to +refrain entirely from distribution of the Program. - 0) Convey the Minimal Corresponding Source under the terms of this - License, and the Corresponding Application Code in a form - suitable for, and under terms that permit, the user to - recombine or relink the Application with a modified version of - the Linked Version to produce a modified Combined Work, in the - manner specified by section 6 of the GNU GPL for conveying - Corresponding Source. +If any portion of this section is held invalid or unenforceable under +any particular circumstance, the balance of the section is intended to +apply and the section as a whole is intended to apply in other +circumstances. + +It is not the purpose of this section to induce you to infringe any +patents or other property right claims or to contest validity of any +such claims; this section has the sole purpose of protecting the +integrity of the free software distribution system, which is +implemented by public license practices. Many people have made +generous contributions to the wide range of software distributed +through that system in reliance on consistent application of that +system; it is up to the author/donor to decide if he or she is willing +to distribute software through any other system and a licensee cannot +impose that choice. + +This section is intended to make thoroughly clear what is believed to +be a consequence of the rest of this License. - 1) Use a suitable shared library mechanism for linking with the - Library. A suitable mechanism is one that (a) uses at run time - a copy of the Library already present on the user's computer - system, and (b) will operate properly with a modified version - of the Library that is interface-compatible with the Linked - Version. + 8. If the distribution and/or use of the Program is restricted in +certain countries either by patents or by copyrighted interfaces, the +original copyright holder who places the Program under this License +may add an explicit geographical distribution limitation excluding +those countries, so that distribution is permitted only in or among +countries not thus excluded. In such case, this License incorporates +the limitation as if written in the body of this License. + + 9. The Free Software Foundation may publish revised and/or new versions +of the General Public License from time to time. Such new versions will +be similar in spirit to the present version, but may differ in detail to +address new problems or concerns. + +Each version is given a distinguishing version number. If the Program +specifies a version number of this License which applies to it and "any +later version", you have the option of following the terms and conditions +either of that version or of any later version published by the Free +Software Foundation. If the Program does not specify a version number of +this License, you may choose any version ever published by the Free Software +Foundation. - e) Provide Installation Information, but only if you would otherwise - be required to provide such information under section 6 of the - GNU GPL, and only to the extent that such information is - necessary to install and execute a modified version of the - Combined Work produced by recombining or relinking the - Application with a modified version of the Linked Version. (If - you use option 4d0, the Installation Information must accompany - the Minimal Corresponding Source and Corresponding Application - Code. If you use option 4d1, you must provide the Installation - Information in the manner specified by section 6 of the GNU GPL - for conveying Corresponding Source.) + 10. If you wish to incorporate parts of the Program into other free +programs whose distribution conditions are different, write to the author +to ask for permission. For software which is copyrighted by the Free +Software Foundation, write to the Free Software Foundation; we sometimes +make exceptions for this. Our decision will be guided by the two goals +of preserving the free status of all derivatives of our free software and +of promoting the sharing and reuse of software generally. + + NO WARRANTY + + 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY +FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN +OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES +PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED +OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS +TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE +PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, +REPAIR OR CORRECTION. - 5. Combined Libraries. + 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING +WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR +REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, +INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING +OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED +TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY +YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER +PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE +POSSIBILITY OF SUCH DAMAGES. - You may place library facilities that are a work based on the -Library side by side in a single library together with other library -facilities that are not Applications and are not covered by this -License, and convey such a combined library under terms of your -choice, if you do both of the following: + END OF TERMS AND CONDITIONS + + How to Apply These Terms to Your New Programs - a) Accompany the combined library with a copy of the same work based - on the Library, uncombined with any other library facilities, - conveyed under the terms of this License. + If you develop a new program, and you want it to be of the greatest +possible use to the public, the best way to achieve this is to make it +free software which everyone can redistribute and change under these terms. + + To do so, attach the following notices to the program. It is safest +to attach them to the start of each source file to most effectively +convey the exclusion of warranty; and each file should have at least +the "copyright" line and a pointer to where the full notice is found. - b) Give prominent notice with the combined library that part of it - is a work based on the Library, and explaining where to find the - accompanying uncombined form of the same work. + + Copyright (C) + + This program is free software; you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation; either version 2 of the License, or + (at your option) any later version. - 6. Revised Versions of the GNU Lesser General Public License. + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. - The Free Software Foundation may publish revised and/or new versions -of the GNU Lesser General Public License from time to time. Such new -versions will be similar in spirit to the present version, but may -differ in detail to address new problems or concerns. + You should have received a copy of the GNU General Public License along + with this program; if not, write to the Free Software Foundation, Inc., + 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. + +Also add information on how to contact you by electronic and paper mail. + +If the program is interactive, make it output a short notice like this +when it starts in an interactive mode: - Each version is given a distinguishing version number. If the -Library as you received it specifies that a certain numbered version -of the GNU Lesser General Public License "or any later version" -applies to it, you have the option of following the terms and -conditions either of that published version or of any later version -published by the Free Software Foundation. If the Library as you -received it does not specify a version number of the GNU Lesser -General Public License, you may choose any version of the GNU Lesser -General Public License ever published by the Free Software Foundation. + Gnomovision version 69, Copyright (C) year name of author + Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'. + This is free software, and you are welcome to redistribute it + under certain conditions; type `show c' for details. + +The hypothetical commands `show w' and `show c' should show the appropriate +parts of the General Public License. Of course, the commands you use may +be called something other than `show w' and `show c'; they could even be +mouse-clicks or menu items--whatever suits your program. - If the Library as you received it specifies that a proxy can decide -whether future versions of the GNU Lesser General Public License shall -apply, that proxy's public statement of acceptance of any version is -permanent authorization for you to choose that version for the -Library. +You should also get your employer (if you work as a programmer) or your +school, if any, to sign a "copyright disclaimer" for the program, if +necessary. Here is a sample; alter the names: + + Yoyodyne, Inc., hereby disclaims all copyright interest in the program + `Gnomovision' (which makes passes at compilers) written by James Hacker. + + , 1 April 1989 + Ty Coon, President of Vice + +This General Public License does not permit incorporating your program into +proprietary programs. If your program is a subroutine library, you may +consider it more useful to permit linking proprietary applications with the +library. If this is what you want to do, use the GNU Lesser General +Public License instead of this License. diff -r 02b52bf9f5f8 -r 0865e1e90674 COPYING.LESSER --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/COPYING.LESSER Wed Apr 28 11:54:13 2010 +0200 @@ -0,0 +1,510 @@ + + GNU LESSER GENERAL PUBLIC LICENSE + Version 2.1, February 1999 + + Copyright (C) 1991, 1999 Free Software Foundation, Inc. + 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + +[This is the first released version of the Lesser GPL. It also counts + as the successor of the GNU Library Public License, version 2, hence + the version number 2.1.] + + Preamble + + The licenses for most software are designed to take away your +freedom to share and change it. By contrast, the GNU General Public +Licenses are intended to guarantee your freedom to share and change +free software--to make sure the software is free for all its users. + + This license, the Lesser General Public License, applies to some +specially designated software packages--typically libraries--of the +Free Software Foundation and other authors who decide to use it. You +can use it too, but we suggest you first think carefully about whether +this license or the ordinary General Public License is the better +strategy to use in any particular case, based on the explanations +below. + + When we speak of free software, we are referring to freedom of use, +not price. Our General Public Licenses are designed to make sure that +you have the freedom to distribute copies of free software (and charge +for this service if you wish); that you receive source code or can get +it if you want it; that you can change the software and use pieces of +it in new free programs; and that you are informed that you can do +these things. + + To protect your rights, we need to make restrictions that forbid +distributors to deny you these rights or to ask you to surrender these +rights. These restrictions translate to certain responsibilities for +you if you distribute copies of the library or if you modify it. + + For example, if you distribute copies of the library, whether gratis +or for a fee, you must give the recipients all the rights that we gave +you. You must make sure that they, too, receive or can get the source +code. If you link other code with the library, you must provide +complete object files to the recipients, so that they can relink them +with the library after making changes to the library and recompiling +it. And you must show them these terms so they know their rights. + + We protect your rights with a two-step method: (1) we copyright the +library, and (2) we offer you this license, which gives you legal +permission to copy, distribute and/or modify the library. + + To protect each distributor, we want to make it very clear that +there is no warranty for the free library. Also, if the library is +modified by someone else and passed on, the recipients should know +that what they have is not the original version, so that the original +author's reputation will not be affected by problems that might be +introduced by others. + + Finally, software patents pose a constant threat to the existence of +any free program. We wish to make sure that a company cannot +effectively restrict the users of a free program by obtaining a +restrictive license from a patent holder. Therefore, we insist that +any patent license obtained for a version of the library must be +consistent with the full freedom of use specified in this license. + + Most GNU software, including some libraries, is covered by the +ordinary GNU General Public License. This license, the GNU Lesser +General Public License, applies to certain designated libraries, and +is quite different from the ordinary General Public License. We use +this license for certain libraries in order to permit linking those +libraries into non-free programs. + + When a program is linked with a library, whether statically or using +a shared library, the combination of the two is legally speaking a +combined work, a derivative of the original library. The ordinary +General Public License therefore permits such linking only if the +entire combination fits its criteria of freedom. The Lesser General +Public License permits more lax criteria for linking other code with +the library. + + We call this license the "Lesser" General Public License because it +does Less to protect the user's freedom than the ordinary General +Public License. It also provides other free software developers Less +of an advantage over competing non-free programs. These disadvantages +are the reason we use the ordinary General Public License for many +libraries. However, the Lesser license provides advantages in certain +special circumstances. + + For example, on rare occasions, there may be a special need to +encourage the widest possible use of a certain library, so that it +becomes a de-facto standard. To achieve this, non-free programs must +be allowed to use the library. A more frequent case is that a free +library does the same job as widely used non-free libraries. In this +case, there is little to gain by limiting the free library to free +software only, so we use the Lesser General Public License. + + In other cases, permission to use a particular library in non-free +programs enables a greater number of people to use a large body of +free software. For example, permission to use the GNU C Library in +non-free programs enables many more people to use the whole GNU +operating system, as well as its variant, the GNU/Linux operating +system. + + Although the Lesser General Public License is Less protective of the +users' freedom, it does ensure that the user of a program that is +linked with the Library has the freedom and the wherewithal to run +that program using a modified version of the Library. + + The precise terms and conditions for copying, distribution and +modification follow. Pay close attention to the difference between a +"work based on the library" and a "work that uses the library". The +former contains code derived from the library, whereas the latter must +be combined with the library in order to run. + + GNU LESSER GENERAL PUBLIC LICENSE + TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION + + 0. This License Agreement applies to any software library or other +program which contains a notice placed by the copyright holder or +other authorized party saying it may be distributed under the terms of +this Lesser General Public License (also called "this License"). +Each licensee is addressed as "you". + + A "library" means a collection of software functions and/or data +prepared so as to be conveniently linked with application programs +(which use some of those functions and data) to form executables. + + The "Library", below, refers to any such software library or work +which has been distributed under these terms. A "work based on the +Library" means either the Library or any derivative work under +copyright law: that is to say, a work containing the Library or a +portion of it, either verbatim or with modifications and/or translated +straightforwardly into another language. (Hereinafter, translation is +included without limitation in the term "modification".) + + "Source code" for a work means the preferred form of the work for +making modifications to it. For a library, complete source code means +all the source code for all modules it contains, plus any associated +interface definition files, plus the scripts used to control +compilation and installation of the library. + + Activities other than copying, distribution and modification are not +covered by this License; they are outside its scope. The act of +running a program using the Library is not restricted, and output from +such a program is covered only if its contents constitute a work based +on the Library (independent of the use of the Library in a tool for +writing it). Whether that is true depends on what the Library does +and what the program that uses the Library does. + + 1. You may copy and distribute verbatim copies of the Library's +complete source code as you receive it, in any medium, provided that +you conspicuously and appropriately publish on each copy an +appropriate copyright notice and disclaimer of warranty; keep intact +all the notices that refer to this License and to the absence of any +warranty; and distribute a copy of this License along with the +Library. + + You may charge a fee for the physical act of transferring a copy, +and you may at your option offer warranty protection in exchange for a +fee. + + 2. You may modify your copy or copies of the Library or any portion +of it, thus forming a work based on the Library, and copy and +distribute such modifications or work under the terms of Section 1 +above, provided that you also meet all of these conditions: + + a) The modified work must itself be a software library. + + b) You must cause the files modified to carry prominent notices + stating that you changed the files and the date of any change. + + c) You must cause the whole of the work to be licensed at no + charge to all third parties under the terms of this License. + + d) If a facility in the modified Library refers to a function or a + table of data to be supplied by an application program that uses + the facility, other than as an argument passed when the facility + is invoked, then you must make a good faith effort to ensure that, + in the event an application does not supply such function or + table, the facility still operates, and performs whatever part of + its purpose remains meaningful. + + (For example, a function in a library to compute square roots has + a purpose that is entirely well-defined independent of the + application. Therefore, Subsection 2d requires that any + application-supplied function or table used by this function must + be optional: if the application does not supply it, the square + root function must still compute square roots.) + +These requirements apply to the modified work as a whole. If +identifiable sections of that work are not derived from the Library, +and can be reasonably considered independent and separate works in +themselves, then this License, and its terms, do not apply to those +sections when you distribute them as separate works. But when you +distribute the same sections as part of a whole which is a work based +on the Library, the distribution of the whole must be on the terms of +this License, whose permissions for other licensees extend to the +entire whole, and thus to each and every part regardless of who wrote +it. + +Thus, it is not the intent of this section to claim rights or contest +your rights to work written entirely by you; rather, the intent is to +exercise the right to control the distribution of derivative or +collective works based on the Library. + +In addition, mere aggregation of another work not based on the Library +with the Library (or with a work based on the Library) on a volume of +a storage or distribution medium does not bring the other work under +the scope of this License. + + 3. You may opt to apply the terms of the ordinary GNU General Public +License instead of this License to a given copy of the Library. To do +this, you must alter all the notices that refer to this License, so +that they refer to the ordinary GNU General Public License, version 2, +instead of to this License. (If a newer version than version 2 of the +ordinary GNU General Public License has appeared, then you can specify +that version instead if you wish.) Do not make any other change in +these notices. + + Once this change is made in a given copy, it is irreversible for +that copy, so the ordinary GNU General Public License applies to all +subsequent copies and derivative works made from that copy. + + This option is useful when you wish to copy part of the code of +the Library into a program that is not a library. + + 4. You may copy and distribute the Library (or a portion or +derivative of it, under Section 2) in object code or executable form +under the terms of Sections 1 and 2 above provided that you accompany +it with the complete corresponding machine-readable source code, which +must be distributed under the terms of Sections 1 and 2 above on a +medium customarily used for software interchange. + + If distribution of object code is made by offering access to copy +from a designated place, then offering equivalent access to copy the +source code from the same place satisfies the requirement to +distribute the source code, even though third parties are not +compelled to copy the source along with the object code. + + 5. A program that contains no derivative of any portion of the +Library, but is designed to work with the Library by being compiled or +linked with it, is called a "work that uses the Library". Such a +work, in isolation, is not a derivative work of the Library, and +therefore falls outside the scope of this License. + + However, linking a "work that uses the Library" with the Library +creates an executable that is a derivative of the Library (because it +contains portions of the Library), rather than a "work that uses the +library". The executable is therefore covered by this License. +Section 6 states terms for distribution of such executables. + + When a "work that uses the Library" uses material from a header file +that is part of the Library, the object code for the work may be a +derivative work of the Library even though the source code is not. +Whether this is true is especially significant if the work can be +linked without the Library, or if the work is itself a library. The +threshold for this to be true is not precisely defined by law. + + If such an object file uses only numerical parameters, data +structure layouts and accessors, and small macros and small inline +functions (ten lines or less in length), then the use of the object +file is unrestricted, regardless of whether it is legally a derivative +work. (Executables containing this object code plus portions of the +Library will still fall under Section 6.) + + Otherwise, if the work is a derivative of the Library, you may +distribute the object code for the work under the terms of Section 6. +Any executables containing that work also fall under Section 6, +whether or not they are linked directly with the Library itself. + + 6. As an exception to the Sections above, you may also combine or +link a "work that uses the Library" with the Library to produce a +work containing portions of the Library, and distribute that work +under terms of your choice, provided that the terms permit +modification of the work for the customer's own use and reverse +engineering for debugging such modifications. + + You must give prominent notice with each copy of the work that the +Library is used in it and that the Library and its use are covered by +this License. You must supply a copy of this License. If the work +during execution displays copyright notices, you must include the +copyright notice for the Library among them, as well as a reference +directing the user to the copy of this License. Also, you must do one +of these things: + + a) Accompany the work with the complete corresponding + machine-readable source code for the Library including whatever + changes were used in the work (which must be distributed under + Sections 1 and 2 above); and, if the work is an executable linked + with the Library, with the complete machine-readable "work that + uses the Library", as object code and/or source code, so that the + user can modify the Library and then relink to produce a modified + executable containing the modified Library. (It is understood + that the user who changes the contents of definitions files in the + Library will not necessarily be able to recompile the application + to use the modified definitions.) + + b) Use a suitable shared library mechanism for linking with the + Library. A suitable mechanism is one that (1) uses at run time a + copy of the library already present on the user's computer system, + rather than copying library functions into the executable, and (2) + will operate properly with a modified version of the library, if + the user installs one, as long as the modified version is + interface-compatible with the version that the work was made with. + + c) Accompany the work with a written offer, valid for at least + three years, to give the same user the materials specified in + Subsection 6a, above, for a charge no more than the cost of + performing this distribution. + + d) If distribution of the work is made by offering access to copy + from a designated place, offer equivalent access to copy the above + specified materials from the same place. + + e) Verify that the user has already received a copy of these + materials or that you have already sent this user a copy. + + For an executable, the required form of the "work that uses the +Library" must include any data and utility programs needed for +reproducing the executable from it. However, as a special exception, +the materials to be distributed need not include anything that is +normally distributed (in either source or binary form) with the major +components (compiler, kernel, and so on) of the operating system on +which the executable runs, unless that component itself accompanies +the executable. + + It may happen that this requirement contradicts the license +restrictions of other proprietary libraries that do not normally +accompany the operating system. Such a contradiction means you cannot +use both them and the Library together in an executable that you +distribute. + + 7. You may place library facilities that are a work based on the +Library side-by-side in a single library together with other library +facilities not covered by this License, and distribute such a combined +library, provided that the separate distribution of the work based on +the Library and of the other library facilities is otherwise +permitted, and provided that you do these two things: + + a) Accompany the combined library with a copy of the same work + based on the Library, uncombined with any other library + facilities. This must be distributed under the terms of the + Sections above. + + b) Give prominent notice with the combined library of the fact + that part of it is a work based on the Library, and explaining + where to find the accompanying uncombined form of the same work. + + 8. You may not copy, modify, sublicense, link with, or distribute +the Library except as expressly provided under this License. Any +attempt otherwise to copy, modify, sublicense, link with, or +distribute the Library is void, and will automatically terminate your +rights under this License. However, parties who have received copies, +or rights, from you under this License will not have their licenses +terminated so long as such parties remain in full compliance. + + 9. You are not required to accept this License, since you have not +signed it. However, nothing else grants you permission to modify or +distribute the Library or its derivative works. These actions are +prohibited by law if you do not accept this License. Therefore, by +modifying or distributing the Library (or any work based on the +Library), you indicate your acceptance of this License to do so, and +all its terms and conditions for copying, distributing or modifying +the Library or works based on it. + + 10. Each time you redistribute the Library (or any work based on the +Library), the recipient automatically receives a license from the +original licensor to copy, distribute, link with or modify the Library +subject to these terms and conditions. You may not impose any further +restrictions on the recipients' exercise of the rights granted herein. +You are not responsible for enforcing compliance by third parties with +this License. + + 11. If, as a consequence of a court judgment or allegation of patent +infringement or for any other reason (not limited to patent issues), +conditions are imposed on you (whether by court order, agreement or +otherwise) that contradict the conditions of this License, they do not +excuse you from the conditions of this License. If you cannot +distribute so as to satisfy simultaneously your obligations under this +License and any other pertinent obligations, then as a consequence you +may not distribute the Library at all. For example, if a patent +license would not permit royalty-free redistribution of the Library by +all those who receive copies directly or indirectly through you, then +the only way you could satisfy both it and this License would be to +refrain entirely from distribution of the Library. + +If any portion of this section is held invalid or unenforceable under +any particular circumstance, the balance of the section is intended to +apply, and the section as a whole is intended to apply in other +circumstances. + +It is not the purpose of this section to induce you to infringe any +patents or other property right claims or to contest validity of any +such claims; this section has the sole purpose of protecting the +integrity of the free software distribution system which is +implemented by public license practices. Many people have made +generous contributions to the wide range of software distributed +through that system in reliance on consistent application of that +system; it is up to the author/donor to decide if he or she is willing +to distribute software through any other system and a licensee cannot +impose that choice. + +This section is intended to make thoroughly clear what is believed to +be a consequence of the rest of this License. + + 12. If the distribution and/or use of the Library is restricted in +certain countries either by patents or by copyrighted interfaces, the +original copyright holder who places the Library under this License +may add an explicit geographical distribution limitation excluding those +countries, so that distribution is permitted only in or among +countries not thus excluded. In such case, this License incorporates +the limitation as if written in the body of this License. + + 13. The Free Software Foundation may publish revised and/or new +versions of the Lesser General Public License from time to time. +Such new versions will be similar in spirit to the present version, +but may differ in detail to address new problems or concerns. + +Each version is given a distinguishing version number. If the Library +specifies a version number of this License which applies to it and +"any later version", you have the option of following the terms and +conditions either of that version or of any later version published by +the Free Software Foundation. If the Library does not specify a +license version number, you may choose any version ever published by +the Free Software Foundation. + + 14. If you wish to incorporate parts of the Library into other free +programs whose distribution conditions are incompatible with these, +write to the author to ask for permission. For software which is +copyrighted by the Free Software Foundation, write to the Free +Software Foundation; we sometimes make exceptions for this. Our +decision will be guided by the two goals of preserving the free status +of all derivatives of our free software and of promoting the sharing +and reuse of software generally. + + NO WARRANTY + + 15. BECAUSE THE LIBRARY IS LICENSED FREE OF CHARGE, THERE IS NO +WARRANTY FOR THE LIBRARY, TO THE EXTENT PERMITTED BY APPLICABLE LAW. +EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR +OTHER PARTIES PROVIDE THE LIBRARY "AS IS" WITHOUT WARRANTY OF ANY +KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR +PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE +LIBRARY IS WITH YOU. SHOULD THE LIBRARY PROVE DEFECTIVE, YOU ASSUME +THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION. + + 16. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN +WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY +AND/OR REDISTRIBUTE THE LIBRARY AS PERMITTED ABOVE, BE LIABLE TO YOU +FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR +CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE +LIBRARY (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING +RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A +FAILURE OF THE LIBRARY TO OPERATE WITH ANY OTHER SOFTWARE), EVEN IF +SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH +DAMAGES. + + END OF TERMS AND CONDITIONS + + How to Apply These Terms to Your New Libraries + + If you develop a new library, and you want it to be of the greatest +possible use to the public, we recommend making it free software that +everyone can redistribute and change. You can do so by permitting +redistribution under these terms (or, alternatively, under the terms +of the ordinary General Public License). + + To apply these terms, attach the following notices to the library. +It is safest to attach them to the start of each source file to most +effectively convey the exclusion of warranty; and each file should +have at least the "copyright" line and a pointer to where the full +notice is found. + + + + Copyright (C) + + This library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + This library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with this library; if not, write to the Free Software + Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA + +Also add information on how to contact you by electronic and paper mail. + +You should also get your employer (if you work as a programmer) or +your school, if any, to sign a "copyright disclaimer" for the library, +if necessary. Here is a sample; alter the names: + + Yoyodyne, Inc., hereby disclaims all copyright interest in the + library `Frob' (a library for tweaking knobs) written by James + Random Hacker. + + , 1 April 1990 + Ty Coon, President of Vice + +That's all there is to it! + + diff -r 02b52bf9f5f8 -r 0865e1e90674 MANIFEST.in --- a/MANIFEST.in Wed Mar 24 10:23:31 2010 +0100 +++ b/MANIFEST.in Wed Apr 28 11:54:13 2010 +0200 @@ -1,4 +1,6 @@ include README +include COPYING +include COPYING.LESSER include pylintrc include bin/cubicweb-* include man/cubicweb-ctl.1 @@ -15,7 +17,7 @@ recursive-include etwist *.xml *.html recursive-include i18n *.pot *.po -recursive-include schemas *.py *.sql.* +recursive-include schemas *.py *.sql recursive-include entities/test/data * recursive-include sobjects/test/data * diff -r 02b52bf9f5f8 -r 0865e1e90674 __init__.py --- a/__init__.py Wed Mar 24 10:23:31 2010 +0100 +++ b/__init__.py Wed Apr 28 11:54:13 2010 +0200 @@ -1,10 +1,23 @@ +# copyright 2003-2010 LOGILAB S.A. (Paris, FRANCE), all rights reserved. +# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr +# +# This file is part of CubicWeb. +# +# CubicWeb is free software: you can redistribute it and/or modify it under the +# terms of the GNU Lesser General Public License as published by the Free +# Software Foundation, either version 2.1 of the License, or (at your option) +# any later version. +# +# logilab-common is distributed in the hope that it will be useful, but WITHOUT +# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS +# FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more +# details. +# +# You should have received a copy of the GNU Lesser General Public License along +# with CubicWeb. If not, see . """CubicWeb is a generic framework to quickly build applications which describes relations between entitites. -:organization: Logilab -:copyright: 2001-2010 LOGILAB S.A. (Paris, FRANCE), license is LGPL v2. -:contact: http://www.logilab.fr/ -- mailto:contact@logilab.fr -:license: Library General Public License version 2 - http://www.gnu.org/licenses """ __docformat__ = "restructuredtext en" @@ -112,7 +125,7 @@ CW_EVENT_MANAGER = CubicWebEventManager() -def onevent(event): +def onevent(event, *args, **kwargs): """decorator to ease event / callback binding >>> from cubicweb import onevent @@ -123,6 +136,6 @@ >>> """ def _decorator(func): - CW_EVENT_MANAGER.bind(event, func) + CW_EVENT_MANAGER.bind(event, func, *args, **kwargs) return func return _decorator diff -r 02b52bf9f5f8 -r 0865e1e90674 __pkginfo__.py --- a/__pkginfo__.py Wed Mar 24 10:23:31 2010 +0100 +++ b/__pkginfo__.py Wed Apr 28 11:54:13 2010 +0200 @@ -1,13 +1,29 @@ # pylint: disable-msg=W0622,C0103 +# copyright 2003-2010 LOGILAB S.A. (Paris, FRANCE), all rights reserved. +# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr +# +# This file is part of CubicWeb. +# +# CubicWeb is free software: you can redistribute it and/or modify it under the +# terms of the GNU Lesser General Public License as published by the Free +# Software Foundation, either version 2.1 of the License, or (at your option) +# any later version. +# +# logilab-common is distributed in the hope that it will be useful, but WITHOUT +# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS +# FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more +# details. +# +# You should have received a copy of the GNU Lesser General Public License along +# with CubicWeb. If not, see . """cubicweb global packaging information for the cubicweb knowledge management software -:license: GNU Lesser General Public License, v2.1 - http://www.gnu.org/licenses """ distname = "cubicweb" modname = "cubicweb" -numversion = (3, 6, 3) +numversion = (3, 7, 4) version = '.'.join(str(num) for num in numversion) license = 'LGPL' @@ -30,7 +46,7 @@ web = 'http://www.cubicweb.org' ftp = 'ftp://ftp.logilab.org/pub/cubicweb' -pyversions = ['2.4', '2.5'] +pyversions = ['2.5', '2.6'] classifiers = [ 'Environment :: Web Environment', @@ -89,6 +105,8 @@ [join(data_dir, fname) for fname in listdir(data_dir) if not isdir(join(data_dir, fname))]], [join('share', 'cubicweb', 'cubes', 'shared', 'data', 'timeline'), [join(data_dir, 'timeline', fname) for fname in listdir(join(data_dir, 'timeline'))]], + [join('share', 'cubicweb', 'cubes', 'shared', 'data', 'images'), + [join(data_dir, 'images', fname) for fname in listdir(join(data_dir, 'images'))]], [join('share', 'cubicweb', 'cubes', 'shared', 'wdoc'), [join(wdoc_dir, fname) for fname in listdir(wdoc_dir) if not isdir(join(wdoc_dir, fname))]], [join('share', 'cubicweb', 'cubes', 'shared', 'wdoc', 'images'), diff -r 02b52bf9f5f8 -r 0865e1e90674 _exceptions.py --- a/_exceptions.py Wed Mar 24 10:23:31 2010 +0100 +++ b/_exceptions.py Wed Apr 28 11:54:13 2010 +0200 @@ -1,10 +1,23 @@ +# copyright 2003-2010 LOGILAB S.A. (Paris, FRANCE), all rights reserved. +# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr +# +# This file is part of CubicWeb. +# +# CubicWeb is free software: you can redistribute it and/or modify it under the +# terms of the GNU Lesser General Public License as published by the Free +# Software Foundation, either version 2.1 of the License, or (at your option) +# any later version. +# +# logilab-common is distributed in the hope that it will be useful, but WITHOUT +# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS +# FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more +# details. +# +# You should have received a copy of the GNU Lesser General Public License along +# with CubicWeb. If not, see . """Exceptions shared by different cubicweb packages. -:organization: Logilab -:copyright: 2001-2010 LOGILAB S.A. (Paris, FRANCE), license is LGPL v2. -:contact: http://www.logilab.fr/ -- mailto:contact@logilab.fr -:license: GNU Lesser General Public License, v2.1 - http://www.gnu.org/licenses """ __docformat__ = "restructuredtext en" @@ -45,21 +58,19 @@ class ConnectionError(RepositoryError): """raised when a bad connection id is given or when an attempt to establish - a connection failed""" + a connection failed + """ class AuthenticationError(ConnectionError): - """raised when a bad connection id is given or when an attempt to establish - a connection failed + """raised when when an attempt to establish a connection failed do to wrong + connection information (login / password or other authentication token) """ def __init__(self, *args, **kwargs): super(AuthenticationError, self).__init__(*args) self.__dict__.update(kwargs) class BadConnectionId(ConnectionError): - """raised when a bad connection id is given or when an attempt to establish - a connection failed""" - -BadSessionId = BadConnectionId # XXX bw compat for pyro connections + """raised when a bad connection id is given""" class UnknownEid(RepositoryError): """the eid is not defined in the system tables""" @@ -127,8 +138,6 @@ class UnknownProperty(RegistryException): """property found in database but unknown in registry""" -class RegistryOutOfDate(RegistryException): - """raised when a source file modification is detected""" # query exception ############################################################# diff -r 02b52bf9f5f8 -r 0865e1e90674 _gcdebug.py --- a/_gcdebug.py Wed Mar 24 10:23:31 2010 +0100 +++ b/_gcdebug.py Wed Apr 28 11:54:13 2010 +0200 @@ -1,3 +1,20 @@ +# copyright 2003-2010 LOGILAB S.A. (Paris, FRANCE), all rights reserved. +# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr +# +# This file is part of CubicWeb. +# +# CubicWeb is free software: you can redistribute it and/or modify it under the +# terms of the GNU Lesser General Public License as published by the Free +# Software Foundation, either version 2.1 of the License, or (at your option) +# any later version. +# +# logilab-common is distributed in the hope that it will be useful, but WITHOUT +# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS +# FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more +# details. +# +# You should have received a copy of the GNU Lesser General Public License along +# with CubicWeb. If not, see . import gc, types, weakref diff -r 02b52bf9f5f8 -r 0865e1e90674 appobject.py --- a/appobject.py Wed Mar 24 10:23:31 2010 +0100 +++ b/appobject.py Wed Apr 28 11:54:13 2010 +0200 @@ -1,11 +1,33 @@ -"""Base class for dynamically loaded objects accessible through the vregistry. - -You'll also find some convenience classes to build selectors. +# copyright 2003-2010 LOGILAB S.A. (Paris, FRANCE), all rights reserved. +# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr +# +# This file is part of CubicWeb. +# +# CubicWeb is free software: you can redistribute it and/or modify it under the +# terms of the GNU Lesser General Public License as published by the Free +# Software Foundation, either version 2.1 of the License, or (at your option) +# any later version. +# +# logilab-common is distributed in the hope that it will be useful, but WITHOUT +# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS +# FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more +# details. +# +# You should have received a copy of the GNU Lesser General Public License along +# with CubicWeb. If not, see . +""" +.. _appobject: -:organization: Logilab -:copyright: 2001-2010 LOGILAB S.A. (Paris, FRANCE), license is LGPL v2. -:contact: http://www.logilab.fr/ -- mailto:contact@logilab.fr -:license: GNU Lesser General Public License, v2.1 - http://www.gnu.org/licenses +The `AppObject` class +--------------------- + +The AppObject class is the base class for all dynamically loaded objects +(application objects) accessible through the vregistry. + +We can find a certain number of attributes and methods defined in this class and +common to all the application objects. + +.. autoclass:: AppObject """ __docformat__ = "restructuredtext en" @@ -14,19 +36,24 @@ from warnings import warn from logilab.common.deprecation import deprecated +from logilab.common.decorators import classproperty from logilab.common.logging_ext import set_log_methods # selector base classes and operations ######################################## def objectify_selector(selector_func): - """convenience decorator for simple selectors where a class definition - would be overkill:: + """Most of the time, a simple score function is enough to build a selector. + The :func:`objectify_selector` decorator turn it into a proper selector + class:: @objectify_selector - def one(cls, *args, **kwargs): + def one(cls, req, rset=None, **kwargs): return 1 + class MyView(View): + __select__ = View.__select__ & one() + """ return type(selector_func.__name__, (Selector,), {'__doc__': selector_func.__doc__, @@ -48,7 +75,7 @@ class Selector(object): """base class for selector classes providing implementation - for operators ``&`` and ``|`` + for operators ``&``, ``|`` and ``~`` This class is only here to give access to binary operators, the selector logic itself should be implemented in the __call__ method @@ -204,47 +231,88 @@ selected according to a context (usually at least a request and a result set). - Concrete application objects classes are designed to be loaded by the - vregistry and should be accessed through it, not by direct instantiation. + The following attributes should be set on concret appobject classes: - The following attributes should be set on concret appobject classes: - :__registry__: + :attr:`__registry__` name of the registry for this object (string like 'views', 'templates'...) - :__regid__: + + :attr:`__regid__` object's identifier in the registry (string like 'main', 'primary', 'folder_box') - :__select__: + + :attr:`__select__` class'selector - Moreover, the `__abstract__` attribute may be set to True to indicate - that a appobject is abstract and should not be registered. + Moreover, the `__abstract__` attribute may be set to True to indicate that a + class is abstract and should not be registered. At selection time, the following attributes are set on the instance: - :_cw: + :attr:`_cw` current request - :cw_extra_kwargs: + :attr:`cw_extra_kwargs` other received arguments - only if rset is found in arguments (in which case rset/row/col will be - removed from cwextra_kwargs): + And also the following, only if `rset` is found in arguments (in which case + rset/row/col will be removed from `cwextra_kwargs`): - :cw_rset: + :attr:`cw_rset` context result set or None - :cw_row: + + :attr:`cw_row` if a result set is set and the context is about a particular cell in the result set, and not the result set as a whole, specify the row number we are interested in, else None - :cw_col: + + :attr:`cw_col` if a result set is set and the context is about a particular cell in the result set, and not the result set as a whole, specify the col number we are interested in, else None + + + .. Note:: + + * do not inherit directly from this class but from a more specific class + such as `AnyEntity`, `EntityView`, `AnyRsetView`, `Action`... + + * to be recordable, a subclass has to define its registry (attribute + `__registry__`) and its identifier (attribute `__regid__`). Usually + you don't have to take care of the registry since it's set by the base + class, only the identifier `id` + + * application objects are designed to be loaded by the vregistry and + should be accessed through it, not by direct instantiation, besides + to use it as base classe. + + + * When we inherit from `AppObject` (even not directly), you *always* have + to use **super()** to get the methods and attributes of the superclasses, + and not use the class identifier. + + For example, instead of writting:: + + class Truc(PrimaryView): + def f(self, arg1): + PrimaryView.f(self, arg1) + + You must write:: + + class Truc(PrimaryView): + def f(self, arg1): + super(Truc, self).f(arg1) + """ __registry__ = None __regid__ = None __select__ = yes() + @classproperty + def __registries__(cls): + if cls.__registry__ is None: + return () + return (cls.__registry__,) + @classmethod def __registered__(cls, registry): """called by the registry when the appobject has been registered. @@ -258,7 +326,7 @@ except AttributeError: pdefs = getattr(cls, 'cw_property_defs', {}) else: - warn('property_defs is deprecated, use cw_property_defs in %s' + warn('[3.6] property_defs is deprecated, use cw_property_defs in %s' % cls, DeprecationWarning) for propid, pdef in pdefs.items(): pdef = pdef.copy() # may be shared diff -r 02b52bf9f5f8 -r 0865e1e90674 common/__init__.py --- a/common/__init__.py Wed Mar 24 10:23:31 2010 +0100 +++ b/common/__init__.py Wed Apr 28 11:54:13 2010 +0200 @@ -1,9 +1,22 @@ +# copyright 2003-2010 LOGILAB S.A. (Paris, FRANCE), all rights reserved. +# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr +# +# This file is part of CubicWeb. +# +# CubicWeb is free software: you can redistribute it and/or modify it under the +# terms of the GNU Lesser General Public License as published by the Free +# Software Foundation, either version 2.1 of the License, or (at your option) +# any later version. +# +# logilab-common is distributed in the hope that it will be useful, but WITHOUT +# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS +# FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more +# details. +# +# You should have received a copy of the GNU Lesser General Public License along +# with CubicWeb. If not, see . """Common subpackage of cubicweb : defines library functions used both on the hg stserver side and on the client side -:organization: Logilab -:copyright: 2001-2010 LOGILAB S.A. (Paris, FRANCE), license is LGPL v2. -:contact: http://www.logilab.fr/ -- mailto:contact@logilab.fr -:license: GNU Lesser General Public License, v2.1 - http://www.gnu.org/licenses """ diff -r 02b52bf9f5f8 -r 0865e1e90674 common/mail.py --- a/common/mail.py Wed Mar 24 10:23:31 2010 +0100 +++ b/common/mail.py Wed Apr 28 11:54:13 2010 +0200 @@ -1,3 +1,20 @@ +# copyright 2003-2010 LOGILAB S.A. (Paris, FRANCE), all rights reserved. +# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr +# +# This file is part of CubicWeb. +# +# CubicWeb is free software: you can redistribute it and/or modify it under the +# terms of the GNU Lesser General Public License as published by the Free +# Software Foundation, either version 2.1 of the License, or (at your option) +# any later version. +# +# logilab-common is distributed in the hope that it will be useful, but WITHOUT +# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS +# FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more +# details. +# +# You should have received a copy of the GNU Lesser General Public License along +# with CubicWeb. If not, see . """pre 3.6 bw compat""" # pylint: disable-msg=W0614,W0401 from warnings import warn diff -r 02b52bf9f5f8 -r 0865e1e90674 common/mixins.py --- a/common/mixins.py Wed Mar 24 10:23:31 2010 +0100 +++ b/common/mixins.py Wed Apr 28 11:54:13 2010 +0200 @@ -1,3 +1,20 @@ +# copyright 2003-2010 LOGILAB S.A. (Paris, FRANCE), all rights reserved. +# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr +# +# This file is part of CubicWeb. +# +# CubicWeb is free software: you can redistribute it and/or modify it under the +# terms of the GNU Lesser General Public License as published by the Free +# Software Foundation, either version 2.1 of the License, or (at your option) +# any later version. +# +# logilab-common is distributed in the hope that it will be useful, but WITHOUT +# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS +# FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more +# details. +# +# You should have received a copy of the GNU Lesser General Public License along +# with CubicWeb. If not, see . """pre 3.6 bw compat""" # pylint: disable-msg=W0614,W0401 from warnings import warn diff -r 02b52bf9f5f8 -r 0865e1e90674 common/mttransforms.py --- a/common/mttransforms.py Wed Mar 24 10:23:31 2010 +0100 +++ b/common/mttransforms.py Wed Apr 28 11:54:13 2010 +0200 @@ -1,3 +1,20 @@ +# copyright 2003-2010 LOGILAB S.A. (Paris, FRANCE), all rights reserved. +# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr +# +# This file is part of CubicWeb. +# +# CubicWeb is free software: you can redistribute it and/or modify it under the +# terms of the GNU Lesser General Public License as published by the Free +# Software Foundation, either version 2.1 of the License, or (at your option) +# any later version. +# +# logilab-common is distributed in the hope that it will be useful, but WITHOUT +# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS +# FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more +# details. +# +# You should have received a copy of the GNU Lesser General Public License along +# with CubicWeb. If not, see . """pre 3.6 bw compat""" # pylint: disable-msg=W0614,W0401 from warnings import warn diff -r 02b52bf9f5f8 -r 0865e1e90674 common/tags.py --- a/common/tags.py Wed Mar 24 10:23:31 2010 +0100 +++ b/common/tags.py Wed Apr 28 11:54:13 2010 +0200 @@ -1,3 +1,20 @@ +# copyright 2003-2010 LOGILAB S.A. (Paris, FRANCE), all rights reserved. +# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr +# +# This file is part of CubicWeb. +# +# CubicWeb is free software: you can redistribute it and/or modify it under the +# terms of the GNU Lesser General Public License as published by the Free +# Software Foundation, either version 2.1 of the License, or (at your option) +# any later version. +# +# logilab-common is distributed in the hope that it will be useful, but WITHOUT +# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS +# FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more +# details. +# +# You should have received a copy of the GNU Lesser General Public License along +# with CubicWeb. If not, see . """pre 3.6 bw compat""" # pylint: disable-msg=W0614,W0401 from warnings import warn diff -r 02b52bf9f5f8 -r 0865e1e90674 common/uilib.py --- a/common/uilib.py Wed Mar 24 10:23:31 2010 +0100 +++ b/common/uilib.py Wed Apr 28 11:54:13 2010 +0200 @@ -1,3 +1,20 @@ +# copyright 2003-2010 LOGILAB S.A. (Paris, FRANCE), all rights reserved. +# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr +# +# This file is part of CubicWeb. +# +# CubicWeb is free software: you can redistribute it and/or modify it under the +# terms of the GNU Lesser General Public License as published by the Free +# Software Foundation, either version 2.1 of the License, or (at your option) +# any later version. +# +# logilab-common is distributed in the hope that it will be useful, but WITHOUT +# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS +# FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more +# details. +# +# You should have received a copy of the GNU Lesser General Public License along +# with CubicWeb. If not, see . """pre 3.6 bw compat""" # pylint: disable-msg=W0614,W0401 from warnings import warn diff -r 02b52bf9f5f8 -r 0865e1e90674 crypto.py --- a/crypto.py Wed Mar 24 10:23:31 2010 +0100 +++ b/crypto.py Wed Apr 28 11:54:13 2010 +0200 @@ -1,9 +1,22 @@ +# copyright 2003-2010 LOGILAB S.A. (Paris, FRANCE), all rights reserved. +# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr +# +# This file is part of CubicWeb. +# +# CubicWeb is free software: you can redistribute it and/or modify it under the +# terms of the GNU Lesser General Public License as published by the Free +# Software Foundation, either version 2.1 of the License, or (at your option) +# any later version. +# +# logilab-common is distributed in the hope that it will be useful, but WITHOUT +# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS +# FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more +# details. +# +# You should have received a copy of the GNU Lesser General Public License along +# with CubicWeb. If not, see . """Simple cryptographic routines, based on python-crypto. -:organization: Logilab -:copyright: 2009-2010 LOGILAB S.A. (Paris, FRANCE), license is LGPL v2. -:contact: http://www.logilab.fr/ -- mailto:contact@logilab.fr -:license: GNU Lesser General Public License, v2.1 - http://www.gnu.org/licenses """ __docformat__ = "restructuredtext en" diff -r 02b52bf9f5f8 -r 0865e1e90674 cwconfig.py --- a/cwconfig.py Wed Mar 24 10:23:31 2010 +0100 +++ b/cwconfig.py Wed Apr 28 11:54:13 2010 +0200 @@ -1,46 +1,53 @@ # -*- coding: utf-8 -*- -"""common configuration utilities for cubicweb +# copyright 2003-2010 LOGILAB S.A. (Paris, FRANCE), all rights reserved. +# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr +# +# This file is part of CubicWeb. +# +# CubicWeb is free software: you can redistribute it and/or modify it under the +# terms of the GNU Lesser General Public License as published by the Free +# Software Foundation, either version 2.1 of the License, or (at your option) +# any later version. +# +# logilab-common is distributed in the hope that it will be useful, but WITHOUT +# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS +# FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more +# details. +# +# You should have received a copy of the GNU Lesser General Public License along +# with CubicWeb. If not, see . +""" +.. _ResourceMode: -:organization: Logilab -:copyright: 2001-2010 LOGILAB S.A. (Paris, FRANCE), license is LGPL v2. -:contact: http://www.logilab.fr/ -- mailto:contact@logilab.fr +Resource mode +------------- + +A resource *mode* is a predifined set of settings for various resources +directories, such as cubes, instances, etc. to ease development with the +framework. There are two running modes with *CubicWeb*: + +* 'user', resources are searched / created in the user home directory: + + - instances are stored in :file:`~/etc/cubicweb.d` + - temporary files (such as pid file) in :file:`/tmp` + +* 'system', resources are searched / created in the system directories (eg + usually requiring root access): + + - instances are stored in :file:`/etc/cubicweb.d` + - temporary files (such as pid file) in :file:`/var/run/cubicweb` + + where `` is the detected installation prefix ('/usr/local' for + instance). -If cubicweb is a mercurial checkout (eg `CWDEV` is true), located in -`CW_SOFTWARE_ROOT`: - - * main cubes directory is `/../cubes`. You can specify - another one with `CW_INSTANCES_DIR` environment variable or simply add some - other directories by using `CW_CUBES_PATH`. - - * cubicweb migration files are by default searched in - `/misc/migration` instead of - `/usr/share/cubicweb/migration/`(unless another emplacement is specified - using `CW_MIGRATION_DIR`. - - * Cubicweb will start in 'user' mode (see below) - - -On startup, Cubicweb is using a specific *mode*. A mode corresponds to some -default setting for various resource directories. There are currently 2 main -modes : 'system', for system wide installation, and 'user', fur user local -installation (e.g. no root privileges). - -'user' mode is activated automatically when cubicweb is a mercurial checkout -(e.g. has a .hg directory). You can also force mode by using the `CW_MODE` -environment variable, to: - -* use system wide installation but user specific instances and all, without root - privileges on the system (`export CW_MODE=user`) - -* use local checkout of cubicweb on system wide instances (requires root - privileges on the system (`export CW_MODE=system`) - - Here is the default resource directories settings according to mode: +Notice that each resource path may be explicitly set using an environment +variable if the default doesn't suit your needs. Here are the default resource +directories that are affected according to mode: * 'system': :: - CW_INSTANCES_DIR = /etc/cubicweb.d/ + CW_INSTANCES_DIR = /etc/cubicweb.d/ CW_INSTANCES_DATA_DIR = /var/lib/cubicweb/instances/ CW_RUNTIME_DIR = /var/run/cubicweb/ @@ -50,27 +57,79 @@ CW_INSTANCES_DATA_DIR = ~/etc/cubicweb.d/ CW_RUNTIME_DIR = /tmp +Cubes search path is also affected, see the :ref:Cube section. + +By default, the mode automatically set to 'user' if a :file:`.hg` directory is found +in the cubicweb package, else it's set to 'system'. You can force this by setting +the :envvar:`CW_MODE` environment variable to either 'user' or 'system' so you can +easily: + +* use system wide installation but user specific instances and all, without root + privileges on the system (`export CW_MODE=user`) + +* use local checkout of cubicweb on system wide instances (requires root + privileges on the system (`export CW_MODE=system`) + +If you've a doubt about the mode you're currently running, check the first line +outputed by the :command:`cubicweb-ctl list` command. + +Also, if cubicweb is a mercurial checkout located in ``: + +* main cubes directory is `/../cubes`. You can specify + another one with :envvar:`CW_INSTANCES_DIR` environment variable or simply + add some other directories by using :envvar:`CW_CUBES_PATH` + +* cubicweb migration files are searched in `/misc/migration` + instead of `/share/cubicweb/migration/`. + + +.. _ConfigurationEnv: + +Environment configuration +------------------------- + +Python +`````` + +If you installed *CubicWeb* by cloning the Mercurial forest or from source +distribution, then you will need to update the environment variable PYTHONPATH by +adding the path to the forest `cubicweb`: + +Add the following lines to either :file:`.bashrc` or :file:`.bash_profile` to +configure your development environment :: + + export PYTHONPATH=/full/path/to/cubicweb-forest + +If you installed *CubicWeb* with packages, no configuration is required and your +new cubes will be placed in `/usr/share/cubicweb/cubes` and your instances will +be placed in `/etc/cubicweb.d`. + + +CubicWeb +```````` + +Here are all environment variables that may be used to configure *CubicWeb*: .. envvar:: CW_MODE - Resource mode: user or system + + Resource mode: user or system, as explained in :ref:`ResourceMode`. .. envvar:: CW_CUBES_PATH - Augments the default search path for cubes + + Augments the default search path for cubes. You may specify several + directories using ':' as separator (';' under windows environment). .. envvar:: CW_INSTANCES_DIR - Directory where cubicweb instances will be found + + Directory where cubicweb instances will be found. .. envvar:: CW_INSTANCES_DATA_DIR - Directory where cubicweb instances data will be written + + Directory where cubicweb instances data will be written (backup file...) .. envvar:: CW_RUNTIME_DIR + Directory where pid files will be written - -.. envvar:: CW_MIGRATION_DIR - Directory where cubicweb migration files will be found - - -:license: GNU Lesser General Public License, v2.1 - http://www.gnu.org/licenses """ __docformat__ = "restructuredtext en" _ = unicode @@ -90,7 +149,8 @@ from logilab.common.configuration import (Configuration, Method, ConfigurationMixIn, merge_options) -from cubicweb import CW_SOFTWARE_ROOT, CW_MIGRATION_MAP, ConfigurationError +from cubicweb import (CW_SOFTWARE_ROOT, CW_MIGRATION_MAP, + ConfigurationError, Binary) from cubicweb.toolsutils import env_path, create_dir CONFIGURATIONS = [] @@ -179,7 +239,7 @@ }), ('short-line-size', {'type' : 'int', - 'default': 40, + 'default': 80, 'help': _('maximum number of characters in short description'), 'group': 'navigation', }), @@ -329,7 +389,7 @@ def available_cubes(cls): cubes = set() for directory in cls.cubes_search_path(): - if not os.path.exists(directory): + if not exists(directory): cls.error('unexistant directory in cubes search path: %s' % directory) continue @@ -668,8 +728,6 @@ # for some commands (creation...) we don't want to initialize gettext set_language = True - # set this to true to avoid false error message while creating an instance - creating = False # set this to true to allow somethings which would'nt be possible repairing = False @@ -972,7 +1030,9 @@ return i18n.compile_i18n_catalogs(sourcedirs, i18ndir, langs) def sendmails(self, msgs): - """msgs: list of 2-uple (message object, recipients)""" + """msgs: list of 2-uple (message object, recipients). Return False + if connection to the smtp server failed, else True. + """ server, port = self['smtp-host'], self['smtp-port'] SMTP_LOCK.acquire() try: @@ -981,7 +1041,7 @@ except Exception, ex: self.exception("can't connect to smtp server %s:%s (%s)", server, port, ex) - return + return False heloaddr = '%s <%s>' % (self['sender-name'], self['sender-addr']) for msg, recipients in msgs: try: @@ -992,6 +1052,7 @@ smtp.close() finally: SMTP_LOCK.release() + return True set_log_methods(CubicWebConfiguration, logging.getLogger('cubicweb.configuration')) @@ -1002,7 +1063,7 @@ _EXT_REGISTERED = False def register_stored_procedures(): - from logilab.common.adbh import FunctionDescr + from logilab.database import FunctionDescr from rql.utils import register_function, iter_funcnode_variables global _EXT_REGISTERED @@ -1014,8 +1075,7 @@ supported_backends = ('postgres', 'sqlite',) rtype = 'String' - @classmethod - def st_description(cls, funcnode, mainindex, tr): + def st_description(self, funcnode, mainindex, tr): return ', '.join(sorted(term.get_description(mainindex, tr) for term in iter_funcnode_variables(funcnode))) @@ -1027,6 +1087,7 @@ register_function(CONCAT_STRINGS) # XXX bw compat + class GROUP_CONCAT(CONCAT_STRINGS): supported_backends = ('mysql', 'postgres', 'sqlite',) @@ -1037,8 +1098,7 @@ supported_backends = ('postgres', 'sqlite',) rtype = 'String' - @classmethod - def st_description(cls, funcnode, mainindex, tr): + def st_description(self, funcnode, mainindex, tr): return funcnode.children[0].get_description(mainindex, tr) register_function(LIMIT_SIZE) @@ -1050,9 +1110,25 @@ register_function(TEXT_LIMIT_SIZE) + class FSPATH(FunctionDescr): + """return path of some bytes attribute stored using the Bytes + File-System Storage (bfss) + """ + rtype = 'Bytes' # XXX return a String? potential pb with fs encoding - class FSPATH(FunctionDescr): - supported_backends = ('postgres', 'sqlite',) - rtype = 'Bytes' + def update_cb_stack(self, stack): + assert len(stack) == 1 + stack[0] = self.source_execute + + def as_sql(self, backend, args): + raise NotImplementedError('source only callback') + + def source_execute(self, source, value): + fpath = source.binary_to_str(value) + try: + return Binary(fpath) + except OSError, ex: + self.critical("can't open %s: %s", fpath, ex) + return None register_function(FSPATH) diff -r 02b52bf9f5f8 -r 0865e1e90674 cwctl.py --- a/cwctl.py Wed Mar 24 10:23:31 2010 +0100 +++ b/cwctl.py Wed Apr 28 11:54:13 2010 +0200 @@ -1,11 +1,24 @@ +# copyright 2003-2010 LOGILAB S.A. (Paris, FRANCE), all rights reserved. +# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr +# +# This file is part of CubicWeb. +# +# CubicWeb is free software: you can redistribute it and/or modify it under the +# terms of the GNU Lesser General Public License as published by the Free +# Software Foundation, either version 2.1 of the License, or (at your option) +# any later version. +# +# logilab-common is distributed in the hope that it will be useful, but WITHOUT +# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS +# FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more +# details. +# +# You should have received a copy of the GNU Lesser General Public License along +# with CubicWeb. If not, see . """the cubicweb-ctl tool, based on logilab.common.clcommands to provide a pluggable commands system. -:organization: Logilab -:copyright: 2001-2010 LOGILAB S.A. (Paris, FRANCE), license is LGPL v2. -:contact: http://www.logilab.fr/ -- mailto:contact@logilab.fr -:license: GNU Lesser General Public License, v2.1 - http://www.gnu.org/licenses """ __docformat__ = "restructuredtext en" @@ -232,9 +245,11 @@ if not isinstance(use, dict): use = dict((key, None) for key in use) self.warnings.append('cube %s should define __depends_cubes__ as a dict not a list') - else: + elif hasattr(info, '__use__'): self.warnings.append('cube %s should define __depends_cubes__' % cube) use = dict((key, None) for key in info.__use__) + else: + continue for name, constraint in use.items(): self.constraints.setdefault(name,set()) if constraint: @@ -383,7 +398,6 @@ cubes = splitstrip(pop_arg(args, 1)) appid = pop_arg(args) # get the configuration and helper - cwcfg.creating = True config = cwcfg.config_for(appid, configname) config.set_language = False cubes = config.expand_cubes(cubes) @@ -797,7 +811,7 @@ # handle i18n upgrade: # * install new languages # * recompile catalogs - # in the first componant given + # XXX search available language in the first cube given from cubicweb import i18n templdir = cwcfg.cube_dir(config.cubes()[0]) langs = [lang for lang, _ in i18n.available_catalogs(join(templdir, 'i18n'))] @@ -812,7 +826,12 @@ print print '-> instance migrated.' if not (CWDEV or self.config.nostartstop): - StartInstanceCommand().start_instance(appid) + # restart instance through fork to get a proper environment, avoid + # uicfg pb (and probably gettext catalogs, to check...) + forkcmd = '%s start %s' % (sys.argv[0], appid) + status = system(forkcmd) + if status: + print '%s exited with status %s' % (forkcmd, status) print @@ -932,20 +951,11 @@ def i18ninstance_instance(appid): """recompile instance's messages catalogs""" config = cwcfg.config_for(appid) - try: - config.bootstrap_cubes() - except IOError, ex: - import errno - if ex.errno != errno.ENOENT: - raise - # bootstrap_cubes files doesn't exist - # notify this is not a regular start - config.repairing = True - # create an in-memory repository, will call config.init_cubes() - config.repository() - except AttributeError: + config.quick_start = True # notify this is not a regular start + repo = config.repository() + if config._cubes is None: # web only config - config.init_cubes(config.repository().get_cubes()) + config.init_cubes(repo.get_cubes()) errors = config.i18ncompile() if errors: print '\n'.join(errors) diff -r 02b52bf9f5f8 -r 0865e1e90674 cwvreg.py --- a/cwvreg.py Wed Mar 24 10:23:31 2010 +0100 +++ b/cwvreg.py Wed Apr 28 11:54:13 2010 +0200 @@ -1,9 +1,193 @@ -"""extend the generic VRegistry with some cubicweb specific stuff +# copyright 2003-2010 LOGILAB S.A. (Paris, FRANCE), all rights reserved. +# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr +# +# This file is part of CubicWeb. +# +# CubicWeb is free software: you can redistribute it and/or modify it under the +# terms of the GNU Lesser General Public License as published by the Free +# Software Foundation, either version 2.1 of the License, or (at your option) +# any later version. +# +# logilab-common is distributed in the hope that it will be useful, but WITHOUT +# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS +# FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more +# details. +# +# You should have received a copy of the GNU Lesser General Public License along +# with CubicWeb. If not, see . +""".. VRegistry: + +The `VRegistry` +--------------- + +The `VRegistry` can be seen as a two levels dictionary. It contains +all dynamically loaded objects (subclasses of :ref:`appobject`) to +build a |cubicweb| application. Basically: + +* the first level key returns a *registry*. This key corresponds to the + `__registry__` attribute of application object classes + +* the second level key returns a list of application objects which + share the same identifier. This key corresponds to the `__regid__` + attribute of application object classes. + +A *registry* holds a specific kind of application objects. There is +for instance a registry for entity classes, another for views, etc... + +The `VRegistry` has two main responsibilities: + +- being the access point to all registries + +- handling the registration process at startup time, and during automatic + reloading in debug mode. + +.. _AppObjectRecording: + +Details of the recording process +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. index:: + vregistry: registration_callback + +On startup |cubicweb| loads application objects defined in its library +and in cubes used by the instance. Application objects from the +library are loaded first, then those provided by cubes are loaded in +dependency order (e.g. if your cube depends on an other, objects from +the dependency will be loaded first). Cube's modules or packages where +appobject are looked for is explained in :ref:`cubelayout`. + +For each module: + +* by default all objects are registered automatically + +* if some objects have to replace other objects, or have to be + included only if some condition is met, you'll have to define a + `registration_callback(vreg)` function in your module and explicitly + register **all objects** in this module, using the api defined + below. + +.. Note:: + Once the function `registration_callback(vreg)` is implemented in a module, + all the objects from this module have to be explicitly registered as it + disables the automatic objects registration. + + +API for objects registration +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Here are the registration methods that you can use in the `registration_callback` +to register your objects to the `VRegistry` instance given as argument (usually +named `vreg`): + +.. automethod:: cubicweb.cwvreg.CubicWebVRegistry.register_all +.. automethod:: cubicweb.cwvreg.CubicWebVRegistry.register_and_replace +.. automethod:: cubicweb.cwvreg.CubicWebVRegistry.register +.. automethod:: cubicweb.cwvreg.CubicWebVRegistry.register_if_interface_found +.. automethod:: cubicweb.cwvreg.CubicWebVRegistry.unregister + +Examples: + +.. sourcecode:: python -:organization: Logilab -:copyright: 2001-2010 LOGILAB S.A. (Paris, FRANCE), license is LGPL v2. -:contact: http://www.logilab.fr/ -- mailto:contact@logilab.fr -:license: GNU Lesser General Public License, v2.1 - http://www.gnu.org/licenses + # web/views/basecomponents.py + def registration_callback(vreg): + # register everything in the module except SeeAlsoComponent + vreg.register_all(globals().values(), __name__, (SeeAlsoVComponent,)) + # conditionally register SeeAlsoVComponent + if 'see_also' in vreg.schema: + vreg.register(SeeAlsoVComponent) + +In this example, we register all application object classes defined in the module +except `SeeAlsoVComponent`. This class is then registered only if the 'see_also' +relation type is defined in the instance'schema. + +.. sourcecode:: python + + # goa/appobjects/sessions.py + def registration_callback(vreg): + vreg.register(SessionsCleaner) + # replace AuthenticationManager by GAEAuthenticationManager + vreg.register_and_replace(GAEAuthenticationManager, AuthenticationManager) + # replace PersistentSessionManager by GAEPersistentSessionManager + vreg.register_and_replace(GAEPersistentSessionManager, PersistentSessionManager) + +In this example, we explicitly register classes one by one: + +* the `SessionCleaner` class +* the `GAEAuthenticationManager` to replace the `AuthenticationManager` +* the `GAEPersistentSessionManager` to replace the `PersistentSessionManager` + +If at some point we register a new appobject class in this module, it won't be +registered at all without modification to the `registration_callback` +implementation. The previous example will register it though, thanks to the call +to the `register_all` method. + + +.. _Selection: + +Runtime objects selection +~~~~~~~~~~~~~~~~~~~~~~~~~ + +Now that we have all application objects loaded, the question is : when +I want some specific object, for instance the primary view for a given +entity, how do I get the proper object ? This is what we call the +**selection mechanism**. + +As explained in the :ref:`Concepts` section: + +* each application object has a **selector**, defined by its + `__select__` class attribute + +* this selector is responsible to return a **score** for a given context + + - 0 score means the object doesn't apply to this context + + - else, the higher the score, the better the object suits the context + +* the object with the higher score is selected. + +.. Note:: + + When no score is higher than the others, an exception is raised in development + mode to let you know that the engine was not able to identify the view to + apply. This error is silenced in production mode and one of the objects with + the higher score is picked. + + In such cases you would need to review your design and make sure your selectors + or appobjects are properly defined. + +For instance, if you are selecting the primary (eg `__regid__ = +'primary'`) view (eg `__registry__ = 'views'`) for a result set +containing a `Card` entity, two objects will probably be selectable: + +* the default primary view (`__select__ = implements('Any')`), meaning + that the object is selectable for any kind of entity type + +* the specific `Card` primary view (`__select__ = implements('Card')`, + meaning that the object is selectable for Card entities + +Other primary views specific to other entity types won't be selectable in this +case. Among selectable objects, the implements selector will return a higher +score than the second view since it's more specific, so it will be selected as +expected. + +.. _SelectionAPI: + +API for objects selections +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Here is the selection API you'll get on every registry. Some of them, as the +'etypes' registry, containing entity classes, extend it. In those methods, +`*args, **kwargs` is what we call the **context**. Those arguments are given to +selectors that will inspect there content and return a score accordingly. + +.. automethod:: cubicweb.vregistry.Registry.select + +.. automethod:: cubicweb.vregistry.Registry.select_or_none + +.. automethod:: cubicweb.vregistry.Registry.possible_objects + +.. automethod:: cubicweb.vregistry.Registry.object_by_id """ __docformat__ = "restructuredtext en" _ = unicode @@ -16,18 +200,15 @@ from cubicweb import (ETYPE_NAME_MAP, Binary, UnknownProperty, UnknownEid, ObjectNotFound, NoSelectableObject, RegistryNotFound, - RegistryOutOfDate, CW_EVENT_MANAGER, onevent) + CW_EVENT_MANAGER, onevent) from cubicweb.utils import dump_class from cubicweb.vregistry import VRegistry, Registry, class_regid from cubicweb.rtags import RTAGS - -@onevent('before-registry-reload') def clear_rtag_objects(): for rtag in RTAGS: rtag.clear() - def use_interfaces(obj): """return interfaces used by the given object by searching for implements selectors, with a bw compat fallback to accepts_interfaces attribute @@ -265,6 +446,11 @@ self.schema = None self.initialized = False self.reset() + # XXX give force_reload (or refactor [re]loading...) + if self.config.mode != 'test': + # don't clear rtags during test, this may cause breakage with + # manually imported appobject modules + CW_EVENT_MANAGER.bind('before-registry-reload', clear_rtag_objects) def setdefault(self, regid): try: @@ -285,8 +471,8 @@ def itervalues(self): return (value for key, value in self.items()) - def reset(self, path=None, force_reload=None): - super(CubicWebVRegistry, self).reset(path, force_reload) + def reset(self): + super(CubicWebVRegistry, self).reset() self._needs_iface = {} # two special registries, propertydefs which care all the property # definitions, and propertyvals which contains values for those @@ -296,7 +482,29 @@ self['propertyvalues'] = self.eprop_values = {} for key, propdef in self.config.eproperty_definitions(): self.register_property(key, **propdef) - if path is not None and force_reload: + + def set_schema(self, schema): + """set instance'schema and load application objects""" + self._set_schema(schema) + # now we can load application's web objects + self.reload(self.config.vregistry_path(), force_reload=False) + # map lowered entity type names to their actual name + self.case_insensitive_etypes = {} + for eschema in self.schema.entities(): + etype = str(eschema) + self.case_insensitive_etypes[etype.lower()] = etype + clear_cache(eschema, 'ordered_relations') + clear_cache(eschema, 'meta_attributes') + + def reload_if_needed(self): + path = self.config.vregistry_path() + if self.is_reload_needed(path): + self.reload(path) + + def reload(self, path, force_reload=True): + """modification detected, reset and reload the vreg""" + CW_EVENT_MANAGER.emit('before-registry-reload') + if force_reload: cleanup_sys_modules(path) cubes = self.config.cubes() # if the fs code use some cubes not yet registered into the instance @@ -307,21 +515,9 @@ if not cube in cubes: cpath = cfg.build_vregistry_cube_path([cfg.cube_dir(cube)]) cleanup_sys_modules(cpath) - - def set_schema(self, schema): - """set instance'schema and load application objects""" - self._set_schema(schema) - # now we can load application's web objects - searchpath = self.config.vregistry_path() - self.reset(searchpath, force_reload=False) - self.register_objects(searchpath, force_reload=False) - # map lowered entity type names to their actual name - self.case_insensitive_etypes = {} - for eschema in self.schema.entities(): - etype = str(eschema) - self.case_insensitive_etypes[etype.lower()] = etype - clear_cache(eschema, 'ordered_relations') - clear_cache(eschema, 'meta_attributes') + self.reset() + self.register_objects(path, force_reload) + CW_EVENT_MANAGER.emit('after-registry-reload') def _set_schema(self, schema): """set instance'schema""" @@ -339,8 +535,11 @@ obj.schema = schema def register_if_interface_found(self, obj, ifaces, **kwargs): - """register an object but remove it if no entity class implements one of - the given interfaces at the end of the registration process + """register `obj` but remove it if no entity class implements one of + the given `ifaces` interfaces at the end of the registration process. + + Extra keyword arguments are given to the + :meth:`~cubicweb.cwvreg.CubicWebVRegistry.register` function. """ self.register(obj, **kwargs) if not isinstance(ifaces, (tuple, list)): @@ -349,26 +548,23 @@ self._needs_iface[obj] = ifaces def register(self, obj, *args, **kwargs): + """register `obj` application object into `registryname` or + `obj.__registry__` if not specified, with identifier `oid` or + `obj.__regid__` if not specified. + + If `clear` is true, all objects with the same identifier will be + previously unregistered. + """ super(CubicWebVRegistry, self).register(obj, *args, **kwargs) # XXX bw compat ifaces = use_interfaces(obj) if ifaces: self._needs_iface[obj] = ifaces - def register_objects(self, path, force_reload=None): + def register_objects(self, path, force_reload=False): """overriden to remove objects requiring a missing interface""" - if force_reload is None: - force_reload = self.config.debugmode - try: - super(CubicWebVRegistry, self).register_objects( - path, force_reload, self.config.extrapath) - except RegistryOutOfDate: - CW_EVENT_MANAGER.emit('before-registry-reload') - # modification detected, reset and reload - self.reset(path, force_reload) - super(CubicWebVRegistry, self).register_objects( - path, force_reload, self.config.extrapath) - CW_EVENT_MANAGER.emit('after-registry-reload') + super(CubicWebVRegistry, self).register_objects( + path, force_reload, self.config.extrapath) def initialization_completed(self): """cw specific code once vreg initialization is completed: diff -r 02b52bf9f5f8 -r 0865e1e90674 dataimport.py --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/dataimport.py Wed Apr 28 11:54:13 2010 +0200 @@ -0,0 +1,708 @@ +# -*- coding: utf-8 -*- +# copyright 2003-2010 LOGILAB S.A. (Paris, FRANCE), all rights reserved. +# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr +# +# This file is part of CubicWeb. +# +# CubicWeb is free software: you can redistribute it and/or modify it under the +# terms of the GNU Lesser General Public License as published by the Free +# Software Foundation, either version 2.1 of the License, or (at your option) +# any later version. +# +# logilab-common is distributed in the hope that it will be useful, but WITHOUT +# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS +# FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more +# details. +# +# You should have received a copy of the GNU Lesser General Public License along +# with CubicWeb. If not, see . +"""This module provides tools to import tabular data. + + + +Example of use (run this with `cubicweb-ctl shell instance import-script.py`): + +.. sourcecode:: python + + from cubicweb.devtools.dataimport import * + # define data generators + GENERATORS = [] + + USERS = [('Prenom', 'firstname', ()), + ('Nom', 'surname', ()), + ('Identifiant', 'login', ()), + ] + + def gen_users(ctl): + for row in ctl.get_data('utilisateurs'): + entity = mk_entity(row, USERS) + entity['upassword'] = u'motdepasse' + ctl.check('login', entity['login'], None) + ctl.store.add('CWUser', entity) + email = {'address': row['email']} + ctl.store.add('EmailAddress', email) + ctl.store.relate(entity['eid'], 'use_email', email['eid']) + ctl.store.rql('SET U in_group G WHERE G name "users", U eid %(x)s', {'x':entity['eid']}) + + CHK = [('login', check_doubles, 'Utilisateurs Login', + 'Deux utilisateurs ne devraient pas avoir le même login.'), + ] + + GENERATORS.append( (gen_users, CHK) ) + + # create controller + ctl = CWImportController(RQLObjectStore(cnx)) + ctl.askerror = 1 + ctl.generators = GENERATORS + ctl.data['utilisateurs'] = lazytable(utf8csvreader(open('users.csv'))) + # run + ctl.run() + +.. BUG file with one column are not parsable +.. TODO rollback() invocation is not possible yet +""" +__docformat__ = "restructuredtext en" + +import sys +import csv +import traceback +import os.path as osp +from StringIO import StringIO +from copy import copy + +from logilab.common import shellutils +from logilab.common.date import strptime +from logilab.common.decorators import cached +from logilab.common.deprecation import deprecated + +from cubicweb.server.utils import eschema_eid + +def ucsvreader_pb(filepath, encoding='utf-8', separator=',', quote='"', + skipfirst=False, withpb=True): + """same as ucsvreader but a progress bar is displayed as we iter on rows""" + if not osp.exists(filepath): + raise Exception("file doesn't exists: %s" % filepath) + rowcount = int(shellutils.Execute('wc -l "%s"' % filepath).out.strip().split()[0]) + if skipfirst: + rowcount -= 1 + if withpb: + pb = shellutils.ProgressBar(rowcount, 50) + for urow in ucsvreader(file(filepath), encoding, separator, quote, skipfirst): + yield urow + if withpb: + pb.update() + print ' %s rows imported' % rowcount + +def ucsvreader(stream, encoding='utf-8', separator=',', quote='"', + skipfirst=False): + """A csv reader that accepts files with any encoding and outputs unicode + strings + """ + it = iter(csv.reader(stream, delimiter=separator, quotechar=quote)) + if skipfirst: + it.next() + for row in it: + yield [item.decode(encoding) for item in row] + +def commit_every(nbit, store, it): + for i, x in enumerate(it): + yield x + if nbit is not None and i % nbit: + store.commit() + if nbit is not None: + store.commit() + +def lazytable(reader): + """The first row is taken to be the header of the table and + used to output a dict for each row of data. + + >>> data = lazytable(utf8csvreader(open(filename))) + """ + header = reader.next() + for row in reader: + yield dict(zip(header, row)) + +def mk_entity(row, map): + """Return a dict made from sanitized mapped values. + + ValueError can be raised on unexpected values found in checkers + + >>> row = {'myname': u'dupont'} + >>> map = [('myname', u'name', (call_transform_method('title'),))] + >>> mk_entity(row, map) + {'name': u'Dupont'} + >>> row = {'myname': u'dupont', 'optname': u''} + >>> map = [('myname', u'name', (call_transform_method('title'),)), + ... ('optname', u'MARKER', (optional,))] + >>> mk_entity(row, map) + {'name': u'Dupont', 'optname': None} + """ + res = {} + assert isinstance(row, dict) + assert isinstance(map, list) + for src, dest, funcs in map: + res[dest] = row[src] + try: + for func in funcs: + res[dest] = func(res[dest]) + if res[dest] is None: + break + except ValueError, err: + raise ValueError('error with %r field: %s' % (src, err)) + return res + + +# user interactions ############################################################ + +def tell(msg): + print msg + +def confirm(question): + """A confirm function that asks for yes/no/abort and exits on abort.""" + answer = shellutils.ASK.ask(question, ('Y', 'n', 'abort'), 'Y') + if answer == 'abort': + sys.exit(1) + return answer == 'Y' + + +class catch_error(object): + """Helper for @contextmanager decorator.""" + + def __init__(self, ctl, key='unexpected error', msg=None): + self.ctl = ctl + self.key = key + self.msg = msg + + def __enter__(self): + return self + + def __exit__(self, type, value, traceback): + if type is not None: + if issubclass(type, (KeyboardInterrupt, SystemExit)): + return # re-raise + if self.ctl.catcherrors: + self.ctl.record_error(self.key, None, type, value, traceback) + return True # silent + + +# base sanitizing/coercing functions ########################################### + +def optional(value): + """checker to filter optional field + + If value is undefined (ex: empty string), return None that will + break the checkers validation chain + + General use is to add 'optional' check in first condition to avoid + ValueError by further checkers + + >>> MAPPER = [(u'value', 'value', (optional, int))] + >>> row = {'value': u'XXX'} + >>> mk_entity(row, MAPPER) + {'value': None} + >>> row = {'value': u'100'} + >>> mk_entity(row, MAPPER) + {'value': 100} + """ + if value: + return value + return None + +def required(value): + """raise ValueError is value is empty + + This check should be often found in last position in the chain. + """ + if value: + return value + raise ValueError("required") + +def todatetime(format='%d/%m/%Y'): + """return a transformation function to turn string input value into a + `datetime.datetime` instance, using given format. + + Follow it by `todate` or `totime` functions from `logilab.common.date` if + you want a `date`/`time` instance instead of `datetime`. + """ + def coerce(value): + return strptime(value, format) + return coerce + +def call_transform_method(methodname, *args, **kwargs): + """return value returned by calling the given method on input""" + def coerce(value): + return getattr(value, methodname)(*args, **kwargs) + return coerce + +def call_check_method(methodname, *args, **kwargs): + """check value returned by calling the given method on input is true, + else raise ValueError + """ + def check(value): + if getattr(value, methodname)(*args, **kwargs): + return value + raise ValueError('%s not verified on %r' % (methodname, value)) + return check + +# base integrity checking functions ############################################ + +def check_doubles(buckets): + """Extract the keys that have more than one item in their bucket.""" + return [(k, len(v)) for k, v in buckets.items() if len(v) > 1] + +def check_doubles_not_none(buckets): + """Extract the keys that have more than one item in their bucket.""" + return [(k, len(v)) for k, v in buckets.items() + if k is not None and len(v) > 1] + + +# object stores ################################################################# + +class ObjectStore(object): + """Store objects in memory for *faster* validation (development mode) + + But it will not enforce the constraints of the schema and hence will miss some problems + + >>> store = ObjectStore() + >>> user = {'login': 'johndoe'} + >>> store.add('CWUser', user) + >>> group = {'name': 'unknown'} + >>> store.add('CWUser', group) + >>> store.relate(user['eid'], 'in_group', group['eid']) + """ + def __init__(self): + self.items = [] + self.eids = {} + self.types = {} + self.relations = set() + self.indexes = {} + self._rql = None + self._commit = None + + def _put(self, type, item): + self.items.append(item) + return len(self.items) - 1 + + def add(self, type, item): + assert isinstance(item, dict), 'item is not a dict but a %s' % type(item) + eid = item['eid'] = self._put(type, item) + self.eids[eid] = item + self.types.setdefault(type, []).append(eid) + + def relate(self, eid_from, rtype, eid_to, inlined=False): + """Add new relation""" + relation = eid_from, rtype, eid_to + self.relations.add(relation) + return relation + + def commit(self): + """this commit method do nothing by default + + This is voluntary to use the frequent autocommit feature in CubicWeb + when you are using hooks or another + + If you want override commit method, please set it by the + constructor + """ + pass + + def rql(self, *args): + if self._rql is not None: + return self._rql(*args) + + @property + def nb_inserted_entities(self): + return len(self.eids) + @property + def nb_inserted_types(self): + return len(self.types) + @property + def nb_inserted_relations(self): + return len(self.relations) + + @deprecated("[3.7] index support will disappear") + def build_index(self, name, type, func=None, can_be_empty=False): + """build internal index for further search""" + index = {} + if func is None or not callable(func): + func = lambda x: x['eid'] + for eid in self.types[type]: + index.setdefault(func(self.eids[eid]), []).append(eid) + if not can_be_empty: + assert index, "new index '%s' cannot be empty" % name + self.indexes[name] = index + + @deprecated("[3.7] index support will disappear") + def build_rqlindex(self, name, type, key, rql, rql_params=False, + func=None, can_be_empty=False): + """build an index by rql query + + rql should return eid in first column + ctl.store.build_index('index_name', 'users', 'login', 'Any U WHERE U is CWUser') + """ + self.types[type] = [] + rset = self.rql(rql, rql_params or {}) + if not can_be_empty: + assert rset, "new index type '%s' cannot be empty (0 record found)" % type + for entity in rset.entities(): + getattr(entity, key) # autopopulate entity with key attribute + self.eids[entity.eid] = dict(entity) + if entity.eid not in self.types[type]: + self.types[type].append(entity.eid) + + # Build index with specified key + func = lambda x: x[key] + self.build_index(name, type, func, can_be_empty=can_be_empty) + + @deprecated("[3.7] index support will disappear") + def fetch(self, name, key, unique=False, decorator=None): + """index fetcher method + + decorator is a callable method or an iterator of callable methods (usually a lambda function) + decorator=lambda x: x[:1] (first value is returned) + decorator=lambda x: x.lower (lowercased value is returned) + + decorator is handy when you want to improve index keys but without + changing the original field + + Same check functions can be reused here. + """ + eids = self.indexes[name].get(key, []) + if decorator is not None: + if not hasattr(decorator, '__iter__'): + decorator = (decorator,) + for f in decorator: + eids = f(eids) + if unique: + assert len(eids) == 1, u'expected a single one value for key "%s" in index "%s". Got %i' % (key, name, len(eids)) + eids = eids[0] + return eids + + @deprecated("[3.7] index support will disappear") + def find(self, type, key, value): + for idx in self.types[type]: + item = self.items[idx] + if item[key] == value: + yield item + + @deprecated("[3.7] checkpoint() deprecated. use commit() instead") + def checkpoint(self): + self.commit() + + +class RQLObjectStore(ObjectStore): + """ObjectStore that works with an actual RQL repository (production mode)""" + _rql = None # bw compat + + def __init__(self, session=None, commit=None): + ObjectStore.__init__(self) + if session is not None: + if not hasattr(session, 'set_pool'): + # connection + cnx = session + session = session.request() + session.set_pool = lambda : None + commit = commit or cnx.commit + else: + session.set_pool() + self.session = session + self._commit = commit or session.commit + elif commit is not None: + self._commit = commit + # XXX .session + + @deprecated("[3.7] checkpoint() deprecated. use commit() instead") + def checkpoint(self): + self.commit() + + def commit(self): + txuuid = self._commit() + self.session.set_pool() + return txuuid + + def rql(self, *args): + if self._rql is not None: + return self._rql(*args) + return self.session.execute(*args) + + def create_entity(self, *args, **kwargs): + entity = self.session.create_entity(*args, **kwargs) + self.eids[entity.eid] = entity + self.types.setdefault(args[0], []).append(entity.eid) + return entity + + def _put(self, type, item): + query = ('INSERT %s X: ' % type) + ', '.join('X %s %%(%s)s' % (k, k) + for k in item) + return self.rql(query, item)[0][0] + + def relate(self, eid_from, rtype, eid_to, inlined=False): + eid_from, rtype, eid_to = super(RQLObjectStore, self).relate( + eid_from, rtype, eid_to) + self.rql('SET X %s Y WHERE X eid %%(x)s, Y eid %%(y)s' % rtype, + {'x': int(eid_from), 'y': int(eid_to)}, ('x', 'y')) + + +# the import controller ######################################################## + +class CWImportController(object): + """Controller of the data import process. + + >>> ctl = CWImportController(store) + >>> ctl.generators = list_of_data_generators + >>> ctl.data = dict_of_data_tables + >>> ctl.run() + """ + + def __init__(self, store, askerror=0, catcherrors=None, tell=tell, + commitevery=50): + self.store = store + self.generators = None + self.data = {} + self.errors = None + self.askerror = askerror + if catcherrors is None: + catcherrors = askerror + self.catcherrors = catcherrors + self.commitevery = commitevery # set to None to do a single commit + self._tell = tell + + def check(self, type, key, value): + self._checks.setdefault(type, {}).setdefault(key, []).append(value) + + def check_map(self, entity, key, map, default): + try: + entity[key] = map[entity[key]] + except KeyError: + self.check(key, entity[key], None) + entity[key] = default + + def record_error(self, key, msg=None, type=None, value=None, tb=None): + tmp = StringIO() + if type is None: + traceback.print_exc(file=tmp) + else: + traceback.print_exception(type, value, tb, file=tmp) + print tmp.getvalue() + # use a list to avoid counting a errors instead of one + errorlog = self.errors.setdefault(key, []) + if msg is None: + errorlog.append(tmp.getvalue().splitlines()) + else: + errorlog.append( (msg, tmp.getvalue().splitlines()) ) + + def run(self): + self.errors = {} + for func, checks in self.generators: + self._checks = {} + func_name = func.__name__ + self.tell("Run import function '%s'..." % func_name) + try: + func(self) + except: + if self.catcherrors: + self.record_error(func_name, 'While calling %s' % func.__name__) + else: + self._print_stats() + raise + for key, func, title, help in checks: + buckets = self._checks.get(key) + if buckets: + err = func(buckets) + if err: + self.errors[title] = (help, err) + txuuid = self.store.commit() + self._print_stats() + if self.errors: + if self.askerror == 2 or (self.askerror and confirm('Display errors ?')): + from pprint import pformat + for errkey, error in self.errors.items(): + self.tell("\n%s (%s): %d\n" % (error[0], errkey, len(error[1]))) + self.tell(pformat(sorted(error[1]))) + if txuuid is not None: + print 'transaction id:', txuuid + def _print_stats(self): + nberrors = sum(len(err[1]) for err in self.errors.values()) + self.tell('\nImport statistics: %i entities, %i types, %i relations and %i errors' + % (self.store.nb_inserted_entities, + self.store.nb_inserted_types, + self.store.nb_inserted_relations, + nberrors)) + + def get_data(self, key): + return self.data.get(key) + + def index(self, name, key, value, unique=False): + """create a new index + + If unique is set to True, only first occurence will be kept not the following ones + """ + if unique: + try: + if value in self.store.indexes[name][key]: + return + except KeyError: + # we're sure that one is the first occurence; so continue... + pass + self.store.indexes.setdefault(name, {}).setdefault(key, []).append(value) + + def tell(self, msg): + self._tell(msg) + + def iter_and_commit(self, datakey): + """iter rows, triggering commit every self.commitevery iterations""" + return commit_every(self.commitevery, self.store, self.get_data(datakey)) + + + +from datetime import datetime +from cubicweb.schema import META_RTYPES, VIRTUAL_RTYPES + + +class NoHookRQLObjectStore(RQLObjectStore): + """ObjectStore that works with an actual RQL repository (production mode)""" + _rql = None # bw compat + + def __init__(self, session, metagen=None, baseurl=None): + super(NoHookRQLObjectStore, self).__init__(session) + self.source = session.repo.system_source + self.rschema = session.repo.schema.rschema + self.add_relation = self.source.add_relation + if metagen is None: + metagen = MetaGenerator(session, baseurl) + self.metagen = metagen + self._nb_inserted_entities = 0 + self._nb_inserted_types = 0 + self._nb_inserted_relations = 0 + self.rql = session.execute + # deactivate security + session.set_read_security(False) + session.set_write_security(False) + + def create_entity(self, etype, **kwargs): + for k, v in kwargs.iteritems(): + kwargs[k] = getattr(v, 'eid', v) + entity, rels = self.metagen.base_etype_dicts(etype) + entity = copy(entity) + entity._related_cache = {} + self.metagen.init_entity(entity) + entity.update(kwargs) + entity.edited_attributes = set(entity) + session = self.session + self.source.add_entity(session, entity) + self.source.add_info(session, entity, self.source, None, complete=False) + for rtype, targeteids in rels.iteritems(): + # targeteids may be a single eid or a list of eids + inlined = self.rschema(rtype).inlined + try: + for targeteid in targeteids: + self.add_relation(session, entity.eid, rtype, targeteid, + inlined) + except TypeError: + self.add_relation(session, entity.eid, rtype, targeteids, + inlined) + self._nb_inserted_entities += 1 + return entity + + def relate(self, eid_from, rtype, eid_to): + assert not rtype.startswith('reverse_') + self.add_relation(self.session, eid_from, rtype, eid_to, + self.rschema(rtype).inlined) + self._nb_inserted_relations += 1 + + @property + def nb_inserted_entities(self): + return self._nb_inserted_entities + @property + def nb_inserted_types(self): + return self._nb_inserted_types + @property + def nb_inserted_relations(self): + return self._nb_inserted_relations + + def _put(self, type, item): + raise RuntimeError('use create entity') + + +class MetaGenerator(object): + def __init__(self, session, baseurl=None): + self.session = session + self.source = session.repo.system_source + self.time = datetime.now() + if baseurl is None: + config = session.vreg.config + baseurl = config['base-url'] or config.default_base_url() + if not baseurl[-1] == '/': + baseurl += '/' + self.baseurl = baseurl + # attributes/relations shared by all entities of the same type + self.etype_attrs = [] + self.etype_rels = [] + # attributes/relations specific to each entity + self.entity_attrs = ['cwuri'] + #self.entity_rels = [] XXX not handled (YAGNI?) + schema = session.vreg.schema + rschema = schema.rschema + for rtype in META_RTYPES: + if rtype in ('eid', 'cwuri') or rtype in VIRTUAL_RTYPES: + continue + if rschema(rtype).final: + self.etype_attrs.append(rtype) + else: + self.etype_rels.append(rtype) + if not schema._eid_index: + # test schema loaded from the fs + self.gen_is = self.test_gen_is + self.gen_is_instance_of = self.test_gen_is_instanceof + + @cached + def base_etype_dicts(self, etype): + entity = self.session.vreg['etypes'].etype_class(etype)(self.session) + # entity are "surface" copied, avoid shared dict between copies + del entity.cw_extra_kwargs + for attr in self.etype_attrs: + entity[attr] = self.generate(entity, attr) + rels = {} + for rel in self.etype_rels: + rels[rel] = self.generate(entity, rel) + return entity, rels + + def init_entity(self, entity): + entity.eid = self.source.create_eid(self.session) + for attr in self.entity_attrs: + entity[attr] = self.generate(entity, attr) + + def generate(self, entity, rtype): + return getattr(self, 'gen_%s' % rtype)(entity) + + def gen_cwuri(self, entity): + return u'%seid/%s' % (self.baseurl, entity.eid) + + def gen_creation_date(self, entity): + return self.time + def gen_modification_date(self, entity): + return self.time + + def gen_is(self, entity): + return entity.e_schema.eid + def gen_is_instance_of(self, entity): + eids = [] + for etype in entity.e_schema.ancestors() + [entity.e_schema]: + eids.append(entity.e_schema.eid) + return eids + + def gen_created_by(self, entity): + return self.session.user.eid + def gen_owned_by(self, entity): + return self.session.user.eid + + # implementations of gen_is / gen_is_instance_of to use during test where + # schema has been loaded from the fs (hence entity type schema eids are not + # known) + def test_gen_is(self, entity): + return eschema_eid(self.session, entity.e_schema) + def test_gen_is_instanceof(self, entity): + eids = [] + for eschema in entity.e_schema.ancestors() + [entity.e_schema]: + eids.append(eschema_eid(self.session, eschema)) + return eids diff -r 02b52bf9f5f8 -r 0865e1e90674 dbapi.py --- a/dbapi.py Wed Mar 24 10:23:31 2010 +0100 +++ b/dbapi.py Wed Apr 28 11:54:13 2010 +0200 @@ -1,13 +1,26 @@ +# copyright 2003-2010 LOGILAB S.A. (Paris, FRANCE), all rights reserved. +# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr +# +# This file is part of CubicWeb. +# +# CubicWeb is free software: you can redistribute it and/or modify it under the +# terms of the GNU Lesser General Public License as published by the Free +# Software Foundation, either version 2.1 of the License, or (at your option) +# any later version. +# +# logilab-common is distributed in the hope that it will be useful, but WITHOUT +# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS +# FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more +# details. +# +# You should have received a copy of the GNU Lesser General Public License along +# with CubicWeb. If not, see . """DB-API 2.0 compliant module Take a look at http://www.python.org/peps/pep-0249.html (most parts of this document are reported here in docstrings) -:organization: Logilab -:copyright: 2001-2010 LOGILAB S.A. (Paris, FRANCE), license is LGPL v2. -:contact: http://www.logilab.fr/ -- mailto:contact@logilab.fr -:license: GNU Lesser General Public License, v2.1 - http://www.gnu.org/licenses """ __docformat__ = "restructuredtext en" @@ -57,6 +70,7 @@ etypescls = cwvreg.VRegistry.REGISTRY_FACTORY['etypes'] etypescls.etype_class = etypescls.orig_etype_class + class ConnectionProperties(object): def __init__(self, cnxtype=None, lang=None, close=True, log=False): self.cnxtype = cnxtype or 'pyro' @@ -105,11 +119,50 @@ def connect(database=None, login=None, host=None, group=None, cnxprops=None, setvreg=True, mulcnx=True, initlog=True, **kwargs): """Constructor for creating a connection to the CubicWeb repository. - Returns a Connection object. + Returns a :class:`Connection` object. + + Typical usage:: + + cnx = connect('myinstance', login='me', password='toto') + + Arguments: + + :database: + the instance's pyro identifier. + + :login: + the user login to use to authenticate. + + :host: + the pyro nameserver host. Will be detected using broadcast query if + unspecified. + + :group: + the instance's pyro nameserver group. You don't have to specify it unless + tweaked in instance's configuration. - When method is 'pyro', setvreg is True, try to deal with connections to - differents instances in the same process unless specified otherwise by - setting the mulcnx to False. + :cnxprops: + an optional :class:`ConnectionProperties` instance, allowing to specify + the connection method (eg in memory or pyro). A Pyro connection will be + established if you don't specify that argument. + + :setvreg: + flag telling if a registry should be initialized for the connection. + Don't change this unless you know what you're doing. + + :mulcnx: + Will disappear at some point. Try to deal with connections to differents + instances in the same process unless specified otherwise by setting this + flag to False. Don't change this unless you know what you're doing. + + :initlog: + flag telling if logging should be initialized. You usually don't want + logging initialization when establishing the connection from a process + where it's already initialized. + + :kwargs: + there goes authentication tokens. You usually have to specify for + instance a password for the given user, using a named 'password' argument. """ config = cwconfig.CubicWebNoAppConfiguration() if host: @@ -203,11 +256,6 @@ self.pgettext = lambda x, y: y self.debug('request default language: %s', self.lang) - def decorate_rset(self, rset): - rset.vreg = self.vreg - rset.req = self - return rset - def describe(self, eid): """return a tuple (type, sourceuri, extid) for the entity with id """ return self.cnx.describe(eid) @@ -242,7 +290,7 @@ def get_session_data(self, key, default=None, pop=False): """return value associated to `key` in session data""" if self.cnx is None: - return None # before the connection has been established + return default # before the connection has been established return self.cnx.get_session_data(key, default, pop) def set_session_data(self, key, value): @@ -371,6 +419,16 @@ return '' % self.sessionid return '' % self.sessionid + def __enter__(self): + return self.cursor() + + def __exit__(self, exc_type, exc_val, exc_tb): + if exc_type is None: + self.commit() + else: + self.rollback() + return False #propagate the exception + def request(self): return DBAPIRequest(self.vreg, self) @@ -397,15 +455,21 @@ pass def check(self): - """raise `BadSessionId` if the connection is no more valid""" + """raise `BadConnectionId` if the connection is no more valid""" + if self._closed is not None: + raise ProgrammingError('Closed connection') self._repo.check_session(self.sessionid) def set_session_props(self, **props): - """raise `BadSessionId` if the connection is no more valid""" + """raise `BadConnectionId` if the connection is no more valid""" + if self._closed is not None: + raise ProgrammingError('Closed connection') self._repo.set_session_props(self.sessionid, props) def get_shared_data(self, key, default=None, pop=False): """return value associated to `key` in shared data""" + if self._closed is not None: + raise ProgrammingError('Closed connection') return self._repo.get_shared_data(self.sessionid, key, default, pop) def set_shared_data(self, key, value, querydata=False): @@ -416,6 +480,8 @@ transaction, and won't be available through the connexion, only on the repository side. """ + if self._closed is not None: + raise ProgrammingError('Closed connection') return self._repo.set_shared_data(self.sessionid, key, value, querydata) def get_schema(self): @@ -501,6 +567,8 @@ def user(self, req=None, props=None): """return the User object associated to this connection""" # cnx validity is checked by the call to .user_info + if self._closed is not None: + raise ProgrammingError('Closed connection') eid, login, groups, properties = self._repo.user_info(self.sessionid, props) if req is None: @@ -521,6 +589,8 @@ pass def describe(self, eid): + if self._closed is not None: + raise ProgrammingError('Closed connection') return self._repo.describe(self.sessionid, eid) def close(self): @@ -535,19 +605,20 @@ if self._closed: raise ProgrammingError('Connection is already closed') self._repo.close(self.sessionid) + del self._repo # necessary for proper garbage collection self._closed = 1 def commit(self): - """Commit any pending transaction to the database. Note that if the - database supports an auto-commit feature, this must be initially off. An - interface method may be provided to turn it back on. + """Commit pending transaction for this connection to the repository. - Database modules that do not support transactions should implement this - method with void functionality. + may raises `Unauthorized` or `ValidationError` if we attempted to do + something we're not allowed to for security or integrity reason. + + If the transaction is undoable, a transaction id will be returned. """ if not self._closed is None: raise ProgrammingError('Connection is already closed') - self._repo.commit(self.sessionid) + return self._repo.commit(self.sessionid) def rollback(self): """This method is optional since not all databases provide transaction @@ -574,11 +645,78 @@ req = self.request() return self.cursor_class(self, self._repo, req=req) + # undo support ############################################################ + + def undoable_transactions(self, ueid=None, req=None, **actionfilters): + """Return a list of undoable transaction objects by the connection's + user, ordered by descendant transaction time. + + Managers may filter according to user (eid) who has done the transaction + using the `ueid` argument. Others will only see their own transactions. + + Additional filtering capabilities is provided by using the following + named arguments: + + * `etype` to get only transactions creating/updating/deleting entities + of the given type + + * `eid` to get only transactions applied to entity of the given eid + + * `action` to get only transactions doing the given action (action in + 'C', 'U', 'D', 'A', 'R'). If `etype`, action can only be 'C', 'U' or + 'D'. + + * `public`: when additional filtering is provided, their are by default + only searched in 'public' actions, unless a `public` argument is given + and set to false. + """ + txinfos = self._repo.undoable_transactions(self.sessionid, ueid, + **actionfilters) + if req is None: + req = self.request() + for txinfo in txinfos: + txinfo.req = req + return txinfos + + def transaction_info(self, txuuid, req=None): + """Return transaction object for the given uid. + + raise `NoSuchTransaction` if not found or if session's user is not + allowed (eg not in managers group and the transaction doesn't belong to + him). + """ + txinfo = self._repo.transaction_info(self.sessionid, txuuid) + if req is None: + req = self.request() + txinfo.req = req + return txinfo + + def transaction_actions(self, txuuid, public=True): + """Return an ordered list of action effectued during that transaction. + + If public is true, return only 'public' actions, eg not ones triggered + under the cover by hooks, else return all actions. + + raise `NoSuchTransaction` if the transaction is not found or if + session's user is not allowed (eg not in managers group and the + transaction doesn't belong to him). + """ + return self._repo.transaction_actions(self.sessionid, txuuid, public) + + def undo_transaction(self, txuuid): + """Undo the given transaction. Return potential restoration errors. + + raise `NoSuchTransaction` if not found or if session's user is not + allowed (eg not in managers group and the transaction doesn't belong to + him). + """ + return self._repo.undo_transaction(self.sessionid, txuuid) + # cursor object ############################################################### class Cursor(object): - """These objects represent a database cursor, which is used to manage the + """This represents a database cursor, which is used to manage the context of a fetch operation. Cursors created from the same connection are not isolated, i.e., any changes done to the database by a cursor are immediately visible by the other cursors. Cursors created from different @@ -588,21 +726,18 @@ """ def __init__(self, connection, repo, req=None): - """This read-only attribute return a reference to the Connection - object on which the cursor was created. - """ + # This read-only attribute returns a reference to the Connection + # object on which the cursor was created. self.connection = connection - """optionnal issuing request instance""" + # optionnal issuing request instance self.req = req - """This read/write attribute specifies the number of rows to fetch at a - time with fetchmany(). It defaults to 1 meaning to fetch a single row - at a time. - - Implementations must observe this value with respect to the fetchmany() - method, but are free to interact with the database a single row at a - time. It may also be used in the implementation of executemany(). - """ + # This read/write attribute specifies the number of rows to fetch at a + # time with fetchmany(). It defaults to 1 meaning to fetch a single row + # at a time. + # Implementations must observe this value with respect to the fetchmany() + # method, but are free to interact with the database a single row at a + # time. It may also be used in the implementation of executemany(). self.arraysize = 1 self._repo = repo @@ -611,7 +746,6 @@ self._closed = None self._index = 0 - def close(self): """Close the cursor now (rather than whenever __del__ is called). The cursor will be unusable from this point forward; an Error (or subclass) @@ -646,11 +780,11 @@ Return values are not defined by the DB-API, but this here it returns a ResultSet object. """ - self._res = res = self._repo.execute(self._sessid, operation, - parameters, eid_key, build_descr) - self.req.decorate_rset(res) + self._res = rset = self._repo.execute(self._sessid, operation, + parameters, eid_key, build_descr) + rset.req = self.req self._index = 0 - return res + return rset def executemany(self, operation, seq_of_parameters): @@ -781,4 +915,3 @@ self.connection.executed_queries.append((operation, parameters, time() - tstart, clock() - cstart)) return rset - diff -r 02b52bf9f5f8 -r 0865e1e90674 debian/changelog --- a/debian/changelog Wed Mar 24 10:23:31 2010 +0100 +++ b/debian/changelog Wed Apr 28 11:54:13 2010 +0200 @@ -1,4 +1,38 @@ -cubicweb (3.6.3-1) unstable; urgency=low +cubicweb (3.7.4-1) unstable; urgency=low + + * new upstream release + + -- Sylvain Thénault Thu, 15 Apr 2010 18:20:39 +0200 + +cubicweb (3.7.3-1) unstable; urgency=low + + * new upstream release + + -- Sylvain Thénault Wed, 31 Mar 2010 14:55:21 +0200 + +cubicweb (3.7.2-1) unstable; urgency=low + + * new upstream release + + -- Sylvain Thénault Fri, 26 Mar 2010 15:53:01 +0100 + +cubicweb (3.7.1-1) unstable; urgency=low + + * new upstream release + + -- Sylvain Thénault Fri, 19 Mar 2010 14:47:23 +0100 + +cubicweb (3.7.0-1) unstable; urgency=low + + * remove postgresql-contrib from cubicweb dependency (using tsearch + which is included with postgres >= 8.3) + * add postgresql-client | mysql-client to cubicweb-server dependencies using two + new cubicweb-[postgresql|mysql]-support virtual packages (necessary for + dump/restore of database) + + -- Sylvain Thénault Tue, 16 Mar 2010 17:55:37 +0100 + + cubicweb (3.6.3-1) unstable; urgency=low * remove postgresql-contrib from cubicweb dependency (using tsearch which is included with postgres >= 8.3) diff -r 02b52bf9f5f8 -r 0865e1e90674 debian/control --- a/debian/control Wed Mar 24 10:23:31 2010 +0100 +++ b/debian/control Wed Apr 28 11:54:13 2010 +0200 @@ -7,10 +7,10 @@ Adrien Di Mascio , Aurélien Campéas , Nicolas Chauvat -Build-Depends: debhelper (>= 5), python-dev (>=2.4), python-central (>= 0.5) +Build-Depends: debhelper (>= 5), python-dev (>=2.5), python-central (>= 0.5) Standards-Version: 3.8.0 Homepage: http://www.cubicweb.org -XS-Python-Version: >= 2.4, << 2.6 +XS-Python-Version: >= 2.5, << 2.6 Package: cubicweb Architecture: all @@ -33,8 +33,7 @@ Conflicts: cubicweb-multisources Replaces: cubicweb-multisources Provides: cubicweb-multisources -# postgresql/mysql -client packages for backup/restore of non local database -Depends: ${python:Depends}, cubicweb-common (= ${source:Version}), cubicweb-ctl (= ${source:Version}), python-indexer (>= 0.6.1), cubicweb-postgresql-support | cubicweb-mysql-support | python-pysqlite2 +Depends: ${python:Depends}, cubicweb-common (= ${source:Version}), cubicweb-ctl (= ${source:Version}), python-logilab-database (>= 1.0.2), cubicweb-postgresql-support | cubicweb-mysql-support | python-pysqlite2 Recommends: pyro, cubicweb-documentation (= ${source:Version}) Description: server part of the CubicWeb framework CubicWeb is a semantic web application framework. @@ -51,7 +50,7 @@ Description: postgres support for the CubicWeb framework CubicWeb is a semantic web application framework. . - This virtual package provides dependancies to use postgres for the + This virtual package provides dependencies to use postgres for the cubicweb repository. Package: cubicweb-mysql-support @@ -61,7 +60,7 @@ Description: mysql support for the CubicWeb framework CubicWeb is a semantic web application framework. . - This virtual package provides dependancies to use mysql for the + This virtual package provides dependencies to use mysql for the cubicweb repository. @@ -98,7 +97,7 @@ Package: cubicweb-common Architecture: all XB-Python-Version: ${python:Versions} -Depends: ${python:Depends}, graphviz, gettext, python-logilab-mtconverter (>= 0.6.0), python-logilab-common (>= 0.48.1), python-yams (>= 0.28.0), python-rql (>= 0.24.0), python-lxml +Depends: ${python:Depends}, graphviz, gettext, python-logilab-mtconverter (>= 0.6.0), python-logilab-common (>= 0.49.0), python-yams (>= 0.28.1), python-rql (>= 0.25.0), python-lxml Recommends: python-simpletal (>= 4.0), python-crypto Conflicts: cubicweb-core Replaces: cubicweb-core diff -r 02b52bf9f5f8 -r 0865e1e90674 debian/copyright --- a/debian/copyright Wed Mar 24 10:23:31 2010 +0100 +++ b/debian/copyright Wed Apr 28 11:54:13 2010 +0200 @@ -1,27 +1,31 @@ This package was debianized by Logilab . +It was downloaded from ftp://ftp.logilab.org/pub/cubicweb -Upstream Author: +Upstream Author: - Logilab + Logilab Copyright: -Copyright (c) 2003-2008 LOGILAB S.A. (Paris, FRANCE). -http://www.logilab.fr/ -- mailto:contact@logilab.fr + Copyright (c) 2003-2010 LOGILAB S.A. (Paris, FRANCE). + http://www.logilab.fr/ -- mailto:contact@logilab.fr -This program is free software; you can redistribute it and/or modify it under -the terms of the GNU Lesser General Public License as published by the Free -Software Foundation; either version 2 of the License, or (at your option) any -later version. +License: -This program is distributed in the hope that it will be useful, but WITHOUT -ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS -FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. + This program is free software; you can redistribute it and/or modify it + under the terms of the GNU Lesser General Public License as published by the + Free Software Foundation; either version 2.1 of the License, or (at your + option) any later version. -You should have received a copy of the GNU General Public License along with -this program; if not, write to the Free Software Foundation, Inc., -51 Franklin St, Fifth Floor, Boston, MA 02110-1301, USA. + This program is distributed in the hope that it will be useful, but WITHOUT + ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License + for more details. + + You should have received a copy of the GNU Lessser General Public License + along with this program; if not, write to the Free Software Foundation, + Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301, USA. On Debian systems, the complete text of the GNU Lesser General Public License -may be found in '/usr/share/common-licenses/LGPL'. +may be found in '/usr/share/common-licenses/LGPL-2.1'. diff -r 02b52bf9f5f8 -r 0865e1e90674 devtools/__init__.py --- a/devtools/__init__.py Wed Mar 24 10:23:31 2010 +0100 +++ b/devtools/__init__.py Wed Apr 28 11:54:13 2010 +0200 @@ -1,9 +1,22 @@ +# copyright 2003-2010 LOGILAB S.A. (Paris, FRANCE), all rights reserved. +# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr +# +# This file is part of CubicWeb. +# +# CubicWeb is free software: you can redistribute it and/or modify it under the +# terms of the GNU Lesser General Public License as published by the Free +# Software Foundation, either version 2.1 of the License, or (at your option) +# any later version. +# +# logilab-common is distributed in the hope that it will be useful, but WITHOUT +# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS +# FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more +# details. +# +# You should have received a copy of the GNU Lesser General Public License along +# with CubicWeb. If not, see . """Test tools for cubicweb -:organization: Logilab -:copyright: 2001-2010 LOGILAB S.A. (Paris, FRANCE), license is LGPL v2. -:contact: http://www.logilab.fr/ -- mailto:contact@logilab.fr -:license: GNU Lesser General Public License, v2.1 - http://www.gnu.org/licenses """ __docformat__ = "restructuredtext en" @@ -81,7 +94,6 @@ mode = 'test' set_language = False read_instance_schema = False - bootstrap_schema = False init_repository = True options = cwconfig.merge_options(ServerConfiguration.options + ( ('anonymous-user', @@ -106,8 +118,6 @@ self.init_log(log_threshold, force=True) # need this, usually triggered by cubicweb-ctl self.load_cwctl_plugins() - self.global_set_option('anonymous-user', 'anon') - self.global_set_option('anonymous-password', 'anon') anonymous_user = TwistedConfiguration.anonymous_user.im_func @@ -123,6 +133,8 @@ super(TestServerConfiguration, self).load_configuration() self.global_set_option('anonymous-user', 'anon') self.global_set_option('anonymous-password', 'anon') + # no undo support in tests + self.global_set_option('undo-support', '') def main_config_file(self): """return instance's control configuration file""" @@ -201,6 +213,8 @@ init_test_database_sqlite(config) elif driver == 'postgres': init_test_database_postgres(config) + elif driver == 'sqlserver2005': + init_test_database_sqlserver2005(config) else: raise ValueError('no initialization function for driver %r' % driver) config._cubes = None # avoid assertion error @@ -216,6 +230,10 @@ driver = config.sources()['system']['db-driver'] if driver == 'sqlite': reset_test_database_sqlite(config) + elif driver in ('sqlserver2005', 'postgres'): + # XXX do something with dump/restore ? + print 'resetting the database is not done for', driver + print 'you should handle it manually' else: raise ValueError('no reset function for driver %r' % driver) @@ -223,11 +241,18 @@ ### postgres test database handling ############################################ def init_test_database_postgres(config): - """initialize a fresh sqlite databse used for testing purpose""" + """initialize a fresh postgresql databse used for testing purpose""" if config.init_repository: from cubicweb.server import init_repository init_repository(config, interactive=False, drop=True) +### sqlserver2005 test database handling ############################################ + +def init_test_database_sqlserver2005(config): + """initialize a fresh sqlserver databse used for testing purpose""" + if config.init_repository: + from cubicweb.server import init_repository + init_repository(config, interactive=False, drop=True, vreg=vreg) ### sqlite test database handling ############################################## diff -r 02b52bf9f5f8 -r 0865e1e90674 devtools/cwtwill.py --- a/devtools/cwtwill.py Wed Mar 24 10:23:31 2010 +0100 +++ b/devtools/cwtwill.py Wed Apr 28 11:54:13 2010 +0200 @@ -1,9 +1,22 @@ +# copyright 2003-2010 LOGILAB S.A. (Paris, FRANCE), all rights reserved. +# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr +# +# This file is part of CubicWeb. +# +# CubicWeb is free software: you can redistribute it and/or modify it under the +# terms of the GNU Lesser General Public License as published by the Free +# Software Foundation, either version 2.1 of the License, or (at your option) +# any later version. +# +# logilab-common is distributed in the hope that it will be useful, but WITHOUT +# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS +# FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more +# details. +# +# You should have received a copy of the GNU Lesser General Public License along +# with CubicWeb. If not, see . """cubicweb extensions for twill -:organization: Logilab -:copyright: 2001-2010 LOGILAB S.A. (Paris, FRANCE), license is LGPL v2. -:contact: http://www.logilab.fr/ -- mailto:contact@logilab.fr -:license: GNU Lesser General Public License, v2.1 - http://www.gnu.org/licenses """ import re diff -r 02b52bf9f5f8 -r 0865e1e90674 devtools/dataimport.py --- a/devtools/dataimport.py Wed Mar 24 10:23:31 2010 +0100 +++ b/devtools/dataimport.py Wed Apr 28 11:54:13 2010 +0200 @@ -1,752 +1,21 @@ -# -*- coding: utf-8 -*- -"""This module provides tools to import tabular data. - -:organization: Logilab -:copyright: 2001-2010 LOGILAB S.A. (Paris, FRANCE), license is LGPL v2. -:contact: http://www.logilab.fr/ -- mailto:contact@logilab.fr -:license: GNU Lesser General Public License, v2.1 - http://www.gnu.org/licenses - - -Example of use (run this with `cubicweb-ctl shell instance import-script.py`): - -.. sourcecode:: python - - from cubicweb.devtools.dataimport import * - # define data generators - GENERATORS = [] - - USERS = [('Prenom', 'firstname', ()), - ('Nom', 'surname', ()), - ('Identifiant', 'login', ()), - ] - - def gen_users(ctl): - for row in ctl.get_data('utilisateurs'): - entity = mk_entity(row, USERS) - entity['upassword'] = u'motdepasse' - ctl.check('login', entity['login'], None) - ctl.store.add('CWUser', entity) - email = {'address': row['email']} - ctl.store.add('EmailAddress', email) - ctl.store.relate(entity['eid'], 'use_email', email['eid']) - ctl.store.rql('SET U in_group G WHERE G name "users", U eid %(x)s', {'x':entity['eid']}) - - CHK = [('login', check_doubles, 'Utilisateurs Login', - 'Deux utilisateurs ne devraient pas avoir le même login.'), - ] - - GENERATORS.append( (gen_users, CHK) ) - - # create controller - ctl = CWImportController(RQLObjectStore()) - ctl.askerror = 1 - ctl.generators = GENERATORS - ctl.store._checkpoint = checkpoint - ctl.store._rql = rql - ctl.data['utilisateurs'] = lazytable(utf8csvreader(open('users.csv'))) - # run - ctl.run() - sys.exit(0) - - -.. BUG fichier à une colonne pose un problème de parsing -.. TODO rollback() -""" -__docformat__ = "restructuredtext en" - -import sys -import csv -import traceback -import os.path as osp -from StringIO import StringIO -from copy import copy - -from logilab.common import shellutils -from logilab.common.date import strptime -from logilab.common.decorators import cached -from logilab.common.deprecation import deprecated - - -def ucsvreader_pb(filepath, encoding='utf-8', separator=',', quote='"', - skipfirst=False, withpb=True): - """same as ucsvreader but a progress bar is displayed as we iter on rows""" - if not osp.exists(filepath): - raise Exception("file doesn't exists: %s" % filepath) - rowcount = int(shellutils.Execute('wc -l "%s"' % filepath).out.strip().split()[0]) - if skipfirst: - rowcount -= 1 - if withpb: - pb = shellutils.ProgressBar(rowcount, 50) - for urow in ucsvreader(file(filepath), encoding, separator, quote, skipfirst): - yield urow - if withpb: - pb.update() - print ' %s rows imported' % rowcount - -def ucsvreader(stream, encoding='utf-8', separator=',', quote='"', - skipfirst=False): - """A csv reader that accepts files with any encoding and outputs unicode - strings - """ - it = iter(csv.reader(stream, delimiter=separator, quotechar=quote)) - if skipfirst: - it.next() - for row in it: - yield [item.decode(encoding) for item in row] - -def commit_every(nbit, store, it): - for i, x in enumerate(it): - yield x - if nbit is not None and i % nbit: - store.checkpoint() - if nbit is not None: - store.checkpoint() - -def lazytable(reader): - """The first row is taken to be the header of the table and - used to output a dict for each row of data. - - >>> data = lazytable(utf8csvreader(open(filename))) - """ - header = reader.next() - for row in reader: - yield dict(zip(header, row)) - -def mk_entity(row, map): - """Return a dict made from sanitized mapped values. - - ValidationError can be raised on unexpected values found in checkers - - >>> row = {'myname': u'dupont'} - >>> map = [('myname', u'name', (capitalize_if_unicase,))] - >>> mk_entity(row, map) - {'name': u'Dupont'} - >>> row = {'myname': u'dupont', 'optname': u''} - >>> map = [('myname', u'name', (capitalize_if_unicase,)), - ... ('optname', u'MARKER', (optional,))] - >>> mk_entity(row, map) - {'name': u'Dupont'} - """ - res = {} - assert isinstance(row, dict) - assert isinstance(map, list) - for src, dest, funcs in map: - assert not (required in funcs and optional in funcs), \ - "optional and required checks are exclusive" - res[dest] = row[src] - try: - for func in funcs: - res[dest] = func(res[dest]) - if res[dest] is None: - break - except ValueError, err: - raise ValueError('error with %r field: %s' % (src, err)) - return res - - -# user interactions ############################################################ - -def tell(msg): - print msg - -def confirm(question): - """A confirm function that asks for yes/no/abort and exits on abort.""" - answer = shellutils.ASK.ask(question, ('Y', 'n', 'abort'), 'Y') - if answer == 'abort': - sys.exit(1) - return answer == 'Y' - - -class catch_error(object): - """Helper for @contextmanager decorator.""" - - def __init__(self, ctl, key='unexpected error', msg=None): - self.ctl = ctl - self.key = key - self.msg = msg - - def __enter__(self): - return self - - def __exit__(self, type, value, traceback): - if type is not None: - if issubclass(type, (KeyboardInterrupt, SystemExit)): - return # re-raise - if self.ctl.catcherrors: - self.ctl.record_error(self.key, None, type, value, traceback) - return True # silent - - -# base sanitizing/coercing functions ########################################### - -def optional(value): - """validation error will not been raised if you add this checker in chain""" - if value: - return value - return None - -def required(value): - """raise ValueError is value is empty - - This check should be often found in last position in the chain. - """ - if value: - return value - raise ValueError("required") - -def todatetime(format='%d/%m/%Y'): - """return a transformation function to turn string input value into a - `datetime.datetime` instance, using given format. - - Follow it by `todate` or `totime` functions from `logilab.common.date` if - you want a `date`/`time` instance instead of `datetime`. - """ - def coerce(value): - return strptime(value, format) - return coerce - -def call_transform_method(methodname, *args, **kwargs): - """return value returned by calling the given method on input""" - def coerce(value): - return getattr(value, methodname)(*args, **kwargs) - return coerce - -def call_check_method(methodname, *args, **kwargs): - """check value returned by calling the given method on input is true, - else raise ValueError - """ - def check(value): - if getattr(value, methodname)(*args, **kwargs): - return value - raise ValueError('%s not verified on %r' % (methodname, value)) - return check - -# base integrity checking functions ############################################ - -def check_doubles(buckets): - """Extract the keys that have more than one item in their bucket.""" - return [(k, len(v)) for k, v in buckets.items() if len(v) > 1] - -def check_doubles_not_none(buckets): - """Extract the keys that have more than one item in their bucket.""" - return [(k, len(v)) for k, v in buckets.items() - if k is not None and len(v) > 1] - - -# object stores ################################################################# - -class ObjectStore(object): - """Store objects in memory for *faster* validation (development mode) - - But it will not enforce the constraints of the schema and hence will miss some problems - - >>> store = ObjectStore() - >>> user = {'login': 'johndoe'} - >>> store.add('CWUser', user) - >>> group = {'name': 'unknown'} - >>> store.add('CWUser', group) - >>> store.relate(user['eid'], 'in_group', group['eid']) - """ - def __init__(self): - self.items = [] - self.eids = {} - self.types = {} - self.relations = set() - self.indexes = {} - self._rql = None - self._checkpoint = None - - def _put(self, type, item): - self.items.append(item) - return len(self.items) - 1 - - def add(self, type, item): - assert isinstance(item, dict), 'item is not a dict but a %s' % type(item) - eid = item['eid'] = self._put(type, item) - self.eids[eid] = item - self.types.setdefault(type, []).append(eid) - - def relate(self, eid_from, rtype, eid_to, inlined=False): - """Add new relation (reverse type support is available) - - >>> 1,2 = eid_from, eid_to - >>> self.relate(eid_from, 'in_group', eid_to) - 1, 'in_group', 2 - >>> self.relate(eid_from, 'reverse_in_group', eid_to) - 2, 'in_group', 1 - """ - if rtype.startswith('reverse_'): - eid_from, eid_to = eid_to, eid_from - rtype = rtype[8:] - relation = eid_from, rtype, eid_to - self.relations.add(relation) - return relation - - def build_index(self, name, type, func=None): - index = {} - if func is None or not callable(func): - func = lambda x: x['eid'] - for eid in self.types[type]: - index.setdefault(func(self.eids[eid]), []).append(eid) - assert index, "new index '%s' cannot be empty" % name - self.indexes[name] = index - - def build_rqlindex(self, name, type, key, rql, rql_params=False, func=None): - """build an index by rql query - - rql should return eid in first column - ctl.store.build_index('index_name', 'users', 'login', 'Any U WHERE U is CWUser') - """ - rset = self.rql(rql, rql_params or {}) - for entity in rset.entities(): - getattr(entity, key) # autopopulate entity with key attribute - self.eids[entity.eid] = dict(entity) - if entity.eid not in self.types.setdefault(type, []): - self.types[type].append(entity.eid) - assert self.types[type], "new index type '%s' cannot be empty (0 record found)" % type - - # Build index with specified key - func = lambda x: x[key] - self.build_index(name, type, func) - - def fetch(self, name, key, unique=False, decorator=None): - """ - decorator is a callable method or an iterator of callable methods (usually a lambda function) - decorator=lambda x: x[:1] (first value is returned) - - We can use validation check function available in _entity - """ - eids = self.indexes[name].get(key, []) - if decorator is not None: - if not hasattr(decorator, '__iter__'): - decorator = (decorator,) - for f in decorator: - eids = f(eids) - if unique: - assert len(eids) == 1, u'expected a single one value for key "%s" in index "%s". Got %i' % (key, name, len(eids)) - eids = eids[0] # FIXME maybe it's better to keep an iterator here ? - return eids - - def find(self, type, key, value): - for idx in self.types[type]: - item = self.items[idx] - if item[key] == value: - yield item - - def rql(self, *args): - if self._rql is not None: - return self._rql(*args) - - def checkpoint(self): - pass - - @property - def nb_inserted_entities(self): - return len(self.eids) - @property - def nb_inserted_types(self): - return len(self.types) - @property - def nb_inserted_relations(self): - return len(self.relations) - - @deprecated('[3.6] get_many() deprecated. Use fetch() instead') - def get_many(self, name, key): - return self.fetch(name, key, unique=False) - - @deprecated('[3.6] get_one() deprecated. Use fetch(..., unique=True) instead') - def get_one(self, name, key): - return self.fetch(name, key, unique=True) - - -class RQLObjectStore(ObjectStore): - """ObjectStore that works with an actual RQL repository (production mode)""" - _rql = None # bw compat - - def __init__(self, session=None, checkpoint=None): - ObjectStore.__init__(self) - if session is not None: - if not hasattr(session, 'set_pool'): - # connection - cnx = session - session = session.request() - session.set_pool = lambda : None - checkpoint = checkpoint or cnx.commit - else: - session.set_pool() - self.session = session - self._checkpoint = checkpoint or session.commit - elif checkpoint is not None: - self._checkpoint = checkpoint - # XXX .session - - def checkpoint(self): - self._checkpoint() - self.session.set_pool() - - def rql(self, *args): - if self._rql is not None: - return self._rql(*args) - return self.session.execute(*args) - - def create_entity(self, *args, **kwargs): - entity = self.session.create_entity(*args, **kwargs) - self.eids[entity.eid] = entity - self.types.setdefault(args[0], []).append(entity.eid) - return entity - - def _put(self, type, item): - query = ('INSERT %s X: ' % type) + ', '.join('X %s %%(%s)s' % (k, k) - for k in item) - return self.rql(query, item)[0][0] - - def relate(self, eid_from, rtype, eid_to, inlined=False): - # if reverse relation is found, eids are exchanged - eid_from, rtype, eid_to = super(RQLObjectStore, self).relate( - eid_from, rtype, eid_to) - self.rql('SET X %s Y WHERE X eid %%(x)s, Y eid %%(y)s' % rtype, - {'x': int(eid_from), 'y': int(eid_to)}, ('x', 'y')) - - -# the import controller ######################################################## - -class CWImportController(object): - """Controller of the data import process. - - >>> ctl = CWImportController(store) - >>> ctl.generators = list_of_data_generators - >>> ctl.data = dict_of_data_tables - >>> ctl.run() - """ - - def __init__(self, store, askerror=0, catcherrors=None, tell=tell, - commitevery=50): - self.store = store - self.generators = None - self.data = {} - self.errors = None - self.askerror = askerror - if catcherrors is None: - catcherrors = askerror - self.catcherrors = catcherrors - self.commitevery = commitevery # set to None to do a single commit - self._tell = tell - - def check(self, type, key, value): - self._checks.setdefault(type, {}).setdefault(key, []).append(value) - - def check_map(self, entity, key, map, default): - try: - entity[key] = map[entity[key]] - except KeyError: - self.check(key, entity[key], None) - entity[key] = default - - def record_error(self, key, msg=None, type=None, value=None, tb=None): - tmp = StringIO() - if type is None: - traceback.print_exc(file=tmp) - else: - traceback.print_exception(type, value, tb, file=tmp) - print tmp.getvalue() - # use a list to avoid counting a errors instead of one - errorlog = self.errors.setdefault(key, []) - if msg is None: - errorlog.append(tmp.getvalue().splitlines()) - else: - errorlog.append( (msg, tmp.getvalue().splitlines()) ) - - def run(self): - self.errors = {} - for func, checks in self.generators: - self._checks = {} - func_name = func.__name__[4:] # XXX - self.tell("Import '%s'..." % func_name) - try: - func(self) - except: - if self.catcherrors: - self.record_error(func_name, 'While calling %s' % func.__name__) - else: - raise - for key, func, title, help in checks: - buckets = self._checks.get(key) - if buckets: - err = func(buckets) - if err: - self.errors[title] = (help, err) - self.store.checkpoint() - nberrors = sum(len(err[1]) for err in self.errors.values()) - self.tell('\nImport completed: %i entities, %i types, %i relations and %i errors' - % (self.store.nb_inserted_entities, - self.store.nb_inserted_types, - self.store.nb_inserted_relations, - nberrors)) - if self.errors: - if self.askerror == 2 or (self.askerror and confirm('Display errors ?')): - from pprint import pformat - for errkey, error in self.errors.items(): - self.tell("\n%s (%s): %d\n" % (error[0], errkey, len(error[1]))) - self.tell(pformat(sorted(error[1]))) - - def get_data(self, key): - return self.data.get(key) - - def index(self, name, key, value, unique=False): - """create a new index - - If unique is set to True, only first occurence will be kept not the following ones - """ - if unique: - try: - if value in self.store.indexes[name][key]: - return - except KeyError: - # we're sure that one is the first occurence; so continue... - pass - self.store.indexes.setdefault(name, {}).setdefault(key, []).append(value) - - def tell(self, msg): - self._tell(msg) - - def iter_and_commit(self, datakey): - """iter rows, triggering commit every self.commitevery iterations""" - return commit_every(self.commitevery, self.store, self.get_data(datakey)) - - - -from datetime import datetime -from cubicweb.schema import META_RTYPES, VIRTUAL_RTYPES - - -class NoHookRQLObjectStore(RQLObjectStore): - """ObjectStore that works with an actual RQL repository (production mode)""" - _rql = None # bw compat - - def __init__(self, session, metagen=None, baseurl=None): - super(NoHookRQLObjectStore, self).__init__(session) - self.source = session.repo.system_source - self.rschema = session.repo.schema.rschema - self.add_relation = self.source.add_relation - if metagen is None: - metagen = MetaGenerator(session, baseurl) - self.metagen = metagen - self._nb_inserted_entities = 0 - self._nb_inserted_types = 0 - self._nb_inserted_relations = 0 - self.rql = session.unsafe_execute - - def create_entity(self, etype, **kwargs): - for k, v in kwargs.iteritems(): - kwargs[k] = getattr(v, 'eid', v) - entity, rels = self.metagen.base_etype_dicts(etype) - entity = copy(entity) - entity._related_cache = {} - self.metagen.init_entity(entity) - entity.update(kwargs) - session = self.session - self.source.add_entity(session, entity) - self.source.add_info(session, entity, self.source, complete=False) - for rtype, targeteids in rels.iteritems(): - # targeteids may be a single eid or a list of eids - inlined = self.rschema(rtype).inlined - try: - for targeteid in targeteids: - self.add_relation(session, entity.eid, rtype, targeteid, - inlined) - except TypeError: - self.add_relation(session, entity.eid, rtype, targeteids, - inlined) - self._nb_inserted_entities += 1 - return entity - - def relate(self, eid_from, rtype, eid_to): - assert not rtype.startswith('reverse_') - self.add_relation(self.session, eid_from, rtype, eid_to, - self.rschema(rtype).inlined) - self._nb_inserted_relations += 1 - - @property - def nb_inserted_entities(self): - return self._nb_inserted_entities - @property - def nb_inserted_types(self): - return self._nb_inserted_types - @property - def nb_inserted_relations(self): - return self._nb_inserted_relations - - def _put(self, type, item): - raise RuntimeError('use create entity') - - -class MetaGenerator(object): - def __init__(self, session, baseurl=None): - self.session = session - self.source = session.repo.system_source - self.time = datetime.now() - if baseurl is None: - config = session.vreg.config - baseurl = config['base-url'] or config.default_base_url() - if not baseurl[-1] == '/': - baseurl += '/' - self.baseurl = baseurl - # attributes/relations shared by all entities of the same type - self.etype_attrs = [] - self.etype_rels = [] - # attributes/relations specific to each entity - self.entity_attrs = ['eid', 'cwuri'] - #self.entity_rels = [] XXX not handled (YAGNI?) - schema = session.vreg.schema - rschema = schema.rschema - for rtype in META_RTYPES: - if rtype in ('eid', 'cwuri') or rtype in VIRTUAL_RTYPES: - continue - if rschema(rtype).final: - self.etype_attrs.append(rtype) - else: - self.etype_rels.append(rtype) - if not schema._eid_index: - # test schema loaded from the fs - self.gen_is = self.test_gen_is - self.gen_is_instance_of = self.test_gen_is_instanceof - - @cached - def base_etype_dicts(self, etype): - entity = self.session.vreg['etypes'].etype_class(etype)(self.session) - # entity are "surface" copied, avoid shared dict between copies - del entity.cw_extra_kwargs - for attr in self.etype_attrs: - entity[attr] = self.generate(entity, attr) - rels = {} - for rel in self.etype_rels: - rels[rel] = self.generate(entity, rel) - return entity, rels - - def init_entity(self, entity): - for attr in self.entity_attrs: - entity[attr] = self.generate(entity, attr) - entity.eid = entity['eid'] - - def generate(self, entity, rtype): - return getattr(self, 'gen_%s' % rtype)(entity) - - def gen_eid(self, entity): - return self.source.create_eid(self.session) - - def gen_cwuri(self, entity): - return u'%seid/%s' % (self.baseurl, entity['eid']) - - def gen_creation_date(self, entity): - return self.time - def gen_modification_date(self, entity): - return self.time - - def gen_is(self, entity): - return entity.e_schema.eid - def gen_is_instance_of(self, entity): - eids = [] - for etype in entity.e_schema.ancestors() + [entity.e_schema]: - eids.append(entity.e_schema.eid) - return eids - - def gen_created_by(self, entity): - return self.session.user.eid - def gen_owned_by(self, entity): - return self.session.user.eid - - # implementations of gen_is / gen_is_instance_of to use during test where - # schema has been loaded from the fs (hence entity type schema eids are not - # known) - def test_gen_is(self, entity): - from cubicweb.hooks.metadata import eschema_eid - return eschema_eid(self.session, entity.e_schema) - def test_gen_is_instanceof(self, entity): - from cubicweb.hooks.metadata import eschema_eid - eids = [] - for eschema in entity.e_schema.ancestors() + [entity.e_schema]: - eids.append(eschema_eid(self.session, eschema)) - return eids - - -################################################################################ - -utf8csvreader = deprecated('[3.6] use ucsvreader instead')(ucsvreader) - -@deprecated('[3.6] use required') -def nonempty(value): - return required(value) - -@deprecated("[3.6] use call_check_method('isdigit')") -def alldigits(txt): - if txt.isdigit(): - return txt - else: - return u'' - -@deprecated("[3.7] too specific, will move away, copy me") -def capitalize_if_unicase(txt): - if txt.isupper() or txt.islower(): - return txt.capitalize() - return txt - -@deprecated("[3.7] too specific, will move away, copy me") -def yesno(value): - """simple heuristic that returns boolean value - - >>> yesno("Yes") - True - >>> yesno("oui") - True - >>> yesno("1") - True - >>> yesno("11") - True - >>> yesno("") - False - >>> yesno("Non") - False - >>> yesno("blablabla") - False - """ - if value: - return value.lower()[0] in 'yo1' - return False - -@deprecated("[3.7] use call_check_method('isalpha')") -def isalpha(value): - if value.isalpha(): - return value - raise ValueError("not all characters in the string alphabetic") - -@deprecated("[3.7] use call_transform_method('upper')") -def uppercase(txt): - return txt.upper() - -@deprecated("[3.7] use call_transform_method('lower')") -def lowercase(txt): - return txt.lower() - -@deprecated("[3.7] use call_transform_method('replace', ' ', '')") -def no_space(txt): - return txt.replace(' ','') - -@deprecated("[3.7] use call_transform_method('replace', u'\xa0', '')") -def no_uspace(txt): - return txt.replace(u'\xa0','') - -@deprecated("[3.7] use call_transform_method('replace', '-', '')") -def no_dash(txt): - return txt.replace('-','') - -@deprecated("[3.7] use call_transform_method('strip')") -def strip(txt): - return txt.strip() - -@deprecated("[3.7] use call_transform_method('replace', ',', '.'), float") -def decimal(value): - return comma_float(value) - -@deprecated('[3.7] use int builtin') -def integer(value): - return int(value) +# pylint: disable-msg=W0614,W0401 +# copyright 2003-2010 LOGILAB S.A. (Paris, FRANCE), all rights reserved. +# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr +# +# This file is part of CubicWeb. +# +# CubicWeb is free software: you can redistribute it and/or modify it under the +# terms of the GNU Lesser General Public License as published by the Free +# Software Foundation, either version 2.1 of the License, or (at your option) +# any later version. +# +# logilab-common is distributed in the hope that it will be useful, but WITHOUT +# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS +# FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more +# details. +# +# You should have received a copy of the GNU Lesser General Public License along +# with CubicWeb. If not, see . +from warnings import warn +warn('moved to cubicweb.dataimport', DeprecationWarning, stacklevel=2) +from cubicweb.dataimport import * diff -r 02b52bf9f5f8 -r 0865e1e90674 devtools/devctl.py --- a/devtools/devctl.py Wed Mar 24 10:23:31 2010 +0100 +++ b/devtools/devctl.py Wed Apr 28 11:54:13 2010 +0200 @@ -1,10 +1,23 @@ +# copyright 2003-2010 LOGILAB S.A. (Paris, FRANCE), all rights reserved. +# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr +# +# This file is part of CubicWeb. +# +# CubicWeb is free software: you can redistribute it and/or modify it under the +# terms of the GNU Lesser General Public License as published by the Free +# Software Foundation, either version 2.1 of the License, or (at your option) +# any later version. +# +# logilab-common is distributed in the hope that it will be useful, but WITHOUT +# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS +# FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more +# details. +# +# You should have received a copy of the GNU Lesser General Public License along +# with CubicWeb. If not, see . """additional cubicweb-ctl commands and command handlers for cubicweb and cubicweb's cubes development -:organization: Logilab -:copyright: 2001-2010 LOGILAB S.A. (Paris, FRANCE), license is LGPL v2. -:contact: http://www.logilab.fr/ -- mailto:contact@logilab.fr -:license: GNU Lesser General Public License, v2.1 - http://www.gnu.org/licenses """ __docformat__ = "restructuredtext en" @@ -515,19 +528,14 @@ longdesc = shortdesc = raw_input('Enter a short description for your cube: ') if verbose: longdesc = raw_input('Enter a long description (leave empty to reuse the short one): ') + dependencies = {} if verbose: - includes = self._ask_for_dependancies() - if len(includes) == 1: - dependancies = '%r,' % includes[0] - else: - dependancies = ', '.join(repr(cube) for cube in includes) - else: - dependancies = '' + dependencies = self._ask_for_dependencies() context = {'cubename' : cubename, 'distname' : distname, 'shortdesc' : shortdesc, 'longdesc' : longdesc or shortdesc, - 'dependancies' : dependancies, + 'dependencies' : dict((dep, None) for dep in dependencies), 'version' : cubicwebversion, 'year' : str(datetime.now().year), 'author': self['author'], @@ -536,7 +544,7 @@ } copy_skeleton(skeldir, cubedir, context) - def _ask_for_dependancies(self): + def _ask_for_dependencies(self): from logilab.common.shellutils import ASK from logilab.common.textutils import splitstrip includes = [] @@ -546,7 +554,7 @@ if answer == 'y': includes.append(stdtype) if answer == 'type': - includes = splitstrip(raw_input('type dependancies: ')) + includes = splitstrip(raw_input('type dependencies: ')) break elif answer == 'skip': break @@ -565,47 +573,48 @@ chances are the lines at the top are the ones that will bring the higher benefit after optimisation. Start there. """ - arguments = '< rql.log' + arguments = 'rql.log' name = 'exlog' options = ( ) def run(self, args): - if args: - raise BadCommandUsage("no argument expected") import re requests = {} - for lineno, line in enumerate(sys.stdin): - if not ' WHERE ' in line: - continue - #sys.stderr.write( line ) + for filepath in args: try: - rql, time = line.split('--') - rql = re.sub("(\'\w+': \d*)", '', rql) - if '{' in rql: - rql = rql[:rql.index('{')] - req = requests.setdefault(rql, []) - time.strip() - chunks = time.split() - clocktime = float(chunks[0][1:]) - cputime = float(chunks[-3]) - req.append( (clocktime, cputime) ) - except Exception, exc: - sys.stderr.write('Line %s: %s (%s)\n' % (lineno, exc, line)) - + stream = file(filepath) + except OSError, ex: + raise BadCommandUsage("can't open rql log file %s: %s" + % (filepath, ex)) + for lineno, line in enumerate(file): + if not ' WHERE ' in line: + continue + try: + rql, time = line.split('--') + rql = re.sub("(\'\w+': \d*)", '', rql) + if '{' in rql: + rql = rql[:rql.index('{')] + req = requests.setdefault(rql, []) + time.strip() + chunks = time.split() + clocktime = float(chunks[0][1:]) + cputime = float(chunks[-3]) + req.append( (clocktime, cputime) ) + except Exception, exc: + sys.stderr.write('Line %s: %s (%s)\n' % (lineno, exc, line)) stat = [] - for rql, times in requests.items(): + for rql, times in requests.iteritems(): stat.append( (sum(time[0] for time in times), sum(time[1] for time in times), len(times), rql) ) - stat.sort() stat.reverse() - - total_time = sum(clocktime for clocktime, cputime, occ, rql in stat)*0.01 + total_time = sum(clocktime for clocktime, cputime, occ, rql in stat) * 0.01 print 'Percentage;Cumulative Time (clock);Cumulative Time (CPU);Occurences;Query' for clocktime, cputime, occ, rql in stat: - print '%.2f;%.2f;%.2f;%s;%s' % (clocktime/total_time, clocktime, cputime, occ, rql) + print '%.2f;%.2f;%.2f;%s;%s' % (clocktime/total_time, clocktime, + cputime, occ, rql) class GenerateSchema(Command): diff -r 02b52bf9f5f8 -r 0865e1e90674 devtools/fake.py --- a/devtools/fake.py Wed Mar 24 10:23:31 2010 +0100 +++ b/devtools/fake.py Wed Apr 28 11:54:13 2010 +0200 @@ -1,15 +1,26 @@ +# copyright 2003-2010 LOGILAB S.A. (Paris, FRANCE), all rights reserved. +# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr +# +# This file is part of CubicWeb. +# +# CubicWeb is free software: you can redistribute it and/or modify it under the +# terms of the GNU Lesser General Public License as published by the Free +# Software Foundation, either version 2.1 of the License, or (at your option) +# any later version. +# +# logilab-common is distributed in the hope that it will be useful, but WITHOUT +# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS +# FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more +# details. +# +# You should have received a copy of the GNU Lesser General Public License along +# with CubicWeb. If not, see . """Fake objects to ease testing of cubicweb without a fully working environment -:organization: Logilab -:copyright: 2001-2010 LOGILAB S.A. (Paris, FRANCE), license is LGPL v2. -:contact: http://www.logilab.fr/ -- mailto:contact@logilab.fr -:license: GNU Lesser General Public License, v2.1 - http://www.gnu.org/licenses """ __docformat__ = "restructuredtext en" -from logilab.common.adbh import get_adv_func_helper - -from indexer import get_indexer +from logilab.database import get_db_helper from cubicweb.req import RequestSessionBase from cubicweb.cwvreg import CubicWebVRegistry @@ -118,17 +129,6 @@ def validate_cache(self): pass - # session compatibility (in some test are using this class to test server - # side views...) - def actual_session(self): - """return the original parent session if any, else self""" - return self - - def unsafe_execute(self, *args, **kwargs): - """return the original parent session if any, else self""" - kwargs.pop('propagate', None) - return self.execute(*args, **kwargs) - class FakeUser(object): login = 'toto' @@ -138,18 +138,19 @@ class FakeSession(RequestSessionBase): + read_security = write_security = True + set_read_security = set_write_security = lambda *args, **kwargs: None + def __init__(self, repo=None, user=None): self.repo = repo self.vreg = getattr(self.repo, 'vreg', CubicWebVRegistry(FakeConfig(), initlog=False)) self.pool = FakePool() self.user = user or FakeUser() self.is_internal_session = False - self.is_super_session = self.user.eid == -1 self.transaction_data = {} - def execute(self, *args): + def execute(self, *args, **kwargs): pass - unsafe_execute = execute def commit(self, *args): self.transaction_data.clear() @@ -158,11 +159,6 @@ def system_sql(self, sql, args=None): pass - def decorate_rset(self, rset, propagate=False): - rset.vreg = self.vreg - rset.req = self - return rset - def set_entity_cache(self, entity): pass @@ -200,12 +196,7 @@ class FakeSource(object): - dbhelper = get_adv_func_helper('sqlite') - indexer = get_indexer('sqlite', 'UTF8') - dbhelper.fti_uid_attr = indexer.uid_attr - dbhelper.fti_table = indexer.table - dbhelper.fti_restriction_sql = indexer.restriction_sql - dbhelper.fti_need_distinct_query = indexer.need_distinct + dbhelper = get_db_helper('sqlite') def __init__(self, uri): self.uri = uri diff -r 02b52bf9f5f8 -r 0865e1e90674 devtools/fill.py --- a/devtools/fill.py Wed Mar 24 10:23:31 2010 +0100 +++ b/devtools/fill.py Wed Apr 28 11:54:13 2010 +0200 @@ -1,10 +1,23 @@ # -*- coding: iso-8859-1 -*- +# copyright 2003-2010 LOGILAB S.A. (Paris, FRANCE), all rights reserved. +# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr +# +# This file is part of CubicWeb. +# +# CubicWeb is free software: you can redistribute it and/or modify it under the +# terms of the GNU Lesser General Public License as published by the Free +# Software Foundation, either version 2.1 of the License, or (at your option) +# any later version. +# +# logilab-common is distributed in the hope that it will be useful, but WITHOUT +# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS +# FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more +# details. +# +# You should have received a copy of the GNU Lesser General Public License along +# with CubicWeb. If not, see . """This modules defines func / methods for creating test repositories -:organization: Logilab -:copyright: 2001-2010 LOGILAB S.A. (Paris, FRANCE), license is LGPL v2. -:contact: http://www.logilab.fr/ -- mailto:contact@logilab.fr -:license: GNU Lesser General Public License, v2.1 - http://www.gnu.org/licenses """ __docformat__ = "restructuredtext en" diff -r 02b52bf9f5f8 -r 0865e1e90674 devtools/htmlparser.py --- a/devtools/htmlparser.py Wed Mar 24 10:23:31 2010 +0100 +++ b/devtools/htmlparser.py Wed Apr 28 11:54:13 2010 +0200 @@ -1,9 +1,22 @@ +# copyright 2003-2010 LOGILAB S.A. (Paris, FRANCE), all rights reserved. +# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr +# +# This file is part of CubicWeb. +# +# CubicWeb is free software: you can redistribute it and/or modify it under the +# terms of the GNU Lesser General Public License as published by the Free +# Software Foundation, either version 2.1 of the License, or (at your option) +# any later version. +# +# logilab-common is distributed in the hope that it will be useful, but WITHOUT +# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS +# FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more +# details. +# +# You should have received a copy of the GNU Lesser General Public License along +# with CubicWeb. If not, see . """defines a validating HTML parser used in web application tests -:organization: Logilab -:copyright: 2001-2010 LOGILAB S.A. (Paris, FRANCE), license is LGPL v2. -:contact: http://www.logilab.fr/ -- mailto:contact@logilab.fr -:license: GNU Lesser General Public License, v2.1 - http://www.gnu.org/licenses """ import re @@ -74,6 +87,21 @@ Validator.__init__(self) self.parser = etree.XMLParser() +class XMLDemotingValidator(SaxOnlyValidator): + """ some views produce html instead of xhtml, using demote_to_html + + this is typically related to the use of external dependencies + which do not produce valid xhtml (google maps, ...) + """ + + def preprocess_data(self, data): + if data.startswith('. """provide utilies for web (live) unit testing -:organization: Logilab -:copyright: 2001-2010 LOGILAB S.A. (Paris, FRANCE), license is LGPL v2. -:contact: http://www.logilab.fr/ -- mailto:contact@logilab.fr -:license: GNU Lesser General Public License, v2.1 - http://www.gnu.org/licenses """ import os diff -r 02b52bf9f5f8 -r 0865e1e90674 devtools/realdbtest.py --- a/devtools/realdbtest.py Wed Mar 24 10:23:31 2010 +0100 +++ b/devtools/realdbtest.py Wed Apr 28 11:54:13 2010 +0200 @@ -1,3 +1,20 @@ +# copyright 2003-2010 LOGILAB S.A. (Paris, FRANCE), all rights reserved. +# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr +# +# This file is part of CubicWeb. +# +# CubicWeb is free software: you can redistribute it and/or modify it under the +# terms of the GNU Lesser General Public License as published by the Free +# Software Foundation, either version 2.1 of the License, or (at your option) +# any later version. +# +# logilab-common is distributed in the hope that it will be useful, but WITHOUT +# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS +# FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more +# details. +# +# You should have received a copy of the GNU Lesser General Public License along +# with CubicWeb. If not, see . from cubicweb import toolsutils from cubicweb.devtools import DEFAULT_SOURCES, BaseApptestConfiguration diff -r 02b52bf9f5f8 -r 0865e1e90674 devtools/repotest.py --- a/devtools/repotest.py Wed Mar 24 10:23:31 2010 +0100 +++ b/devtools/repotest.py Wed Apr 28 11:54:13 2010 +0200 @@ -1,11 +1,24 @@ +# copyright 2003-2010 LOGILAB S.A. (Paris, FRANCE), all rights reserved. +# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr +# +# This file is part of CubicWeb. +# +# CubicWeb is free software: you can redistribute it and/or modify it under the +# terms of the GNU Lesser General Public License as published by the Free +# Software Foundation, either version 2.1 of the License, or (at your option) +# any later version. +# +# logilab-common is distributed in the hope that it will be useful, but WITHOUT +# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS +# FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more +# details. +# +# You should have received a copy of the GNU Lesser General Public License along +# with CubicWeb. If not, see . """some utilities to ease repository testing This module contains functions to initialize a new repository. -:organization: Logilab -:copyright: 2003-2010 LOGILAB S.A. (Paris, FRANCE), license is LGPL v2. -:contact: http://www.logilab.fr/ -- mailto:contact@logilab.fr -:license: GNU Lesser General Public License, v2.1 - http://www.gnu.org/licenses """ __docformat__ = "restructuredtext en" @@ -95,6 +108,31 @@ def __iter__(self): return iter(sorted(self.origdict, key=self.sortkey)) +def schema_eids_idx(schema): + """return a dictionary mapping schema types to their eids so we can reread + it from the fs instead of the db (too costly) between tests + """ + schema_eids = {} + for x in schema.entities(): + schema_eids[x] = x.eid + for x in schema.relations(): + schema_eids[x] = x.eid + for rdef in x.rdefs.itervalues(): + schema_eids[(rdef.subject, rdef.rtype, rdef.object)] = rdef.eid + return schema_eids + +def restore_schema_eids_idx(schema, schema_eids): + """rebuild schema eid index""" + for x in schema.entities(): + x.eid = schema_eids[x] + schema._eid_index[x.eid] = x + for x in schema.relations(): + x.eid = schema_eids[x] + schema._eid_index[x.eid] = x + for rdef in x.rdefs.itervalues(): + rdef.eid = schema_eids[(rdef.subject, rdef.rtype, rdef.object)] + schema._eid_index[rdef.eid] = rdef + from logilab.common.testlib import TestCase from rql import RQLHelper @@ -150,17 +188,23 @@ self.pool = self.session.set_pool() self.maxeid = self.get_max_eid() do_monkey_patch() + self._dumb_sessions = [] def get_max_eid(self): - return self.session.unsafe_execute('Any MAX(X)')[0][0] + return self.session.execute('Any MAX(X)')[0][0] def cleanup(self): - self.session.unsafe_execute('DELETE Any X WHERE X eid > %s' % self.maxeid) + self.session.set_pool() + self.session.execute('DELETE Any X WHERE X eid > %s' % self.maxeid) def tearDown(self): undo_monkey_patch() self.session.rollback() self.cleanup() self.commit() + # properly close dumb sessions + for session in self._dumb_sessions: + session.rollback() + session.close() self.repo._free_pool(self.pool) assert self.session.user.eid != -1 @@ -198,6 +242,8 @@ u._groups = set(groups) s = Session(u, self.repo) s._threaddata.pool = self.pool + # register session to ensure it gets closed + self._dumb_sessions.append(s) return s def execute(self, rql, args=None, eid_key=None, build_descr=True): @@ -223,6 +269,7 @@ self.sources = self.o._repo.sources self.system = self.sources[-1] do_monkey_patch() + self._dumb_sessions = [] # by hi-jacked parent setup def add_source(self, sourcecls, uri): self.sources.append(sourcecls(self.repo, self.o.schema, @@ -237,6 +284,9 @@ del self.repo.sources_by_uri[source.uri] self.newsources -= 1 undo_monkey_patch() + for session in self._dumb_sessions: + session._threaddata.pool = None + session.close() def _prepare_plan(self, rql, kwargs=None): rqlst = self.o.parse(rql, annotate=True) diff -r 02b52bf9f5f8 -r 0865e1e90674 devtools/stresstester.py --- a/devtools/stresstester.py Wed Mar 24 10:23:31 2010 +0100 +++ b/devtools/stresstester.py Wed Apr 28 11:54:13 2010 +0200 @@ -1,3 +1,20 @@ +# copyright 2003-2010 LOGILAB S.A. (Paris, FRANCE), all rights reserved. +# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr +# +# This file is part of CubicWeb. +# +# CubicWeb is free software: you can redistribute it and/or modify it under the +# terms of the GNU Lesser General Public License as published by the Free +# Software Foundation, either version 2.1 of the License, or (at your option) +# any later version. +# +# logilab-common is distributed in the hope that it will be useful, but WITHOUT +# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS +# FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more +# details. +# +# You should have received a copy of the GNU Lesser General Public License along +# with CubicWeb. If not, see . """ Usage: %s [OPTIONS] Stress test a CubicWeb repository @@ -23,7 +40,6 @@ Copyright (c) 2003-2010 LOGILAB S.A. (Paris, FRANCE), license is LGPL v2. http://www.logilab.fr/ -- mailto:contact@logilab.fr -:license: GNU Lesser General Public License, v2.1 - http://www.gnu.org/licenses """ import os diff -r 02b52bf9f5f8 -r 0865e1e90674 devtools/test/data/schema.py --- a/devtools/test/data/schema.py Wed Mar 24 10:23:31 2010 +0100 +++ b/devtools/test/data/schema.py Wed Apr 28 11:54:13 2010 +0200 @@ -1,9 +1,22 @@ +# copyright 2003-2010 LOGILAB S.A. (Paris, FRANCE), all rights reserved. +# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr +# +# This file is part of CubicWeb. +# +# CubicWeb is free software: you can redistribute it and/or modify it under the +# terms of the GNU Lesser General Public License as published by the Free +# Software Foundation, either version 2.1 of the License, or (at your option) +# any later version. +# +# logilab-common is distributed in the hope that it will be useful, but WITHOUT +# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS +# FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more +# details. +# +# You should have received a copy of the GNU Lesser General Public License along +# with CubicWeb. If not, see . """ -:organization: Logilab -:copyright: 2001-2010 LOGILAB S.A. (Paris, FRANCE), license is LGPL v2. -:contact: http://www.logilab.fr/ -- mailto:contact@logilab.fr -:license: GNU Lesser General Public License, v2.1 - http://www.gnu.org/licenses """ from yams.buildobjs import EntityType, SubjectRelation, String, Int, Date diff -r 02b52bf9f5f8 -r 0865e1e90674 devtools/test/data/views.py --- a/devtools/test/data/views.py Wed Mar 24 10:23:31 2010 +0100 +++ b/devtools/test/data/views.py Wed Apr 28 11:54:13 2010 +0200 @@ -1,9 +1,22 @@ +# copyright 2003-2010 LOGILAB S.A. (Paris, FRANCE), all rights reserved. +# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr +# +# This file is part of CubicWeb. +# +# CubicWeb is free software: you can redistribute it and/or modify it under the +# terms of the GNU Lesser General Public License as published by the Free +# Software Foundation, either version 2.1 of the License, or (at your option) +# any later version. +# +# logilab-common is distributed in the hope that it will be useful, but WITHOUT +# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS +# FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more +# details. +# +# You should have received a copy of the GNU Lesser General Public License along +# with CubicWeb. If not, see . """only for unit tests ! -:organization: Logilab -:copyright: 2001-2010 LOGILAB S.A. (Paris, FRANCE), license is LGPL v2. -:contact: http://www.logilab.fr/ -- mailto:contact@logilab.fr -:license: GNU Lesser General Public License, v2.1 - http://www.gnu.org/licenses """ from cubicweb.view import EntityView diff -r 02b52bf9f5f8 -r 0865e1e90674 devtools/test/unittest_dbfill.py --- a/devtools/test/unittest_dbfill.py Wed Mar 24 10:23:31 2010 +0100 +++ b/devtools/test/unittest_dbfill.py Wed Apr 28 11:54:13 2010 +0200 @@ -1,10 +1,23 @@ # -*- coding: iso-8859-1 -*- +# copyright 2003-2010 LOGILAB S.A. (Paris, FRANCE), all rights reserved. +# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr +# +# This file is part of CubicWeb. +# +# CubicWeb is free software: you can redistribute it and/or modify it under the +# terms of the GNU Lesser General Public License as published by the Free +# Software Foundation, either version 2.1 of the License, or (at your option) +# any later version. +# +# logilab-common is distributed in the hope that it will be useful, but WITHOUT +# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS +# FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more +# details. +# +# You should have received a copy of the GNU Lesser General Public License along +# with CubicWeb. If not, see . """unit tests for database value generator -:organization: Logilab -:copyright: 2001-2010 LOGILAB S.A. (Paris, FRANCE), license is LGPL v2. -:contact: http://www.logilab.fr/ -- mailto:contact@logilab.fr -:license: GNU Lesser General Public License, v2.1 - http://www.gnu.org/licenses """ import os.path as osp diff -r 02b52bf9f5f8 -r 0865e1e90674 devtools/test/unittest_fill.py --- a/devtools/test/unittest_fill.py Wed Mar 24 10:23:31 2010 +0100 +++ b/devtools/test/unittest_fill.py Wed Apr 28 11:54:13 2010 +0200 @@ -1,9 +1,22 @@ +# copyright 2003-2010 LOGILAB S.A. (Paris, FRANCE), all rights reserved. +# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr +# +# This file is part of CubicWeb. +# +# CubicWeb is free software: you can redistribute it and/or modify it under the +# terms of the GNU Lesser General Public License as published by the Free +# Software Foundation, either version 2.1 of the License, or (at your option) +# any later version. +# +# logilab-common is distributed in the hope that it will be useful, but WITHOUT +# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS +# FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more +# details. +# +# You should have received a copy of the GNU Lesser General Public License along +# with CubicWeb. If not, see . """unit tests for cubicweb.devtools.fill module -:organization: Logilab -:copyright: 2001-2010 LOGILAB S.A. (Paris, FRANCE), license is LGPL v2. -:contact: http://www.logilab.fr/ -- mailto:contact@logilab.fr -:license: GNU Lesser General Public License, v2.1 - http://www.gnu.org/licenses """ from logilab.common.testlib import TestCase, unittest_main diff -r 02b52bf9f5f8 -r 0865e1e90674 devtools/test/unittest_testlib.py --- a/devtools/test/unittest_testlib.py Wed Mar 24 10:23:31 2010 +0100 +++ b/devtools/test/unittest_testlib.py Wed Apr 28 11:54:13 2010 +0200 @@ -1,20 +1,33 @@ +# copyright 2003-2010 LOGILAB S.A. (Paris, FRANCE), all rights reserved. +# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr +# +# This file is part of CubicWeb. +# +# CubicWeb is free software: you can redistribute it and/or modify it under the +# terms of the GNU Lesser General Public License as published by the Free +# Software Foundation, either version 2.1 of the License, or (at your option) +# any later version. +# +# logilab-common is distributed in the hope that it will be useful, but WITHOUT +# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS +# FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more +# details. +# +# You should have received a copy of the GNU Lesser General Public License along +# with CubicWeb. If not, see . """unittests for gct.apptest module -:organization: Logilab -:copyright: 2001-2010 LOGILAB S.A. (Paris, FRANCE), license is LGPL v2. -:contact: http://www.logilab.fr/ -- mailto:contact@logilab.fr -:license: GNU Lesser General Public License, v2.1 - http://www.gnu.org/licenses """ from cStringIO import StringIO from unittest import TestSuite - -from logilab.common.testlib import (TestCase, unittest_main, +from logilab.common.testlib import (TestCase, unittest_main, SkipAwareTextTestRunner) from cubicweb.devtools import htmlparser from cubicweb.devtools.testlib import CubicWebTC +from cubicweb.pytestconf import clean_repo_test_cls class WebTestTC(TestCase): @@ -37,7 +50,7 @@ self.assertEquals(result.testsRun, 2) self.assertEquals(len(result.errors), 0) self.assertEquals(len(result.failures), 1) - + clean_repo_test_cls(MyWebTest) HTML_PAGE = u""" diff -r 02b52bf9f5f8 -r 0865e1e90674 devtools/testlib.py --- a/devtools/testlib.py Wed Mar 24 10:23:31 2010 +0100 +++ b/devtools/testlib.py Wed Apr 28 11:54:13 2010 +0200 @@ -1,9 +1,22 @@ +# copyright 2003-2010 LOGILAB S.A. (Paris, FRANCE), all rights reserved. +# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr +# +# This file is part of CubicWeb. +# +# CubicWeb is free software: you can redistribute it and/or modify it under the +# terms of the GNU Lesser General Public License as published by the Free +# Software Foundation, either version 2.1 of the License, or (at your option) +# any later version. +# +# logilab-common is distributed in the hope that it will be useful, but WITHOUT +# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS +# FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more +# details. +# +# You should have received a copy of the GNU Lesser General Public License along +# with CubicWeb. If not, see . """this module contains base classes and utilities for cubicweb tests -:organization: Logilab -:copyright: 2001-2010 LOGILAB S.A. (Paris, FRANCE), license is LGPL v2. -:contact: http://www.logilab.fr/ -- mailto:contact@logilab.fr -:license: GNU Lesser General Public License, v2.1 - http://www.gnu.org/licenses """ __docformat__ = "restructuredtext en" @@ -12,8 +25,7 @@ import re from urllib import unquote from math import log - -import simplejson +from contextlib import contextmanager import yams.schema @@ -31,7 +43,7 @@ from cubicweb.web import Redirect, application from cubicweb.devtools import SYSTEM_ENTITIES, SYSTEM_RELATIONS, VIEW_VALIDATORS from cubicweb.devtools import fake, htmlparser - +from cubicweb.utils import json # low-level utilities ########################################################## @@ -131,17 +143,16 @@ """abstract class for test using an apptest environment attributes: - `vreg`, the vregistry - `schema`, self.vreg.schema - `config`, cubicweb configuration - `cnx`, dbapi connection to the repository using an admin user - `session`, server side session associated to `cnx` - `app`, the cubicweb publisher (for web testing) - `repo`, the repository object - `admlogin`, login of the admin user - `admpassword`, password of the admin user - + * `vreg`, the vregistry + * `schema`, self.vreg.schema + * `config`, cubicweb configuration + * `cnx`, dbapi connection to the repository using an admin user + * `session`, server side session associated to `cnx` + * `app`, the cubicweb publisher (for web testing) + * `repo`, the repository object + * `admlogin`, login of the admin user + * `admpassword`, password of the admin user """ appid = 'data' configcls = devtools.ApptestConfiguration @@ -207,6 +218,7 @@ def _build_repo(cls): cls.repo, cls.cnx = devtools.init_test_database(config=cls.config) cls.init_config(cls.config) + cls.repo.hm.call_hooks('server_startup', repo=cls.repo) cls.vreg = cls.repo.vreg cls._orig_cnx = cls.cnx cls.config.repository = lambda x=None: cls.repo @@ -228,7 +240,9 @@ @property def session(self): """return current server side session (using default manager account)""" - return self.repo._sessions[self.cnx.sessionid] + session = self.repo._sessions[self.cnx.sessionid] + session.set_pool() + return session @property def adminsession(self): @@ -245,12 +259,25 @@ def setUp(self): pause_tracing() - self._init_repo() + previous_failure = self.__class__.__dict__.get('_repo_init_failed') + if previous_failure is not None: + self.skip('repository is not initialised: %r' % previous_failure) + try: + self._init_repo() + except Exception, ex: + self.__class__._repo_init_failed = ex + raise resume_tracing() + self._cnxs = [] self.setup_database() self.commit() MAILBOX[:] = [] # reset mailbox + def tearDown(self): + for cnx in self._cnxs: + if not cnx._closed: + cnx.close() + def setup_database(self): """add your database setup code by overriding this method""" @@ -265,20 +292,20 @@ return req.user def create_user(self, login, groups=('users',), password=None, req=None, - commit=True): + commit=True, **kwargs): """create and return a new user entity""" if password is None: password = login.encode('utf8') - cursor = self._orig_cnx.cursor(req or self.request()) - rset = cursor.execute('INSERT CWUser X: X login %(login)s, X upassword %(passwd)s', - {'login': unicode(login), 'passwd': password}) - user = rset.get_entity(0, 0) - cursor.execute('SET X in_group G WHERE X eid %%(x)s, G name IN(%s)' - % ','.join(repr(g) for g in groups), - {'x': user.eid}, 'x') + if req is None: + req = self._orig_cnx.request() + user = req.create_entity('CWUser', login=unicode(login), + upassword=password, **kwargs) + req.execute('SET X in_group G WHERE X eid %%(x)s, G name IN(%s)' + % ','.join(repr(g) for g in groups), + {'x': user.eid}, 'x') user.clear_related_cache('in_group', 'subject') if commit: - self._orig_cnx.commit() + req.cnx.commit() return user def login(self, login, **kwargs): @@ -291,6 +318,7 @@ self.cnx = repo_connect(self.repo, unicode(login), cnxprops=ConnectionProperties('inmemory'), **kwargs) + self._cnxs.append(self.cnx) if login == self.vreg.config.anonymous_user()[0]: self.cnx.anonymous_connection = True return self.cnx @@ -299,6 +327,7 @@ if not self.cnx is self._orig_cnx: try: self.cnx.close() + self._cnxs.remove(self.cnx) except ProgrammingError: pass # already closed self.cnx = self._orig_cnx @@ -319,7 +348,10 @@ @nocoverage def commit(self): - self.cnx.commit() + try: + return self.cnx.commit() + finally: + self.session.set_pool() # ensure pool still set after commit @nocoverage def rollback(self): @@ -327,6 +359,8 @@ self.cnx.rollback() except ProgrammingError: pass + finally: + self.session.set_pool() # ensure pool still set after commit # # server side db api ####################################################### @@ -339,6 +373,17 @@ def entity(self, rql, args=None, eidkey=None, req=None): return self.execute(rql, args, eidkey, req=req).get_entity(0, 0) + @contextmanager + def temporary_appobjects(self, *appobjects): + self.vreg._loadedmods.setdefault(self.__module__, {}) + for obj in appobjects: + self.vreg.register(obj) + try: + yield + finally: + for obj in appobjects: + self.vreg.unregister(obj) + # vregistry inspection utilities ########################################### def pviews(self, req, rset): @@ -449,7 +494,7 @@ def remote_call(self, fname, *args): """remote json call simulation""" - dump = simplejson.dumps + dump = json.dumps args = [dump(arg) for arg in args] req = self.request(fname=fname, pageid='123', arg=args) ctrl = self.vreg['controllers'].select('json', req) @@ -458,9 +503,9 @@ def app_publish(self, req, path='view'): return self.app.publish(path, req) - def ctrl_publish(self, req): + def ctrl_publish(self, req, ctrl='edit'): """call the publish method of the edit controller""" - ctrl = self.vreg['controllers'].select('edit', req) + ctrl = self.vreg['controllers'].select(ctrl, req) try: result = ctrl.publish() req.cnx.commit() @@ -484,7 +529,8 @@ else: cleanup = lambda p: (p[0], unquote(p[1])) params = dict(cleanup(p.split('=', 1)) for p in params.split('&') if p) - path = path[len(req.base_url()):] + if path.startswith(req.base_url()): # may be relative + path = path[len(req.base_url()):] return path, params else: self.fail('expected a Redirect exception') diff -r 02b52bf9f5f8 -r 0865e1e90674 doc/book/en/.static/sphinx-default.css --- a/doc/book/en/.static/sphinx-default.css Wed Mar 24 10:23:31 2010 +0100 +++ b/doc/book/en/.static/sphinx-default.css Wed Apr 28 11:54:13 2010 +0200 @@ -3,7 +3,7 @@ */ html, body { - background: white; + background: white; } body { @@ -115,7 +115,7 @@ } div.sphinxsidebar h3 { - font-family: 'Verdanda', sans-serif; + font-family: Verdana, sans-serif; color: black; font-size: 1.2em; font-weight: normal; @@ -126,7 +126,7 @@ } div.sphinxsidebar h4 { - font-family: 'Verdana', sans-serif; + font-family: Verdana, sans-serif; color: black; font-size: 1.1em; font-weight: normal; diff -r 02b52bf9f5f8 -r 0865e1e90674 doc/book/en/admin/additional-tips.rst --- a/doc/book/en/admin/additional-tips.rst Wed Mar 24 10:23:31 2010 +0100 +++ b/doc/book/en/admin/additional-tips.rst Wed Apr 28 11:54:13 2010 +0200 @@ -1,55 +1,64 @@ .. _Additional Tips: -Additional Tips ---------------- - -Here are some additional tips as far as administration of a CubicWeb is concerned. - -Backup, backup, backup -`````````````````````` +Backups (mostly with postgresql) +-------------------------------- It is always a good idea to backup. If your system does not do that, you should set it up. Note that whenever you do an upgrade, -`cubicweb-ctl` offers you to backup your database. +`cubicweb-ctl` offers you to backup your database. There are a number +of ways for doing backups. -There are a number of ways for doing backups. Before you go ahead, -make sure the following permissions are correct :: +Using postgresql (and only that) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Before you +go ahead, make sure the following permissions are correct :: # chgrp postgres /var/lib/cubicweb/backup - # chmod g+ws /var/lib/cubicweb/backup - # chgrp postgres /etc/cubicweb.d/**/sources - # chmod g+r /etc/cubicweb.d/**/sources -**Classic way** +Simply use the pg_dump in a cron installed for `postgres` user on the database server:: -Simply use the pg_dump in a cron :: - - su -c "pg_dump -Fc --username=cubicweb --no-owner" postgres > -$(date '+%Y-%m-%d_%H:%M:%S').dump + # m h dom mon dow command + 0 2 * * * pg_dump -Fc --username=cubicweb --no-owner > /var/backups/-$(date '+%Y-%m-%d_%H:%M:%S').dump -**CubicWeb way** +Using :command:`cubicweb-ctl db-dump` +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -The CubicWeb way is to use the `db-dump` command. For that, you have to put your passwords in a user-only-readable file at the -root of the postgres user. The file is `.pgpass` (`chmod 0600`), in this case for a socket run connection to postgres :: +The CubicWeb way is to use the :command:`db-dump` command. For that, +you have to put your passwords in a user-only-readable file at the +home directory of root user. The file is `.pgpass` (`chmod 0600`), in +this case for a socket run connection to PostgreSQL :: - /var/run/postgresql:5432::cubicweb: + /var/run/postgresql:5432::: The postgres documentation for the `.pgpass` format can be found `here`_ -Then add the following command to the crontab of the postgres user (`su posgres 'crontab -e'`):: +Then add the following command to the crontab of the user (`crontab -e`):: # m h dom mon dow command 0 2 * * * cubicweb-ctl db-dump -**The automated sysadmin way** + +Backup ninja +~~~~~~~~~~~~ -You can use a combination `backup-ninja`_ (which has a postgres script in the example directory), `backuppc`)_ (for versionning). +You can use a combination `backup-ninja`_ (which has a postgres script in the +example directory), `backuppc`)_ (for versionning). -Please note that in the *CubicWeb way* it adds a second location for your password which is error-prone. +Please note that in the *CubicWeb way* it adds a second location for your +password which is error-prone. .. _`here` : http://www.postgresql.org/docs/current/static/libpq-pgpass.html .. _`backup-ninja` : https://labs.riseup.net/code/projects/show/backupninja/ .. _`backuppc` : http://backuppc.sourceforge.net/ + +.. warning:: + + Remember that these indications will fail you whenever you use + another database backend than postgres. Also it does properly handle + externally managed data such as files (using the Bytes File System + Storage). diff -r 02b52bf9f5f8 -r 0865e1e90674 doc/book/en/admin/create-instance.rst --- a/doc/book/en/admin/create-instance.rst Wed Mar 24 10:23:31 2010 +0100 +++ b/doc/book/en/admin/create-instance.rst Wed Apr 28 11:54:13 2010 +0200 @@ -20,7 +20,7 @@ sufficient. You can anyway modify the configuration later on by editing configuration files. When a user/psswd is requested to access the database please use the login you create at the time you configured the database -(:ref:`ConfigurationPostgresql`). +(:ref:`PostgresqlConfiguration`). It is important to distinguish here the user used to access the database and the user used to login to the cubicweb instance. When an instance starts, it uses @@ -36,17 +36,20 @@ start / stop ~~~~~~~~~~~~ + When this command is completed, the definition of your instance is -located in :file:`~/etc/cubicweb.d/myinstance/*`. To launch it, you just type :: +located in :file:`~/etc/cubicweb.d/myinstance/*`. To launch it, you +just type :: cubicweb-ctl start -D myinstance -The option `-D` specify the *debug mode* : the instance is not running in -server mode and does not disconnect from the termnial, which simplifies debugging -in case the instance is not properly launched. You can see how it looks by -visiting the URL `http://localhost:8080` (the port number depends of your -configuration). To login, please use the cubicweb administrator login/psswd you -defined when you created the instance. +The option `-D` specifies the *debug mode* : the instance is not +running in server mode and does not disconnect from the terminal, +which simplifies debugging in case the instance is not properly +launched. You can see how it looks by visiting the URL +`http://localhost:8080` (the port number depends of your +configuration). To login, please use the cubicweb administrator +login/psswd you defined when you created the instance. To shutdown the instance, Crtl-C in the terminal window is enough. If you did not use the option `-D`, then type :: @@ -55,6 +58,12 @@ This is it! All is settled down to start developping your data model... +.. note:: + + The output of `cubicweb-ctl start -D myinstance` can be + overwhelming. It is possible to reduce the log level with the + `--loglevel` parameter as in `cubicweb-ctl start -D myinstance -l + info` to filter out all logs under `info` gravity. upgrade ~~~~~~~ @@ -63,5 +72,27 @@ cubicweb-ctl upgrade myinstance -XXX write me +A series of questions will be asked. It always starts with a proposal +to make a backup of your sources (where it applies). Unless you know +exactly what you are doing (i.e. typically fiddling in debug mode, but +definitely NOT migrating a production instance), you should answer YES +to that. + +The remaining questions concern the migration steps of |cubicweb|, +then of the cubes that form the whole application, in reverse +dependency order. +In principle, if the migration scripts have been properly written and +tested, you should answer YES to all questions. + +Somtimes, typically while debugging a migration script, something goes +wrong and the migration fails. Unfortunately the databse may be in an +incoherent state. You have two options here: + +* fix the bug, restore the database and restart the migration process + from scratch (quite recommended in a production environement) + +* try to replay the migration up to the last successful commit, that + is answering NO to all question up to the step that failed, and + finish by answering YES to the remaining questions. + diff -r 02b52bf9f5f8 -r 0865e1e90674 doc/book/en/admin/gae.rst --- a/doc/book/en/admin/gae.rst Wed Mar 24 10:23:31 2010 +0100 +++ b/doc/book/en/admin/gae.rst Wed Apr 28 11:54:13 2010 +0200 @@ -210,7 +210,7 @@ This cookie values needs to be provided to ``laxctl`` commands in order to handle datastore administration requests. -.. image:: ../images/lax-book.02-cookie-values.en.png +.. image:: ../images/lax-book_02-cookie-values_en.png :alt: displaying the detailed view of the cookie values returned diff -r 02b52bf9f5f8 -r 0865e1e90674 doc/book/en/admin/index.rst --- a/doc/book/en/admin/index.rst Wed Mar 24 10:23:31 2010 +0100 +++ b/doc/book/en/admin/index.rst Wed Apr 28 11:54:13 2010 +0200 @@ -2,9 +2,9 @@ .. _Part3: -------------------------- -Part III - Administration -------------------------- +-------------- +Administration +-------------- This part is for installation and administration of the *CubicWeb* framework and instances based on that framework. @@ -21,6 +21,7 @@ ldap pyro gae + migration additional-tips RQL logs diff -r 02b52bf9f5f8 -r 0865e1e90674 doc/book/en/admin/instance-config.rst --- a/doc/book/en/admin/instance-config.rst Wed Mar 24 10:23:31 2010 +0100 +++ b/doc/book/en/admin/instance-config.rst Wed Apr 28 11:54:13 2010 +0200 @@ -157,7 +157,7 @@ :`ui.encoding`: Character encoding to use for the web -:`navigation.short-line-size`: # XXX should be in ui +:`navigation.short-line-size`: number of characters for "short" display :`navigation.page-size`: maximum number of entities to show per results page diff -r 02b52bf9f5f8 -r 0865e1e90674 doc/book/en/admin/migration.rst --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/doc/book/en/admin/migration.rst Wed Apr 28 11:54:13 2010 +0200 @@ -0,0 +1,46 @@ +.. -*- coding: utf-8 -*- + +Migrating cubicweb instances - benefits from a distributed architecture +======================================================================= + +Migrate apache & cubicweb +------------------------- + +**Aim** : do the migration for N cubicweb instances hosted on a server to another with no downtime. + +**Prerequisites** : have an explicit definition of the database host (not default or localhost). In our case, the database is hosted on another host. You are not migrating your pyro server. You are not using multisource (more documentation on that soon). + +**Steps** : + +1. *on new machine* : install your environment (*pseudocode*) :: + + apt-get install cubicweb cubicweb-applications apache2 + +2. *on old machine* : copy your cubicweb and apache configuration to the new machine :: + + scp /etc/cubicweb.d/ newmachine:/etc/cubicweb.d/ + scp /etc/apache2/sites-available/ newmachine:/etc/apache2/sites-available/ + +3. *on new machine* : give new ids to pyro registration so the new instances can register :: + + cd /etc/cubicweb.d/ ; sed -i.bck 's/^pyro-instance-id=.*$/\02/' */all-in-one.conf + +4. *on new machine* : start your instances :: + + cubicweb start + +5. *on new machine* : enable sites and modules for apache and start it, test it using by modifying your /etc/host file. + +6. change dns entry from your oldmachine to newmachine + +7. shutdown your *old machine* (if it doesn't host other services or your database) + +8. That's it. + +**Possible enhancements** : use right from the start a pound server behind your apache, that way you can add backends and smoothily migrate by shuting down backends that pound will take into account. + +Migrate apache & cubicweb with pyro +----------------------------------- + +FIXME TODO + diff -r 02b52bf9f5f8 -r 0865e1e90674 doc/book/en/admin/multisources.rst --- a/doc/book/en/admin/multisources.rst Wed Mar 24 10:23:31 2010 +0100 +++ b/doc/book/en/admin/multisources.rst Wed Apr 28 11:54:13 2010 +0200 @@ -3,4 +3,4 @@ Data sources include SQL, LDAP, RQL, mercurial and subversion. -XXX feed me +.. XXX feed me diff -r 02b52bf9f5f8 -r 0865e1e90674 doc/book/en/admin/setup.rst --- a/doc/book/en/admin/setup.rst Wed Mar 24 10:23:31 2010 +0100 +++ b/doc/book/en/admin/setup.rst Wed Apr 28 11:54:13 2010 +0200 @@ -8,16 +8,17 @@ Installation of `Cubicweb` and its dependencies ----------------------------------------------- -*CubicWeb* is packaged for Debian and Ubuntu, but can be installed from source +|cubicweb| is packaged for Debian and Ubuntu, but can be installed from source using a tarball or the Mercurial version control system. + .. _DebianInstallation: Debian and Ubuntu packages ``````````````````````````` -Depending on the distribution you are using, add the appropriate line to your list -of sources (for example by editing ``/etc/apt/sources.list``). +Depending on the distribution you are using, add the appropriate line to your +list of sources (for example by editing ``/etc/apt/sources.list``). For Debian Lenny:: @@ -37,21 +38,26 @@ apt-get update apt-get install cubicweb cubicweb-dev -`cubicweb` installs the framework itself, allowing you to create -new instances. + +`cubicweb` installs the framework itself, allowing you to create new instances. + +`cubicweb-dev` installs the development environment allowing you to develop new +cubes. -`cubicweb-dev` installs the development environment allowing you to -develop new cubes. +There is also a wide variety of cubes listed on the `CubicWeb.org Forge`_ +available as debian packages and tarball. -There is also a wide variety of cubes listed on http://www.cubicweb.org/Project available as debian packages and tarball. +The repositories are signed with `Logilab's gnupg key`_. To avoid warning on +"apt-get update": -The repositories are signed with `Logilab's gnupg key`_. To avoid warning on "apt-get update": 1. become root using sudo 2. download http://ftp.logilab.org/dists/logilab-dists-key.asc using e.g. wget 3. run "apt-key add logilab-dists-key.asc" 4. re-run apt-get update (manually or through the package manager, whichever you prefer) .. _`Logilab's gnupg key`: http://ftp.logilab.org/dists/logilab-dists-key.asc +.. _`CubicWeb.org Forge`: http://www.cubicweb.org/project/ + .. _SourceInstallation: @@ -66,6 +72,11 @@ Make sure you have installed the dependencies (see appendixes for the list). +|cubicweb| should soon be pip_ installable, stay tuned (expected in 3.8). + +.. _pip: http://pypi.python.org/pypi/pip + + Install from version control system ``````````````````````````````````` @@ -85,32 +96,31 @@ Make sure you have installed the dependencies (see appendixes for the list). + .. _WindowsInstallation: Windows installation ```````````````````` Base elements -_____________ +~~~~~~~~~~~~~ -Setting up a windows development environment is not too complicated -but requires a series of small steps. What is proposed there is only -an example of what can be done. We assume everything goes into C:\ in -this document. Adjusting the installation drive should be -straightforward. +Setting up a windows development environment is not too complicated but requires +a series of small steps. What is proposed there is only an example of what can be +done. We assume everything goes into `C:\\` in this document. Adjusting the +installation drive should be straightforward. -You should start by downloading and installing the Python(x,y) -distribution. It contains python 2.5 plus numerous useful third-party -modules and applications:: +You should start by downloading and installing the Python(x,y) distribution. It +contains python 2.5 plus numerous useful third-party modules and applications:: http://www.pythonxy.com/download_fr.php -At the time of this writting, one gets version 2.1.15. Among the many -things provided, one finds Eclipse + pydev (an arguably good IDE for -python under windows). +At the time of this writting, one gets version 2.1.15. Among the many things +provided, one finds Eclipse + pydev (an arguably good IDE for python under +windows). -Then you must grab Twisted. There is a windows installer directly -available from this page:: +Then you must grab Twisted. There is a windows installer directly available from +this page:: http://twistedmatrix.com/trac/ @@ -129,11 +139,18 @@ http://www.stickpeople.com/projects/python/win-psycopg/#Version2 -Please be careful to select the right python (2.5) and postgres (8.4) -versions. +Please be careful to select the right python (2.5) and postgres (8.4) versions. + +A windows compiled recent version of gettext:: + + http://ftp.logilab.org/pub/gettext/gettext-0.17-win32-setup.exe -Pyro enables remote access to cubicweb repository instances. Get it -there:: +A pre-compiled version of rql for windows (take care of retrieving the +most recent version available there):: + + http://ftp.logilab.org/pub/rql/rql-0.23.0.win32-py2.5.exe + +Pyro enables remote access to cubicweb repository instances. Get it there:: http://sourceforge.net/projects/pyro/files/ @@ -144,26 +161,26 @@ Check out the latest release. -Having graphviz will allow schema drawings, which is quite recommended -(albeit not mandatory). You should get an msi installer there:: +Having graphviz will allow schema drawings, which is quite recommended (albeit +not mandatory). You should get an msi installer there:: http://www.graphviz.org/Download_windows.php -Simplejson will be provided within the forest, but a win32 compiled -version will run much faster:: +Simplejson will be provided within the forest, but a win32 compiled version will +run much faster:: http://www.osuch.org/python-simplejson%3Awin32 Tools -_____ +~~~~~ -Get mercurial + its standard windows GUI (TortoiseHG) there (the -latest is the greatest):: +Get mercurial + its standard windows GUI (TortoiseHG) there (the latest is the +greatest):: http://bitbucket.org/tortoisehg/stable/wiki/download -If you need to peruse mercurial over ssh, it can be helpful to get an -ssh client like Putty:: +If you need to peruse mercurial over ssh, it can be helpful to get an ssh client +like Putty:: http://www.putty.org/ @@ -173,10 +190,9 @@ http://www.vectrace.com/mercurialeclipse/ Setting up the sources -______________________ +~~~~~~~~~~~~~~~~~~~~~~ -You need to enable the mercurial forest extension. To do this, edit -the file:: +You need to enable the mercurial forest extension. To do this, edit the file:: C:\Program Files\TortoiseHg\Mercurial.ini @@ -185,8 +201,8 @@ forest=C:\Program Files\TortoiseHg\ext\forest\forest.py Now, you need to clone the cubicweb repository. We assume that you use -Eclipse. From the IDE, choose File -> Import. In the box, select -`Mercurial/Clone repository using MercurialEclipse`. +Eclipse. From the IDE, choose File -> Import. In the box, select `Mercurial/Clone +repository using MercurialEclipse`. In the import main panel you just have to: @@ -194,28 +210,26 @@ * check the 'Repository is a forest' box. -Then, click on 'Finish'. It might take some time to get it all. Note -that the `cubicwin32` forest contains additional python packages such -as yapps, vobject, simplejson and twisted-web2 which are not provided -with Python(x,y). This is provided for convenience, as we do not -ensure the up-to-dateness of these packages, especially with respect -to security fixes. +Then, click on 'Finish'. It might take some time to get it all. Note that the +`cubicwin32` forest contains additional python packages such as yapps, vobject, +simplejson and twisted-web2 which are not provided with Python(x,y). This is +provided for convenience, as we do not ensure the up-to-dateness of these +packages, especially with respect to security fixes. Environment variables -_____________________ +~~~~~~~~~~~~~~~~~~~~~ -You will need some convenience environment variables once all is set -up. These variables are settable through the GUI by getting at the -'System properties' window (by righ-clicking on 'My Computer' -> -properties). +You will need some convenience environment variables once all is set up. These +variables are settable through the GUI by getting at the 'System properties' +window (by righ-clicking on 'My Computer' -> properties). -In the 'advanced' tab, there is an 'Environment variables' -button. Click on it. That opens a small window allowing edition of -user-related and system-wide variables. +In the 'advanced' tab, there is an 'Environment variables' button. Click on +it. That opens a small window allowing edition of user-related and system-wide +variables. -We will consider only user variables. First, the PATH variable. You -should ensure it contains, separated by semi-colons, and assuming you -are logged in as user Jane:: +We will consider only user variables. First, the PATH variable. You should ensure +it contains, separated by semi-colons, and assuming you are logged in as user +Jane:: C:\Documents and Settings\Jane\My Documents\Python\cubicweb\cubicweb\bin C:\Program Files\Graphviz2.24\bin @@ -231,13 +245,13 @@ ... and get a meaningful output. Running an instance as a service --------------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -This currently assumes that the instances configurations is located -at C:\\etc\\cubicweb.d. +This currently assumes that the instances configurations is located at +C:\\etc\\cubicweb.d. -For a cube 'my_cube', you will then find C:\\etc\\cubicweb.d\\my_cube\\win32svc.py -that has to be used thusly:: +For a cube 'my_cube', you will then find +C:\\etc\\cubicweb.d\\my_cube\\win32svc.py that has to be used thusly:: win32svc install @@ -248,17 +262,6 @@ should start the service. -PostgreSQL installation -``````````````````````` - -Please refer to the `PostgreSQL project online documentation`_. - -.. _`PostgreSQL project online documentation`: http://www.postgresql.org/ - -You need to install the three following packages: `postgresql-8.3`, -`postgresql-contrib-8.3` and `postgresql-plpython-8.3`. - - Other dependencies `````````````````` @@ -271,103 +274,115 @@ * `python-ldap` if you plan to use a LDAP source on the server -.. _ConfigurationEnv: -Environment configuration -------------------------- - -If you installed *CubicWeb* by cloning the Mercurial forest, then you -will need to update the environment variable PYTHONPATH by adding -the path to the forest ``cubicweb``: - -Add the following lines to either `.bashrc` or `.bash_profile` to configure -your development environment :: - - export PYTHONPATH=/full/path/to/cubicweb-forest - -If you installed *CubicWeb* with packages, no configuration is required and your -new cubes will be placed in `/usr/share/cubicweb/cubes` and your instances -will be placed in `/etc/cubicweb.d`. - -You may run a system-wide install of *CubicWeb* in "user mode" and use it for -development by setting the following environment variable:: - - export CW_MODE=user - export CW_CUBES_PATH=~/lib/cubes - export CW_INSTANCES_DIR=~/etc/cubicweb.d/ - export CW_INSTANCES_DATA_DIR=$CW_INSTANCES_DIR - export CW_RUNTIME_DIR=/tmp - -.. note:: - The values given above are our suggestions but of course - can be different. - +.. _DatabaseInstallation: Databases configuration ----------------------- -.. _ConfigurationPostgresql: +Whatever the backend used, database connection information are stored in the +instance's :file:`sources` file. Currently cubicweb has been tested using +Postgresql (recommanded), MySQL, SQLServer and SQLite. + +.. _PostgresqlConfiguration: PostgreSQL configuration ```````````````````````` -.. note:: - If you already have an existing cluster and PostgreSQL server - running, you do not need to execute the initilization step - of your PostgreSQL database. +For installation, please refer to the `PostgreSQL project online documentation`_. + +.. _`PostgreSQL project online documentation`: http://www.postgresql.org/ + +You need to install the three following packages: `postgresql-8.X`, +`postgresql-client-8.X`, and `postgresql-plpython-8.X`. If you run postgres +version prior to 8.3, you'll also need the `postgresql-contrib-8.X` package for +full-text search extension. -* First, initialize the database PostgreSQL with the command ``initdb``. +If you run postgres on another host than the |cubicweb| repository, you should +install the `postgresql-client` package on the |cubicweb| host, and others on the +database host. + +.. Note:: + + If you already have an existing cluster and PostgreSQL server running, you do + not need to execute the initilization step of your PostgreSQL database unless + you want a specific cluster for |cubicweb| databases or if your existing + cluster doesn't use the UTF8 encoding (see note below). + +* First, initialize a PostgreSQL cluster with the command ``initdb``. :: - $ initdb -D /path/to/pgsql + $ initdb -E UTF8 -D /path/to/pgsql + + Notice the encoding specification. This is necessary since |cubicweb| usually + want UTF8 encoded database. If you use a cluster with the wrong encoding, you'll + get error like:: - Once initialized, start the database server PostgreSQL - with the command:: + new encoding (UTF8) is incompatible with the encoding of the template database (SQL_ASCII) + HINT: Use the same encoding as in the template database, or use template0 as template. + + + Once initialized, start the database server PostgreSQL with the command:: $ postgres -D /path/to/psql - If you cannot execute this command due to permission issues, please - make sure that your username has write access on the database. - :: + If you cannot execute this command due to permission issues, please make sure + that your username has write access on the database. :: $ chown username /path/to/pgsql -* The database authentication can be either set to `ident sameuser` - or `md5`. - If set to `md5`, make sure to use an existing user - of your database. - If set to `ident sameuser`, make sure that your - client's operating system user name has a matching user in - the database. If not, please do as follow to create a user:: +* The database authentication can be either set to `ident sameuser` or `md5`. If + set to `md5`, make sure to use an existing user of your database. If set to + `ident sameuser`, make sure that your client's operating system user name has a + matching user in the database. If not, please do as follow to create a user:: $ su $ su - postgres $ createuser -s -P username - The option `-P` (for password prompt), will encrypt the password with - the method set in the configuration file ``pg_hba.conf``. - If you do not use this option `-P`, then the default value will be null - and you will need to set it with:: + The option `-P` (for password prompt), will encrypt the password with the + method set in the configuration file :file:`pg_hba.conf`. If you do not use this + option `-P`, then the default value will be null and you will need to set it + with:: $ su postgres -c "echo ALTER USER username WITH PASSWORD 'userpasswd' | psql" - This login/password will be requested when you will create an - instance with `cubicweb-ctl create` to initialize the database of - your instance. - -.. note:: - The authentication method can be configured in ``pg_hba.conf``. +.. Note:: + The authentication method can be configured in file:`pg_hba.conf`. -.. FIXME Are these steps really necessary? It seemed to work without. +The above login/password will be requested when you will create an instance with +`cubicweb-ctl create` to initialize the database of your instance. -* Installation of plain-text index extension :: +Notice that the `cubicweb-ctl db-create` does database initialization that +may requires a postgres superuser. That's why a login/password is explicitly asked +at this step, so you can use there a superuser without using this user when running +the instance. Things that require special privileges at this step: + +* database creation, require the 'create database' permission +* install the plpython extension language (require superuser) +* install the tsearch extension for postgres version prior to 8.3 (require superuser) - cat /usr/share/postgresql/8.3/contrib/tsearch2.sql | psql -U username template1 +To avoid using a super user each time you create an install, a nice trick is to +install plpython (and tsearch when needed) on the special `template1` database, +so they will be installed automatically when cubicweb databases are created +without even with needs for special access rights. To do so, run :: + + # Installation of plpythonu language by default :: + $ createlang -U pgadmin plpythonu template1 + $ psql -U pgadmin template1 + template1=# update pg_language set lanpltrusted=TRUE where lanname='plpythonu'; -* Installation of plpythonu language by default :: +Where `pgadmin` is a postgres superuser. The last command is necessary since by +default plpython is an 'untrusted' language and as such can't be used by non +superuser. This update fix that problem by making it trusted. - createlang -U pgadmin plpythonu template1 +To install the tsearch plain-text index extension on postgres prior to 8.3, run:: + + cat /usr/share/postgresql/8.X/contrib/tsearch2.sql | psql -U username template1 + + +.. _MySqlConfiguration: MySql configuration ``````````````````` @@ -378,19 +393,22 @@ default-character-set=utf8 max_allowed_packet = 128M -.. note:: +.. Note:: It is unclear whether mysql supports indexed string of arbitrary lenght or not. + +.. _SQLServerConfiguration: + SQLServer configuration ------------------------ +``````````````````````` -As of this writing, sqlserver support is in progress. You should be -able to connect, create a database and go quite far, but some of the -generated SQL is still currently not accepted by the backend. +As of this writing, sqlserver support is in progress. You should be able to +connect, create a database and go quite far, but some of the generated SQL is +still currently not accepted by the backend. -The `source` configuration file may look like this (specific parts -only are shown):: +The `source` configuration file may look like this (specific parts only are +shown):: [system] db-driver=sqlserver2005 @@ -402,17 +420,40 @@ db-encoding=utf8 + +.. _SQLiteConfiguration: + +SQLite configuration +```````````````````` +SQLite has the great advantage of requiring almost no configuration. Simply +use 'sqlite' as db-driver, and set path to the dabase as db-name. Don't specify +anything for db-user and db-password, they will be ignore anyway. + +.. Note:: + SQLite is great for testing and to play with cubicweb but is not suited for + production environments. + + +.. _PyroConfiguration: + Pyro configuration ------------------ -If you use Pyro, it is required to have a name server Pyro running on your -network (by default it is detected by a broadcast request). +If you want to use Pyro to access your instance remotly, or to have multi-source +or distributed configuration, it is required to have a name server Pyro running +on your network. By by default it is detected by a broadcast request, but you can +specify a location in the instance's configuration file. To do so, you need to : -* launch the server manually before starting cubicweb as a server with - `pyro-nsd start` +* launch the server manually before starting cubicweb as a server with `pyro-nsd + start` + +* under debian, edit the file :file:`/etc/default/pyro-nsd` so that the name + server pyro will be launched automatically when the machine fire up -* edit the file ``/etc/default/pyro-nsd`` so that the name server pyro - will be launched automatically when the machine fire up +Cubicweb resources configuration +-------------------------------- + +.. autodocstring:: cubicweb.cwconfig diff -r 02b52bf9f5f8 -r 0865e1e90674 doc/book/en/admin/site-config.rst --- a/doc/book/en/admin/site-config.rst Wed Mar 24 10:23:31 2010 +0100 +++ b/doc/book/en/admin/site-config.rst Wed Apr 28 11:54:13 2010 +0200 @@ -3,7 +3,7 @@ User interface for web site configuration ========================================= -.. image:: ../images/lax-book.03-site-config-panel.en.png +.. image:: ../images/lax-book_03-site-config-panel_en.png This panel allows you to configure the appearance of your instance site. Six menus are available and we will go through each of them to explain how diff -r 02b52bf9f5f8 -r 0865e1e90674 doc/book/en/annexes/cookbook.rst --- a/doc/book/en/annexes/cookbook.rst Wed Mar 24 10:23:31 2010 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,85 +0,0 @@ -.. -*- coding: utf-8 -*- - -Cook book -========= - -We gathered together some of our tricks and scripts that could make -life easier. - - -* How to import LDAP users in *CubicWeb*? - - [XXX distribute this script with cubicweb instead] - - Here is a very useful script which enables you to import LDAP users - into your *CubicWeb* instance by running the following: - -.. sourcecode:: python - - import os - import pwd - import sys - - from logilab.common.db import get_connection - - def getlogin(): - """avoid usinng os.getlogin() because of strange tty / stdin problems - (man 3 getlogin) - Another solution would be to use $LOGNAME, $USER or $USERNAME - """ - return pwd.getpwuid(os.getuid())[0] - - - try: - database = sys.argv[1] - except IndexError: - print 'USAGE: python ldap2system.py ' - sys.exit(1) - - if raw_input('update %s db ? [y/n]: ' % database).strip().lower().startswith('y'): - cnx = get_connection(user=getlogin(), database=database) - cursor = cnx.cursor() - - insert = ('INSERT INTO euser (creation_date, eid, modification_date, login, firstname, surname, last_login_time, upassword) ' - "VALUES (%(mtime)s, %(eid)s, %(mtime)s, %(login)s, %(firstname)s, %(surname)s, %(mtime)s, './fqEz5LeZnT6');") - update = "UPDATE entities SET source='system' WHERE eid=%(eid)s;" - cursor.execute("SELECT eid,type,source,extid,mtime FROM entities WHERE source!='system'") - for eid, type, source, extid, mtime in cursor.fetchall(): - if type != 'CWUser': - print "don't know what to do with entity type", type - continue - if source != 'ldapuser': - print "don't know what to do with source type", source - continue - ldapinfos = dict(x.strip().split('=') for x in extid.split(',')) - login = ldapinfos['uid'] - firstname = ldapinfos['uid'][0].upper() - surname = ldapinfos['uid'][1:].capitalize() - if login != 'jcuissinat': - args = dict(eid=eid, type=type, source=source, login=login, - firstname=firstname, surname=surname, mtime=mtime) - print args - cursor.execute(insert, args) - cursor.execute(update, args) - - cnx.commit() - cnx.close() - - -* How to load data from a script? - - The following script aims at loading data within a script assuming pyro-nsd is - running and your instance is configured with ``pyro-server=yes``, otherwise - you would not be able to use dbapi. - -.. sourcecode:: python - - from cubicweb import dbapi - - cnx = dbapi.connection(database='instance-id', user='admin', password='admin') - cur = cnx.cursor() - for name in ('Personal', 'Professional', 'Computers'): - cur.execute('INSERT Blog B: B name %s', name) - cnx.commit() - - diff -r 02b52bf9f5f8 -r 0865e1e90674 doc/book/en/annexes/faq.rst --- a/doc/book/en/annexes/faq.rst Wed Mar 24 10:23:31 2010 +0100 +++ b/doc/book/en/annexes/faq.rst Wed Apr 28 11:54:13 2010 +0200 @@ -6,11 +6,26 @@ [XXX 'copy answer from forum' means reusing text from http://groups.google.com/group/google-appengine/browse_frm/thread/c9476925f5f66ec6 and -http://groups.google.com/group/google-appengine/browse_frm/thread/d791ce17e2716147/eb078f8cfe8426e0 -and http://groups.google.com/group/google-appengine/browse_frm/thread/f48cf6099973aef5/c28cd6934dd72457 ] +Generalities +```````````` + +Why do you use the LGPL license to prevent me from doing X ? +------------------------------------------------------------ + +LGPL means that *if* you redistribute your application, you need to +redistribute the changes you made to CubicWeb under the LGPL licence. + +Publishing a web site has nothing to do with redistributing source +code according to the terms of the LGPL. A fair amount of companies +use modified LGPL code for internal use. And someone could publish a +*CubicWeb* component under a BSD licence for others to plug into a +LGPL framework without any problem. The only thing we are trying to +prevent here is someone taking the framework and packaging it as +closed source to his own clients. + Why does not CubicWeb have a template language ? ------------------------------------------------ @@ -26,8 +41,13 @@ Python is the templating language that we use in *CubicWeb*, but again, it does not prevent you from using a templating language. -The reason template languages are not used in this book is that -experience has proved us that using pure python was less cumbersome. +Moreover, CubicWeb currently supports `simpletal`_ out of the box and +it is also possible to use the `cwtags`_ library to build html trees +using the `with statement`_ with more comfort than raw strings. + +.. _`simpletal`: http://www.owlfish.com/software/simpleTAL/ +.. _`cwtags`: http://www.cubicweb.org/project/cwtags +.. _`with statement`: http://www.python.org/dev/peps/pep-0343/ Why do you think using pure python is better than using a template language ? ----------------------------------------------------------------------------- @@ -36,33 +56,11 @@ already provides a consistent and strong architecture and syntax a templating language would not reach. -When doing development, you need a real language and template -languages are not real languages. - Using Python instead of a template langage for describing the user interface makes it to maintain with real functions/classes/contexts without the need of learning a new dialect. By using Python, we use standard OOP techniques and this is a key factor in a robust application. -The `cwtags` (http://www.cubicweb.org/project/cwtags) package can be -used in cubes to help generate html from Python with more comfort than -raw strings. - -Why do you use the LGPL license to prevent me from doing X ? ------------------------------------------------------------- - -LGPL means that *if* you redistribute your application, you need to -redistribute the changes you made to CubicWeb under the LGPL licence. - -Publishing a web site has nothing to do with redistributing -source code. A fair amount of companies use modified LGPL code -for internal use. And someone could publish a *CubicWeb* component -under a BSD licence for others to plug into a LGPL framework without -any problem. The only thing we are trying to prevent here is someone -taking the framework and packaging it as closed source to his own -clients. - - CubicWeb looks pretty recent. Is it stable ? -------------------------------------------- @@ -70,6 +68,10 @@ 2001 and data has been migrated from one schema to the other ever since. There is a well-defined way to handle data and schema migration. +You can see the roadmap there: +http://www.cubicweb.org/project/cubicweb?tab=projectroadmap_tab. + + Why is the RQL query language looking similar to X ? ----------------------------------------------------- @@ -77,7 +79,7 @@ SPARQL. Except that SPARQL did not exist when we started the project. With version 3.4, CubicWeb has support for SPARQL. -That RQL language is what is going to make a difference with django- +The RQL language is what is going to make a difference with django- like frameworks for several reasons. 1. accessing data is *much* easier with it. One can write complex @@ -99,6 +101,315 @@ that. Additionally, some jQuery plugins are provided (some are provided in specific cubes). +Development +``````````` + +How to load data from a script ? +-------------------------------- + +The following script aims at loading data within a script assuming pyro-nsd is +running and your instance is configured with ``pyro-server=yes``, otherwise +you would not be able to use dbapi. + +.. sourcecode:: python + + from cubicweb import dbapi + + cnx = dbapi.connection(database='instance-id', user='admin', password='admin') + cur = cnx.cursor() + for name in ('Personal', 'Professional', 'Computers'): + cur.execute('INSERT Blog B: B name %s', name) + cnx.commit() + + +How to format an entity date attribute ? +---------------------------------------- + +If your schema has an attribute of type Date or Datetime, you might +want to format it. First, you should define your preferred format using +the site configuration panel ``http://appurl/view?vid=systempropertiesform`` +and then set ``ui.date`` and/or ``ui.datetime``. +Then in the view code, use: + +.. sourcecode:: python + + self.format_date(entity.date_attribute) + +What is the CubicWeb datatype corresponding to GAE datastore's UserProperty ? +----------------------------------------------------------------------------- + +If you take a look at your instance schema and +click on "display detailed view of metadata" you will see that there +is a Euser entity in there. That's the one that is modeling users. The +thing that corresponds to a UserProperty is a relationship between +your entity and the Euser entity. As in: + +.. sourcecode:: python + + class TodoItem(EntityType): + text = String() + todo_by = SubjectRelation('Euser') + +[XXX check that cw handle users better by mapping Google Accounts to local Euser +entities automatically] + + +How do I translate an msg id defined (and translated) in another cube ? +----------------------------------------------------------------------- + +You should put these translations in the `i18n/static-messages.pot` +file of your own cube. + + +What is `Error while publishing rest text ...` ? +------------------------------------------------ + +While modifying the description of an entity, you get an error message in +the instance `Error while publishing ...` for Rest text and plain text. +The server returns a traceback like as follows :: + + 2008-10-06 15:05:08 - (cubicweb.rest) ERROR: error while publishing ReST text + Traceback (most recent call last): + File "/home/user/src/blogdemo/cubicweb/common/rest.py", line 217, in rest_publish + File "/usr/lib/python2.5/codecs.py", line 817, in open + file = __builtin__.open(filename, mode, buffering) + TypeError: __init__() takes at most 3 arguments (4 given) + +This can be fixed by applying the patch described in : +http://code.google.com/p/googleappengine/issues/detail?id=48 + +What are hooks used for ? +------------------------- + +Hooks are executed around (actually before or after) events. The +most common events are data creation, update and deletion. They +permit additional constraint checking (those not expressible at the +schema level), pre and post computations depending on data +movements. + +As such, they are a vital part of the framework. + +Other kinds of hooks, called Operations, are available +for execution just before commit. + +When should you define an HTML template rather than define a graphical component ? +---------------------------------------------------------------------------------- + +An HTML template cannot contain code, hence it is only about static +content. A component is made of code and operations that apply on a +well defined context (request, result set). It enables much more +dynamic views. + +How to update a database after a schema modification ? +------------------------------------------------------ + +It depends on what has been modified in the schema. + +* update the permissions and properties of an entity or a relation: + ``sync_schema_props_perms('MyEntityOrRelation')``. + +* add an attribute: ``add_attribute('MyEntityType', 'myattr')``. + +* add a relation: ``add_relation_definition('SubjRelation', 'MyRelation', 'ObjRelation')``. + + +How to create an anonymous user ? +--------------------------------- + +This allows to bypass authentication for your site. In the +``all-in-one.conf`` file of your instance, define the anonymous user +as follows :: + + # login of the CubicWeb user account to use for anonymous user (if you want to + # allow anonymous) + anonymous-user=anon + + # password of the CubicWeb user account matching login + anonymous-password=anon + +You also must ensure that this `anon` user is a registered user of +the DB backend. If not, you can create through the administation +interface of your instance by adding a user with the role `guests`. +This could be the admin account (for development +purposes, of course). + +.. note:: + While creating a new instance, you can decide to allow access + to anonymous user, which will automatically execute what is + decribed above. + + +How to change the instance logo ? +------------------------------------ + +There are two ways of changing the logo. + +1. The easiest way to use a different logo is to replace the existing + ``logo.png`` in ``myapp/data`` by your prefered icon and refresh. + By default all instance will look for a ``logo.png`` to be + rendered in the logo section. + + .. image:: ../images/lax-book_06-main-template-logo_en.png + +2. In your cube directory, you can specify which file to use for the logo. + This is configurable in ``mycube/data/external_resources``: :: + + LOGO = DATADIR/path/to/mylogo.gif + + where DATADIR is ``mycube/data``. + +Configuration +````````````` + +How to configure a LDAP source ? +-------------------------------- + +Your instance's sources are defined in ``/etc/cubicweb.d/myapp/sources``. +Configuring an LDAP source is about declaring that source in your +instance configuration file such as: :: + + [ldapuser] + adapter=ldapuser + # ldap host + host=myhost + # base DN to lookup for usres + user-base-dn=ou=People,dc=mydomain,dc=fr + # user search scope + user-scope=ONELEVEL + # classes of user + user-classes=top,posixAccount + # attribute used as login on authentication + user-login-attr=uid + # name of a group in which ldap users will be by default + user-default-group=users + # map from ldap user attributes to cubicweb attributes + user-attrs-map=gecos:email,uid:login + +Any change applied to configuration file requires to restart your +instance. + +You can find additional information in the section :ref:`LDAP`. + +How to import LDAP users in |cubicweb| ? +---------------------------------------- + + Here is a useful script which enables you to import LDAP users + into your *CubicWeb* instance by running the following: + +.. sourcecode:: python + + import os + import pwd + import sys + + from logilab.common.db import get_connection + + def getlogin(): + """avoid usinng os.getlogin() because of strange tty / stdin problems + (man 3 getlogin) + Another solution would be to use $LOGNAME, $USER or $USERNAME + """ + return pwd.getpwuid(os.getuid())[0] + + + try: + database = sys.argv[1] + except IndexError: + print 'USAGE: python ldap2system.py ' + sys.exit(1) + + if raw_input('update %s db ? [y/n]: ' % database).strip().lower().startswith('y'): + cnx = get_connection(user=getlogin(), database=database) + cursor = cnx.cursor() + + insert = ('INSERT INTO euser (creation_date, eid, modification_date, login, ' + ' firstname, surname, last_login_time, upassword) ' + "VALUES (%(mtime)s, %(eid)s, %(mtime)s, %(login)s, %(firstname)s, " + "%(surname)s, %(mtime)s, './fqEz5LeZnT6');") + update = "UPDATE entities SET source='system' WHERE eid=%(eid)s;" + cursor.execute("SELECT eid,type,source,extid,mtime FROM entities WHERE source!='system'") + for eid, type, source, extid, mtime in cursor.fetchall(): + if type != 'CWUser': + print "don't know what to do with entity type", type + continue + if source != 'ldapuser': + print "don't know what to do with source type", source + continue + ldapinfos = dict(x.strip().split('=') for x in extid.split(',')) + login = ldapinfos['uid'] + firstname = ldapinfos['uid'][0].upper() + surname = ldapinfos['uid'][1:].capitalize() + if login != 'jcuissinat': + args = dict(eid=eid, type=type, source=source, login=login, + firstname=firstname, surname=surname, mtime=mtime) + print args + cursor.execute(insert, args) + cursor.execute(update, args) + + cnx.commit() + cnx.close() + + +I get NoSelectableObject exceptions, how do I debug selectors ? +--------------------------------------------------------------- + +You just need to put the appropriate context manager around view/component +selection (one standard place in in vreg.py): + +.. sourcecode:: python + + def possible_objects(self, registry, *args, **kwargs): + """return an iterator on possible objects in a registry for this result set + + actions returned are classes, not instances + """ + from cubicweb.selectors import traced_selection + with traced_selection(): + for vobjects in self.registry(registry).values(): + try: + yield self.select(vobjects, *args, **kwargs) + except NoSelectableObject: + continue + +Don't forget the 'from __future__ import with_statement' at the module +top-level. + +This will yield additional WARNINGs, like this:: + + 2009-01-09 16:43:52 - (cubicweb.selectors) WARNING: selector one_line_rset returned 0 for + +Security +```````` + +How to reset the password for user joe ? +---------------------------------------- + +If you want to reset the admin password for ``myinstance``, do:: + + $ cubicweb-ctl reset-admin-pwd myinstance + +You need to generate a new encrypted password:: + + $ python + >>> from cubicweb.server.utils import crypt_password + >>> crypt_password('joepass') + 'qHO8282QN5Utg' + >>> + +and paste it in the database:: + + $ psql mydb + mydb=> update cw_cwuser set cw_upassword='qHO8282QN5Utg' where cw_login='joe'; + UPDATE 1 + +I've just created a user in a group and it doesn't work ! +--------------------------------------------------------- + +You are probably getting errors such as :: + + remove {'PR': 'Project', 'C': 'CWUser'} from solutions since your_user has no read access to cost + +This is because you have to put your user in the "users" group. The user has to be in both groups. How is security implemented ? ------------------------------ @@ -152,181 +463,6 @@ You can find additional information in the section :ref:`securitymodel`. - -What is `Error while publishing rest text ...` ? ------------------------------------------------- - -While modifying the description of an entity, you get an error message in -the instance `Error while publishing ...` for Rest text and plain text. -The server returns a traceback like as follows :: - - 2008-10-06 15:05:08 - (cubicweb.rest) ERROR: error while publishing ReST text - Traceback (most recent call last): - File "/home/user/src/blogdemo/cubicweb/common/rest.py", line 217, in rest_publish - File "/usr/lib/python2.5/codecs.py", line 817, in open - file = __builtin__.open(filename, mode, buffering) - TypeError: __init__() takes at most 3 arguments (4 given) - -This can be fixed by applying the patch described in : -http://code.google.com/p/googleappengine/issues/detail?id=48 - -What are hooks used for ? -------------------------- - -Hooks are executed around (actually before or after) events. The -most common events are data creation, update and deletion. They -permit additional constraint checking (those not expressible at the -schema level), pre and post computations depending on data -movements. - -As such, they are a vital part of the framework. - -Other kinds of hooks, called Operations, are available -for execution just before commit. - -When should you define an HTML template rather than define a graphical component ? ----------------------------------------------------------------------------------- - -An HTML template cannot contain code, hence it is only about static -content. A component is made of code and operations that apply on a -well defined context (request, result set). It enables much more -dynamic views. - -What is the difference between `AppRsetObject` and `AppObject` ? ----------------------------------------------------------------- - -`AppRsetObject` instances are selected on a request and a result -set. `AppObject` instances are directly selected by id. - -How to update a database after a schema modification ? ------------------------------------------------------- - -It depends on what has been modified in the schema. - -* Update the permissions and properties of an entity or a relation: - ``sync_schema_props_perms('MyEntityOrRelation')``. - -* Add an attribute: ``add_attribute('MyEntityType', 'myattr')``. - -* Add a relation: ``add_relation_definition('SubjRelation', 'MyRelation', 'ObjRelation')``. - - -How to create an anonymous user ? ---------------------------------- - -This allows to bypass authentication for your site. In the -``all-in-one.conf`` file of your instance, define the anonymous user -as follows :: - - # login of the CubicWeb user account to use for anonymous user (if you want to - # allow anonymous) - anonymous-user=anon - - # password of the CubicWeb user account matching login - anonymous-password=anon - -You also must ensure that this `anon` user is a registered user of -the DB backend. If not, you can create through the administation -interface of your instance by adding a user with the role `guests`. -This could be the admin account (for development -purposes, of course). - -.. note:: - While creating a new instance, you can decide to allow access - to anonymous user, which will automatically execute what is - decribed above. - - -How to change the instance logo ? ------------------------------------- - -There are two ways of changing the logo. - -1. The easiest way to use a different logo is to replace the existing - ``logo.png`` in ``myapp/data`` by your prefered icon and refresh. - By default all instance will look for a ``logo.png`` to be - rendered in the logo section. - - .. image:: ../images/lax-book.06-main-template-logo.en.png - -2. In your cube directory, you can specify which file to use for the logo. - This is configurable in ``mycube/data/external_resources``: :: - - LOGO = DATADIR/path/to/mylogo.gif - - where DATADIR is ``mycube/data``. - - -How to configure a LDAP source ? --------------------------------- - -Your instance's sources are defined in ``/etc/cubicweb.d/myapp/sources``. -Configuring an LDAP source is about declaring that source in your -instance configuration file such as: :: - - [ldapuser] - adapter=ldapuser - # ldap host - host=myhost - # base DN to lookup for usres - user-base-dn=ou=People,dc=mydomain,dc=fr - # user search scope - user-scope=ONELEVEL - # classes of user - user-classes=top,posixAccount - # attribute used as login on authentication - user-login-attr=uid - # name of a group in which ldap users will be by default - user-default-group=users - # map from ldap user attributes to cubicweb attributes - user-attrs-map=gecos:email,uid:login - -Any change applied to configuration file requires to restart your -instance. - -You can find additional information in the section :ref:`LDAP`. - -I get NoSelectableObject exceptions, how do I debug selectors ? ---------------------------------------------------------------- - -You just need to put the appropriate context manager around view/component -selection (one standard place in in vreg.py): - -.. sourcecode:: python - - def possible_objects(self, registry, *args, **kwargs): - """return an iterator on possible objects in a registry for this result set - - actions returned are classes, not instances - """ - from cubicweb.selectors import traced_selection - with traced_selection(): - for vobjects in self.registry(registry).values(): - try: - yield self.select(vobjects, *args, **kwargs) - except NoSelectableObject: - continue - -Don't forget the 'from __future__ import with_statement' at the module -top-level. - -This will yield additional WARNINGs, like this:: - - 2009-01-09 16:43:52 - (cubicweb.selectors) WARNING: selector one_line_rset returned 0 for - -How to format an entity date attribute ? ----------------------------------------- - -If your schema has an attribute of type Date or Datetime, you might -want to format it. First, you should define your preferred format using -the site configuration panel ``http://appurl/view?vid=systempropertiesform`` -and then set ``ui.date`` and/or ``ui.datetime``. -Then in the view code, use: - -.. sourcecode:: python - - self.format_date(entity.date_attribute) - Can PostgreSQL and CubicWeb authentication work with kerberos ? ---------------------------------------------------------------- @@ -336,68 +472,3 @@ instance to connect to postgresql with a kerberos ticket. -How to load data from a script ? --------------------------------- - -The following script aims at loading data within a script assuming pyro-nsd is -running and your instance is configured with ``pyro-server=yes``, otherwise -you would not be able to use dbapi. - -.. sourcecode:: python - - from cubicweb import dbapi - - cnx = dbapi.connection(database='instance-id', user='admin', password='admin') - cur = cnx.cursor() - for name in ('Personal', 'Professional', 'Computers'): - cur.execute('INSERT Blog B: B name %s', name) - cnx.commit() - -What is the CubicWeb datatype corresponding to GAE datastore's UserProperty ? ------------------------------------------------------------------------------ - -If you take a look at your instance schema and -click on "display detailed view of metadata" you will see that there -is a Euser entity in there. That's the one that is modeling users. The -thing that corresponds to a UserProperty is a relationship between -your entity and the Euser entity. As in: - -.. sourcecode:: python - - class TodoItem(EntityType): - text = String() - todo_by = SubjectRelation('Euser') - -[XXX check that cw handle users better by mapping Google Accounts to local Euser -entities automatically] - - -How to reset the password for user joe ? ----------------------------------------- - -If you want to reset the admin password for ``myinstance``, do:: - - $ cubicweb-ctl reset-admin-pwd myinstance - -You need to generate a new encrypted password:: - - $ python - >>> from cubicweb.server.utils import crypt_password - >>> crypt_password('joepass') - 'qHO8282QN5Utg' - >>> - -and paste it in the database:: - - $ psql mydb - mydb=> update cw_cwuser set cw_upassword='qHO8282QN5Utg' where cw_login='joe'; - UPDATE 1 - -I've just created a user in a group and it doesn't work ! ---------------------------------------------------------- - -You are probably getting errors such as :: - - remove {'PR': 'Project', 'C': 'CWUser'} from solutions since your_user has no read access to cost - -This is because you have to put your user in the "users" group. The user has to be in both groups. diff -r 02b52bf9f5f8 -r 0865e1e90674 doc/book/en/annexes/index.rst --- a/doc/book/en/annexes/index.rst Wed Mar 24 10:23:31 2010 +0100 +++ b/doc/book/en/annexes/index.rst Wed Apr 28 11:54:13 2010 +0200 @@ -2,9 +2,9 @@ .. _Part4: --------------------- -Part IV - Appendixes --------------------- +---------- +Appendixes +---------- The following chapters are reference material. @@ -13,7 +13,6 @@ :numbered: faq - cookbook cubicweb-ctl rql/index mercurial diff -r 02b52bf9f5f8 -r 0865e1e90674 doc/book/en/annexes/rql/implementation.rst --- a/doc/book/en/annexes/rql/implementation.rst Wed Mar 24 10:23:31 2010 +0100 +++ b/doc/book/en/annexes/rql/implementation.rst Wed Apr 28 11:54:13 2010 +0200 @@ -2,6 +2,7 @@ Implementation -------------- + BNF grammar ~~~~~~~~~~~ @@ -126,8 +127,8 @@ ~~~~~~~~~~~~~~~~~ - The current implementation does not support linking two relations of type 'is' - with a OR. I do not think that the negation is supported on this type of - relation (XXX FIXME to be confirmed). + with an OR. I do not think that the negation is supported on this type of + relation (XXX to be confirmed). - Relations defining the variables must be left to those using them. For example:: @@ -140,11 +141,11 @@ is not. -- missing proper explicit type conversion, COALESCE and certainly other things... +- missing proper explicit type conversion, COALESCE and certainly other things... -- writing a rql query require knowledge of the schema used (with real relation - names and entities, not those viewing in the user interface). On the other - hand, we can not really bypass that, and it is the job of a user interface to +- writing an rql query requires knowledge of the used schema (with real relation + names and entities, not those viewed in the user interface). On the other + hand, we cannot really bypass that, and it is the job of a user interface to hide the RQL. diff -r 02b52bf9f5f8 -r 0865e1e90674 doc/book/en/annexes/rql/intro.rst --- a/doc/book/en/annexes/rql/intro.rst Wed Mar 24 10:23:31 2010 +0100 +++ b/doc/book/en/annexes/rql/intro.rst Wed Apr 28 11:54:13 2010 +0200 @@ -1,3 +1,5 @@ + +.. _rql_intro: Introduction ------------ @@ -5,10 +7,10 @@ Goals of RQL ~~~~~~~~~~~~ -The goal is to have a language emphasizing the way of browsing relations. As -such, attributes will be regarded as cases of special relations (in terms of -implementation, the user should see no difference between an attribute and a -relation). +The goal is to have a language making relations browsing easy. As +such, attributes will be regarded as cases of special relations (in +terms of usage, the user should see no syntactic difference between an +attribute and a relation). Comparison with existing languages ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -32,10 +34,10 @@ We should look in more detail, but here are already some ideas for the moment ... Versa_ is the language most similar to what we wanted to do, but the model -underlying data being RDF, there is some number of things such as namespaces or +underlying data being RDF, there are some things such as namespaces or handling of the RDF types which does not interest us. On the functionality level, Versa_ is very comprehensive including through many functions of -conversion and basic types manipulation, which may need to be guided at one time +conversion and basic types manipulation, which we may want to look at one time or another. Finally, the syntax is a little esoteric. diff -r 02b52bf9f5f8 -r 0865e1e90674 doc/book/en/annexes/rql/language.rst --- a/doc/book/en/annexes/rql/language.rst Wed Mar 24 10:23:31 2010 +0100 +++ b/doc/book/en/annexes/rql/language.rst Wed Apr 28 11:54:13 2010 +0200 @@ -92,8 +92,7 @@ Any X WHERE X name IN ( 'chauvat', 'fayolle', 'di mascio', 'thenault') -XXX nico: "A trick <> 'bar'" wouldn't it be more convenient than -"NOT A trick 'bar'" ? +.. XXX nico: "A trick <> 'bar'" wouldn't it be more convenient than "NOT A trick 'bar'" ? .. _PriorityOperators: @@ -373,3 +372,15 @@ DELETE X friend Y WHERE X is Person, X name 'foo' +Virtual RQL relations +~~~~~~~~~~~~~~~~~~~~~ + +Those relations may only be used in RQL query and are not actual +attributes of your entities. + +* `has_text`: relation to use to query the full text index (only for + entities having fulltextindexed attributes). + +* `identity`: relation to use to tell that a RQL variable should be + the same as another (but you've to use two different rql variables + for querying purpose) diff -r 02b52bf9f5f8 -r 0865e1e90674 doc/book/en/conf.py --- a/doc/book/en/conf.py Wed Mar 24 10:23:31 2010 +0100 +++ b/doc/book/en/conf.py Wed Apr 28 11:54:13 2010 +0200 @@ -1,10 +1,23 @@ # -*- coding: utf-8 -*- +# copyright 2003-2010 LOGILAB S.A. (Paris, FRANCE), all rights reserved. +# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr +# +# This file is part of CubicWeb. +# +# CubicWeb is free software: you can redistribute it and/or modify it under the +# terms of the GNU Lesser General Public License as published by the Free +# Software Foundation, either version 2.1 of the License, or (at your option) +# any later version. +# +# logilab-common is distributed in the hope that it will be useful, but WITHOUT +# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS +# FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more +# details. +# +# You should have received a copy of the GNU Lesser General Public License along +# with CubicWeb. If not, see . """ -:organization: Logilab -:copyright: 2001-2010 LOGILAB S.A. (Paris, FRANCE), license is LGPL v2. -:contact: http://www.logilab.fr/ -- mailto:contact@logilab.fr -:license: GNU Lesser General Public License, v2.1 - http://www.gnu.org/licenses """ # # Cubicweb documentation build configuration file, created by @@ -19,6 +32,7 @@ # serve to show the default value. import sys, os + from cubicweb import __pkginfo__ as cw # If your extensions are in another directory, add it here. If the directory @@ -31,10 +45,10 @@ # Add any Sphinx extension module names here, as strings. They can be extensions # coming with Sphinx (named 'sphinx.ext.*') or your custom ones. -extensions = ['sphinx.ext.autodoc'] +extensions = ['sphinx.ext.autodoc', 'logilab.common.sphinx_ext'] autoclass_content = 'both' # Add any paths that contain templates here, relative to this directory. -templates_path = ['.templates'] +#templates_path = [] # The suffix of source filenames. source_suffix = '.rst' @@ -44,7 +58,7 @@ # General substitutions. project = 'CubicWeb' -copyright = '2008-2010, Logilab' +copyright = '2001-2010, Logilab' # The default replacements for |version| and |release|, also used in various # other places throughout the built documents. @@ -91,11 +105,13 @@ # The style sheet to use for HTML and HTML Help pages. A file of that name # must exist either in Sphinx' static/ path, or in one of the custom paths # given in html_static_path. -html_style = 'sphinx-default.css' +#html_style = 'sphinx-default.css' # The name for this set of Sphinx documents. If None, it defaults to # " v documentation". html_title = '%s %s' % (project, release) +html_theme = 'standard_theme' +html_theme_path = ['.'] # A shorter title for the navigation bar. Default is the same as html_title. #html_short_title = None @@ -185,3 +201,11 @@ # If false, no module index is generated. #latex_use_modindex = True + +#aafig_format = dict(latex='pdf', html='svg', text=None) + +rst_epilog = """ +.. |cubicweb| replace:: *CubicWeb* +.. |yams| replace:: *Yams* +.. |rql| replace:: *RQL* +""" diff -r 02b52bf9f5f8 -r 0865e1e90674 doc/book/en/development/cubes/available-cubes.rst --- a/doc/book/en/development/cubes/available-cubes.rst Wed Mar 24 10:23:31 2010 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,64 +0,0 @@ - -Available cubes ---------------- - -An instance is based on several basic cubes. In the set of available -basic cubes we can find for example : - -Base entity types -~~~~~~~~~~~~~~~~~ -* addressbook_: PhoneNumber and PostalAddress -* card_: Card, generic documenting card -* event_: Event (define events, display them in calendars) -* file_: File (to allow users to upload and store binary or text files) -* link_: Link (to collect links to web resources) -* mailinglist_: MailingList (to reference a mailing-list and the URLs - for its archives and its admin interface) -* person_: Person (easily mixed with addressbook) -* task_: Task (something to be done between start and stop date) -* zone_: Zone (to define places within larger places, for example a - city in a state in a country) - - -Classification -~~~~~~~~~~~~~~ -* folder_: Folder (to organize things but grouping them in folders) -* keyword_: Keyword (to define classification schemes) -* tag_: Tag (to tag anything) - -Other features -~~~~~~~~~~~~~~ -* basket_: Basket (like a shopping cart) -* blog_: a blogging system uxing Blog and BlogEntry entity types -* comment_: system to attach comment threads to entities) -* email_: archiving management for emails (`Email`, `Emailpart`, - `Emailthread`), trigger action in cubicweb through email - - - - - -.. _addressbook: http://www.cubicweb.org/project/cubicweb-addressbook -.. _basket: http://www.cubicweb.org/project/cubicweb-basket -.. _card: http://www.cubicweb.org/project/cubicweb-card -.. _blog: http://www.cubicweb.org/project/cubicweb-blog -.. _comment: http://www.cubicweb.org/project/cubicweb-comment -.. _email: http://www.cubicweb.org/project/cubicweb-email -.. _event: http://www.cubicweb.org/project/cubicweb-event -.. _file: http://www.cubicweb.org/project/cubicweb-file -.. _folder: http://www.cubicweb.org/project/cubicweb-folder -.. _keyword: http://www.cubicweb.org/project/cubicweb-keyword -.. _link: http://www.cubicweb.org/project/cubicweb-link -.. _mailinglist: http://www.cubicweb.org/project/cubicweb-mailinglist -.. _person: http://www.cubicweb.org/project/cubicweb-person -.. _tag: http://www.cubicweb.org/project/cubicweb-tag -.. _task: http://www.cubicweb.org/project/cubicweb-task -.. _zone: http://www.cubicweb.org/project/cubicweb-zone - -To declare the use of a component, once installed, add the name of the component -to the variable `__use__` in the file `__pkginfo__.py` of your own component. - -.. note:: - The listed cubes above are available as debian-packages on `CubicWeb's forge`_. - -.. _`CubicWeb's forge`: http://www.cubicweb.org/project?vtitle=All%20cubicweb%20projects diff -r 02b52bf9f5f8 -r 0865e1e90674 doc/book/en/development/cubes/cc-newcube.rst --- a/doc/book/en/development/cubes/cc-newcube.rst Wed Mar 24 10:23:31 2010 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,48 +0,0 @@ -Creating a new cube from scratch using :command:`cubicweb-ctl newcube` ----------------------------------------------------------------------- - -Let's start by creating the cube environment in which we will develop :: - - cd ~/hg - # use cubicweb-ctl to generate a template for the cube - cubicweb-ctl newcube mycube # will ask some questions, most with nice default - # makes the cube source code managed by mercurial - cd mycube - hg init - hg add . - hg ci - -If all went well, you should see the cube you just created in the list -returned by ``cubicweb-ctl list`` in the section *Available cubes*, -and if it is not the case please refer to :ref:`ConfigurationEnv`. - -To reuse an existing cube, add it to the list named ``__use__`` and defined in -:file:`__pkginfo__.py`. This variable is used for the instance packaging -(dependencies handled by system utility tools such as APT) and the usable cubes -at the time the base is created (import_erschema('MyCube') will not properly -work otherwise). - -.. note:: - - Please note that if you do not wish to use default directory for your cubes - library, you should set the :envvar:`CW_CUBES_PATH` environment variable to - add extra directories where cubes will be search, and you'll then have to use - the option `--directory` to specify where you would like to place the source - code of your cube: - - ``cubicweb-ctl newcube --directory=/path/to/cubes/library mycube`` - - -.. XXX resurrect once live-server is back -.. Usage of :command:`cubicweb-ctl liveserver` -.. ------------------------------------------- - -.. To quickly test a new cube, you can also use the `liveserver` command for cubicweb-ctl -.. which allows to create an instance in memory (using an SQLite database by -.. default) and make it accessible through a web server :: - -.. cubicweb-ctl live-server mycube - -.. or by using an existing database (SQLite or Postgres):: - -.. cubicweb-ctl live-server -s myfile_sources mycube diff -r 02b52bf9f5f8 -r 0865e1e90674 doc/book/en/development/cubes/index.rst --- a/doc/book/en/development/cubes/index.rst Wed Mar 24 10:23:31 2010 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,11 +0,0 @@ -Cubes -===== - -This chapter describes how to define your own cubes and reuse already available cubes. - -.. toctree:: - :maxdepth: 1 - - layout.rst - cc-newcube.rst - available-cubes.rst diff -r 02b52bf9f5f8 -r 0865e1e90674 doc/book/en/development/cubes/layout.rst --- a/doc/book/en/development/cubes/layout.rst Wed Mar 24 10:23:31 2010 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,139 +0,0 @@ - -.. _foundationsCube: - -.. _cubelayout: - -Standard structure for a cube ------------------------------ - -A cube is structured as follows: - -:: - - mycube/ - | - |-- data/ - | |-- cubes.mycube.css - | |-- cubes.mycube.js - | `-- external_resources - | - |-- debian/ - | |-- changelog - | |-- compat - | |-- control - | |-- copyright - | |-- cubicweb-mycube.prerm - | `-- rules - | - |-- entities.py - | - |-- i18n/ - | |-- en.po - | |-- es.po - | `-- fr.po - | - |-- __init__.py - | - |-- MANIFEST.in - | - |-- migration/ - | |-- postcreate.py - | `-- precreate.py - | - |-- __pkginfo__.py - | - |-- schema.py - | - |-- setup.py - | - |-- site_cubicweb.py - | - |-- hooks.py - | - |-- test/ - | |-- data/ - | | `-- bootstrap_cubes - | |-- pytestconf.py - | |-- realdb_test_mycube.py - | `-- test_mycube.py - | - `-- views.py - - -We can use subpackages instead of python modules for ``views.py``, ``entities.py``, -``schema.py`` or ``hooks.py``. For example, we could have: - -:: - - mycube/ - | - |-- entities.py - |-- hooks.py - `-- views/ - |-- forms.py - |-- primary.py - `-- widgets.py - - -where : - -* ``schema`` contains the schema definition (server side only) -* ``entities`` contains the entities definition (server side and web interface) -* ``hooks`` contains hooks and/or views notifications (server side only) -* ``views`` contains the web interface components (web interface only) -* ``test`` contains tests related to the cube (not installed) -* ``i18n`` contains message catalogs for supported languages (server side and - web interface) -* ``data`` contains data files for static content (images, css, javascripts) - ...(web interface only) -* ``migration`` contains initialization files for new instances (``postcreate.py``) - and a file containing dependencies of the component depending on the version - (``depends.map``) -* ``debian`` contains all the files managing debian packaging (you will find - the usual files ``control``, ``rules``, ``changelog``... not installed) -* file ``__pkginfo__.py`` provides component meta-data, especially the distribution - and the current version (server side and web interface) or sub-cubes used by - the cube. - - -At least you should have: - -* the file ``__pkginfo__.py`` -* the schema definition - XXX false, we may want to have cubes which are only adding a service, - no persistent data (eg embedding for instance) - - - -The :file:`__init__.py` and :file:`site_cubicweb.py` files -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The :file:`__pkginfo__.py` file -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -XXX contains metadata describing your cubes - distname / modname - version / numversion - __use__ - __recommend__ - - -:file:`migration/precreate.py` and :file:`migration/postcreate.py` -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -XXX detail steps of instance creation - - -External resources such as image, javascript and css files -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -XXX naming convention external_resources file - - -Out-of the box testing -~~~~~~~~~~~~~~~~~~~~~~ -XXX MANIFEST.in, __pkginfo__.include_dirs, debian - - - -Packaging and distribution -~~~~~~~~~~~~~~~~~~~~~~~~~~ -XXX MANIFEST.in, __pkginfo__.include_dirs, debian - diff -r 02b52bf9f5f8 -r 0865e1e90674 doc/book/en/development/datamodel/baseschema.rst --- a/doc/book/en/development/datamodel/baseschema.rst Wed Mar 24 10:23:31 2010 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,43 +0,0 @@ -.. _CWBaseEntityTypes: - -Pre-defined entities in the library ------------------------------------ - -The library defines a set of entity schemas that are required by the system -or commonly used in *CubicWeb* instances. - - -Entity types used to store the schema -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -* _`CWEType`, entity type -* _`CWRType`, relation type -* _`CWRelation`, relation definition -* _`CWAttribute`, attribute relation definition -* _`CWConstraint`, `CWConstraintType`, `RQLExpression` - -Entity types used to manage users and permissions -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -* _`CWUser`, system users -* _`CWGroup`, users groups -* _`CWPermission`, used to configure the security of the instance - -Entity types used to manage workflows -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -* _`Workflow`, workflow entity, linked to some entity types which may use this workflow -* _`State`, workflow state -* _`Transition`, workflow transition -* _`TrInfo`, record of a transition trafic for an entity - -Other entity types -~~~~~~~~~~~~~~~~~~ -* _`CWCache`, cache entities used to improve performances -* _`CWProperty`, used to configure the instance - -* _`EmailAddress`, email address, used by the system to send notifications - to the users and also used by others optionnals schemas - -* _`Bookmark`, an entity type used to allow a user to customize his links within - the instance - -* _`ExternalUri`, used for semantic web site to indicate that an entity is the - same as another from an external site diff -r 02b52bf9f5f8 -r 0865e1e90674 doc/book/en/development/datamodel/define-workflows.rst --- a/doc/book/en/development/datamodel/define-workflows.rst Wed Mar 24 10:23:31 2010 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,159 +0,0 @@ -.. -*- coding: utf-8 -*- - -.. _Workflow: - -Define a Workflow -================= - -General -------- - -A workflow describes how certain entities have to evolve between -different states. Hence we have a set of states, and a "transition graph", -i.e. a set of possible transitions from one state to another state. - -We will define a simple workflow for a blog, with only the following -two states: `submitted` and `published`. So first, we create a simple -*CubicWeb* instance in ten minutes (see :ref:`BlogFiveMinutes`). - -Set-up a workflow ------------------ - -We want to create a workflow to control the quality of the BlogEntry -submitted on the instance. When a BlogEntry is created by a user -its state should be `submitted`. To be visible to all, it has to -be in the state `published`. To move it from `submitted` to `published`, -we need a transition that we can call `approve_blogentry`. - -A BlogEntry state should not be modifiable by every user. -So we have to define a group of users, `moderators`, and -this group will have appropriate permissions to publish a BlogEntry. - -There are two ways to create a workflow: from the user interface, or -by defining it in ``migration/postcreate.py``. This script is executed -each time a new ``cubicweb-ctl db-init`` is done. We strongly -recommend to create the workflow in ``migration/postcreate.py`` and we -will now show you how. Read `Two bits of warning`_ to understand why. - -The state of an entity is managed by the `in_state` attribute which -can be added to your entity schema by inheriting from -`cubicweb.schema.WorkflowableEntityType`. - - -About our example of BlogEntry, we must have: - -.. sourcecode:: python - - from cubicweb.schema import WorkflowableEntityType - - class BlogEntry(WorkflowableEntityType): - ... - - -Create states, transitions and group permissions -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The ``postcreate.py`` script is executed in a special environment, adding -several *CubicWeb* primitives that can be used. - -They are all defined in the ``class ServerMigrationHelper``. -We will only discuss the methods we use to create a workflow in this example. - -A workflow is a collection of entities of type ``State`` and of type -``Transition`` which are standard *CubicWeb* entity types. - -To define a workflow for BlogDemo, please add the following lines -to ``migration/postcreate.py``: - -.. sourcecode:: python - - _ = unicode - - moderators = add_entity('CWGroup', name=u"moderators") - -This adds the `moderators` user group. - -.. sourcecode:: python - - wf = add_workflow(u'blog publication workflow', 'BlogEntry') - -At first, instanciate a new workflow object with a gentle description -and the concerned entity types (this one can be a tuple for multiple -value). - -.. sourcecode:: python - - submitted = wf.add_state(_('submitted'), initial=True) - published = wf.add_state(_('published')) - -This will create two entities of type ``State``, one with name -'submitted', and the other with name 'published'. - -``add_state`` expects as first argument the name of the state you want -to create and an optional argument to say if it is supposed to be the -initial state of the entity type. - -.. sourcecode:: python - - wf.add_transition(_('approve_blogentry'), (submitted,), published, ('moderators', 'managers'),) - -This will create an entity of type ``Transition`` with name -`approve_blogentry` which will be linked to the ``State`` entities -created before. - -``add_transition`` expects - - * as the first argument: the name of the transition - * then the list of states on which the transition can be triggered, - * the target state of the transition, - * and the permissions - (e.g. a list of user groups who can apply the transition; the user - has to belong to at least one of the listed group to perform the action). - -.. sourcecode:: python - - checkpoint() - -.. note:: - Do not forget to add the `_()` in front of all states and transitions names while creating - a workflow so that they will be identified by the i18n catalog scripts. - -In addition to the user groups (one of which the user needs to belong -to), we could have added a RQL condition. In this case, the user can -only perform the action if the two conditions are satisfied. - -If we use an RQL condition on a transition, we can use the following variables: - -* `X`, the entity on which we may pass the transition -* `U`, the user executing that may pass the transition - - -.. image:: ../../images/03-transitions-view.en.png - -You can notice that in the action box of a BlogEntry, the state is now -listed as well as the possible transitions for the current state -defined by the workflow. - -The transitions will only be displayed for users having the right permissions. -In our example, the transition `approve_blogentry` will only be displayed -for the users belonging to the group `moderators` or `managers`. - - -Two bits of warning -~~~~~~~~~~~~~~~~~~~ - -We could perfectly use the administration interface to do these -operations. It is a convenient thing to do at times (when doing -development, to quick-check things). But it is not recommended beyond -that because it is a bit complicated to do it right and it will be -only local to your instance (or, said a bit differently, such a -workflow only exists in an instance database). Furthermore, you cannot -write unit tests against deployed instances, and experience shows it -is mandatory to have tests for any mildly complicated workflow -setup. - -Indeed, if you create the states and transitions through the user -interface, next time you initialize the database you will have to -re-create all the workflow entities. The user interface should only be -a reference for you to view the states and transitions, but is not the -appropriate interface to define your application workflow. diff -r 02b52bf9f5f8 -r 0865e1e90674 doc/book/en/development/datamodel/definition.rst --- a/doc/book/en/development/datamodel/definition.rst Wed Mar 24 10:23:31 2010 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,512 +0,0 @@ - .. -*- coding: utf-8 -*- - -Yams *schema* -------------- - -The **schema** is the core piece of a *CubicWeb* instance as it defines -the handled data model. It is based on entity types that are either already -defined in the *CubicWeb* standard library; or more specific types defined -in cubes. The schema for a cube is defined in a :file:schema.py file or in -one or more Python files under the :file:`schema` directory (python package). - -At this point, it is important to make clear the difference between -*relation type* and *relation definition*: a *relation type* is only a relation -name with potentially other additionnal properties (see below), whereas a -*relation definition* is a complete triplet -" ". -A relation type could have been implied if none is related to a -relation definition of the schema. - -Also, it should be clear that to properly handle data migration, an instance'schema -is stored in the database, so the python schema file used to defined it are only readen -when the instance is created or upgraded. - -The following built-in types are available : `String`, `Int`, `Float`, -`Decimal`, `Boolean`, `Date`, `Datetime`, `Time`, `Interval`, `Byte` -and `Password`. - -You'll also have access to :ref:`base cubicweb entity types `. - -The instance schema is accessible through the .schema attribute of the -`vregistry`. It's an instance of :class:`cubicweb.schema.Schema`, which -extends :class:`yams.schema.Schema`. - -:note: - In previous yams versions, almost all classes where available without - any import, but the should now be explicitely imported. - - -Entity type -~~~~~~~~~~~ -It's an instance of :class:`yams.schema.EntitySchema`. Each entity types has -a set of attributes and relation and some permissions, defining who can add, read, -update or delete entities of this type. - -XXX yams inheritance - -Relation type -~~~~~~~~~~~~~ -It's an instance of :class:`yams.schema.RelationSchema`. A relation type is simply -a semantic definition of a kind of relationship that may occurs in your application. - -It's important to choose a good name, at least to avoid conflicts with some semantically -different relation defined in other cubes (since we've no namespace yet). - -A relation type hold the following properties (which are hence shared between all -relation definitions of that type): - -* `inlined` : boolean handling the physical optimization for archiving - the relation in the subject entity table, instead of creating a specific - table for the relation. This applies to relations where cardinality - of subject->relation->object is 0..1 (`?`) or 1..1 (`1`) for *all* its relation - definitions. - -* `symmetric` : boolean indicating that the relation is symmetrical, which - means that `X relation Y` implies `Y relation X`. - - -Relation definition -~~~~~~~~~~~~~~~~~~~ -It's an instance of :class:`yams.schema.RelationDefinition`. It is a complete triplet -" ". - -Properties -`````````` - -* Optional properties for attributes and relations : - - - `description` : a string describing an attribute or a relation. By default - this string will be used in the editing form of the entity, which means - that it is supposed to help the end-user and should be flagged by the - function `_` to be properly internationalized. - - - `constraints` : a list of conditions/constraints that the relation has to - satisfy (c.f. `Constraints`_) - - - `cardinality` : a two character string which specify the cardinality of the - relation. The first character defines the cardinality of the relation on - the subject, and the second on the object. When a relation can have - multiple subjects or objects, the cardinality applies to all, - not on a one-to-one basis (so it must be consistent...). The possible - values are inspired from regular expression syntax : - - * `1`: 1..1 - * `?`: 0..1 - * `+`: 1..n - * `*`: 0..n - -* optional properties for attributes : - - - `unique` : boolean indicating if the value of the attribute has to be unique - or not within all entities of the same type (false by default) - - - `indexed` : boolean indicating if an index needs to be created for this - attribute in the database (false by default). This is useful only if - you know that you will have to run numerous searches on the value of this - attribute. - - - `default` : default value of the attribute. In case of date types, the values - which could be used correspond to the RQL keywords `TODAY` and `NOW`. - -* optional properties of type `String` : - - - `fulltextindexed` : boolean indicating if the attribute is part of - the full text index (false by default) (*applicable on the type `Byte` - as well*) - - - `internationalizable` : boolean indicating if the value of the attribute - is internationalizable (false by default) - -* optional properties for relations : - - - `composite` : string indicating that the subject (composite == 'subject') - is composed of the objects of the relations. For the opposite case (when - the object is composed of the subjects of the relation), we just set - 'object' as value. The composition implies that when the relation - is deleted (so when the composite is deleted, at least), the composed are also deleted. - - - `fti_container`: XXX feed me - -Constraints -``````````` - -By default, the available constraint types are : - -General Constraints -...................... - -* `SizeConstraint` : allows to specify a minimum and/or maximum size on - string (generic case of `maxsize`) - -* `BoundConstraint` : allows to specify a minimum and/or maximum value on - numeric types - -* `UniqueConstraint` : identical to "unique=True" - -* `StaticVocabularyConstraint` : identical to "vocabulary=(...)" - -XXX Attribute, TODAY, NOW - -RQL Based Constraints -...................... - -RQL based constraints may take three arguments. The first one is the ``WHERE`` -clause of a RQL query used by the constraint. The second argument ``mainvars`` -is the ``Any`` clause of the query. By default this include `S` reserved for the -subject of the relation and `O` for the object. Additional variables could be -specified using ``mainvars``. The argument expects a single string with all -variable's name separated by spaces. The last one, ``msg``, is the error message -displayed when the constraint fails. As RQLVocabularyConstraint never fails the -third argument is not available. - -* `RQLConstraint` : allows to specify a RQL query that has to be satisfied - by the subject and/or the object of relation. In this query the variables - `S` and `O` are reserved for the entities subject and object of the - relation. - -* `RQLVocabularyConstraint` : similar to the previous type of constraint except - that it does not express a "strong" constraint, which means it is only used to - restrict the values listed in the drop-down menu of editing form, but it does - not prevent another entity to be selected. - -* `RQLUniqueConstraint` : allows to the specify a RQL query that ensure that an - attribute is unique in a specific context. The Query must **never** return more - than a single result to be satisfied. In this query the variables `S` is - reserved for the entity subject of the relation. The other variable should be - specified with the second constructor argument (mainvars). This constraints - should be used when UniqueConstraint doesn't fit. Here is a simple example :: - - # Check that in the same Workflow each state's name is unique. Using - # UniqueConstraint (or unique=True) here would prevent states in different - # workflows to have the same name. - - # With: State S, Workflow W, String N ; S state_of W, S name N - - RQLUniqueConstraint('S name N, S state_of WF, Y state_of WF, Y name N', - mainvars='Y', - msg=_('workflow already have a state of that name')) - - - -* `RQLUniqueConstraint` : allows to the specify a RQL query that ensure that an - attribute is unique in a specific context. The Query must **never** return more - than a single result to be satisfied. In this query the variables `S` is - reserved for the entity subject of the relation. The other variable should be - specified with the second constructor argument (mainvars). This constraints - should be used when UniqueConstraint doesn't fit. Here is a simple example :: - - # Check that in the same Workflow each state's name is unique. Using - # UniqueConstraint (or unique=True) here would prevent states in different - # workflows to have the same name. - - # With: State S, Workflow W, String N ; S state_of W, S name N - - RQLUniqueConstraint('S name N, S state_of WF, Y state_of WF, Y name N', - mainvars='Y', - msg=_('workflow already have a state of that name')) - - - -XXX note about how to add new constraint - -.. _securitymodel: - - -The security model -~~~~~~~~~~~~~~~~~~ - -The security model of `cubicWeb` is based on `Access Control List`. -The main principles are: - -* users and groups of users -* a user belongs to at least one group of user -* permissions (read, update, create, delete) -* permissions are assigned to groups (and not to users) - -For *CubicWeb* in particular: - -* we associate rights at the enttities/relations schema level -* for each entity, we distinguish four kind of permissions: read, - add, update and delete -* for each relation, we distinguish three kinds of permissions: read, - add and delete (we can not modify a relation) -* the basic groups are: Administrators, Users and Guests -* by default, users belong to the group Users -* there is a virtual group called `Owners` to which we - can associate only deletion and update permissions -* we can not add users to the `Owners` group, they are - implicitly added to it according to the context of the objects - they own -* the permissions of this group are only checked on update/deletion - actions if all the other groups the user belongs to does not provide - those permissions - -Setting permissions is done with the attribute `__permissions__` of entities and -relation types. It defines a dictionary where the keys are the access types -(action), and the values are the authorized groups or expressions. - -For an entity type, the possible actions are `read`, `add`, `update` and -`delete`. - -For a relation type, the possible actions are `read`, `add`, and `delete`. - -For each access type, a tuple indicates the name of the authorized groups and/or -one or multiple RQL expressions to satisfy to grant access. The access is -provided if the user is in one of the listed groups or one of if the RQL condition -is satisfied. - -The standard user groups -```````````````````````` - -* `guests` - -* `users` - -* `managers` - -* `owners` : virtual group corresponding to the entity's owner. - This can only be used for the actions `update` and `delete` of an entity - type. - -It is also possible to use specific groups if they are defined in the -precreate of the cube (``migration/precreate.py``). Defining groups in -postcreate or even later makes them NOT available for security -purposes (in this case, an `sync_schema_props_perms` command have to -be issued in a CubicWeb shell). - - -Use of RQL expression for write permissions -``````````````````````````````````````````` -It is possible to define RQL expression to provide update permission -(`add`, `delete` and `update`) on relation and entity types. - -RQL expression for entity type permission : - -* you have to use the class `ERQLExpression` - -* the used expression corresponds to the WHERE statement of an RQL query - -* in this expression, the variables X and U are pre-defined references - respectively on the current entity (on which the action is verified) and - on the user who send the request - -* it is possible to use, in this expression, a special relation - "has__permission" where the subject is the user and the - object is any variable, meaning that the user needs to have - permission to execute the action on the entities related - to this variable - -For RQL expressions on a relation type, the principles are the same except -for the following : - -* you have to use the class `RRQLExpression` in the case of a non-final relation - -* in the expression, the variables S, O and U are pre-defined references - to respectively the subject and the object of the current relation (on - which the action is being verified) and the user who executed the query - -* we can also define rights over attributes of an entity (non-final relation), - knowing that : - - - to define RQL expression, we have to use the class `ERQLExpression` - in which X represents the entity the attribute belongs to - - - the permissions `add` and `delete` are equivalent. Only `add`/`read` - are actually taken in consideration. - -:Note on the use of RQL expression for `add` permission: - - Potentially, the use of an RQL expression to add an entity or a - relation can cause problems for the user interface, because if the - expression uses the entity or the relation to create, then we are - not able to verify the permissions before we actually add the entity - (please note that this is not a problem for the RQL server at all, - because the permissions checks are done after the creation). In such - case, the permission check methods (CubicWebEntitySchema.check_perm - and has_perm) can indicate that the user is not allowed to create - this entity but can obtain the permission. - To compensate this problem, it is usually necessary, for such case, - to use an action that reflects the schema permissions but which enables - to check properly the permissions so that it would show up if necessary. - - -Use of RQL expression for reading rights -```````````````````````````````````````` - -The principles are the same but with the following restrictions : - -* we can not use `RRQLExpression` on relation types for reading - -* special relations "has__permission" can not be used - - - - -Defining your schema using yams -------------------------------- - -Entity type definition -~~~~~~~~~~~~~~~~~~~~~~ - -An entity type is defined by a Python class which inherits from `EntityType`. -The class definition contains the description of attributes and relations -for the defined entity type. -The class name corresponds to the entity type name. It is exepected to be -defined in the module ``mycube.schema``. - -When defining a schema using python files, you may use the following shortcuts: - -- `required` : boolean indicating if the attribute is required, eg subject cardinality is '1' - -- `vocabulary` : specify static possible values of an attribute - -- `maxsize` : integer providing the maximum size of a string (no limit by default) - -For example: - -.. sourcecode:: python - - class Person(EntityType): - """A person with the properties and the relations necessary for my - application""" - - last_name = String(required=True, fulltextindexed=True) - first_name = String(required=True, fulltextindexed=True) - title = String(vocabulary=('Mr', 'Mrs', 'Miss')) - date_of_birth = Date() - works_for = SubjectRelation('Company', cardinality='?*') - - -The entity described above defines three attributes of type String, -last_name, first_name and title, an attribute of type Date for the date of -birth and a relation that connects a `Person` to another entity of type -`Company` through the semantic `works_for`. - -The name of the Python attribute corresponds to the name of the attribute -or the relation in *CubicWeb* application. - -An attribute is defined in the schema as follows:: - - attr_name = attr_type(properties) - -where `attr_type` is one of the type listed above and `properties` is -a list of the attribute needs to statisfy (see `Properties`_ -for more details). - - -* relations can be defined by using `ObjectRelation` or `SubjectRelation`. - The first argument of `SubjectRelation` or `ObjectRelation` gives respectively - the object/subject entity type of the relation. This could be : - - * a string corresponding to an entity type - - * a tuple of string corresponding to multiple entity types - - * special string such as follows : - - - "**" : all types of entities - - "*" : all types of non-meta entities - - "@" : all types of meta entities but not system entities (e.g. used for - the basic schema description) - -* it is possible to use the attribute `meta` to flag an entity type as a `meta` - (e.g. used to describe/categorize other entities) - -*Note* : if you end up with an `if` in the definition of your entity, this probably -means that you need two separate entities that implement the `ITree` interface and -get the result from `.children()` which ever entity is concerned. - -Inheritance -``````````` -XXX feed me - - -Definition of relations -~~~~~~~~~~~~~~~~~~~~~~~ - -XXX add note about defining relation type / definition - -A relation is defined by a Python class heriting `RelationType`. The name -of the class corresponds to the name of the type. The class then contains -a description of the properties of this type of relation, and could as well -contain a string for the subject and a string for the object. This allows to create -new definition of associated relations, (so that the class can have the -definition properties from the relation) for example :: - - class locked_by(RelationType): - """relation on all entities indicating that they are locked""" - inlined = True - cardinality = '?*' - subject = '*' - object = 'CWUser' - -In the case of simultaneous relations definitions, `subject` and `object` -can both be equal to the value of the first argument of `SubjectRelation` -and `ObjectRelation`. - -When a relation is not inlined and not symmetrical, and it does not require -specific permissions, its definition (by using `SubjectRelation` and -`ObjectRelation`) is all we need. - - -Definition of permissions -~~~~~~~~~~~~~~~~~~~~~~~~~~ -The entity type `CWPermission` from the standard library -allows to build very complex and dynamic security architectures. The schema of -this entity type is as follow : - -.. sourcecode:: python - - class CWPermission(EntityType): - """entity type that may be used to construct some advanced security configuration - """ - name = String(required=True, indexed=True, internationalizable=True, maxsize=100) - require_group = SubjectRelation('CWGroup', cardinality='+*', - description=_('groups to which the permission is granted')) - require_state = SubjectRelation('State', - description=_("entity's state in which the permission is applicable")) - # can be used on any entity - require_permission = ObjectRelation('**', cardinality='*1', composite='subject', - description=_("link a permission to the entity. This " - "permission should be used in the security " - "definition of the entity's type to be useful.")) - - -Example of configuration: - -.. sourcecode:: python - - class Version(EntityType): - """a version is defining the content of a particular project's release""" - - __permissions__ = {'read': ('managers', 'users', 'guests',), - 'update': ('managers', 'logilab', 'owners',), - 'delete': ('managers', ), - 'add': ('managers', 'logilab', - ERQLExpression('X version_of PROJ, U in_group G,' - 'PROJ require_permission P, P name "add_version",' - 'P require_group G'),)} - - - class version_of(RelationType): - """link a version to its project. A version is necessarily linked to one and only one project. - """ - __permissions__ = {'read': ('managers', 'users', 'guests',), - 'delete': ('managers', ), - 'add': ('managers', 'logilab', - RRQLExpression('O require_permission P, P name "add_version",' - 'U in_group G, P require_group G'),) - } - inlined = True - - -This configuration indicates that an entity `CWPermission` named -"add_version" can be associated to a project and provides rights to create -new versions on this project to specific groups. It is important to notice that : - -* in such case, we have to protect both the entity type "Version" and the relation - associating a version to a project ("version_of") - -* because of the genericity of the entity type `CWPermission`, we have to execute - a unification with the groups and/or the states if necessary in the expression - ("U in_group G, P require_group G" in the above example) diff -r 02b52bf9f5f8 -r 0865e1e90674 doc/book/en/development/datamodel/index.rst --- a/doc/book/en/development/datamodel/index.rst Wed Mar 24 10:23:31 2010 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,12 +0,0 @@ -Data model -========== - -This chapter describes how you define a schema and how to make it evolves as the time goes. - -.. toctree:: - :maxdepth: 1 - - definition - metadata - baseschema - define-workflows diff -r 02b52bf9f5f8 -r 0865e1e90674 doc/book/en/development/datamodel/metadata.rst --- a/doc/book/en/development/datamodel/metadata.rst Wed Mar 24 10:23:31 2010 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,54 +0,0 @@ - -Meta-data ---------- - -.. index:: - schema: meta-data; - schema: eid; creation_date; modification_data; cwuri - schema: created_by; owned_by; is; is_instance; - -Each entity type in |cubicweb| has at least the following meta-data attributes and relations: - -eid - entity's identifier which is unique in an instance. We usually call this identifier `eid` for historical reason. - -creation_date - Date and time of the creation of the entity. - -modification_date - Date and time of the latest modification of an entity. - -cwuri - Reference URL of the entity, which is not expected to change. - -created_by - Relation to the :ref:`users ` who has created the entity - -owned_by - Relation to :ref:`users ` whom the entity belongs; usually the creator but not - necessary, and it could have multiple owners notably for permission control - -is - Relation to the :ref:`entity type ` of which type the entity is. - -is_instance - Relation to the :ref:`entity types ` of which type the - entity is an instance of. - - -Virtual RQL relations ---------------------- -XXX move this to the RQL chapter. - -Those relations may only be used in RQL query and are not actual attributes of your entities... - -has_text - Relation to use to query the full text index (only for entities having fulltextindexed attributes). - -identity - Relation to use to tell that a RQL variable should be the same as entity another - (but you've to use two different rql variables for querying purpose) - - - -.. |cubicweb| replace:: *CubicWeb* \ No newline at end of file diff -r 02b52bf9f5f8 -r 0865e1e90674 doc/book/en/development/devcore/appobject.rst --- a/doc/book/en/development/devcore/appobject.rst Wed Mar 24 10:23:31 2010 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,71 +0,0 @@ - - -The `AppObject` class -~~~~~~~~~~~~~~~~~~~~~ - -In general: - -* we do not inherit directly from this class but from a more specific - class such as `AnyEntity`, `EntityView`, `AnyRsetView`, - `Action`... - -* to be recordable, a subclass has to define its own register (attribute - `__registry__`) and its identifier (attribute `id`). Usually we do not have - to take care of the register, only the identifier `id`. - -We can find a certain number of attributes and methods defined in this class -and common to all the application objects. - -At recording time, the following attributes are dynamically added to -the *subclasses*: - -* `vreg`, the `vregistry` of the instance -* `schema`, the instance schema -* `config`, the instance configuration - -We also find on instances, the following attributes: - -* ._cw`, `Request` instance -* `rset`, the *result set* associated to the object if necessary - -:URL handling: - * `build_url(*args, **kwargs)`, returns an absolute URL based on the - given arguments. The *controller* supposed to handle the response, - can be specified through the first positional parameter (the - connection is theoretically done automatically :). - -:Data manipulation: - - * `entity(row, col=0)`, returns the entity corresponding to the data position - in the *result set* associated to the object - - * `complete_entity(row, col=0, skip_bytes=True)`, is equivalent to `entity` but - also call the method `complete()` on the entity before returning it - -:Data formatting: - * `format_date(date, date_format=None, time=False)` returns a string for a - date time according to instance's configuration - * `format_time(time)` returns a string for a date time according to - instance's configuration - -:And more...: - - * `tal_render(template, variables)`, renders a precompiled page template with - variables in the given dictionary as context - -.. note:: - When we inherit from `AppObject` (even not directly), you *always* have to use - **super()** to get the methods and attributes of the superclasses, and not - use the class identifier. - - For example, instead of writting: :: - - class Truc(PrimaryView): - def f(self, arg1): - PrimaryView.f(self, arg1) - - You must write: :: - - class Truc(PrimaryView): - def f(self, arg1): - super(Truc, self).f(arg1) diff -r 02b52bf9f5f8 -r 0865e1e90674 doc/book/en/development/devcore/cwconfig.rst --- a/doc/book/en/development/devcore/cwconfig.rst Wed Mar 24 10:23:31 2010 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,5 +0,0 @@ -:mod:`Configuration ` ----------------------------------------- - -.. automodule:: cubicweb.cwconfig - :members: diff -r 02b52bf9f5f8 -r 0865e1e90674 doc/book/en/development/devcore/dbapi.rst --- a/doc/book/en/development/devcore/dbapi.rst Wed Mar 24 10:23:31 2010 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,82 +0,0 @@ - - -API Python/RQL -~~~~~~~~~~~~~~ - -The Python API developped to interface with RQL is inspired from the standard db-api, -with a Connection object having the methods cursor, rollback and commit essentially. -The most important method is the `execute` method of a cursor : - -`execute(rqlstring, args=None, cachekey=None, build_descr=True)` - -:rqlstring: the RQL query to execute (unicode) -:args: if the query contains substitutions, a dictionary containing the values to use -:cachekey: - an implementation detail of the RQL cache implies that if a substitution - is used to introduce an eid *susceptible to raise the ambiguities in the query - type resolution*, then we have to specify the corresponding key in the dictionary - through this argument - - -The `Connection` object owns the methods `commit` and `rollback`. You *should -never need to use them* during the development of the web interface based on -the *CubicWeb* framework as it determines the end of the transaction depending -on the query execution success. - -.. note:: - While executing update queries (SET, INSERT, DELETE), if a query generates - an error related to security, a rollback is automatically done on the current - transaction. - -Executing RQL queries from a view or a hook -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -When you're within code of the web interface, the db-api like connexion is -handled by the request object. You should not have to access it directly, but -use the `execute` method directly available on the request, eg: - - rset = self._cw.execute(rqlstring, kwargs) - -Similarly, on the server side (eg in hooks), there is no db-api connexion (since -you're directly inside the data-server), so you'll have to use the execute method -of the session object. - - -Important note about proper usage of .execute -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Let's say you want to get T which is in configuration C, this translates to: - -.. sourcecode:: python - - self._cw.execute('Any T WHERE T in_conf C, C eid %s' % entity.eid) - -But it can also be written in a syntax that will benefit from the use -of a cache on the RQL server side: - -.. sourcecode:: python - - self._cw.execute('Any T WHERE T in_conf C, C eid %(x)s', {'x': entity.eid}, 'x') - -Beside proper usage of the `args` argument, notice the latest argument: this is what's called -the cache key. The cache key should be either a string or a tuple containing the names of keys -in args which are referencing eids. *YOU MUST SET THIS PROPERLY* if you don't want weird result -on queries which have ambigous solutions deambiguified by specifing an eid. So the good habit is: -*always put in the cache key all eid keys*. - -The syntax tree is build once for the "generic" RQL and can be re-used -with a number of different eid. - -Alternativelly, some of the common data related to an entity can be obtained from -the top-level `entity.related()` method (which is used under the hood by the orm -when you use attribute access notation on an entity to get a relation. The above -would then be translated to: - -.. sourcecode:: python - - entity.related('in_conf', 'object') - -The `related()` method, as more generally others orm methods, makes extensive use -of the cache mechanisms so you don't have to worry about them. Additionnaly this -use will get you commonly used attributes that you will be able to use in your -view generation without having to ask the data backend. - diff -r 02b52bf9f5f8 -r 0865e1e90674 doc/book/en/development/devcore/index.rst --- a/doc/book/en/development/devcore/index.rst Wed Mar 24 10:23:31 2010 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,12 +0,0 @@ -Core APIs -========= - -.. toctree:: - :maxdepth: 1 - - vreg.rst - appobject.rst - selectors.rst - dbapi.rst - cwconfig.rst - diff -r 02b52bf9f5f8 -r 0865e1e90674 doc/book/en/development/devcore/selectors.rst --- a/doc/book/en/development/devcore/selectors.rst Wed Mar 24 10:23:31 2010 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,89 +0,0 @@ -Base selectors --------------- - -Selectors are scoring functions that are called by the registry to tell whenever -an appobject can be selected in a given context. Selector sets are for instance -the glue that tie views to the data model. Using them appropriately is an -essential part of the construction of well behaved cubes. - -Of course you may have to write your own set of selectors as your needs grows and -you get familiar with the framework (see :ref:`CustomSelectors`). - -Here is a description of generic selectors provided by CubicWeb that should suit -most of your needs. - -Bare selectors -~~~~~~~~~~~~~~ -Those selectors are somewhat dumb, which doesn't mean they're not (very) useful. - -.. autoclass:: cubicweb.appobject.yes -.. autoclass:: cubicweb.selectors.match_kwargs -.. autoclass:: cubicweb.selectors.appobject_selectable - - -Result set selectors -~~~~~~~~~~~~~~~~~~~~~ -Those selectors are looking for a result set in the context ('rset' argument or -the input context) and match or not according to its shape. Some of these -selectors have different behaviour if a particular cell of the result set is -specified using 'row' and 'col' arguments of the input context or not. - -.. autoclass:: cubicweb.selectors.none_rset -.. autoclass:: cubicweb.selectors.any_rset -.. autoclass:: cubicweb.selectors.nonempty_rset -.. autoclass:: cubicweb.selectors.empty_rset -.. autoclass:: cubicweb.selectors.one_line_rset -.. autoclass:: cubicweb.selectors.multi_lines_rset -.. autoclass:: cubicweb.selectors.multi_columns_rset -.. autoclass:: cubicweb.selectors.paginated_rset -.. autoclass:: cubicweb.selectors.sorted_rset -.. autoclass:: cubicweb.selectors.one_etype_rset -.. autoclass:: cubicweb.selectors.multi_etypes_rset - - -Entity selectors -~~~~~~~~~~~~~~~~ -Those selectors are looking for either an `entity` argument in the input context, -or entity found in the result set ('rset' argument or the input context) and -match or not according to entity's (instance or class) properties. - -.. autoclass:: cubicweb.selectors.non_final_entity -.. autoclass:: cubicweb.selectors.implements -.. autoclass:: cubicweb.selectors.score_entity -.. autoclass:: cubicweb.selectors.rql_condition -.. autoclass:: cubicweb.selectors.relation_possible -.. autoclass:: cubicweb.selectors.partial_relation_possible -.. autoclass:: cubicweb.selectors.has_related_entities -.. autoclass:: cubicweb.selectors.partial_has_related_entities -.. autoclass:: cubicweb.selectors.has_permission -.. autoclass:: cubicweb.selectors.has_add_permission - - -Logged user selectors -~~~~~~~~~~~~~~~~~~~~~ -Those selectors are looking for properties of the user issuing the request. - -.. autoclass:: cubicweb.selectors.anonymous_user -.. autoclass:: cubicweb.selectors.authenticated_user -.. autoclass:: cubicweb.selectors.match_user_groups - - -Web request selectors -~~~~~~~~~~~~~~~~~~~~~ -Those selectors are looking for properties of *web* request, they can not be -used on the data repository side. - -.. autoclass:: cubicweb.selectors.match_form_params -.. autoclass:: cubicweb.selectors.match_search_state -.. autoclass:: cubicweb.selectors.match_context_prop -.. autoclass:: cubicweb.selectors.match_view -.. autoclass:: cubicweb.selectors.primary_view -.. autoclass:: cubicweb.selectors.specified_etype_implements - - -Other selectors -~~~~~~~~~~~~~~~ -.. autoclass:: cubicweb.selectors.match_transition - -You'll also find some other (very) specific selectors hidden in other modules -than :mod:`cubicweb.selectors`. diff -r 02b52bf9f5f8 -r 0865e1e90674 doc/book/en/development/devcore/vreg.rst --- a/doc/book/en/development/devcore/vreg.rst Wed Mar 24 10:23:31 2010 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,220 +0,0 @@ -The VRegistry --------------- - -The recording process on startup -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Details of the recording process -```````````````````````````````` - -.. index:: - vregistry: registration_callback - -On startup, |cubicweb| have to fill the vregistry with appobjects defined -in its library and in cubes used by the instance. Appobjects from the library -are loaded first, then appobjects provided by cubes are loaded in an ordered -way (e.g. if your cube depends on an other, appobjects from the dependancy will -be loaded first). Cube's modules or packages where appobject are looked at is explained -in :ref:`cubelayout`. - -For each module: - -* by default all objects are registered automatically - -* if some objects have to replace other objects or be included only if a - condition is true, you'll have to define a `registration_callback(vreg)` - function in your module and explicitly register *all objects* in this - module, using the vregistry api defined below. - -.. note:: - Once the function `registration_callback(vreg)` is implemented, all the objects - have to be explicitly registered as it disables the automatic object registering. - - -API d'enregistrement des objets -``````````````````````````````` -.. automethod:: cubicweb.cwvreg.CubicWebVRegistry.register_all -.. automethod:: cubicweb.cwvreg.CubicWebVRegistry.register_and_replace -.. automethod:: cubicweb.cwvreg.CubicWebVRegistry.register -.. automethod:: cubicweb.cwvreg.CubicWebVRegistry.register_if_interface_found -.. automethod:: cubicweb.cwvreg.CubicWebVRegistry.unregister - - -Examples -```````` -.. sourcecode:: python - - # web/views/basecomponents.py - def registration_callback(vreg): - # register everything in the module except SeeAlsoComponent - vreg.register_all(globals().values(), __name__, (SeeAlsoVComponent,)) - # conditionally register SeeAlsoVComponent - if 'see_also' in vreg.schema: - vreg.register(SeeAlsoVComponent) - - # goa/appobjects/sessions.py - def registration_callback(vreg): - vreg.register(SessionsCleaner) - # replace AuthenticationManager by GAEAuthenticationManager - vreg.register_and_replace(GAEAuthenticationManager, AuthenticationManager) - # replace PersistentSessionManager by GAEPersistentSessionManager - vreg.register_and_replace(GAEPersistentSessionManager, PersistentSessionManager) - - -Runtime objects selection -~~~~~~~~~~~~~~~~~~~~~~~~~ - -Using and combining existant selectors -`````````````````````````````````````` - -The object's selector is defined by its `__select__` class attribute. - -When two selectors are combined using the `&` operator (formerly `chainall`), it -means that both should return a positive score. On success, the sum of scores is returned. - -When two selectors are combined using the `|` operator (former `chainfirst`), it -means that one of them should return a positive score. On success, the first -positive score is returned. - -You can also "negate" a selector by precedeing it by the `~` operator. - -Of course you can use paren to balance expressions. - - -For instance, if you are selecting the primary (eg `__regid__ = 'primary'`) view (eg -`__registry__ = 'view'`) for a result set containing a `Card` entity, 2 objects -will probably be selectable: - -* the default primary view (`__select__ = implements('Any')`), meaning - that the object is selectable for any kind of entity type - -* the specific `Card` primary view (`__select__ = implements('Card')`, - meaning that the object is selectable for Card entities - -Other primary views specific to other entity types won't be selectable -in this case. Among selectable objects, the implements selector will -return a higher score than the second view since it's more specific, -so it will be selected as expected. - - -Example -```````` - -The goal: when on a Blog, one wants the RSS link to refer to blog -entries, not to the blog entity itself. - -To do that, one defines a method on entity classes that returns the -RSS stream url for a given entity. The default implementation on -AnyEntity and a specific implementation on Blog will do what we want. - -But when we have a result set containing several Blog entities (or -different entities), we don't know on which entity to call the -aforementioned method. In this case, we keep the current behaviour -(e.g : call to limited_rql). - -Hence we have two cases here, one for a single-entity rsets, the other -for multi-entities rsets. - -In web/views/boxes.py lies the RSSIconBox class. Look at its selector :: - - class RSSIconBox(ExtResourcesBoxTemplate): - """just display the RSS icon on uniform result set""" - __select__ = ExtResourcesBoxTemplate.__select__ & non_final_entity() - -It takes into account: - -* the inherited selection criteria (one has to look them up in the - class hierarchy to know the details) - -* non_final_entity, which filters on rsets containing non final - entities (a 'final entity' being synonym for entity attribute) - -This matches our second case. Hence we have to provide a specific -component for the first case:: - - class EntityRSSIconBox(RSSIconBox): - """just display the RSS icon on uniform result set for a single entity""" - __select__ = RSSIconBox.__select__ & one_line_rset() - -Here, one adds the one_line_rset selector, which filters result sets -of size 1. When one chains selectors, the final score is the sum of -the score of each individual selector (unless one of them returns 0, -in which case the object is non selectable). Thus, on a multiple -entities selector, one_line_rset makes the EntityRSSIconBox class non -selectable. For an rset with one entity, the EntityRSSIconBox class -will have a higher score then RSSIconBox, which is what we wanted. - -Of course, once this is done, you have to: - -* fill in the call method of EntityRSSIconBox - -* provide the default implementation of the method returning the RSS - stream url on AnyEntity - -* redefine this method on Blog. - -When to use selectors? -`````````````````````` - -Selectors are to be used whenever arises the need of dispatching on the shape or -content of a result set or whatever else context (value in request form params, -authenticated user groups, etc...). That is, almost all the time. - -XXX add and example of a single view w/ big "if" inside splitted into two views -with appropriate selectors. - - -.. CustomSelectors_ - -Defining your own selectors -``````````````````````````` -.. autoclass:: cubicweb.appobject.Selector - :members: __call__ - -.. autofunction:: cubicweb.appobject.objectify_selector -.. autofunction:: cubicweb.selectors.lltrace - -Selectors __call__ should *always* return a positive integer, and shall never -return `None`. - -Useful abstract base classes for 'entity' selectors: - -.. autoclass:: cubicweb.selectors.EClassSelector -.. autoclass:: cubicweb.selectors.EntitySelector - - -Debugging -````````` - -Once in a while, one needs to understand why a view (or any AppObject) -is, or is not selected appropriately. Looking at which selectors fired -(or did not) is the way. There exists a traced_selection context -manager to help with that, *if you're running your instance in debug mode*. - -Here is an example: - -.. sourcecode:: python - - from cubicweb.selectors import traced_selection - with traced_selection(): - mycomp = self._cw.vreg['views'].select('wfhistory', self._cw, rset=rset) - -Don't forget the 'from __future__ import with_statement' at the module -top-level if you're using python 2.5. - -This will yield additional WARNINGs in the logs, like this:: - - 2009-01-09 16:43:52 - (cubicweb.selectors) WARNING: selector one_line_rset returned 0 for - -You can also give to traced_selection the registry ids of objects on which to debug -you want to debug selection ('wfhistory' in the example above). - -Also, if you're using python 2.4, which as no 'with' yet, you'll have to to it -the following way: - -.. sourcecode:: python - - from cubicweb import selectors - selectors.TRACED_OIDS = ('wfhistory',) - mycomp = self._cw.vreg['views'].select('wfhistory', self._cw, rset=rset) - selectors.TRACED_OIDS = () diff -r 02b52bf9f5f8 -r 0865e1e90674 doc/book/en/development/devrepo/hooks.rst --- a/doc/book/en/development/devrepo/hooks.rst Wed Mar 24 10:23:31 2010 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,32 +0,0 @@ -.. -*- coding: utf-8 -*- - -.. _hooks: - -Hooks -===== - -XXX FILLME - -*Hooks* are executed before or after updating an entity or a relation in the -repository. - -Their prototypes are as follows: - - * after_add_entity (session, entity) - * after_update_entity (session, entity) - * after_delete_entity (session, eid) - * before_add_entity (session, entity) - * before_update_entity (session, entity) - * before_delete_entity (session, eid) - - * after_add_relation (session, fromeid, rtype, toeid) - * after_delete_relation (session, fromeid, rtype, toeid) - * before_add_relation (session, fromeid, rtype, toeid) - * before_delete_relation (session, fromeid, rtype, toeid) - - * server_startup - * server_shutdown - - * session_open - * session_close - diff -r 02b52bf9f5f8 -r 0865e1e90674 doc/book/en/development/devrepo/index.rst --- a/doc/book/en/development/devrepo/index.rst Wed Mar 24 10:23:31 2010 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,14 +0,0 @@ -.. -*- coding: utf-8 -*- - -Repository customization -++++++++++++++++++++++++ -.. toctree:: - :maxdepth: 1 - - sessions - hooks - notifications - operations - tasks - - diff -r 02b52bf9f5f8 -r 0865e1e90674 doc/book/en/development/devrepo/notifications.rst --- a/doc/book/en/development/devrepo/notifications.rst Wed Mar 24 10:23:31 2010 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,6 +0,0 @@ -.. -*- coding: utf-8 -*- - -Notifications management -======================== - -XXX FILLME diff -r 02b52bf9f5f8 -r 0865e1e90674 doc/book/en/development/devrepo/operations.rst --- a/doc/book/en/development/devrepo/operations.rst Wed Mar 24 10:23:31 2010 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,38 +0,0 @@ -.. -*- coding: utf-8 -*- - -Repository operations -====================== - -When one needs to perform operations (real world operations like mail -notifications, file operations, real-world side-effects) at -transaction commit time, Operations are the way to go. - -Possible events are: - -* precommit: the pool is preparing to commit. You shouldn't do - anything things which has to be reverted if the commit fail at this - point, but you can freely do any heavy computation or raise an - exception if the commit can't go. You can add some new operation - during this phase but their precommit event won't be triggered - -* commit: the pool is preparing to commit. You should avoid to do to - expensive stuff or something that may cause an exception in this - event - -* revertcommit: if an operation failed while commited, this event is - triggered for all operations which had their commit event already to - let them revert things (including the operation which made fail the - commit) - -* rollback: the transaction has been either rollbacked either - - - intentionaly - - a precommit event failed, all operations are rollbacked - - a commit event failed, all operations which are not been triggered - for commit are rollbacked - -Exceptions signaled from within a rollback are logged and swallowed. - -The order of operations may be important, and is controlled according -to operation's class (see : Operation, LateOperation, SingleOperation, -SingleLastOperation). diff -r 02b52bf9f5f8 -r 0865e1e90674 doc/book/en/development/devrepo/sessions.rst --- a/doc/book/en/development/devrepo/sessions.rst Wed Mar 24 10:23:31 2010 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,27 +0,0 @@ -.. -*- coding: utf-8 -*- - -Sessions -======== - -There are three kinds of sessions. - -* `user sessions` are the most common: they are related to users and - carry security checks coming with user credentials - -* `super sessions` are children of ordinary user sessions and allow to - bypass security checks (they are created by calling unsafe_execute - on a user session); this is often convenient in hooks which may - touch data that is not directly updatable by users - -* `internal sessions` have all the powers; they are also used in only a - few situations where you don't already have an adequate session at - hand, like: user authentication, data synchronisation in - multi-source contexts - -.. note:: - Do not confuse the session type with their connection mode, for - instance : 'in memory' or 'pyro'. - -[WRITE ME] - -* authentication and management of sessions diff -r 02b52bf9f5f8 -r 0865e1e90674 doc/book/en/development/devrepo/tasks.rst --- a/doc/book/en/development/devrepo/tasks.rst Wed Mar 24 10:23:31 2010 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,9 +0,0 @@ -.. -*- coding: utf-8 -*- - -Tasks -========= - -[WRITE ME] - -* repository tasks - diff -r 02b52bf9f5f8 -r 0865e1e90674 doc/book/en/development/devweb/controllers.rst --- a/doc/book/en/development/devweb/controllers.rst Wed Mar 24 10:23:31 2010 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,79 +0,0 @@ -Controllers ------------ - -Overview -++++++++ - -Controllers are responsible for taking action upon user requests -(loosely following the terminology of the MVC meta pattern). - -The following controllers are provided out-of-the box in CubicWeb. We -list them by category. - -Browsing: - -* the View controller (web/views/basecontrollers.py) is associated - with most browsing actions within a CubicWeb application: it always - instantiates a `main template` and lets the ResultSet/Views dispatch - system build up the whole content; it handles ObjectNotFound and - NoSelectableObject errors that may bubble up to its entry point, in - an end-user-friendly way (but other programming errors will slip - through) - -* the JSon controller (web/views/basecontrollers.py) provides services - for Ajax calls, typically using JSON as a serialization format for - input, and sometimes using either JSON or XML for output; - -* the Login/Logout controllers (web/views/basecontrollers.py) make - effective user login or logout requests - -Edition: - -* the Edit controller (web/views/editcontroller.py) handles CRUD - operations in response to a form being submitted; it works in close - association with the Forms, to which it delegates some of the work - -* the Form validator controller (web/views/basecontrollers.py) - provides form validation from Ajax context, using the Edit - controller, to implement the classic form handling loop (user edits, - hits 'submit/apply', validation occurs server-side by way of the - Form validator controller, and the UI is decorated with failure - information, either global or per-field , until it is valid) - -Other: - -* the SendMail controller (web/views/basecontrollers.py) is reponsible - for outgoing email notifications - -* the MailBugReport controller (web/views/basecontrollers.py) allows - to quickly have a `repotbug` feature in one's application - -Registration -++++++++++++ - -All controllers (should) live in the 'controllers' namespace within -the global registry. - -API -+++ - -Most API details should be resolved by source code inspection, as the -various controllers have differing goals. - -`web/controller.py` contains the top-level abstract Controller class and -its (NotImplemented) entry point `publish(rset=None)` method. - -A handful of helpers are also provided there: - -* process_rql builds a result set from an rql query typically issued - from the browser (and available through _cw.form['rql']) - -* validate_cache will force cache validation handling with respect to - the HTTP Cache directives (that were typically originally issued - from a previous server -> client response); concrete Controller - implementations dealing with HTTP (thus, for instance, not the - SendMail controller) may very well call this in their publication - process. - - - diff -r 02b52bf9f5f8 -r 0865e1e90674 doc/book/en/development/devweb/css.rst --- a/doc/book/en/development/devweb/css.rst Wed Mar 24 10:23:31 2010 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,29 +0,0 @@ -.. -*- coding: utf-8 -*- - -CSS Stylesheet ---------------- -Conventions -~~~~~~~~~~~ - -XXX external_resources variable - naming convention - request.add_css - - -Extending / overriding existing styles -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -We cannot modify the order in which the application is reading the CSS. In -the case we want to create new CSS style, the best is to define it a in a new -CSS located under ``myapp/data/`` and use those new styles while writing -customized views and templates. - -If you want to modify an existing CSS styling property, you will have to use -``!important`` declaration to override the existing property. The application -apply a higher priority on the default CSS and you can not change that. -Customized CSS will not be read first. - - -CubicWeb stylesheets -~~~~~~~~~~~~~~~~~~~~ -XXX explain diffenrent files and main classes diff -r 02b52bf9f5f8 -r 0865e1e90674 doc/book/en/development/devweb/facets.rst --- a/doc/book/en/development/devweb/facets.rst Wed Mar 24 10:23:31 2010 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,129 +0,0 @@ -The facets system ------------------ -XXX feed me more (below is the extracted of adim blog) - - -Recently, for internal purposes, we've made a little cubicweb application to -help us -organizing visits to find new office locations. Here's an *excerpt* of the -schema: - -.. sourcecode:: python - - class Office(WorkflowableEntityType): - price = Int(description='euros / m2 / HC / HT') - surface = Int(description='m2') - description = RichString(fulltextindexed=True) - has_address = SubjectRelation('PostalAddress', cardinality='1?', composite='subject') - proposed_by = SubjectRelation('Agency') - comments = ObjectRelation('Comment', cardinality='1*', composite='object') - screenshots = SubjectRelation(('File', 'Image'), cardinality='*1', - composite='subject') - -The two other entity types defined in the schema are `Visit` and `Agency` but we -can also guess from the above that this application uses the two cubes -`comment`_ and -`addressbook`_ (remember, cubicweb is only a game where you assemble cubes !). - -While we know that just defining the schema in enough to have a full, usable, -(testable !) application, we also know that every application needs to be -customized to fulfill the needs it was built for. So in this case, what we -needed most was some custom filters that would let us restrict searches -according -to surfaces, prices or zipcodes. Fortunately for us, Cubicweb provides the -**facets** (image_) mechanism and a few base classes that make the task quite -easy: - -.. sourcecode:: python - - class PostalCodeFacet(RelationFacet): - __regid__ = 'postalcode-facet' # every registered class must have an id - __select__ = implements('Office') # this facet should only be selected when - # visualizing offices - rtype = 'has_address' # this facet is a filter on the entity linked to - # the office thrhough the relation - # has_address - target_attr = 'postalcode' # the filter's key is the attribute "postal_code" - # of the target PostalAddress entity - -This is a typical `RelationFacet`: we want to be able to filter offices -according -to the attribute `postalcode` of their associated `PostalAdress`. Each line in -the class is explained by the comment on its right. - -Now, here is the code to define a filter based on the `surface` attribute of the -`Office`: - -.. sourcecode:: python - - class SurfaceFacet(AttributeFacet): - __regid__ = 'surface-facet' # every registered class must have an id - __select__ = implements('Office') # this facet should only be selected when - # visualizing offices - rtype = 'surface' # the filter's key is the attribute "surface" - comparator = '>=' # override the default value of operator since - # we want to filter according to a - # minimal - # value, not an exact one - - def rset_vocabulary(self, ___): - """override the default vocabulary method since we want to hard-code - our threshold values. - Not overriding would generate a filter box with all existing surfaces - defined in the database. - """ - return [('> 200', '200'), ('> 250', '250'), - ('> 275', '275'), ('> 300', '300')] - - -And that's it: we have two filter boxes automatically displayed on each page -presenting more than one office. The `price` facet is basically the same as the -`surface` one but with a different vocabulary and with ``rtype = 'price'``. - -(The cube also benefits from the builtin google map views defined by -cubicweb but that's for another blog). - -.. _image: http://www.cubicweb.org/image/197646?vid=download -.. _comment: http://www.cubicweb.org/project/cubicweb-comment -.. _addressbook: http://www.cubicweb.org/project/cubicweb-addressbook - -CubicWeb has this really nice builtin `facet`_ system to -define restrictions `filters`_ really as easily as possible. - -We've just added two new kind of facets in CubicWeb : - -- The **RangeFacet** which displays a slider using `jquery`_ - to choose a lower bound and an upper bound. The **RangeWidget** - works with either numerical values or date values - -- The **HasRelationFacet** which displays a simple checkbox and - lets you refine your selection in order to get only entities - that actually use this relation. - -.. image :: http://www.cubicweb.org/Image/343498?vid=download - - -Here's an example of code that defines a facet to filter -musical works according to their composition date: - -.. sourcecode:: python - - class CompositionDateFacet(DateRangeFacet): - # 1. make sure this facet is displayed only on Track selection - __select__ = DateRangeFacet.__select__ & implements('Track') - # 2. give the facet an id required by CubicWeb) - __regid__ = 'compdate-facet' - # 3. specify the attribute name that actually stores the date in the DB - rtype = 'composition_date' - -And that's it, on each page displaying tracks, you'll be able to filter them -according to their composition date with a jquery slider. - -All this, brought by CubicWeb (in the next 3.3 version) - -.. _facet: http://en.wikipedia.org/wiki/Faceted_browser -.. _filters: http://www.cubicweb.org/blogentry/154152 -.. _jquery: http://www.jqueryui.com/ - -To use **HasRelationFacet** on a reverse relation add ``role = 'object'`` in -it's definitions. diff -r 02b52bf9f5f8 -r 0865e1e90674 doc/book/en/development/devweb/form.rst --- a/doc/book/en/development/devweb/form.rst Wed Mar 24 10:23:31 2010 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,77 +0,0 @@ -Form construction ------------------- - -CubicWeb provides usual form/field/widget/renderer abstraction to -provde some generic building blocks which will greatly help you in -building forms properly integrated with CubicWeb (coherent display, -error handling, etc...). - -A form basically only holds a set of fields, and has te be bound to a -renderer which is responsible to layout them. Each field is bound to a -widget that will be used to fill in value(s) for that field (at form -generation time) and 'decode' (fetch and give a proper Python type to) -values sent back by the browser. - -The Field class and basic fields -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. autoclass:: cubicweb.web.formfields.Field - -Existing field types are: - -.. autoclass:: cubicweb.web.formfields.StringField -.. autoclass:: cubicweb.web.formfields.PasswordField -.. autoclass:: cubicweb.web.formfields.RichTextField -.. autoclass:: cubicweb.web.formfields.FileField -.. autoclass:: cubicweb.web.formfields.EditableFileField -.. autoclass:: cubicweb.web.formfields.IntField -.. autoclass:: cubicweb.web.formfields.BooleanField -.. autoclass:: cubicweb.web.formfields.FloatField -.. autoclass:: cubicweb.web.formfields.DateField -.. autoclass:: cubicweb.web.formfields.DateTimeField -.. autoclass:: cubicweb.web.formfields.TimeField -.. autoclass:: cubicweb.web.formfields.RelationField -.. autoclass:: cubicweb.web.formfields.CompoundField - - -Widgets -~~~~~~~ -Base class for widget is :class:cubicweb.web.formwidgets.FieldWidget class. - -Existing widget types are: - -.. autoclass:: cubicweb.web.formwidgets.HiddenInput -.. autoclass:: cubicweb.web.formwidgets.TextInput -.. autoclass:: cubicweb.web.formwidgets.PasswordInput -.. autoclass:: cubicweb.web.formwidgets.PasswordSingleInput -.. autoclass:: cubicweb.web.formwidgets.FileInput -.. autoclass:: cubicweb.web.formwidgets.ButtonInput -.. autoclass:: cubicweb.web.formwidgets.TextArea -.. autoclass:: cubicweb.web.formwidgets.FCKEditor -.. autoclass:: cubicweb.web.formwidgets.Select -.. autoclass:: cubicweb.web.formwidgets.CheckBox -.. autoclass:: cubicweb.web.formwidgets.Radio -.. autoclass:: cubicweb.web.formwidgets.DateTimePicker -.. autoclass:: cubicweb.web.formwidgets.JQueryDateTimePicker -.. autoclass:: cubicweb.web.formwidgets.JQueryDatePicker -.. autoclass:: cubicweb.web.formwidgets.JQueryTimePicker -.. autoclass:: cubicweb.web.formwidgets.AjaxWidget -.. autoclass:: cubicweb.web.formwidgets.AutoCompletionWidget -.. autoclass:: cubicweb.web.formwidgets.EditableURLWidget - -Other classes in this module, which are not proper widget (they are not associated to -field) but are used as form controls, may also be useful: Button, SubmitButton, -ResetButton, ImgButton, - - -Of course you can not use any widget with any field... - -Renderers -~~~~~~~~~ - -.. autoclass:: cubicweb.web.views.formrenderers.BaseFormRenderer -.. autoclass:: cubicweb.web.views.formrenderers.HTableFormRenderer -.. autoclass:: cubicweb.web.views.formrenderers.EntityCompositeFormRenderer -.. autoclass:: cubicweb.web.views.formrenderers.EntityFormRenderer -.. autoclass:: cubicweb.web.views.formrenderers.EntityInlinedFormRenderer - diff -r 02b52bf9f5f8 -r 0865e1e90674 doc/book/en/development/devweb/httpcaching.rst --- a/doc/book/en/development/devweb/httpcaching.rst Wed Mar 24 10:23:31 2010 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,3 +0,0 @@ -HTTP cache management ---------------------- -XXX feedme \ No newline at end of file diff -r 02b52bf9f5f8 -r 0865e1e90674 doc/book/en/development/devweb/index.rst --- a/doc/book/en/development/devweb/index.rst Wed Mar 24 10:23:31 2010 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,21 +0,0 @@ -Web development -=============== - -In this chapter, we will describe the core api for web development in the *CubicWeb* framework. - -.. toctree:: - :maxdepth: 1 - - request - publisher - controllers - property - rtags - views - gettingdata - form - facets - httpcaching - js - css - internationalization diff -r 02b52bf9f5f8 -r 0865e1e90674 doc/book/en/development/devweb/internationalization.rst --- a/doc/book/en/development/devweb/internationalization.rst Wed Mar 24 10:23:31 2010 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,145 +0,0 @@ -.. -*- coding: utf-8 -*- - -.. _internationalization: - -Internationalization ---------------------- - -Cubicweb fully supports the internalization of its content and interface. - -Cubicweb's interface internationalization is based on the translation project `GNU gettext`_. - -.. _`GNU gettext`: http://www.gnu.org/software/gettext/ - -Cubicweb' internalization involves two steps: - -* in your Python code and cubicweb-tal templates : mark translatable strings - -* in your instance : handle the translation catalog - -String internationalization -~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -User defined string -``````````````````` - -In the Python code and cubicweb-tal templates translatable strings can be -marked in one of the following ways : - - * by using the *built-in* function `_` :: - - class PrimaryView(EntityView): - """the full view of an non final entity""" - __regid__ = 'primary' - title = _('primary') - - OR - - * by using the equivalent request's method :: - - class NoResultView(EmptyRsetView): - """default view when no result has been found""" - __regid__ = 'noresult' - - def call(self, **kwargs): - self.w(u'
%s
\n' - % self._cw._('No result matching query')) - -The goal of the *built-in* function `_` is only **to mark the -translatable strings**, it will only return the string to translate -itself, but not its translation (it's actually another name for the -`unicode` builtin). - -In the other hand the request's method `self._cw._` is meant to retrieve the -proper translation of translation strings in the requested language. - -Finally you can also use the `__` attribute of request object to get a -translation for a string *which should not itself added to the catalog*, -usually in case where the actual msgid is created by string interpolation :: - - self._cw.__('This %s' % etype) - -In this example ._cw.__` is used instead of ._cw._` so we don't have 'This %s' in -messages catalogs. - - -Translations in cubicweb-tal template can also be done with TAL tags -`i18n:content` and `i18n:replace`. - - -If you need to add messages on top of those that can be found in the source, -you can create a file named `i18n/static-messages.pot`. - -Generated string -```````````````` - -We do not need to mark the translation strings of entities/relations used by a -particular instance's schema as they are generated automatically. String for -various actions are also generated. - -For exemple the following schema :: - - Class EntityA(EntityType): - relationa2b = SubjectRelation('EntityB') - - class EntityB(EntityType): - pass - -May generate the following message :: - - add Execution has_export File subject - -This message will be used in views of ``EntityA`` for creation of a new -``EntityB`` with a preset relation ``relation_a2b`` between the current -``EntityA`` and the new ``EntityB``. The opposite message :: - - add Execution has_export File object - -Is used for similar creation of an ``EntityA`` from a view of ``EntityB``. The -title of they respective creation form will be :: - - creating EntityB (EntityA %(linkto)s relation_a2b EntityB) - - creating EntityA (EntityA relation_a2b %(linkto)s EntityA) - -In the translated string you can use ``%(linkto)s`` for reference to the source -``entity``. - -Handle the translation catalog -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Once the internationalization is done in your code, you need to populate and -update the translation catalog. Cubicweb provides the following commands for this -purpose: - - -* `i18ncubicweb` updates Cubicweb framework's translation - catalogs. Unless you actually work on the framework itself, you - don't need to use this command. - -* `i18ncube` updates the translation catalogs of *one particular - cube* (or of all cubes). After this command is - executed you must update the translation files *.po* in the "i18n" - directory of your template. This command will of course not remove - existing translations still in use. - -* `i18ninstance` recompiles the translation catalogs of *one particular - instance* (or of all instances) after the translation catalogs of - its cubes have been updated. This command is automatically - called every time you create or update your instance. The compiled - catalogs (*.mo*) are stored in the i18n//LC_MESSAGES of - instance where `lang` is the language identifier ('en' or 'fr' - for exemple). - - -Example -``````` -You have added and/or modified some translation strings in your cube -(after creating a new view or modifying the cube's schema for exemple). -To update the translation catalogs you need to do: - -1. `cubicweb-ctl i18ncube ` -2. Edit the /i18n/xxx.po files and add missing translations (empty `msgstr`) -3. `hg ci -m "updated i18n catalogs"` -4. `cubicweb-ctl i18ninstance ` - diff -r 02b52bf9f5f8 -r 0865e1e90674 doc/book/en/development/devweb/js.rst --- a/doc/book/en/development/devweb/js.rst Wed Mar 24 10:23:31 2010 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,72 +0,0 @@ -.. -*- coding: utf-8 -*- - -Javascript ----------- - -*CubicWeb* uses quite a bit of javascript in its user interface and -ships with jquery (1.3.x) and parts of the jquery UI -library, plus a number of homegrown files and also other thirparty -libraries. - -All javascript files are stored in cubicweb/web/data/. There are -around thirty js files there. In a cube it goes to data/. - -Obviously one does not want javascript pieces to be loaded all at -once, hence the framework provides a number of mechanisms and -conventions to deal with javascript resources. - -Conventions -~~~~~~~~~~~ - -It is good practice to name cube specific js files after the name of -the cube, like this : 'cube.mycube.js', so as to avoid name clashes. - -XXX external_resources variable (which needs love) - -CubicWeb javascript api -~~~~~~~~~~~~~~~~~~~~~~~ - -Javascript resources are typically loaded on demand, from views. The -request object (available as self._cw from most application objects, -for instance views and entities objects) has a few methods to do that: - -* `add_js(self, jsfiles, localfile=True)` which takes a sequence of - javascript files and writes proper entries into the HTML header - section. The localfile parameter allows to declare resources which - are not from web/data (for instance, residing on a content delivery - network). - -* `add_onload(self, jscode)` which adds one raw javascript code - snippet inline in the html headers. This is quite useful for setting - up early jQuery(document).ready(...) initialisations. - -Overview of what's available -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -* jquery.* : jquery and jquery UI library - -* cubicweb.python.js : adds a number of practical extension to stdanrd - javascript objects (on Date, Array, String, some list and dictionary - operations), and a pythonesque way to build classes. Defines a - CubicWeb namespace. - -* cubicweb.htmlhelpers.js : a small bag of convenience functions used - in various other cubicweb javascript resources (baseuri, progress - cursor handling, popup login box, html2dom function, etc.) - -* cubicweb.ajax.js : concentrates all ajax related facilities (it - extends jQuery with the loahxhtml function, provides a handfull of - high-level ajaxy operations like asyncRemoteExec, reloadComponent, - replacePageChunk, getDomFromResponse) - -* cubicweb.widgets.js : provides a widget namespace and constructors - and helpers for various widgets (mainly facets and timeline) - -* cubicweb.edition.js : used by edition forms - -* cubicweb.preferences.js : used by the preference form - -* cubicweb.facets.js : used by the facets mechanism - -xxx massmailing, gmap, fckcwconfig, timeline-bundle, timeline-ext, -calendar, goa, flotn tazy, tabs, bookmarks diff -r 02b52bf9f5f8 -r 0865e1e90674 doc/book/en/development/devweb/property.rst --- a/doc/book/en/development/devweb/property.rst Wed Mar 24 10:23:31 2010 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,12 +0,0 @@ -The property mecanism ---------------------- -XXX CWProperty and co - - -Property API -~~~~~~~~~~~~ -XXX feed me - -Registering and using your own property -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -XXX feed me diff -r 02b52bf9f5f8 -r 0865e1e90674 doc/book/en/development/devweb/publisher.rst --- a/doc/book/en/development/devweb/publisher.rst Wed Mar 24 10:23:31 2010 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,4 +0,0 @@ -Publisher ---------- - -XXX cubicweb.web.application; coop diagram for execution of a http query \ No newline at end of file diff -r 02b52bf9f5f8 -r 0865e1e90674 doc/book/en/development/devweb/request.rst --- a/doc/book/en/development/devweb/request.rst Wed Mar 24 10:23:31 2010 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,59 +0,0 @@ - - -The `Request` class (`cubicweb.web`) -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -A request instance is created when an HTTP request is sent to the web server. -It contains informations such as form parameters, user authenticated, etc. - -**Globally, a request represents a user query, either through HTTP or not -(we also talk about RQL queries on the server side for example).** - -An instance of `Request` has the following attributes: - -* `user`, instance of `cubicweb.common.utils.User` corresponding to the authenticated - user -* `form`, dictionary containing the values of a web form -* `encoding`, character encoding to use in the response - -But also: - -:Session data handling: - * `session_data()`, returns a dictionary containing all the session data - * `get_session_data(key, default=None)`, returns a value associated to the given - key or the value `default` if the key is not defined - * `set_session_data(key, value)`, assign a value to a key - * `del_session_data(key)`, suppress the value associated to a key - - -:Cookies handling: - * `get_cookie()`, returns a dictionary containing the value of the header - HTTP 'Cookie' - * `set_cookie(cookie, key, maxage=300)`, adds a header HTTP `Set-Cookie`, - with a minimal 5 minutes length of duration by default (`maxage` = None - returns a *session* cookie which will expire when the user closes the browser - window) - * `remove_cookie(cookie, key)`, forces a value to expire - -:URL handling: - * `url()`, returns the full URL of the HTTP request - * `base_url()`, returns the root URL of the web application - * `relative_path()`, returns the relative path of the request - -:And more...: - * `set_content_type(content_type, filename=None)`, adds the header HTTP - 'Content-Type' - * `get_header(header)`, returns the value associated to an arbitrary header - of the HTTP request - * `set_header(header, value)`, adds an arbitrary header in the response - * `cursor()` returns a RQL cursor on the session - * `execute(*args, **kwargs)`, shortcut to ``.cursor().execute()`` - * `property_value(key)`, properties management (`CWProperty`) - * dictionary `data` to store data to share informations between components - *while a request is executed* - -Please note that this class is abstract and that a concrete implementation -will be provided by the *frontend* web used (in particular *twisted* as of -today). For the views or others that are executed on the server side, -most of the interface of `Request` is defined in the session associated -to the client. diff -r 02b52bf9f5f8 -r 0865e1e90674 doc/book/en/development/devweb/rtags.rst --- a/doc/book/en/development/devweb/rtags.rst Wed Mar 24 10:23:31 2010 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,14 +0,0 @@ -Configuring the generated interface ------------------------------------ - - -Relation tags -~~~~~~~~~~~~~ -.. automodule:: cubicweb.rtags - - - -The ``uicfg`` module -~~~~~~~~~~~~~~~~~~~~ -.. automodule:: cubicweb.web.uicfg - diff -r 02b52bf9f5f8 -r 0865e1e90674 doc/book/en/development/devweb/views.rst --- a/doc/book/en/development/devweb/views.rst Wed Mar 24 10:23:31 2010 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,269 +0,0 @@ - -.. _Views: - -Views ------ - -This chapter aims to describe the concept of a `view` used all along -the development of a web application and how it has been implemented -in *CubicWeb*. - -We'll start with a description of the interface providing you with a basic -understanding of the classes and methods available, then detail the view -selection principle which makes *CubicWeb* web interface very flexible. - -A `View` is an object applied to another object such as an entity. - -Basic class for views -~~~~~~~~~~~~~~~~~~~~~ - -Class `View` (`cubicweb.view`) -````````````````````````````````````` - -This class is an abstraction of a view class, used as a base class for every -renderable object such as views, templates, graphic components, etc. - -A `View` is instantiated to render a result set or part of a result set. `View` -subclasses may be parametrized using the following class attributes: - - * `templatable` indicates if the view may be embeded in a main - template or if it has to be rendered standalone (i.e. XML views - must not be embeded in the main template for HTML pages) - * if the view is not templatable, it should set the `content_type` class - attribute to the correct MIME type (text/xhtml by default) - * the `category` attribute may be used in the interface to regroup related - objects together - -At instantiation time, the standard `_cw` and `cw_rset` attributes are -added and the `w` attribute will be set at rendering time. - -A view writes to its output stream thanks to its attribute `w` (an -`UStreamIO`, except for binary views). - -The basic interface for views is as follows (remember that the result set has a -tabular structure with rows and columns, hence cells): - -* `render(**context)`, render the view by calling `call` or - `cell_call` depending on the given parameters - -* `call(**kwargs)`, call the view for a complete result set or null - (the default implementation calls `cell_call()` on each cell of the - result set) - -* `cell_call(row, col, **kwargs)`, call the view for a given cell of a - result set - -* `url()`, returns the URL enabling us to get the view with the current - result set - -* `view(__vid, rset, __fallback_vid=None, **kwargs)`, call the view of identifier - `__vid` on the given result set. It is possible to give a view identifier - of fallback that will be used if the view requested is not applicable to the - result set. This is actually defined on the AppObject class. - -* `wview(__vid, rset, __fallback_vid=None, **kwargs)`, similar to `view` except - the flow is automatically passed in the parameters - -* `html_headers()`, returns a list of HTML headers to set by the main template - -* `page_title()`, returns the title to use in the HTML header `title` - - -Other basic view classes -```````````````````````` -Here are some of the subclasses of `View` defined in `cubicweb.common.view` -that are more concrete as they relate to data rendering within the application: - -* `EntityView`, view applying to lines or cell containing an entity (e.g. an eid) -* `StartupView`, start view that does not require a result set to apply to -* `AnyRsetView`, view applicable to any result set -* `EmptyRsetView`, view applicable to an empty result set - - -Examples of views class ------------------------ - -- Using `templatable`, `content_type` and HTTP cache configuration - -.. sourcecode:: python - - class RSSView(XMLView): - __regid__ = 'rss' - title = _('rss') - templatable = False - content_type = 'text/xml' - http_cache_manager = MaxAgeHTTPCacheManager - cache_max_age = 60*60*2 # stay in http cache for 2 hours by default - - -- Using custom selector - -.. sourcecode:: python - - class SearchForAssociationView(EntityView): - """view called by the edition view when the user asks - to search for something to link to the edited eid - """ - __regid__ = 'search-associate' - title = _('search for association') - __select__ = one_line_rset() & match_search_state('linksearch') & implements('Any') - - -Example of view customization and creation -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -We'll show you now an example of a ``primary`` view and how to customize it. - -If you want to change the way a ``BlogEntry`` is displayed, just override -the method ``cell_call()`` of the view ``primary`` in ``BlogDemo/views.py``: - -.. sourcecode:: python - - from cubicweb.selectors import implements - from cubicweb.web.views.primary improt Primaryview - - class BlogEntryPrimaryView(PrimaryView): - __select__ = PrimaryView.__select__ & implements('BlogEntry') - - def render_entity_attributes(self, entity): - self.w(u'

published on %s

' % - entity.publish_date.strftime('%Y-%m-%d')) - super(BlogEntryPrimaryView, self).render_entity_attributes(entity) - -The above source code defines a new primary view for -``BlogEntry``. The `id` class attribute is not repeated there since it -is inherited through the `primary.PrimaryView` class. - -The selector for this view chains the selector of the inherited class -with its own specific criterion. - -The view method ``self.w()`` is used to output data. Here `lines -08-09` output HTML for the publication date of the entry. - -.. image:: ../../images/lax-book.09-new-view-blogentry.en.png - :alt: blog entries now look much nicer - -Let us now improve the primary view of a blog - -.. sourcecode:: python - - from logilab.mtconverter import xml_escape - from cubicweb.selectors import implements, one_line_rset - from cubicweb.web.views.primary import Primaryview - - class BlogPrimaryView(PrimaryView): - __regid__ = 'primary' - __select__ = PrimaryView.__select__ & implements('Blog') - rql = 'Any BE ORDERBY D DESC WHERE BE entry_of B, BE publish_date D, B eid %(b)s' - - def render_entity_relations(self, entity): - rset = self._cw.execute(self.rql, {'b' : entity.eid}) - for entry in rset.entities(): - self.w(u'

%s

' % entry.view('inblogcontext')) - - class BlogEntryInBlogView(EntityView): - __regid__ = 'inblogcontext' - __select__ = implements('BlogEntry') - - def cell_call(self, row, col): - entity = self.cw_rset.get_entity(row, col) - self.w(u'%s' % - entity.absolute_url(), - xml_escape(entity.content[:50]), - xml_escape(entity.description)) - -This happens in two places. First we override the -render_entity_relations method of a Blog's primary view. Here we want -to display our blog entries in a custom way. - -At `line 10`, a simple request is made to build a result set with all -the entities linked to the current ``Blog`` entity by the relationship -``entry_of``. The part of the framework handling the request knows -about the schema and infers that such entities have to be of the -``BlogEntry`` kind and retrieves them (in the prescribed publish_date -order). - -The request returns a selection of data called a result set. Result -set objects have an .entities() method returning a generator on -requested entities (going transparently through the `ORM` layer). - -At `line 13` the view 'inblogcontext' is applied to each blog entry to -output HTML. (Note that the 'inblogcontext' view is not defined -whatsoever in *CubicWeb*. You are absolutely free to define whole view -families.) We juste arrange to wrap each blogentry output in a 'p' -html element. - -Next, we define the 'inblogcontext' view. This is NOT a primary view, -with its well-defined sections (title, metadata, attribtues, -relations/boxes). All a basic view has to define is cell_call. - -Since views are applied to result sets which can be tables of data, we -have to recover the entity from its (row,col)-coordinates (`line -20`). Then we can spit some HTML. - -But careful: all strings manipulated in *CubicWeb* are actually -unicode strings. While web browsers are usually tolerant to incoherent -encodings they are being served, we should not abuse it. Hence we have -to properly escape our data. The xml_escape() function has to be used -to safely fill (X)HTML elements from Python unicode strings. - - -**This is to be compared to interfaces and protocols in object-oriented -languages. Applying a given view called 'a_view' to all the entities -of a result set only requires to have for each entity of this result set, -an available view called 'a_view' which accepts the entity.** - -**Instead of merely using type based dispatch, we do predicate dispatch -which is quite more powerful.** - -Assuming we added entries to the blog titled `MyLife`, displaying it -now allows to read its description and all its entries. - -.. image:: ../../images/lax-book.10-blog-with-two-entries.en.png - :alt: a blog and all its entries - -**Before we move forward, remember that the selection/view principle is -at the core of *CubicWeb*. Everywhere in the engine, data is requested -using the RQL language, then HTML/XML/text/PNG is output by applying a -view to the result set returned by the query. That is where most of the -flexibility comes from.** - -[WRITE ME] - -* implementing interfaces, calendar for blog entries -* show that a calendar view can export data to ical - -We will implement the `cubicweb.interfaces.ICalendarable` interfaces on -entities.BlogEntry and apply the OneMonthCalendar and iCalendar views -to result sets like "Any E WHERE E is BlogEntry" - -* create view "blogentry table" with title, publish_date, category - -We will show that by default the view that displays -"Any E,D,C WHERE E publish_date D, E category C" is the table view. -Of course, the same can be obtained by calling -self.wview('table',rset) - -* in view blog, select blogentries and apply view "blogentry table" -* demo ajax by filtering blogentry table on category - -we did the same with 'primary', but with tables we can turn on filters -and show that ajax comes for free. -[FILLME] - - -XML views, binaries views... -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -For views generating other formats than HTML (an image generated dynamically -for example), and which can not simply be included in the HTML page generated -by the main template (see above), you have to: - -* set the attribute `templatable` of the class to `False` -* set, through the attribute `content_type` of the class, the MIME type generated - by the view to `application/octet-stream` - -For views dedicated to binary content creation (like dynamically generated -images), we have to set the attribute `binary` of the class to `True` (which -implies that `templatable == False`, so that the attribute `w` of the view could be -replaced by a binary flow instead of unicode). diff -r 02b52bf9f5f8 -r 0865e1e90674 doc/book/en/development/entityclasses/data-as-objects.rst --- a/doc/book/en/development/entityclasses/data-as-objects.rst Wed Mar 24 10:23:31 2010 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,111 +0,0 @@ -Access to persistent data --------------------------- - -Python-level access to persistent data is provided by the -:class:`Entity ` class. - -An entity class is bound to a schema entity type. Descriptors are added when -classes are registered in order to initialize the class according to its schema: - -* we can access the defined attributes in the schema thanks to the attributes of - the same name on instances (typed value) - -* we can access the defined relations in the schema thanks to the relations of - the same name on instances (entities instances list) - - -:Formatting and output generation: - - * `view(vid, **kwargs)`, applies the given view to the entity - - * `absolute_url(**kwargs)`, returns an absolute URL to access the primary view - of an entity - - * `rest_path()`, returns a relative REST URL to get the entity - - * `printable_value(attr, value=_marker, attrtype=None, format='text/html')`, - returns a string enabling the display of an attribute value in a given format - (the value is automatically recovered if necessary) - -:Data handling: - - * `as_rset()`, converts the entity into an equivalent result set simulating the - request `Any X WHERE X eid _eid_` - - * `complete(skip_bytes=True)`, executes a request that recovers all at once - all the missing attributes of an entity - - * `get_value(name)`, returns the value associated to the attribute name given - in parameter - - * `related(rtype, x='subject', limit=None, entities=False)`, returns a list - of entities related to the current entity by the relation given in parameter - - * `unrelated(rtype, targettype, x='subject', limit=None)`, returns a result set - corresponding to the entities not related to the current entity by the - relation given in parameter and satisfying its constraints - - * `set_attributes(**kwargs)`, updates the attributes list with the corresponding - values given named parameters - - * `copy_relations(ceid)`, copies the relations of the entities having the eid - given in the parameters on the current entity - - * `delete()` allows to delete the entity - - -The :class:`AnyEntity` class ----------------------------- - -To provide a specific behavior for each entity, we have to define a class -inheriting from `cubicweb.entities.AnyEntity`. In general, we define this class -in `mycube.entities` module (or in a submodule if we want to split code among -multiple files) so that it will be available on both server and client side. - -The class `AnyEntity` is a sub-class of Entity that add methods to it, -and helps specializing (by further subclassing) the handling of a -given entity type. - -The methods defined for `AnyEntity`, in addition to `Entity`, are the -following ones: - -:Standard meta-data (Dublin Core): - - * `dc_title()`, returns a unicode string corresponding to the - meta-data `Title` (used by default is the first non-meta attribute - of the entity schema) - - * `dc_long_title()`, same as dc_title but can return a more - detailed title - - * `dc_description(format='text/plain')`, returns a unicode string - corresponding to the meta-data `Description` (looks for a - description attribute by default) - - * `dc_authors()`, returns a unicode string corresponding to the meta-data - `Authors` (owners by default) - - * `dc_date(date_format=None)`, returns a unicode string corresponding to - the meta-data `Date` (update date by default) - - * `dc_type(form='')`, returns a string to display the entity type by - specifying the preferred form (`plural` for a plural form) - - -Inheritance ------------ - -When describing a data model, entities can inherit from other entities as is -common in object-oriented programming. - -You have the possibility to adapt some entity attributes, as follow: - -.. sourcecode:: python - - from cubes.OTHER_CUBE import entities - class EntityExample(entities.EntityExample): - def dc_long_title(self): - return '%s (%s)' % (self.name, self.description) - -Notice this is different than yams schema inheritance. - diff -r 02b52bf9f5f8 -r 0865e1e90674 doc/book/en/development/entityclasses/index.rst --- a/doc/book/en/development/entityclasses/index.rst Wed Mar 24 10:23:31 2010 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,13 +0,0 @@ -Data as objects -=============== - -In this chapter, we will introduce the objects that are used to handle -the logic associated to the data stored in the database. - -.. toctree:: - :maxdepth: 1 - - data-as-objects - load-sort - interfaces - more diff -r 02b52bf9f5f8 -r 0865e1e90674 doc/book/en/development/entityclasses/interfaces.rst --- a/doc/book/en/development/entityclasses/interfaces.rst Wed Mar 24 10:23:31 2010 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,63 +0,0 @@ -Interfaces ----------- - -Same thing as object-oriented programming interfaces. - -Definition of an interface is quite trivial. An example from cubicweb -itself (found in cubicweb/interfaces.py): - -.. sourcecode:: python - - class ITree(Interface): - - def parent(self): - """returns the parent entity""" - - def children(self): - """returns the item's children""" - - def children_rql(self): - """XXX returns RQL to get children""" - - def iterchildren(self): - """iterates over the item's children""" - - def is_leaf(self): - """returns true if this node as no child""" - - def is_root(self): - """returns true if this node has no parent""" - - def root(self): - """returns the root object""" - - -Declaration of interfaces implemented by a class -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. sourcecode:: python - - from cubicweb.interfaces import ITree - from cubicweb.mixins import TreeMixIn - - class MyEntity(TreeMixIn, AnyEntity): - __regid__ = 'MyEntity' - __implements__ = AnyEntity.__implements__ + ('ITree',) - - tree_attribute = 'filed_under' - -The TreeMixIn here provides a default implementation for the -interface. The tree_attribute class attribute is actually used by this -implementation to help implement correct behaviour. - -Interfaces (and some implementations as mixins) defined in the library -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. automodule:: cubicweb.interfaces - :members: - -.. automodule:: cubicweb.mixins - :members: - - - diff -r 02b52bf9f5f8 -r 0865e1e90674 doc/book/en/development/entityclasses/load-sort.rst --- a/doc/book/en/development/entityclasses/load-sort.rst Wed Mar 24 10:23:31 2010 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,53 +0,0 @@ - -.. _FetchAttrs: - -Loaded attributes and default sorting management -```````````````````````````````````````````````` - -* The class attribute `fetch_attrs` allows to define in an entity class a list - of names of attributes or relations that should be automatically loaded when - entities of this type are fetched from the database. In the case of relations, - we are limited to *subject of cardinality `?` or `1`* relations. - -* The class method `fetch_order(attr, var)` expects an attribute (or relation) - name as a parameter and a variable name, and it should return a string - to use in the requirement `ORDERBY` of an RQL query to automatically - sort the list of entities of such type according to this attribute, or - `None` if we do not want to sort on the attribute given in the parameter. - By default, the entities are sorted according to their creation date. - -* The class method `fetch_unrelated_order(attr, var)` is similar to - the method `fetch_order` except that it is essentially used to - control the sorting of drop-down lists enabling relations creation - in the editing view of an entity. The default implementation uses - the modification date. Here's how to adapt it for one entity (sort - on the name attribute): :: - - class MyEntity(AnyEntity): - __regid__ = 'MyEntity' - fetch_attrs = ('modification_date', 'name') - - @classmethod - def fetch_unrelated_order(cls, attr, var): - if attr == 'name': - return '%s ASC' % var - return None - - -The function `fetch_config(fetchattrs, mainattr=None)` simplifies the -definition of the attributes to load and the sorting by returning a -list of attributes to pre-load (considering automatically the -attributes of `AnyEntity`) and a sorting function based on the main -attribute (the second parameter if specified, otherwise the first -attribute from the list `fetchattrs`). This function is defined in -`cubicweb.entities`. - -For example: :: - - class Transition(AnyEntity): - """...""" - __regid__ = 'Transition' - fetch_attrs, fetch_order = fetch_config(['name']) - -Indicates that for the entity type "Transition", you have to pre-load -the attribute `name` and sort by default on this attribute. diff -r 02b52bf9f5f8 -r 0865e1e90674 doc/book/en/development/entityclasses/more.rst --- a/doc/book/en/development/entityclasses/more.rst Wed Mar 24 10:23:31 2010 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,14 +0,0 @@ -Navigation on deletion ----------------------- - -XXX after_deletion_path, pre_web_edit - -Controlling output url ------------------------ - -XXX write me - -Controling notification references ----------------------------------- - -XXX write me diff -r 02b52bf9f5f8 -r 0865e1e90674 doc/book/en/development/index.rst --- a/doc/book/en/development/index.rst Wed Mar 24 10:23:31 2010 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,22 +0,0 @@ -.. _Part2: - ---------------------- -Part II - Development ---------------------- - -This part is about developing web applications with the *CubicWeb* framework. - -.. toctree:: - :maxdepth: 2 - :numbered: - - cubes/index - datamodel/index - entityclasses/index - devcore/index - devweb/index - devrepo/index - testing/index - migration/index - webstdlib/index - profiling/index diff -r 02b52bf9f5f8 -r 0865e1e90674 doc/book/en/development/migration/index.rst --- a/doc/book/en/development/migration/index.rst Wed Mar 24 10:23:31 2010 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,198 +0,0 @@ -.. -*- coding: utf-8 -*- - -.. _migration: - -Migration -========= - -One of the main design goals of *CubicWeb* was to support iterative and agile -development. For this purpose, multiple actions are provided to facilitate the -improvement of an instance, and in particular to handle the changes to be -applied to the data model, without loosing existing data. - -The current version of a cube (and of cubicweb itself) is provided in the file -`__pkginfo__.py` as a tuple of 3 integers. - -Migration scripts management ----------------------------- - -Migration scripts has to be located in the directory `migration` of your -cube and named accordingly: - -:: - - [_]_.py - -in which : - -* X.Y.Z is the model version number to which the script enables to migrate. - -* *mode* (between the last "_" and the extension ".py") is used for - distributed installation. It indicates to which part - of the application (RQL server, web server) the script applies. - Its value could be : - - * `common`, applies to the RQL server as well as the web server and updates - files on the hard drive (configuration files migration for example). - - * `web`, applies only to the web server and updates files on the hard drive. - - * `repository`, applies only to the RQL server and updates files on the - hard drive. - - * `Any`, applies only to the RQL server and updates data in the database - (schema and data migration for example). - -Again in the directory `migration`, the file `depends.map` allows to indicate -that for the migration to a particular model version, you always have to first -migrate to a particular *CubicWeb* version. This file can contain comments (lines -starting by `#`) and a dependancy is listed as follows: :: - - : - -For example: :: - - 0.12.0: 2.26.0 - 0.13.0: 2.27.0 - # 0.14 works with 2.27 <= cubicweb <= 2.28 at least - 0.15.0: 2.28.0 - -Base context ------------- - -The following identifiers are pre-defined in migration scripts: - -* `config`, instance configuration - -* `interactive_mode`, boolean indicating that the script is executed in - an interactive mode or not - -* `versions_map`, dictionary of migrated versions (key are cubes - names, including 'cubicweb', values are (from version, to version) - -* `confirm(question)`, function asking the user and returning true - if the user answers yes, false otherwise (always returns true in - non-interactive mode) - -* the function `_`, it is equivalent to `unicode` allowing to flag the strings - to internationalize in the migration scripts. - -In the `repository` scripts, the following identifiers are also defined: - -* `checkpoint`, request confirming and executing a "commit" at checking point - -* `schema`, instance schema (readen from the database) - -* `fsschema`, installed schema on the file system (e.g. schema of - the updated model and cubicweb) - -* `repo`, repository object - -* `session`, repository session object - - -Schema migration ----------------- -The following functions for schema migration are available in `repository` -scripts: - -* `add_attribute(etype, attrname, attrtype=None, commit=True)`, adds a new - attribute to an existing entity type. If the attribute type is not specified, - then it is extracted from the updated schema. - -* `drop_attribute(etype, attrname, commit=True)`, removes an attribute from an - existing entity type. - -* `rename_attribute(etype, oldname, newname, commit=True)`, renames an attribute - -* `add_entity_type(etype, auto=True, commit=True)`, adds a new entity type. - If `auto` is True, all the relations using this entity type and having a known - entity type on the other hand will automatically be added. - -* `drop_entity_type(etype, commit=True)`, removes an entity type and all the - relations using it. - -* `rename_entity_type(oldname, newname, commit=True)`, renames an entity type - -* `add_relation_type(rtype, addrdef=True, commit=True)`, adds a new relation - type. If `addrdef` is True, all the relations definitions of this type will - be added. - -* `drop_relation_type(rtype, commit=True)`, removes a relation type and all the - definitions of this type. - -* `rename_relation(oldname, newname, commit=True)`, renames a relation. - -* `add_relation_definition(subjtype, rtype, objtype, commit=True)`, adds a new - relation definition. - -* `drop_relation_definition(subjtype, rtype, objtype, commit=True)`, removes - a relation definition. - -* `sync_schema_props_perms(ertype=None, syncperms=True, syncprops=True, syncrdefs=True, commit=True)`, - synchronizes properties and/or permissions on: - - the whole schema if ertype is None - - an entity or relation type schema if ertype is a string - - a relation definition if ertype is a 3-uple (subject, relation, object) - -* `change_relation_props(subjtype, rtype, objtype, commit=True, **kwargs)`, changes - properties of a relation definition by using the named parameters of the properties - to change. - -* `set_widget(etype, rtype, widget, commit=True)`, changes the widget used for the - relation of entity type . - -* `set_size_constraint(etype, rtype, size, commit=True)`, changes the size constraints - for the relation of entity type . - -Data migration --------------- -The following functions for data migration are available in `repository` scripts: - -* `rql(rql, kwargs=None, cachekey=None, ask_confirm=True)`, executes an arbitrary RQL - query, either to interrogate or update. A result set object is returned. - -* `add_entity(etype, *args, **kwargs)`, adds a nes entity type of the given - type. The attribute and relation values are specified using the named and - positionned parameters. - -Workflow creation ------------------ - -The following functions for workflow creation are available in `repository` -scripts: - -* `add_workflow(label, workflowof, initial=False, commit=False, **kwargs)`, adds a new workflow - for a given type(s) - -You can find more details about workflows in the chapter :ref:`Workflow` . - -Configuration migration ------------------------ - -The following functions for configuration migration are available in all -scripts: - -* `option_renamed(oldname, newname)`, indicates that an option has been renamed - -* `option_group_change(option, oldgroup, newgroup)`, indicates that an option does not - belong anymore to the same group. - -* `option_added(oldname, newname)`, indicates that an option has been added. - -* `option_removed(oldname, newname)`, indicates that an option has been deleted. - - -Others migration functions --------------------------- -Those functions are only used for low level operations that could not be -accomplished otherwise or to repair damaged databases during interactive -session. They are available in `repository` scripts: - -* `sql(sql, args=None, ask_confirm=True)`, executes an arbitrary SQL query on the system source -* `add_entity_type_table(etype, commit=True)` -* `add_relation_type_table(rtype, commit=True)` -* `uninline_relation(rtype, commit=True)` - - -[FIXME] Add explanation on how to use cubicweb-ctl shell diff -r 02b52bf9f5f8 -r 0865e1e90674 doc/book/en/development/profiling/index.rst --- a/doc/book/en/development/profiling/index.rst Wed Mar 24 10:23:31 2010 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,55 +0,0 @@ -Profiling and performance -========================= - -If you feel that one of your pages takes more time than it should to be -generated, chances are that you're making too many RQL queries. Obviously, -there are other reasons but experience tends to show this is the first thing to -track down. Luckily, CubicWeb provides a configuration option to log RQL -queries. In your ``all-in-one.conf`` file, set the **query-log-file** option:: - - # web application query log file - query-log-file=~/myapp-rql.log - -Then restart your application, reload your page and stop your application. -The file ``myapp-rql.log`` now contains the list of RQL queries that were -executed during your test. It's a simple text file containing lines such as:: - - Any A WHERE X eid %(x)s, X lastname A {'x': 448} -- (0.002 sec, 0.010 CPU sec) - Any A WHERE X eid %(x)s, X firstname A {'x': 447} -- (0.002 sec, 0.000 CPU sec) - -The structure of each line is:: - - --