[repo] refactor fti operation to use set_operation and a single operation whatever the number of entities to ft index

[querier] introduce RepeatList class, used to optimize size of data returned for result set description When rql query has no ambiguity, we used to return the same description * N where N is the size of the result set. Returning RepeatList class avoid that multiplication. According to quick benchmark this improve performance for result set whose size is ~ > 50, has very small penalty for rset < 50, and in any case improve the size of data to be transfered over the network through pyro connection.

[repo] on add entity, set cache as soon as possible + fill type/source cache

[repo] more efficient eid cache operations handling based on set_operation; refactor * split CleanupEidTypeCache operation into one operation for deleted eids and another for added eids, and put them in cw.server.hook * for consistency, move fti unindexing handling from the repository to the system source * handling of deleted eids is done in plan execution (ssplanner), no needs to redo it in *delete_info -> no more need for _prepare_delete_info method, killed

[repo] move eschema_eid function from hooks.metadata to server.utils

[primary view] skip virtual rtypes anyway

[test] CubicWebTC.create_user expect an actual query, and we don't need to give session here

[dataimport] stop disabling undo ; commit return transaction id

[logging] demote static file served to debug

[cleanup]

[repo] optimize massive insertion/deletion by using the new set_operation function Idea is that on massive insertion, cost of handling the list of operation become non negligeable, so we should minimize the number of operations in that list. The set_operation function ease usage of operation associated to data in session.transaction_data, and we only add the operation when data set isn't initialized yet, else we simply add data to the set. The operation then simply process accumulated data.

[repo] make etype which should go in deleted_entities configurable: we only need this for types imported from other multi-sources instances

[repo] don't execute child step twice in EntitiesDeleteStep

[form] consider autoform_field_kwargs for meta fields; allow required customization

[workflow] support for automatic transition starting from the initial state

[testlib] create_user support extra kwargs to give to create_entity(CWUser)

fix dataimport for 3.7.2

use lgc.testlib.TestCase

backport stable

Added tag cubicweb-debian-version-3.7.2-1 for changeset 8fda29a6c219

Added tag cubicweb-version-3.7.2 for changeset d010f749c21d

take care to empty string in cubes search path

backport stable

[ms] fix name error introduced by rql 0.26 api update

add note

3.7.2

[selectors] new is_in_state select to avoid common error when writing state based selector

[repo config] cleanup read_instance_schema / bootstrap_schema / creating mess

fix #726167: i18ninstance use data in bootstrab_cubes instead of the database one

[cleanup] remove 3.7 deprecation warning

[form controller] clear caches in case some attribute participating to the rest path has been modified, avoid redirection to a no more existant page (fix #753567)

i18nupdate

[form] fix validation error handling * type when possible ValidationError raised by the edit controller * don't rely on repository to check required field * turn css class used for field error message to errorMsg to avoid confusion with the error class added to input * fix css when errors are added by the form renderer * fix form renderer to add the error message

[captcha] handle captcha validation properly in the captcha widget also, avoid error if pil isn't installed (only a recommendation)

[testlib] __errorurl may be a relative path, check we're actually removing base url first

[i18ncube] clear_rtag_objects imported when generating cube's .po file

[maintainance] don't crash if we've no in-memory repository

[db dump/restore] fix '#615840: db-dump must be always runnable' by setting repairing on config. Also, avoid to read the whole schema from the database on dump/restore: the bootstrap schema is enough.

cleanup

__missing__ main

[form] put qualified name on validation error, should fix #784299

[test] fixes when executed under apycot environment (also added ldap1 to test host /etc/hosts)

...

[config] move ConfigurationProblem to migration + refactor it to benefit from config methods

[config] more cleanup

deprecate __use__ and __recommend__

add egg support with dependencies auto-installation

fix dumb name error

fix _cubes dir

handle nicely typical installation other than debian package / mercurial forest * new _find_prefix method to detect installation prefix (maybe specified using CW_INSTALL_PREFIX environment variable) * set various resource accessors according to this, cleanup on the way * remove no more necessary APYCOT_ROOT hacks

When a TC failed to initiliase the database, the following test are SKIPPED When a TC failed to initiliase the database, the other test of the same TestCase are skipped with an explicite message instead of failing later for with an unrelated message.

[c-c] new server_maintenance hook, called on c-c shell / upgrade

[appobject] add version number to deprecation warning

update rql dependency

backport stable

fix migration script

[cleanup] useless try except

[source storage] refactor source sql generation and results handling to allow repository side callbacks for instance with the BytesFileSystemStorage, before this change: * fspath, _fsopen function were stored procedures executed on the database -> files had to be available both on the repository *and* the database host * we needed implementation for each handled database Now, those function are python callbacks executed when necessary on the repository side, on data comming from the database. The litle cons are: * you can't do anymore restriction on mapped attributes * you can't write queries which will return in the same rset column some mapped attributes (or not mapped the same way) / some not This seems much acceptable since: * it's much more easy to handle when you start having the db on another host than the repo * BFSS works seemlessly on any backend now * you don't bother that much about the cons (at least in the bfss case): you usually don't do any restriction on Bytes... Bonus points: BFSS is more efficient (no queries under the cover as it was done in the registered procedure) and we have a much nicer/efficient fspath implementation. IMO, that rocks :D

[bfss] fix name error

[source storage] on deletion, entity has no edited_attributes

[cleanup] rename dbms_helper to dbhelper for consistency

[entity] we also have to override dict.pop to properly maintain edited_attributes consistency

[bfss test] some minor refactoring. Test deletion + creation/update/deletion rollback

don't put hooks deleting orphan composites into the 'integrity' category, we usually want it when integrity is deactivated

[facet] allow to specify a target type on relation facets for case such as Keyword/CodeKeyword where we only targets Keyword

don't rewrite config's url during migration

le patch rql26 a été importé

backport stable

[D] book: update backup procedures in additional tips

[cleanup] remove accidentaly commited print

[test] don't try to reset session manager during test, this leads to weird failures when running multiple tests

don't update dontcheck until everything went fine: see usage in after_update_entity, where if we got an Unauthorized at hook time, we will retry and commit time.

[test] having two eid relations is now forbiden by rql

[web test] a basic test, to be completed

[test] don't clear rtags during tests, may cause breakage with manually import appobjects modules (see unittest_tracker for instance)

[web] move workflow related uicfg stuff to the workflow module

dont get eid from entity's dict

[notification] don't create 'changes' in transaction data until actually needed

backport pending_operations on hi-jacked session, see comment

[entity] fix dumb name error on .set_relations(rel=None)

[security] don't add attribute into skip_security if already in edited_attributes, else we may accidentaly skip a desired security check

[repo entity] we have to provide a __delitem__ implementation as well, see example in the docstring

[yams] on creation, specify relations anyway so we don't get spurious warning for accessing eid through the dict api on yams validation

[repo] fix warning message and implementation of bw compat unsafe_execute

stable is now 3.7

oldstable is now 3.6

backport stable

merge

Added tag cubicweb-debian-version-3.6.3-1 for changeset f9fce56d6a0c

Added tag cubicweb-version-3.6.3 for changeset 9c342fa4f1b7

Added tag cubicweb-debian-version-3.7.1-1 for changeset 9194740f070e

Added tag cubicweb-version-3.7.1 for changeset 6b0832bbd1da

backport debian control changes done in 3.7

3.6.3

ticket #636927, add ajaxhtmlhead in controller js_component

[widget] allow kwargs passed to Select widget

take care to empty breadcrumbs

fix bug with damned transmutation functionality, occuring when using an INSERT query which also add relation to the entity

update skeleton for __pkginfo__.py

don't use optional on final relation

fix security issue introduced by 4967:04543ed0bbdc: attributes explicitly set by hooks should not be checked by security hooks

[form js] apply vgodard patch fixing #750056: reorderTabindex js function crashes when form dom id is not 'entityForm'

missing unittest_main

[server] add unit tests for storages

[registry] trigger reload events on set_schema()

[source] only consider edited_attributes in source.preprocess_entity()

[source] implement storages right in the source rather than in hooks The problem is that Storage objects will most probably change entity's dictionary so that values are correctly set before the source's corresponding method (e.g. entity_added()) is called. For instance, the BFSFileStorage will change the original binary data and replace it with the destination file path in order to store the file path in the database. This change must be local to the source in order not to impact other hooks or attribute access during the transaction, the whole idea being that the same application code should work exactly the same whether or not a BFSStorage is used or not.

i18n update

3.7.1

backport stable

[etwist] cleanup, use cProfile instead of hotshot

[repo] start looping task on repo only config (though I bet more work is needed to get it actually working

[repo] call server_startup event in start_looping_task the repo is fully started (registered in pyro for instance)

[security] take care uidrels may contains a relation targetting something else than a constant node (eg IN function)

[ms] this test is already done before putting a relation in uidrels

cleanup

[security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying

[read security] minor optimizations

[web] get rid of spurious '[Errno 2] No such file or directory' on load This was caused by can_do_pdf_conversion() which tries to use '/usr/bin/fop'. When fop was not available, an OSError was raised by suprocess.Popen. This changeset first checks for /usr/bin/fop presence, then tries to run fop. If an error occurs, log it instead of printing it. NOTE for later: why do we test explicitly for "/usr/bin/fop" rather than simply "fop" ?

backport stable

[schema sync] fix another potential name error on synchronizing rdefs

[schema sync] don't try to synchronize infered relation defs, fixing a name error on the way

cleanup

don't update constraints when it's not necessary

[repo] don't call server_shutdown hooks when creating/repairing instance to be consistence with server_startup hooks

[migration] new build_descr argument to rqlexec on the migration helper

fix migration when upgrading from cw < 3.6 to cw 3.7

disable security when undoing

merge

[sql] fix FSPATH implementation plpy.execute() returns a list of dictionaries with column names as keys and corresponding values as values.

[web deletion] must use set_message

fix limited_rql w/ UNION query

backport stable

fix import

Fixed some of the documentation warnings when building the book with sphinx.

move pdf icon component with it's template so we don't see it when the template isn't available due to missing dependancies

backport stable

add missing warn imports

add missing warn imports

improve documentation for uicfg and rtags

backport stable

merge

[cleanup] objtype not used

[schema sync] test and fix bug when updating multiple constraint for the same rdef in the same transaction

3.6 want __regid__, not id

[migration] disable notification by default during migration

[session] fix hijack_user: most transaction_data should be copied (everything but the entity cache

Added tag cubicweb-debian-version-3.7.0-1 for changeset 0c9ff7e496ce

Added tag cubicweb-version-3.7.0 for changeset 270aba1e6fa2

fix control file by using virtual packages

fix common/rql dependencies

3.7.0

test fixes

merge

fix tests

fix test broken by recent rql rewrite / querier changes

backport stable into default

server/web api for accessing to deleted_entites

[R] dataimport: changes - deprecate checkpoint() in favour of commit() - filter on None value in mk_entity - rearrange log messages order

#750055: make it easier to change post logout url

[web auth] fix authentication pb when anonymous are allowed, avoiding the first authentifier to return an anon connection while a following one may find correct authentication info. This make things simpler (eventually)

[web server] simplify base-url handling on startup. Ensure config['base-url'] is correctly set once started

[rql rewrite] until a better solution is found raise BadSchemaDefinition when two inlined relations with security on an optional variable is used, explaining how to bypass it

[rql rewriting] handle case where we've and optional inlined relation in the original query. Also, we should append EXISTS even in subquery to avoid inserting duplicates in results

[rql rewriting] handle case where we want to insert snippet on a variable that has previously been moved to a subquery

backport stable

[backup] fix misplaced paren

fix bad conflict resolution

backport stable

LazySuggestField : remote version of RestrictedSuggestField

typo

backport stable

Add a rssitem_id attribut to RSSView to easly choose the view used for items.

[web] start a new message system based on id of message stored in session's data instead of using __message as today, which is problematic (allow message injection). Also we can have html in messages. Removed the __createdpath hack used to escape those limitation. The old system should still work though (and will probably for a while, though we should progressivly move to the new system where it's possible). Cleanup request paramaters handling on the way.

[notification] fix dumb name error in hijack user

[dbapi] return default argument instead of None when no connection set yet

[cleanup selectors] use authenticated_user, check for form params instead of handling potential key error on missing params

[db backup] fix name error in backup_to_file: we've to pass .confirm all along the chain as for restore

[cleanup] use iteritems

[url] new rebuild_url method on base request

typo

fix apycot detection

provide logilab.database compat when creating database

update debian changelog

[db creation] fix some bugs introduced by the migration to logilab.database

[debian] packaging fixes * remove postgresql-contrib from cubicweb dependency (using tsearch which is included with postgres >= 8.3) * add postgresql-client | mysql-client to cubicweb-server dependencies, necessary for dump/restore of database

[web] yes this is used

[web] cleanup: move delete_entities to the edit controller

[web] stop using deprecated merge_dicts

[utils] improve dump_class by copying back original class doc and module name

[utils] make_uid imrovment: using .hex already remove '-', make key argument optional, document it

deprecates merge_dicts

kill unused method

merge

Added tag cubicweb-debian-version-3.6.2-1 for changeset f3b4bb9121a0

Added tag cubicweb-version-3.6.2 for changeset bfebe3d14d53

[form] fix bug introduced by 4871:a34b76593ab2 with field using multiple input with different suffix: the suffix should be considered in the cache key

i18nupdate

prepare 3.6.2

[form fields] fix severe memory leak due to @cached on field.input_name occuring on forms with persistent fields. Has been triggered in 3.6.1 on our public sites because: * the login form use persistent field since 3.6 * it's displayed on every anonymous request * another bug (fixed during the latest mini-sprint) made it also displayed for authenticated user see explanation in code for more detail. moral: be very careful with @cached...

[debug] allow to specify maxlevel to gc_info

[write security] we must check perm with read security disabled + add missing eid argument to check_perm

merge

[session] del session's thread local store on close

[debug] a new view to help debugging memory leaks

[debug] minor improvments to the registry view

[web req] check latest url in breadcrumbs is not the same as the current one to keep more valuable information

[pyviews] add thead/tbody to pyvaltable view

[repo stats] add cache size

[javascript] upgrade from version 1.0.2 to 1.1 of jquery.autocomplete.js

[web] fix #736332: iespec functionality for add_css Patch provided by vgodard.

[javascript] use jQuery.one('ajax-loaded') instead of jQuery.bind() in add_onload to avoid multiple callback executions

[widgets] fix typo

merge

req has no more a cursor attribute

when some authentication plugin fail, we may try another one

logilab.db compat

[goa] i know goa is broken but that's still that we won't have to fix later :)

[documentation] fix errors in book chapter instance-config

remove deprecation warning introduced by add_onload api change

[refactor] drop rset.vreg attribute, vreg should be accessed through rset.req. Also kill decorate_rset, simply set rset.req where we were calling this method.

missing rename

rename logilab.db into logilab.database

move devtools.dataimport at the cw level since we don't want cubes using it to depends on cubicweb-dev

[widget] allow to specify hour/minute separator on the JQueryTimePicker (vgodard patch)

backport stable

merge

fix hooks control method name + other litle cleanups

cleanup

move SLINK_DIRECTORIES creation to a function avoid unnecessary import (and kill that damed warning of psyco not being installed)

cleanup

[schema hooks] properly check for changes of schema properties avoid useless costly operation (such as full-text reindexing all entities of a type). Refactor inlined handling and fix a bug trying to reindex a final entity type on the way.

[hook] entity_oldnew_value may cause bug on attributes explicitly set to None

misc fixes to ensure logilab.db compatibility

[sql generation] fti_need_distinct quere was not anymore used, potentially causing duplicates on has_text queries with sqlite backend

[repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.

move hooks activation control on session object, so we can have a per transaction control. Added a new `hooks_control` context manager for usual modification of hooks activation.

3.7 depends on python >= 2.5

rename deprecated file

[repo] use logilab.db instead of lgc.adbh/lgc.db/lgc.sqlgen/indexer, test new date extranction functions