fix #822890, missing argument on abstract source backup method make backup failed on multi-sources instances

merge

[schema] ensure we don't remove has_text relation type unfortunatly

merge

Added tag cubicweb-debian-version-3.7.3-1 for changeset 44c7bf90df71

Added tag cubicweb-version-3.7.3 for changeset 768beb8e15f1

[distrib] fix pkginfo

3.7.3

[hooks] better message on bad .event class attribute

[repository] forbid usage of set_attributes() in before_add_entity set_attributes() will generate a RQL query that will silently fail. An explicit error is better. You can still use the ``entity[attr] = value`` notation that won't generate a SQL query but still update the inner state of the entity and its edited_attributes attribute.

merge

[cleanup] tb already printed by self.exception; add note

[facet] fix 'bookmark this search': * vid wasn't included in the link generated by the facet box * proper url quoting in both js / box code (try an has_text search with '&' inside...)

[widget] more wide textarea on bookmark edition

[distribution] the web/data/images directory has to be handled in pkginfo

[test] use the new temporary_appojects CM to simplfy unittest_storage

[session] try to reconnect on unexpected error in system_sql. We need this for proper reconnection of pool used by hooks or looping task doing sql queries (such as cw_cleanup_transaction)

[source] log attempt to reconnect using warning level. Also, don't relog failed sql (already logged)

[test] avoid critical log by informing the source we consider Personne as a multi-sources type

[test] use new temporary_appobjects context manager

[test] that doesn't smell good but fix the test...

[vreg] we do want the opposite...

[testlib] simple helper method to register temporary appobjects (hooks for instance) with a context manager

[undo] fix transactions table name in cleanup operation

Add context management methode to dbapi.Connection the connection is automaticly closed when exiting the context.

i18n update

[i18n] use named substitution to avoid gettext warnings

[dataimport] print transaction id when we get one

[cleanup] add some notes

[optimisation] _select_best is used very intensivly. Peanuts optimization may count :)

[optimization] traced_selection is used very intensively in debug mode optimizing it is worth the pain. Basically : don't do stuff unnecessary when we don't trace anything.

[optimization] improve massive write performance by optimizing hooks selection profiling on some massive deletion showed up that 2/3 of the time was spent in hooks selection. Those changes make it much more acceptable (through selection is still not negligeable): * use one registry for each event, so we've much less hooks to check when emiting an event as well as no more need for the match_event selector. This required ability to put one appobject into several registries, using a __registries__ class attribute. * check for deprecated .enabled at registry initialization time instead of at selection time A very simple HooksManager class has been reintroduce to choose the right registry on call_hooks. Those optimisations leads to a ~x3 factor of time necessary to delete 16000 entities.

[upgrade] after upgrade, fork command to start instance avoid grave uicfg pb (explicit rules all removed...)

merge

[repo creation] removing existing entities of 'single' cardinality relatino should be considered as 'activeintegrity' hook. Also don't disable that category during repo creation to avoid pb such as two default workflows for one entity types

merge

handle Service Close request correctly (closes #736039) The windows event self._stop_event was not used, and twisted was not getting a chance of terminating cleanly by calling its shutdown event handlers (which would eventually call pyro_unregister). Fixed by calling reactor.stop()

[js form validation] fix case where we've None/null keys in the errors dictionnary

[cleanup] exists imported, use it

[form] CompoundField should not return itself in actual_fields, this avoid error on iliane with 3.7.2 where it's marked as required (from the schema) while value are provided by subfields

[repo] we want a hooks manager and hooks initialized event on quick start (we trigger events such as server_backup / server_restore for instance)

[distribution] sql extensions files now have proper extension, fix manifest file

backport stable

[events manager] onevent decorator may be given extra arguments to give to em.bind

[cleanup] some notes

[reload] on registry reloading, we should change class of users of opened session

[appobject imports] don't include __init__ in package's name. See comment for more info

[rset] cubes such as iliane hi-jack rset description by poping some lines from it. Handle this on RepeatList

[undo] basic support for undoing of entity creation / relation addition

[undo] init entity cache when undoing an entity deletion

[undo] consistent is/is_instance_of processing

[repo] don't need rset description on those queries

[cleanup] style fixes, add nodes, 0.2 cents refactorings

[undo] during undoing, call hooks in the [active]integrity and undo categories

[repo] refactor fti operation to use set_operation and a single operation whatever the number of entities to ft index

[querier] introduce RepeatList class, used to optimize size of data returned for result set description When rql query has no ambiguity, we used to return the same description * N where N is the size of the result set. Returning RepeatList class avoid that multiplication. According to quick benchmark this improve performance for result set whose size is ~ > 50, has very small penalty for rset < 50, and in any case improve the size of data to be transfered over the network through pyro connection.

[repo] on add entity, set cache as soon as possible + fill type/source cache

[repo] more efficient eid cache operations handling based on set_operation; refactor * split CleanupEidTypeCache operation into one operation for deleted eids and another for added eids, and put them in cw.server.hook * for consistency, move fti unindexing handling from the repository to the system source * handling of deleted eids is done in plan execution (ssplanner), no needs to redo it in *delete_info -> no more need for _prepare_delete_info method, killed

[repo] move eschema_eid function from hooks.metadata to server.utils

[primary view] skip virtual rtypes anyway

[test] CubicWebTC.create_user expect an actual query, and we don't need to give session here

[dataimport] stop disabling undo ; commit return transaction id

[logging] demote static file served to debug

[cleanup]

[repo] optimize massive insertion/deletion by using the new set_operation function Idea is that on massive insertion, cost of handling the list of operation become non negligeable, so we should minimize the number of operations in that list. The set_operation function ease usage of operation associated to data in session.transaction_data, and we only add the operation when data set isn't initialized yet, else we simply add data to the set. The operation then simply process accumulated data.

[repo] make etype which should go in deleted_entities configurable: we only need this for types imported from other multi-sources instances

[repo] don't execute child step twice in EntitiesDeleteStep

[form] consider autoform_field_kwargs for meta fields; allow required customization

[workflow] support for automatic transition starting from the initial state

[testlib] create_user support extra kwargs to give to create_entity(CWUser)

fix dataimport for 3.7.2

use lgc.testlib.TestCase

backport stable

Added tag cubicweb-debian-version-3.7.2-1 for changeset 8fda29a6c219

Added tag cubicweb-version-3.7.2 for changeset d010f749c21d

take care to empty string in cubes search path

backport stable

[ms] fix name error introduced by rql 0.26 api update

add note

3.7.2

[selectors] new is_in_state select to avoid common error when writing state based selector

[repo config] cleanup read_instance_schema / bootstrap_schema / creating mess

fix #726167: i18ninstance use data in bootstrab_cubes instead of the database one

[cleanup] remove 3.7 deprecation warning

[form controller] clear caches in case some attribute participating to the rest path has been modified, avoid redirection to a no more existant page (fix #753567)

i18nupdate

[form] fix validation error handling * type when possible ValidationError raised by the edit controller * don't rely on repository to check required field * turn css class used for field error message to errorMsg to avoid confusion with the error class added to input * fix css when errors are added by the form renderer * fix form renderer to add the error message

[captcha] handle captcha validation properly in the captcha widget also, avoid error if pil isn't installed (only a recommendation)

[testlib] __errorurl may be a relative path, check we're actually removing base url first

[i18ncube] clear_rtag_objects imported when generating cube's .po file

[maintainance] don't crash if we've no in-memory repository

[db dump/restore] fix '#615840: db-dump must be always runnable' by setting repairing on config. Also, avoid to read the whole schema from the database on dump/restore: the bootstrap schema is enough.

cleanup

__missing__ main

[form] put qualified name on validation error, should fix #784299

[test] fixes when executed under apycot environment (also added ldap1 to test host /etc/hosts)

...

[config] move ConfigurationProblem to migration + refactor it to benefit from config methods

[config] more cleanup

deprecate __use__ and __recommend__

add egg support with dependencies auto-installation

fix dumb name error

fix _cubes dir

handle nicely typical installation other than debian package / mercurial forest * new _find_prefix method to detect installation prefix (maybe specified using CW_INSTALL_PREFIX environment variable) * set various resource accessors according to this, cleanup on the way * remove no more necessary APYCOT_ROOT hacks

When a TC failed to initiliase the database, the following test are SKIPPED When a TC failed to initiliase the database, the other test of the same TestCase are skipped with an explicite message instead of failing later for with an unrelated message.

[c-c] new server_maintenance hook, called on c-c shell / upgrade

[appobject] add version number to deprecation warning

update rql dependency

backport stable

fix migration script

[cleanup] useless try except

[source storage] refactor source sql generation and results handling to allow repository side callbacks for instance with the BytesFileSystemStorage, before this change: * fspath, _fsopen function were stored procedures executed on the database -> files had to be available both on the repository *and* the database host * we needed implementation for each handled database Now, those function are python callbacks executed when necessary on the repository side, on data comming from the database. The litle cons are: * you can't do anymore restriction on mapped attributes * you can't write queries which will return in the same rset column some mapped attributes (or not mapped the same way) / some not This seems much acceptable since: * it's much more easy to handle when you start having the db on another host than the repo * BFSS works seemlessly on any backend now * you don't bother that much about the cons (at least in the bfss case): you usually don't do any restriction on Bytes... Bonus points: BFSS is more efficient (no queries under the cover as it was done in the registered procedure) and we have a much nicer/efficient fspath implementation. IMO, that rocks :D

[bfss] fix name error

[source storage] on deletion, entity has no edited_attributes

[cleanup] rename dbms_helper to dbhelper for consistency

[entity] we also have to override dict.pop to properly maintain edited_attributes consistency

[bfss test] some minor refactoring. Test deletion + creation/update/deletion rollback

don't put hooks deleting orphan composites into the 'integrity' category, we usually want it when integrity is deactivated

[facet] allow to specify a target type on relation facets for case such as Keyword/CodeKeyword where we only targets Keyword

don't rewrite config's url during migration

le patch rql26 a été importé

backport stable

[D] book: update backup procedures in additional tips

[cleanup] remove accidentaly commited print

[test] don't try to reset session manager during test, this leads to weird failures when running multiple tests

don't update dontcheck until everything went fine: see usage in after_update_entity, where if we got an Unauthorized at hook time, we will retry and commit time.

[test] having two eid relations is now forbiden by rql

[web test] a basic test, to be completed

[test] don't clear rtags during tests, may cause breakage with manually import appobjects modules (see unittest_tracker for instance)

[web] move workflow related uicfg stuff to the workflow module

dont get eid from entity's dict

[notification] don't create 'changes' in transaction data until actually needed

backport pending_operations on hi-jacked session, see comment

[entity] fix dumb name error on .set_relations(rel=None)

[security] don't add attribute into skip_security if already in edited_attributes, else we may accidentaly skip a desired security check

[repo entity] we have to provide a __delitem__ implementation as well, see example in the docstring

[yams] on creation, specify relations anyway so we don't get spurious warning for accessing eid through the dict api on yams validation

[repo] fix warning message and implementation of bw compat unsafe_execute

stable is now 3.7

oldstable is now 3.6

backport stable

merge

Added tag cubicweb-debian-version-3.6.3-1 for changeset f9fce56d6a0c

Added tag cubicweb-version-3.6.3 for changeset 9c342fa4f1b7

Added tag cubicweb-debian-version-3.7.1-1 for changeset 9194740f070e

Added tag cubicweb-version-3.7.1 for changeset 6b0832bbd1da

backport debian control changes done in 3.7

3.6.3

ticket #636927, add ajaxhtmlhead in controller js_component

[widget] allow kwargs passed to Select widget

take care to empty breadcrumbs

fix bug with damned transmutation functionality, occuring when using an INSERT query which also add relation to the entity

update skeleton for __pkginfo__.py

don't use optional on final relation

fix security issue introduced by 4967:04543ed0bbdc: attributes explicitly set by hooks should not be checked by security hooks

[form js] apply vgodard patch fixing #750056: reorderTabindex js function crashes when form dom id is not 'entityForm'

missing unittest_main

[server] add unit tests for storages

[registry] trigger reload events on set_schema()

[source] only consider edited_attributes in source.preprocess_entity()

[source] implement storages right in the source rather than in hooks The problem is that Storage objects will most probably change entity's dictionary so that values are correctly set before the source's corresponding method (e.g. entity_added()) is called. For instance, the BFSFileStorage will change the original binary data and replace it with the destination file path in order to store the file path in the database. This change must be local to the source in order not to impact other hooks or attribute access during the transaction, the whole idea being that the same application code should work exactly the same whether or not a BFSStorage is used or not.

i18n update

3.7.1

backport stable

[etwist] cleanup, use cProfile instead of hotshot

[repo] start looping task on repo only config (though I bet more work is needed to get it actually working

[repo] call server_startup event in start_looping_task the repo is fully started (registered in pyro for instance)

[security] take care uidrels may contains a relation targetting something else than a constant node (eg IN function)

[ms] this test is already done before putting a relation in uidrels

cleanup

[security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying

[read security] minor optimizations

[web] get rid of spurious '[Errno 2] No such file or directory' on load This was caused by can_do_pdf_conversion() which tries to use '/usr/bin/fop'. When fop was not available, an OSError was raised by suprocess.Popen. This changeset first checks for /usr/bin/fop presence, then tries to run fop. If an error occurs, log it instead of printing it. NOTE for later: why do we test explicitly for "/usr/bin/fop" rather than simply "fop" ?

backport stable

[schema sync] fix another potential name error on synchronizing rdefs

[schema sync] don't try to synchronize infered relation defs, fixing a name error on the way

cleanup

don't update constraints when it's not necessary

[repo] don't call server_shutdown hooks when creating/repairing instance to be consistence with server_startup hooks

[migration] new build_descr argument to rqlexec on the migration helper

fix migration when upgrading from cw < 3.6 to cw 3.7

disable security when undoing

merge

[sql] fix FSPATH implementation plpy.execute() returns a list of dictionaries with column names as keys and corresponding values as values.

[web deletion] must use set_message

fix limited_rql w/ UNION query

backport stable

fix import

Fixed some of the documentation warnings when building the book with sphinx.

move pdf icon component with it's template so we don't see it when the template isn't available due to missing dependancies

backport stable

add missing warn imports

add missing warn imports

improve documentation for uicfg and rtags

backport stable

merge

[cleanup] objtype not used

[schema sync] test and fix bug when updating multiple constraint for the same rdef in the same transaction

3.6 want __regid__, not id

[migration] disable notification by default during migration

[session] fix hijack_user: most transaction_data should be copied (everything but the entity cache

Added tag cubicweb-debian-version-3.7.0-1 for changeset 0c9ff7e496ce

Added tag cubicweb-version-3.7.0 for changeset 270aba1e6fa2

fix control file by using virtual packages

fix common/rql dependencies

3.7.0

test fixes

merge

fix tests

fix test broken by recent rql rewrite / querier changes

backport stable into default

server/web api for accessing to deleted_entites

[R] dataimport: changes - deprecate checkpoint() in favour of commit() - filter on None value in mk_entity - rearrange log messages order

#750055: make it easier to change post logout url

[web auth] fix authentication pb when anonymous are allowed, avoiding the first authentifier to return an anon connection while a following one may find correct authentication info. This make things simpler (eventually)

[web server] simplify base-url handling on startup. Ensure config['base-url'] is correctly set once started

[rql rewrite] until a better solution is found raise BadSchemaDefinition when two inlined relations with security on an optional variable is used, explaining how to bypass it

[rql rewriting] handle case where we've and optional inlined relation in the original query. Also, we should append EXISTS even in subquery to avoid inserting duplicates in results

[rql rewriting] handle case where we want to insert snippet on a variable that has previously been moved to a subquery

backport stable

[backup] fix misplaced paren

fix bad conflict resolution

backport stable

LazySuggestField : remote version of RestrictedSuggestField

typo

backport stable

Add a rssitem_id attribut to RSSView to easly choose the view used for items.

[web] start a new message system based on id of message stored in session's data instead of using __message as today, which is problematic (allow message injection). Also we can have html in messages. Removed the __createdpath hack used to escape those limitation. The old system should still work though (and will probably for a while, though we should progressivly move to the new system where it's possible). Cleanup request paramaters handling on the way.

[notification] fix dumb name error in hijack user

[dbapi] return default argument instead of None when no connection set yet

[cleanup selectors] use authenticated_user, check for form params instead of handling potential key error on missing params

[db backup] fix name error in backup_to_file: we've to pass .confirm all along the chain as for restore

[cleanup] use iteritems

[url] new rebuild_url method on base request

typo

fix apycot detection

provide logilab.database compat when creating database

update debian changelog

[db creation] fix some bugs introduced by the migration to logilab.database

[debian] packaging fixes * remove postgresql-contrib from cubicweb dependency (using tsearch which is included with postgres >= 8.3) * add postgresql-client | mysql-client to cubicweb-server dependencies, necessary for dump/restore of database