[server] use sqls_create_multicol_unique_index to index entities.extid SQL Server's UNIQUE is not standards compliant, it doesn't accept two NULL values. Related to #7173702.

[server] fix checkintegrity with pyodbc "msg % row" dies with "not enough arguments for format string" if row is a pyodbc.Row.

[schema] remove unique constraint on unlimited String attribute SQL Server doesn't allow indexing (and thus unique constraints) on nvarchar(max). Related to #7173702.

[sources/native] fix potential NameError (closes #7364249)

[py3k] unicode → six.text_type

[devtools/testlib] Port CubicWebTC.view() to py3k testlib.TestCase.set_description's argument will eventually be printed out to sys.stdout/stderr. On python 2, this object takes bytes-like objects (eg, str) whereas python 3's takes unicode-like objects and handles the encoding on its own. Thus the 'str' type can used in both versions to eventually display the description on screen.

[devtools/testlib] port 'fake_form' test to io.BytesIO

[dataimport] port ucsvreader tests to io.BytesIO

[py3k] io.BytesIO

[py3k] io.StringIO

Port cw.Binary to io.BytesIO And add plenty of tests too!

[py3k] execfile → exec

[py3k] file → open

[py3k] six.integer_types

[py3k] basestring → six.string_types

[py3k] import CookieJar using six.moves

[py3k] import urllib2 from six.moves

[py3k] import range using six.moves

[py3k] import queue using six.moves

[py3k] import filter using six.moves

[py3k] import zip using six.moves

[py3k] import SimpleCookie using six.moves

[py3k] import HTTP client constants and exceptions using six.moves

[py3k] import URL mangling functions using six.moves

[py3k] import 'pickle' using six.moves

[pkg] Add dependency on six 1.4.0

[tests] Replace use of deprecated TestCase.assert_

[predicates] Simplify range(len()) iteration

[dataimport] Use pickle.dump instead of dumps

[web/facet] Simplify reversed/sorted calls

[web/test] Use TestCase.setUpClass instead of setUpModule

Use isinstance() instead of type comparison

Unused imports

[py3k] tuple unpacking in function args

[py3k] ur'' is gone

[py3k] octals and long

[py3k] Fix raise with embedded traceback python 3 removed the raise syntax with 2 and 3 expressions.

[py3k] print function

[py3k] except as

[py3k] backtick to repr

[hooks/syncschema] make sure CWUniqueTogetherConstraintDelOp happens before CWConstraintDelOp SQLServer refuses to index an unlimited text column, so we should drop unique_together constraints (which imply an index) before we drop size constraints. Closes #5560601.

[hooks/syncschema] only call "ALTER TABLE" once when changing a size constraint Until now we would: - remove the old size constraint from the in-memory schema - call update_rdef_column which removes the size restriction from the column's type - add the new constraint object - call update_rdef_column which adds the size restriction back This breaks on SQL Server when the column is involved in an index (e.g. as part of a multi-column unique constraint), because in the intermediate stage the column's type is "nvarchar(max)", which is not indexable. Of course we must still detect the case where a size constraint is really dropped and update the db schema accordingly. Closes #5557633.

[syncschema] only add to the `synchronized` set if all possible updates have been done Otherwise a simple sync_schema(syncprops=False) turns the subsequent sync_schema() calls into a no-op. Closes #6510918.

[qunit] remove unused 'data_files' parameter Its use was removed when we stoppped pointing firefox at file:// web/test/jstests/ajax_url2.html has never actually been used, it's only been ever mentioned in a QUnit comment.

[urlpublish] RESTPathEvaluator now use vid_from_rset This avoid cases where vid may be unexpectedly overwritten. For instance when you define a custom vid for a mime type in VID_BY_MIMETYPE, it will currently be overriden by the URL publisher when you access to /<etype> with the proper Accept header. To do so, a new 'check_table' argument is introduced, which may cause bw compatibility problems if the function has been monkey-patched. Also pep8 a bit the tests. Closes #5705835

[server/sources] use decode_extid in datafeed

[server/sources] add decode_extid method avoids having the same dance in both eid_type_source variants

[server/repo] kill repo.pinfo() session.cnxset is dead, this can't work.

[web/sessions] simplify session cleanup session.cnx.check() is no longer a thing. It dates back to dbapi.

[server/repo] fix typo

[server/sources] AbstractSource.__ne__ Seems confusing to have __eq__ but not __ne__.

[server/sources] fix docstring for eid_type_source

[views] fix ProcessInformationView: SESSION_MANAGER can be None (closes #5753280) eg. with pyramid.

[server/migractions] drop mock_object It doesn't seem to have ever been used.

[test] kill unused lgc.testlib imports

[server/migractions] raising SystemExit sounds like a bad idea Simply skip the action (rename_relation_type) with an early return.

[server/storage] rename 'session' variables to 'cnx' We no longer deal with Session objects here.

[migration] remove repo_connect and use config.repository() instead One less more redundant path to the repo. Related to #3933480.

[repoapi, cwconfig] give a convenience method to get a repository from a config This is a very common need. The private repoapi._get_inmemory_repo becomes config.repository(...)

[cwconfig] remove unused method Not to be confused with serverconfig.migration_handler.

[captcha] Port to io.BytesIO

[web/sessions] the session managers are definitely not components Component (or AppObjects) take a _cw at __init__ time. The session managers want a repository. Related to #1381328.

[web] the AuthenticationManager is no more an appobject It is now explictly instanciated by the session manager. Related to #1381328.

[hooksmanager] remove very old unused module

[devtools] fix CubicWebTC.assertDocTestFile (closes #5930028)

Added tag 3.21.1, debian/3.21.1-1, centos/3.21.1-1 for changeset a8a0de0298a5

[pkg] Let dh_installdocs deal with doc-base

[pkg] various debian packaging fixes - stop using *.install.in files, just use wildcards in *.install - switch to X-Python-Version (preferred for dh_python2) - also call dh_python2 for /usr/share/cubicweb, not just public modules - drop long obsolete ginco and erudi migration code from cubicweb-ctl.postinst - fix cubicweb-ctl cleanup: purge code belongs in postrm, not prerm - don't compress changelog.html

[pkg] 3.21.1

[web] fix AbstractSessionManager.clean_sessions (closes #5757240) sessions were not closed if "last_usage_time = session.cnx.check()" actually raises an AtributeError.

[migration/3.21] ask fewer questions Several questions per table is just unreasonable.

[server] typo fix

[test] check that we reject an rqlexpression in computed relation perms

Fix (de)serialization of ComputedRelation read permissions For normal relation types, permissions don't need to be stored since they're just default values for the relation definitions. However, computed relations are serialized (as CWComputedRType), while their relation definitions are added at schema finalization time, and are only in memory. So add the 'read_permission' relation to CWComputedRType, and the appropriate hooks to save and restore those permissions. To avoid having to touch yams, we drop the 'add' and 'delete' permissions from the default computed relation permissions; this should probably be backported there. The actual permissions (set on the relation definitions) are hardcoded in finalize_computed_relations anyway. In deserialize_schema, the CWComputedRType handling needs to be delayed a little bit, until after we've called deserialize_ertype_permissions. The rql2sql test is adjusted because CWComputedRType has a 'name' attribute and the 'read_permission' relation, which generates ambiguity vs CWEType. We add an explicit CubicWebRelationSchema.check_permission_definitions, since we need to check both that computed and non-computed rtypes are defined properly. Based on report and initial patch from Christophe de Vienne (thanks!). Closes #5706307

Copy read permissions on ComputedRelation rdefs Closes #5694110

[datafeed] fix typo in DataFeedXMLParser.is_deleted (closes #5729755)

[server/sources] make sure entity._cw is a Connection before calling Storages The storage callbacks don't get an explicit Connection objects, so they go through entity._cw to access the repo and schedule operations. Since that entity object comes out of the entity cache, its _cw may be a web request instead of a Connection. So fix it up around the storage callbacks. Closes #5753543

[hooks] base64-encode extid before inserting it into moved_entities If we insert the wrong value we're never going to find these entities again... Closes #5741105.

[hook] remove assumption about entity cache vs cw_edited We were assuming that if an entity is in the connection's cache, it has a cw_edited attribute. This doesn't seem to always be the case. At least in cubicweb 3.21, merging the web and repo-side caches makes this more likely, but Denis says he's also seen this on 3.20.

[hook] don't open-code Connection.entity_cache

[tox] Add missing dependency on webtest for devtools tests

[tox] Add missing dependency on webtest for wsgi tests

[tox] Remove set of PATH environment variable The : concatenation does not work actually so the resulting PATH was just wrong (it did not contain the previous PATH). Anyways, the PATH environment variable is passed by default by tox so it's probably better to let the caller set it (thus avoiding to hard-code the specific postgres version and path). http://tox.readthedocs.org/en/latest/example/basic.html#passing-down-environment-variables

[doc] generalize info about debian/ubuntu repositories That section was not kept up to date with recent distro releases; stop trying.

[doc] fix link to setup info in README ... and fix URL to the repository's gpg key.

[server/session] remove unused cnx_meth function

[server] don't issue a warning for something expected Creating an instance happens all the time in tests, there's no reason to warn every time.

[server] stick a _cw on internal connnection's user Entity._fetch_restrictions goes through user._cw.vreg :( Closes #5705769

[migration/3.21] fix stupid error in migration script str.join doesn't like a list of ints. Closes #5706359.

Added tag 3.21.0, debian/3.21.0-1, centos/3.21.0-1 for changeset 887c6eef8077

[pkg] various fixes - add missing files to MANIFEST - exclude doc/changes from dh_installchangelogs (it's a directory, which dh_installchangelogs doesn't expect) - fix doc build in debian package

[doc] fix broken symlinks

[pkg] update 3.21.0 release date

[doc/3.21] mention the postgresql-only state of the upgrade script

[datafeed] also catch EnvironmentError when trying to load the cwclientlib config file means the config file has not been found, so fall back to the old implementation instead of crashing.

[schema] Make syntax compatible with print_function yams seems to exec() us sometimes (instead of __import__()), and yams.reader uses print_function nowadays.

[pkg] 3.21.0

[test] use FakeFile instead of File in test_fulltextindex File.data is no longer fulltextindexed.

i18n update

[serverctl] fix schema-diff command It hadn't been ported away from dbapi.

[serverct] call server_{maintenance,shutdown} hooks around source-sync command

[serverctl] call server_{maintenance,shutdown} hooks around add-source

Update hgignore

delete unmaintained clone_deps script Closes #2517580.

merge 3.20.9 into 3.21 Fix conflict in unittest_postgres.py by changing sources configuration in PostgresTimeoutConfiguration.__init__ so it happens *after* setUpModule().

Added tag 3.20.9, debian/3.20.9-1, centos/3.20.9-1 for changeset d477e6447582

[pkg] 3.20.9

[forms] don't display 'remove' link of inlined form for the first entity related using a '+' cardinality to its parent entity Closes #5227394

[tox] Update doc testenv to account for documentation restructuring * Build doc from `doc` directory. * Use `sphinx-build` directly instead of `make` as this now works and allows to build within .tox temporary environment (instead of doc/_build), preferable for a "test" action. Related to #5447161.

[schema] drop old backward compat code preventing yams to be itself fully cleaned

[datafeed] allow to use cwclientlib for datafeed's queries (closes #5456849) HTTP based URLs will be handled by a CWProxy if: - cwclientlib is available and - the URL matches a cwclientlibrc config entry Otherwise, fallback to previous implementation. See https://www.cubicweb.org/project/cwclientlib :Warning: This comes with an API modification of DataFeedParser.retrieve_url: it used to accept data and headers arguments to build GET or POST (if data was given) queries, but this was not used by any known code and imply more complicated code.

[cwxml parser] refactor url handling code to use urlparse and urlencode (related to #5456849)

add use_extid_as_cwuri ext entity transform, that will be often necessary and not so easy to write at once

[dataimport] dispatch and deprecate old code previously, the whole dataimport.py content has been copied in dataimport/__init__.py, then dataimport/importer.py API has been introduced. Following these changes, this cset: * dispatch __init__ code into sub-modules * dispatch unittest_dataimport accordingly * deprecates functions and classes that are deprecated by the new API Related to #5414753.

Add tox.ini file * Factor out `commands` for most environments using {envname} substitution to build pytest commande. * Positional arguments are passed on to `pytest` (others may also be passed after a `--`). * Set PATH env variable for tests requiring PostgreSQL. * Also added a `doc` target to build the book. Also update .hgignore exclude .tox directory (where virtualenvs will be created and cubicweb.egg-info created by sdist command). Closes #5447161.

Add requirements.txt files in test directories These files, located in each test directory (when needed), list test dependencies supplementary to install requirements. Some requirements.txt files include psycopg2 because it won't get installed by logilab-database apparently. Related to #5447161.

Spelling fixes in comments and docstrings

[web/views] Clean up confusing backwards compatibility code Dead code since fbb370195199, related to #4943392.

[server/test] avoid lgdb warnings from str to unicode conversion

[web/test] Modernize unittest_propertysheet.py - remove call to os.makedirs in global scope - fold CACHEDIR as an instance attribute - bonus: use unittest directly instead of lgc.testlib

[web/webconfig] Drop pre-3.9 backward compatibility method "has_resource"

[web] Stop using negative indices with os.stat results The python doc says: More items may be added at the end by some implementations. .st_mtime is also more readable than [-2].

[doc] small rst fixes and cleanups in changelog files Related to #4832808

[doc] make the changelog chapter a one html page Related to #4832808

[doc] reformat changes files headers to remove the "what's new" phrasing and add the publication date instead. Related to #4832808

[doc] remove mentions of pyro and reposity instance type from the concepts chapter Related to #4832808

[doc] put the tutorials between the intro and the `Repository development` toc of the book Related to #4832808

[doc] move the documentations from docstring directly to the book. and improve a bit API documentations. Related to #4832808