[test] pull yapps from Logilab's github Fixes a couple of failures, pull request is almost a year old (https://github.com/smurfix/yapps/pull/2).

[pkg] bump yams requirement to 0.41.1 Needed by changeset c9321aedfbf9.

[entities] import range from six.moves

merge head published by accident

[fti] Improve big table reindexation * Slice the reindexing in batches of 1000 entities. * Make the output more verbose. Closes #3621392

[tox] Reference requirements.txt for hooks environment The file existed but was not referenced.

[devtools/testlib] Remove assertions about direct usage of CubicWebTC/AutomaticWebTest The point is to be able to use a test runner other than pytest (e.g. nose2, nosetests or py.test) to run CubicWeb test suite. The issue is that most other test loaders inspects members of all base classes of test case classes to determine test methods and it hits this `config` property (actually executing it), which breaks the assertion. So just return None instead of failing. Closes #7817147.

Add ldap3 to server test requirements

[pkg] bump version number so tox won't use cw from pypi

[schema] add CubicWebRelationDefinitionSchema Stop monkeypatching yams, inherit from RelationDefinitionSchema instead.

[schema] Let RQLConstraint inherit from BaseRQLConstraint instead of RQLVocabularyConstraint There is apparently no reason for RQLConstraint to inherit from RQLVocabularyConstraint as it gets its `repo_check` method from `RepoEnforcedRQLConstraintMixIn` and RQLVocabularyConstraint (which inherits from BaseRQLConstraint) actually only overrides that method. So simplify inheritance by removing a useless level of indirection. Adjust cw_unrelated_rql accordingly, where it was that RQLConstraint inherited from RQLVocabularyConstraint.

[server] unormalize source logger's name to avoid encoding errors

[entities] parameters should be given to fire_transition, else they are simply ignored

[entity] set _cw_adapters_cache in __init__ rather than on first use

fix typos and indentation in docstrings/docs

session.id → session.sessionid

[test] test page_size problem (related to #8123231)

[predicates] guard against non-integer page_size form parameter Closes #8123231

[server/test] add test for db-dump command

[server] fix db-dump with sqlite 'gzip -c dbname > backupfile' needs to go through the shell.

[serverctl] fix db-dump Since changeset af47954c1015 we would die with: AttributeError: 'ServerMigrationHelper' object has no attribute 'repo'

[server/test] sorting vs dict vs python3

[devtools] more sorted() vs dict vs python3

[web/application] return bytes from redirect_handler and validation_error_handler

[devtools] fix ResourceWarning: unclosed file tmpdb-__default_empty_db__.config

[serverconfig] fix ResourceWarning: unclosed file bootstrap_cubes

Use cubicweb._

[syncschema] hashlib.md5() wants bytes

[server/test] use next() builtin instead of next() method

[server/test] dict.iterkeys

[syncschema] don't arbitrarily remove constraints from the in-memory schema For some constraint types, it may make sense to have several constraints of that type simultaneously. In such a case, removing one of them from the in-memory schema is wrong. Closes #7446353

[server/test] don't leak timeout from one test class to the next Reset SQL_CONNECT_HOOKS after we're done with this test class so the next one doesn't inherit a timeout it doesn't want.

[server/test] CWAttribute.extra_props is Bytes

[dataimport] give unicode objects to psycopg2 copy_from

[dataimport] extid must be a bytes object

[dataimport] import StringIO from io

[dataimport] fix ucsvreader for python3 The csv module in python3 wants str, so use a StreamReader to feed csv.reader and don't decode the output

[server/test] import StringIO from io on python3 Don't change the import for python2, as we print a mix of unicode and str.

[server/datafeed] drop URLLibResponseAdapter.info I don't know where it's used

[devtools] pass a key to sort() method python3 dicts are not comparable

[postcreate] properly set is_instance_of for bootstrap entities Entities created before CWEType itself need the is/is_instance_of relations created later. We were actually only inserting "is". Closes #7656606.

[migration] re-try without print_function when compiling a script fails As part of the python3 porting we added the __future__.print_function import to cubicweb.migration. That started affecting all migration scripts it runs, which may not be prepared. So if compiling a script results in SyntaxError, try again without the print_function flag. Closes #7935030

[test] do not overwrite user's CW_CUBES_PATH in unittest_migration Currently when you rely on your CW_CUBES_PATH environment variable to locate cubes necessary for cubicweb's tests, the migration test crashes because it overrides the env var.

[server/test] dict.itervalues() → iter(dict.values())

make sync_schema_props_perms(<computed rtype>) work as expected It currently ends up with an ExecutionError while we want to synchronize permissions. This makes 3.21.1 migration of any application using computed relation crashing. Also, remove pre 3.6 compat code in hooks that prevent living schema update on computed relation's permission changes.

[web/http_headers] use classmethod decorator

[schema] use classmethod decorator

[devtools] delete postgresql socket directory after killing the server ... or when failing to start it.

[skel] stop deleting cubes egg-info It doesn't hurt and provides some metadata.

[dataimport] Update doc to reflect new API relate() looks deprecated.

[server] fix generate_source_config regression on python2 lgc outputs "str" objects on python2 and python3, so we can't just use io.StringIO (which expects unicode objects). Fixes: c36cda9074c5

[tests] Fix config.repository() in tests The config.repository was overriden too late. Closes #5778294

[schema] RQL queries are always unicode objects in python 3 fixes commit 42c5bd7286b7.

[server/test] use relative import Fixes py3k breakage

[server/schemaserial] fix extra_props on py3k json.dumps returns unicode, and json.load eats unicode, so add explicit encoding/decoding.

[server/test] unicode → six.text_type

[web] fix Field.__str__ on py3k

[web] unicode → six.text_type

[web/test] CubicWebTC.view returns bytes

[uilib] fix traceback on python3

[web] integer division

[web/test] Binary eats bytes

[devtools] make httptest module importable under py3k

[web/test] CubicWebPublisher.handle_request returns bytes

[web] base64 works on bytes not str (py3k)

[web] accept unicode in parsed Authorization header

[web] CubicWebPublisher.ajax_error_handler must return bytes

[py3k] fix i18n

Fix md5crypt and crypt_password test for python3 Make sure to always work with bytes for hashing.

[server/test] use bytes for extid

[devtools/test] firstnames.txt is in latin1

[web/test] unicode → cubicweb._

[web/formwidgets] fix for py3k no idea why we encode that key...

[sobjects] unicode → six.text_type

[server] fix unittest_security for py3k - use source.binary_to_str instead of str(), helps when the backend returns memoryviews - fixup str vs bytes

[py3k] make ldapfeed tests pass

[server] import filter from six.moves

[ldapfeed] port to ldap3 python-ldap has no python3 support. XXX I only checked that unittest_ldapfeed passes.

[server/test] str vs bytes

[server/test] list() around map and zip

[server/test] to fold, unicode → six.text_type

[server/test] delete test that doesn't make sense on python3 XXX if PY2?

[server/test] unicode → six.text_type

[server] unicode → six.text_type

[py3k] fix building rset description for INSERT queries

[datafeed] use explicit encode/decode instead of casting to str

[py3k] io.BytesIO

[py3k] get urllib foo from six.moves

[devtools] CubicWebTC._check_html may have to deal with unicode or bytes If called with template=None, CubicWebTC.view will not go through main-template, and return the unicode data from the view directly instead of encoded data.

[ext/test] fix test relying on repr(text)

[req] fix url_parse_qsl on py3 should be folded with previous conversion

[server/session] unicode → six.text_type

[web/test] str vs bytes

[web/ajax] fix AjaxController on python3

[web/test] str → bytes

[wsgi/test] use bytes instead of str for POST body

[devtools] 'dict_values' object does not support indexing

[multipart] decode form data before calling parse_qs in python 3 parse_qs eats and returns 'str' on both python versions.

[multipart] unicode → six.text_type

[web] assert that CubicWebPublisher.handle_request returns bytes

[py3k] fix validateform controller We need to return bytes not str.

[web] unicode → six.text_type

[multipart] make sure form keys are not bytes

[web] unicode → six.text_type

[wsgi] don't convert form values This is done earlier in multipart.py

[server] import input from six.moves

[server] don't convert user.login to bytes to generate sessionid

[cwctl] stop playing tricks with std{out,err} buffering Python3 doesn't allow unbuffered text I/O, because it doesn't make sense.

[py3k] CubicWebTC.app_handle_request returns bytes

[py3k] unicode → six.text_type

[py3k] http headers should be ascii-only unicode objects

[py3k] Clarify IDownloadable expected types

[py3k] IDownloadable.download_data() returns bytes

[py3k] unicode → six.text_type

[py3k] unicode → six.text_type

[py3k] unicode → six.text_type

[py3k] unicode → six.text_type

[py3k] PageInfo.source contains bytes

[py3k] unicode → six.text_type

[py3k] unicode → six.text_type

[py3k] Don't encode URLs Python 3's urllib works with unicode objects.

[tests] Stop counting pickle bytes in ResultSet tests Between python 2's pickle and cPickle implementations, the various protocol versions added in python 3, the length of the pickled string is completely unpredictable. Instead, do a simple round trip and test a few attributes.

[tests] Port unittest_cwctl to py3k On python 2, sys.stdout takes bytes-like objects whereas python 3's takes unicode-like objects and handles the encoding on its own.

[migration] NoneType is no longer comparable in py3k

Rename gettext to cwgettext, use stdlib gettext cubicweb/gettext.py was a copy of Python 2.6's gettext.py with the context patch from bug https://bugs.python.org/issue2504 Instead of keeping that full copy, just inherit from gettext.GNUTranslations and add the context-related stuff.

[py3k] MethodType no longer takes a class argument

[py3k] unicode → six.text_type

[py3k] im_self → __self__

[py3k] Don't sort Action objects In python 3, arbitrary objects are no longer sortable.

[py3k] use inspect module to get a function's arguments

[py3k] __unicode__ and __str__

[test] call rql.parse with unicode objects Allows to get same results from rqlst.as_string() in python 2 and 3

[py3k] replace cmp with key in sorted() And stop sorting dicts in snippet_key()

[tests] remove useless sort Useless in python 2, actively harmful in python 3 where arbitrary objects are no longer sortable.

[py3k] dict methods no longer return indexable objects

[py3k] unicode → six.text_type

[py3k] unicode → six.text_type

[server] Port BFSS to py3k The BFSS API changes in python 3: * 'defaultdir' MUST be a unicode object * 'fsencoding' MUST NOT be set In python 2, fsencoding handles both the encoding of file paths on the file system (utf-8 by default, but the system may actually be using something else) and the encoding of file paths that will be stored in the database. So in python 3, we wipe the slate clean: * rely on sys.getfilesystemencoding() to convert unicode objects to bytes * always encode paths to utf-8 for storage in the database Caveat emptor / here be dragons: * sys.getfilesystemencoding() depends on the current locale, which therefore MUST be set properly * when migrating an existing instance from py2 to py3, one MAY need to reencode file paths stored in the database

[py3k] Binary.getvalue() returns bytes

[py3k] ensure Binary objects are initialized with bytes

[py3k] __unicode__ and __str__

[py3k] unicode → six.text_type

[mail] port to py3k

[web/views] port JSON views to py3k

[py3k] unicode → six.text_type

[web/controller] req.form is supposed to contain unicode-data

[py3k] unicode → six.text_type

[py3k] unicode vs str vs bytes vs the world

[web/views] Port csvexport views to py3k There are a few hoops to jump through since python 3's csv module _is_ unicode aware.

[py3k] unicode vs str vs bytes vs the world

[web/request] don't convert request parameters if py3k

[py3k] unicode → six.text_type

[py3k] __getslice__ → __getitem__ with slice support

[py3k] __nonzero__ → __bool__

[py3k] unicode vs str vs bytes vs the world

[py3k] use six.add_metaclass

[py3k] unicode → six.text_type

[py3k] add list() around dict methods

[py3k] unicode → six.text_type

[py3k] unicode → six.text_type

[py3k] unicode → six.text_type

[py3k] unicode → six.text_type

[server/utils] crypt_password should always return bytes

[py3k] str → six.binary_type

[py3k] unicode → six.text_type

[server/utils] convert to py3k using six

[py3k] six.PY2

[py3k] use text/binary types from six

[py3k] unicode → six.text_type

[py3k] bytes vs str

[py3k] io.BytesIO

[py3k] unicode → six.text_type

[py3k] bytes required for hashlib.{md5,sha1}()

[wsgi/test] add __main__ section

[py3k] use next builtin instead of next method

[web/views] use next builtin instead of next method

[py3k] declare meta classes using six.add_metaclass

[py3k] replace '_ = unicode' in global scope (closes #7589459)

[py3k] introduce cubicweb._ (related to #7589459) The end goal is to stop relying on '_' being in the builtin scope. In the mean time, deprecate the builtin '_' so that consumers are warned.

[py3k] dict.iterkeys → dict

[py3k] dict.itervalues → dict.values

[py3k] dict.iteritems → dict.items

implement __ne__ whenever we implement __eq__

[test] use unicode for rql queries (7/7) Closes #6694446

[test] use unicode for rql queries (6/7) Related to #6694446

[test] use unicode for rql queries (5/7) Related to #6694446

[test] use unicode for rql queries (4/7) Related to #6694446

[test] use unicode for rql queries (3/7) Related to #6694446

[test] use unicode for rql queries (2/7) Related to #6694446

[test] use unicode for rql queries (1/7) Related to #6694446

[schema] improve normalization of RQLExpressions Parse and print back the expression instead of manipulating the string. Among other benefits, it means we don't mangle embedded string constants that contain commas or multiple spaces. Closes #6694426

[server/rql2sql] use VariableRef.is_equivalent explicitly instead of relying on __eq__ What we want to compare is the variable referenced by the VariableRef, we should make that explicit. Closes #6694406