cubicweb/server/test/unittest_checkintegrity.py
author Denis Laxalde <denis.laxalde@logilab.fr>
Fri, 20 Jan 2017 14:32:34 +0100
changeset 11911 37726f66ff82
parent 11768 b8b71dd09a2c
child 12567 26744ad37953
permissions -rw-r--r--
[tox] Drop "exclude" option from flake8 config and adjust flake8-ok-files.txt accordingly Some files listed in flake8-ok-files.txt were excluded of flake8 run because of the "exclude" option in tox.ini. Some of them were non-existent files (moved), some others were actually non-flake8-compliant. In the latter case, we adjust trivial errors (blank lines, module import not on top of file) but remove others (like cubicweb/test/unittest_utils.py) from flake8-ok-files.txt.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
11361
5a857bba1b79 [c-c] Add a command to check database index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11269
diff changeset
     1
# copyright 2003-2016 LOGILAB S.A. (Paris, FRANCE), all rights reserved.
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4766
diff changeset
     2
# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4766
diff changeset
     3
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4766
diff changeset
     4
# This file is part of CubicWeb.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4766
diff changeset
     5
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4766
diff changeset
     6
# CubicWeb is free software: you can redistribute it and/or modify it under the
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4766
diff changeset
     7
# terms of the GNU Lesser General Public License as published by the Free
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4766
diff changeset
     8
# Software Foundation, either version 2.1 of the License, or (at your option)
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4766
diff changeset
     9
# any later version.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4766
diff changeset
    10
#
5424
8ecbcbff9777 replace logilab-common by CubicWeb in disclaimer
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5421
diff changeset
    11
# CubicWeb is distributed in the hope that it will be useful, but WITHOUT
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4766
diff changeset
    12
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4766
diff changeset
    13
# FOR A PARTICULAR PURPOSE.  See the GNU Lesser General Public License for more
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4766
diff changeset
    14
# details.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4766
diff changeset
    15
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4766
diff changeset
    16
# You should have received a copy of the GNU Lesser General Public License along
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4766
diff changeset
    17
# with CubicWeb.  If not, see <http://www.gnu.org/licenses/>.
1977
606923dff11b big bunch of copyright / docstring update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1802
diff changeset
    18
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    19
import sys
11361
5a857bba1b79 [c-c] Add a command to check database index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11269
diff changeset
    20
import unittest
10806
bd98cd3b7869 [server/test] import StringIO from io on python3
Julien Cristau <julien.cristau@logilab.fr>
parents: 9797
diff changeset
    21
bd98cd3b7869 [server/test] import StringIO from io on python3
Julien Cristau <julien.cristau@logilab.fr>
parents: 9797
diff changeset
    22
from six import PY2
bd98cd3b7869 [server/test] import StringIO from io on python3
Julien Cristau <julien.cristau@logilab.fr>
parents: 9797
diff changeset
    23
if PY2:
bd98cd3b7869 [server/test] import StringIO from io on python3
Julien Cristau <julien.cristau@logilab.fr>
parents: 9797
diff changeset
    24
    from StringIO import StringIO
bd98cd3b7869 [server/test] import StringIO from io on python3
Julien Cristau <julien.cristau@logilab.fr>
parents: 9797
diff changeset
    25
else:
bd98cd3b7869 [server/test] import StringIO from io on python3
Julien Cristau <julien.cristau@logilab.fr>
parents: 9797
diff changeset
    26
    from io import StringIO
bd98cd3b7869 [server/test] import StringIO from io on python3
Julien Cristau <julien.cristau@logilab.fr>
parents: 9797
diff changeset
    27
11911
37726f66ff82 [tox] Drop "exclude" option from flake8 config and adjust flake8-ok-files.txt accordingly
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11768
diff changeset
    28
from cubicweb import devtools  # noqa: E402
37726f66ff82 [tox] Drop "exclude" option from flake8 config and adjust flake8-ok-files.txt accordingly
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11768
diff changeset
    29
from cubicweb.devtools.testlib import CubicWebTC  # noqa: E402
37726f66ff82 [tox] Drop "exclude" option from flake8 config and adjust flake8-ok-files.txt accordingly
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11768
diff changeset
    30
from cubicweb.server.checkintegrity import check, check_indexes, reindex_entities  # noqa: E402
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    31
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    32
11361
5a857bba1b79 [c-c] Add a command to check database index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11269
diff changeset
    33
class CheckIntegrityTC(unittest.TestCase):
9797
4e640ab62f51 [tests/checkintegrity] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9576
diff changeset
    34
5954
987086484876 [fti migration] test and fix reindexation of some specific entity types
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5424
diff changeset
    35
    def setUp(self):
11768
b8b71dd09a2c [test] Avoid pytest discovery warnings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11361
diff changeset
    36
        handler = devtools.get_test_db_handler(devtools.TestServerConfiguration('data', __file__))
7186
287f2273917f [test] drop some warnings introduced by new test dabase handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6631
diff changeset
    37
        handler.build_db_cache()
9797
4e640ab62f51 [tests/checkintegrity] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9576
diff changeset
    38
        self.repo, _cnx = handler.get_repo_and_cnx()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    39
        sys.stderr = sys.stdout = StringIO()
5954
987086484876 [fti migration] test and fix reindexation of some specific entity types
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5424
diff changeset
    40
987086484876 [fti migration] test and fix reindexation of some specific entity types
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5424
diff changeset
    41
    def tearDown(self):
987086484876 [fti migration] test and fix reindexation of some specific entity types
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5424
diff changeset
    42
        sys.stderr = sys.__stderr__
987086484876 [fti migration] test and fix reindexation of some specific entity types
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5424
diff changeset
    43
        sys.stdout = sys.__stdout__
987086484876 [fti migration] test and fix reindexation of some specific entity types
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5424
diff changeset
    44
        self.repo.shutdown()
987086484876 [fti migration] test and fix reindexation of some specific entity types
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5424
diff changeset
    45
987086484876 [fti migration] test and fix reindexation of some specific entity types
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5424
diff changeset
    46
    def test_checks(self):
9797
4e640ab62f51 [tests/checkintegrity] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9576
diff changeset
    47
        with self.repo.internal_cnx() as cnx:
4e640ab62f51 [tests/checkintegrity] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9576
diff changeset
    48
            check(self.repo, cnx, ('entities', 'relations', 'text_index', 'metadata'),
9576
5d4f662f5e31 [devtools] make get_repo_and_cnx return a repoapi ClientConnection
Julien Cristau <julien.cristau@logilab.fr>
parents: 8578
diff changeset
    49
                  reindex=False, fix=True, withpb=False)
5954
987086484876 [fti migration] test and fix reindexation of some specific entity types
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5424
diff changeset
    50
987086484876 [fti migration] test and fix reindexation of some specific entity types
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5424
diff changeset
    51
    def test_reindex_all(self):
9797
4e640ab62f51 [tests/checkintegrity] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9576
diff changeset
    52
        with self.repo.internal_cnx() as cnx:
4e640ab62f51 [tests/checkintegrity] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9576
diff changeset
    53
            cnx.execute('INSERT Personne X: X nom "toto", X prenom "tutu"')
4e640ab62f51 [tests/checkintegrity] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9576
diff changeset
    54
            cnx.commit()
4e640ab62f51 [tests/checkintegrity] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9576
diff changeset
    55
            self.assertTrue(cnx.execute('Any X WHERE X has_text "tutu"'))
4e640ab62f51 [tests/checkintegrity] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9576
diff changeset
    56
            reindex_entities(self.repo.schema, cnx, withpb=False)
4e640ab62f51 [tests/checkintegrity] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9576
diff changeset
    57
            self.assertTrue(cnx.execute('Any X WHERE X has_text "tutu"'))
5954
987086484876 [fti migration] test and fix reindexation of some specific entity types
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5424
diff changeset
    58
987086484876 [fti migration] test and fix reindexation of some specific entity types
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5424
diff changeset
    59
    def test_reindex_etype(self):
9797
4e640ab62f51 [tests/checkintegrity] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9576
diff changeset
    60
        with self.repo.internal_cnx() as cnx:
4e640ab62f51 [tests/checkintegrity] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9576
diff changeset
    61
            cnx.execute('INSERT Personne X: X nom "toto", X prenom "tutu"')
4e640ab62f51 [tests/checkintegrity] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9576
diff changeset
    62
            cnx.execute('INSERT Affaire X: X ref "toto"')
4e640ab62f51 [tests/checkintegrity] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9576
diff changeset
    63
            cnx.commit()
4e640ab62f51 [tests/checkintegrity] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9576
diff changeset
    64
            reindex_entities(self.repo.schema, cnx, withpb=False,
4e640ab62f51 [tests/checkintegrity] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9576
diff changeset
    65
                             etypes=('Personne',))
4e640ab62f51 [tests/checkintegrity] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9576
diff changeset
    66
            self.assertTrue(cnx.execute('Any X WHERE X has_text "tutu"'))
4e640ab62f51 [tests/checkintegrity] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9576
diff changeset
    67
            self.assertTrue(cnx.execute('Any X WHERE X has_text "toto"'))
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 0
diff changeset
    68
11361
5a857bba1b79 [c-c] Add a command to check database index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11269
diff changeset
    69
5a857bba1b79 [c-c] Add a command to check database index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11269
diff changeset
    70
class SqliteCheckIndexesTC(CubicWebTC):
5a857bba1b79 [c-c] Add a command to check database index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11269
diff changeset
    71
5a857bba1b79 [c-c] Add a command to check database index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11269
diff changeset
    72
    def test_check_indexes(self):
5a857bba1b79 [c-c] Add a command to check database index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11269
diff changeset
    73
        with self.admin_access.repo_cnx() as cnx:
5a857bba1b79 [c-c] Add a command to check database index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11269
diff changeset
    74
            sys.stdout = stream = StringIO()
5a857bba1b79 [c-c] Add a command to check database index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11269
diff changeset
    75
            try:
5a857bba1b79 [c-c] Add a command to check database index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11269
diff changeset
    76
                status = check_indexes(cnx)
5a857bba1b79 [c-c] Add a command to check database index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11269
diff changeset
    77
            finally:
5a857bba1b79 [c-c] Add a command to check database index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11269
diff changeset
    78
                sys.stdout = sys.__stdout__
5a857bba1b79 [c-c] Add a command to check database index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11269
diff changeset
    79
            self.assertEqual(status, 0, stream.getvalue())
5a857bba1b79 [c-c] Add a command to check database index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11269
diff changeset
    80
5a857bba1b79 [c-c] Add a command to check database index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11269
diff changeset
    81
5a857bba1b79 [c-c] Add a command to check database index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11269
diff changeset
    82
class PGCheckIndexesTC(SqliteCheckIndexesTC):
11768
b8b71dd09a2c [test] Avoid pytest discovery warnings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11361
diff changeset
    83
    configcls = devtools.PostgresApptestConfiguration
11361
5a857bba1b79 [c-c] Add a command to check database index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11269
diff changeset
    84
5a857bba1b79 [c-c] Add a command to check database index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11269
diff changeset
    85
    @classmethod
5a857bba1b79 [c-c] Add a command to check database index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11269
diff changeset
    86
    def setUpClass(cls):
11768
b8b71dd09a2c [test] Avoid pytest discovery warnings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11361
diff changeset
    87
        devtools.startpgcluster(__file__)
11361
5a857bba1b79 [c-c] Add a command to check database index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11269
diff changeset
    88
        super(PGCheckIndexesTC, cls).setUpClass()
5a857bba1b79 [c-c] Add a command to check database index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11269
diff changeset
    89
5a857bba1b79 [c-c] Add a command to check database index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11269
diff changeset
    90
    @classmethod
5a857bba1b79 [c-c] Add a command to check database index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11269
diff changeset
    91
    def tearDownClass(cls):
11768
b8b71dd09a2c [test] Avoid pytest discovery warnings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11361
diff changeset
    92
        devtools.stoppgcluster(__file__)
11361
5a857bba1b79 [c-c] Add a command to check database index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11269
diff changeset
    93
        super(PGCheckIndexesTC, cls).tearDownClass()
5a857bba1b79 [c-c] Add a command to check database index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11269
diff changeset
    94
5a857bba1b79 [c-c] Add a command to check database index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11269
diff changeset
    95
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    96
if __name__ == '__main__':
11361
5a857bba1b79 [c-c] Add a command to check database index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11269
diff changeset
    97
    unittest.main()