cubicweb/server/test/unittest_checkintegrity.py
author Denis Laxalde <denis.laxalde@logilab.fr>
Tue, 21 Feb 2017 11:04:19 +0100
branch3.25
changeset 12141 29d032bb70d8
parent 11911 37726f66ff82
child 12567 26744ad37953
permissions -rw-r--r--
Add a "Contributing" section to README with patch submission guidelines For the CubicWeb project and its dependencies, we now prefer patches submission and review by email on a public mailing list. We are thus moving away from the previous vcreview-based workflow taking place on the forge. This change is motivated by the following points: - the current reviewer assignment mechanism (pick a random reviewer, rely on reviewer availability rather than on willingness to review, send related patches to distinct people, etc.) is inefficient if not counter-productive; - most of the times, discussion only happens between the patch submitter and a reviewer with no easy way to increase the audience; - cubicweb-vcreview has no concept of patch series; - cubicweb-vcreview is not actively maintained anymore and its usability keeps deteriorating. We expect that email-based submission and review of patches will circumvent these limitations. Anybody interested in the project is welcome to subscribed to the mailing list and participate to the review process. This patch documents the basic workflow of patches submissions by email.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
11361
5a857bba1b79 [c-c] Add a command to check database index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11269
diff changeset
     1
# copyright 2003-2016 LOGILAB S.A. (Paris, FRANCE), all rights reserved.
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4766
diff changeset
     2
# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4766
diff changeset
     3
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4766
diff changeset
     4
# This file is part of CubicWeb.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4766
diff changeset
     5
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4766
diff changeset
     6
# CubicWeb is free software: you can redistribute it and/or modify it under the
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4766
diff changeset
     7
# terms of the GNU Lesser General Public License as published by the Free
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4766
diff changeset
     8
# Software Foundation, either version 2.1 of the License, or (at your option)
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4766
diff changeset
     9
# any later version.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4766
diff changeset
    10
#
5424
8ecbcbff9777 replace logilab-common by CubicWeb in disclaimer
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5421
diff changeset
    11
# CubicWeb is distributed in the hope that it will be useful, but WITHOUT
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4766
diff changeset
    12
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4766
diff changeset
    13
# FOR A PARTICULAR PURPOSE.  See the GNU Lesser General Public License for more
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4766
diff changeset
    14
# details.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4766
diff changeset
    15
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4766
diff changeset
    16
# You should have received a copy of the GNU Lesser General Public License along
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4766
diff changeset
    17
# with CubicWeb.  If not, see <http://www.gnu.org/licenses/>.
1977
606923dff11b big bunch of copyright / docstring update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1802
diff changeset
    18
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    19
import sys
11361
5a857bba1b79 [c-c] Add a command to check database index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11269
diff changeset
    20
import unittest
10806
bd98cd3b7869 [server/test] import StringIO from io on python3
Julien Cristau <julien.cristau@logilab.fr>
parents: 9797
diff changeset
    21
bd98cd3b7869 [server/test] import StringIO from io on python3
Julien Cristau <julien.cristau@logilab.fr>
parents: 9797
diff changeset
    22
from six import PY2
bd98cd3b7869 [server/test] import StringIO from io on python3
Julien Cristau <julien.cristau@logilab.fr>
parents: 9797
diff changeset
    23
if PY2:
bd98cd3b7869 [server/test] import StringIO from io on python3
Julien Cristau <julien.cristau@logilab.fr>
parents: 9797
diff changeset
    24
    from StringIO import StringIO
bd98cd3b7869 [server/test] import StringIO from io on python3
Julien Cristau <julien.cristau@logilab.fr>
parents: 9797
diff changeset
    25
else:
bd98cd3b7869 [server/test] import StringIO from io on python3
Julien Cristau <julien.cristau@logilab.fr>
parents: 9797
diff changeset
    26
    from io import StringIO
bd98cd3b7869 [server/test] import StringIO from io on python3
Julien Cristau <julien.cristau@logilab.fr>
parents: 9797
diff changeset
    27
11911
37726f66ff82 [tox] Drop "exclude" option from flake8 config and adjust flake8-ok-files.txt accordingly
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11768
diff changeset
    28
from cubicweb import devtools  # noqa: E402
37726f66ff82 [tox] Drop "exclude" option from flake8 config and adjust flake8-ok-files.txt accordingly
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11768
diff changeset
    29
from cubicweb.devtools.testlib import CubicWebTC  # noqa: E402
37726f66ff82 [tox] Drop "exclude" option from flake8 config and adjust flake8-ok-files.txt accordingly
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11768
diff changeset
    30
from cubicweb.server.checkintegrity import check, check_indexes, reindex_entities  # noqa: E402
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    31
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    32
11361
5a857bba1b79 [c-c] Add a command to check database index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11269
diff changeset
    33
class CheckIntegrityTC(unittest.TestCase):
9797
4e640ab62f51 [tests/checkintegrity] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9576
diff changeset
    34
5954
987086484876 [fti migration] test and fix reindexation of some specific entity types
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5424
diff changeset
    35
    def setUp(self):
11768
b8b71dd09a2c [test] Avoid pytest discovery warnings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11361
diff changeset
    36
        handler = devtools.get_test_db_handler(devtools.TestServerConfiguration('data', __file__))
7186
287f2273917f [test] drop some warnings introduced by new test dabase handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6631
diff changeset
    37
        handler.build_db_cache()
9797
4e640ab62f51 [tests/checkintegrity] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9576
diff changeset
    38
        self.repo, _cnx = handler.get_repo_and_cnx()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    39
        sys.stderr = sys.stdout = StringIO()
5954
987086484876 [fti migration] test and fix reindexation of some specific entity types
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5424
diff changeset
    40
987086484876 [fti migration] test and fix reindexation of some specific entity types
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5424
diff changeset
    41
    def tearDown(self):
987086484876 [fti migration] test and fix reindexation of some specific entity types
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5424
diff changeset
    42
        sys.stderr = sys.__stderr__
987086484876 [fti migration] test and fix reindexation of some specific entity types
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5424
diff changeset
    43
        sys.stdout = sys.__stdout__
987086484876 [fti migration] test and fix reindexation of some specific entity types
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5424
diff changeset
    44
        self.repo.shutdown()
987086484876 [fti migration] test and fix reindexation of some specific entity types
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5424
diff changeset
    45
987086484876 [fti migration] test and fix reindexation of some specific entity types
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5424
diff changeset
    46
    def test_checks(self):
9797
4e640ab62f51 [tests/checkintegrity] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9576
diff changeset
    47
        with self.repo.internal_cnx() as cnx:
4e640ab62f51 [tests/checkintegrity] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9576
diff changeset
    48
            check(self.repo, cnx, ('entities', 'relations', 'text_index', 'metadata'),
9576
5d4f662f5e31 [devtools] make get_repo_and_cnx return a repoapi ClientConnection
Julien Cristau <julien.cristau@logilab.fr>
parents: 8578
diff changeset
    49
                  reindex=False, fix=True, withpb=False)
5954
987086484876 [fti migration] test and fix reindexation of some specific entity types
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5424
diff changeset
    50
987086484876 [fti migration] test and fix reindexation of some specific entity types
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5424
diff changeset
    51
    def test_reindex_all(self):
9797
4e640ab62f51 [tests/checkintegrity] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9576
diff changeset
    52
        with self.repo.internal_cnx() as cnx:
4e640ab62f51 [tests/checkintegrity] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9576
diff changeset
    53
            cnx.execute('INSERT Personne X: X nom "toto", X prenom "tutu"')
4e640ab62f51 [tests/checkintegrity] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9576
diff changeset
    54
            cnx.commit()
4e640ab62f51 [tests/checkintegrity] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9576
diff changeset
    55
            self.assertTrue(cnx.execute('Any X WHERE X has_text "tutu"'))
4e640ab62f51 [tests/checkintegrity] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9576
diff changeset
    56
            reindex_entities(self.repo.schema, cnx, withpb=False)
4e640ab62f51 [tests/checkintegrity] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9576
diff changeset
    57
            self.assertTrue(cnx.execute('Any X WHERE X has_text "tutu"'))
5954
987086484876 [fti migration] test and fix reindexation of some specific entity types
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5424
diff changeset
    58
987086484876 [fti migration] test and fix reindexation of some specific entity types
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5424
diff changeset
    59
    def test_reindex_etype(self):
9797
4e640ab62f51 [tests/checkintegrity] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9576
diff changeset
    60
        with self.repo.internal_cnx() as cnx:
4e640ab62f51 [tests/checkintegrity] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9576
diff changeset
    61
            cnx.execute('INSERT Personne X: X nom "toto", X prenom "tutu"')
4e640ab62f51 [tests/checkintegrity] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9576
diff changeset
    62
            cnx.execute('INSERT Affaire X: X ref "toto"')
4e640ab62f51 [tests/checkintegrity] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9576
diff changeset
    63
            cnx.commit()
4e640ab62f51 [tests/checkintegrity] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9576
diff changeset
    64
            reindex_entities(self.repo.schema, cnx, withpb=False,
4e640ab62f51 [tests/checkintegrity] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9576
diff changeset
    65
                             etypes=('Personne',))
4e640ab62f51 [tests/checkintegrity] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9576
diff changeset
    66
            self.assertTrue(cnx.execute('Any X WHERE X has_text "tutu"'))
4e640ab62f51 [tests/checkintegrity] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9576
diff changeset
    67
            self.assertTrue(cnx.execute('Any X WHERE X has_text "toto"'))
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 0
diff changeset
    68
11361
5a857bba1b79 [c-c] Add a command to check database index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11269
diff changeset
    69
5a857bba1b79 [c-c] Add a command to check database index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11269
diff changeset
    70
class SqliteCheckIndexesTC(CubicWebTC):
5a857bba1b79 [c-c] Add a command to check database index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11269
diff changeset
    71
5a857bba1b79 [c-c] Add a command to check database index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11269
diff changeset
    72
    def test_check_indexes(self):
5a857bba1b79 [c-c] Add a command to check database index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11269
diff changeset
    73
        with self.admin_access.repo_cnx() as cnx:
5a857bba1b79 [c-c] Add a command to check database index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11269
diff changeset
    74
            sys.stdout = stream = StringIO()
5a857bba1b79 [c-c] Add a command to check database index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11269
diff changeset
    75
            try:
5a857bba1b79 [c-c] Add a command to check database index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11269
diff changeset
    76
                status = check_indexes(cnx)
5a857bba1b79 [c-c] Add a command to check database index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11269
diff changeset
    77
            finally:
5a857bba1b79 [c-c] Add a command to check database index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11269
diff changeset
    78
                sys.stdout = sys.__stdout__
5a857bba1b79 [c-c] Add a command to check database index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11269
diff changeset
    79
            self.assertEqual(status, 0, stream.getvalue())
5a857bba1b79 [c-c] Add a command to check database index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11269
diff changeset
    80
5a857bba1b79 [c-c] Add a command to check database index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11269
diff changeset
    81
5a857bba1b79 [c-c] Add a command to check database index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11269
diff changeset
    82
class PGCheckIndexesTC(SqliteCheckIndexesTC):
11768
b8b71dd09a2c [test] Avoid pytest discovery warnings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11361
diff changeset
    83
    configcls = devtools.PostgresApptestConfiguration
11361
5a857bba1b79 [c-c] Add a command to check database index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11269
diff changeset
    84
5a857bba1b79 [c-c] Add a command to check database index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11269
diff changeset
    85
    @classmethod
5a857bba1b79 [c-c] Add a command to check database index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11269
diff changeset
    86
    def setUpClass(cls):
11768
b8b71dd09a2c [test] Avoid pytest discovery warnings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11361
diff changeset
    87
        devtools.startpgcluster(__file__)
11361
5a857bba1b79 [c-c] Add a command to check database index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11269
diff changeset
    88
        super(PGCheckIndexesTC, cls).setUpClass()
5a857bba1b79 [c-c] Add a command to check database index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11269
diff changeset
    89
5a857bba1b79 [c-c] Add a command to check database index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11269
diff changeset
    90
    @classmethod
5a857bba1b79 [c-c] Add a command to check database index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11269
diff changeset
    91
    def tearDownClass(cls):
11768
b8b71dd09a2c [test] Avoid pytest discovery warnings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11361
diff changeset
    92
        devtools.stoppgcluster(__file__)
11361
5a857bba1b79 [c-c] Add a command to check database index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11269
diff changeset
    93
        super(PGCheckIndexesTC, cls).tearDownClass()
5a857bba1b79 [c-c] Add a command to check database index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11269
diff changeset
    94
5a857bba1b79 [c-c] Add a command to check database index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11269
diff changeset
    95
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    96
if __name__ == '__main__':
11361
5a857bba1b79 [c-c] Add a command to check database index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11269
diff changeset
    97
    unittest.main()