cubicweb/devtools/httptest.py
author Sylvain Thénault <sylvain.thenault@logilab.fr>
Fri, 30 Sep 2016 18:25:08 +0200
changeset 11767 432f87a63057
parent 11160 ed048bfd1b88
child 12530 9d88e1177c35
permissions -rw-r--r--
flake8 and all * update some copyright * drop most __docformat__ declaration * fix some flake8 warnings / errors
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
10235
684215aca046 Remove remote repository-access-through-pyro support
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9929
diff changeset
     1
# copyright 2003-2014 LOGILAB S.A. (Paris, FRANCE), all rights reserved.
5654
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
     2
# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
     3
#
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
     4
# This file is part of CubicWeb.
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
     5
#
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
     6
# CubicWeb is free software: you can redistribute it and/or modify it under the
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
     7
# terms of the GNU Lesser General Public License as published by the Free
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
     8
# Software Foundation, either version 2.1 of the License, or (at your option)
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
     9
# any later version.
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    10
#
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    11
# CubicWeb is distributed in the hope that it will be useful, but WITHOUT
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    12
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    13
# FOR A PARTICULAR PURPOSE.  See the GNU Lesser General Public License for more
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    14
# details.
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    15
#
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    16
# You should have received a copy of the GNU Lesser General Public License along
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    17
# with CubicWeb.  If not, see <http://www.gnu.org/licenses/>.
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    18
"""this module contains base classes and utilities for integration with running
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    19
http server
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    20
"""
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10363
diff changeset
    21
from __future__ import print_function
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10363
diff changeset
    22
11767
432f87a63057 flake8 and all
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11160
diff changeset
    23
5654
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    24
9136
eef0388e8fea [devtool] randomise available ports search in http test
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8941
diff changeset
    25
import random
5654
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    26
import threading
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    27
import socket
10603
65ad6980976e [py3k] import URL mangling functions using six.moves
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10589
diff changeset
    28
11160
ed048bfd1b88 [devtools] skip CubicWebServerTC on python3
Julien Cristau <julien.cristau@logilab.fr>
parents: 11158
diff changeset
    29
from six import PY3
10609
e2d8e81bfe68 [py3k] import range using six.moves
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10604
diff changeset
    30
from six.moves import range, http_client
10603
65ad6980976e [py3k] import URL mangling functions using six.moves
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10589
diff changeset
    31
from six.moves.urllib.parse import urlparse
5654
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    32
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    33
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    34
from cubicweb.devtools.testlib import CubicWebTC
6424
f443a2b8a5c7 [devtools] refactor http server initialization in a much saner way
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 6322
diff changeset
    35
from cubicweb.devtools import ApptestConfiguration
5654
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    36
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    37
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    38
def get_available_port(ports_scan):
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    39
    """return the first available port from the given ports range
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    40
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    41
    Try to connect port by looking for refused connection (111) or transport
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    42
    endpoint already connected (106) errors
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    43
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    44
    Raise a RuntimeError if no port can be found
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    45
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    46
    :type ports_range: list
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    47
    :param ports_range: range of ports to test
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    48
    :rtype: int
6424
f443a2b8a5c7 [devtools] refactor http server initialization in a much saner way
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 6322
diff changeset
    49
f443a2b8a5c7 [devtools] refactor http server initialization in a much saner way
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 6322
diff changeset
    50
    .. see:: :func:`test.test_support.bind_port`
5654
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    51
    """
9136
eef0388e8fea [devtool] randomise available ports search in http test
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8941
diff changeset
    52
    ports_scan = list(ports_scan)
eef0388e8fea [devtool] randomise available ports search in http test
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8941
diff changeset
    53
    random.shuffle(ports_scan)  # lower the chance of race condition
5654
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    54
    for port in ports_scan:
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    55
        try:
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    56
            s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    57
            sock = s.connect(("localhost", port))
8695
358d8bed9626 [toward-py3k] rewrite to "except AnException as exc:" (part of #2711624)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 8694
diff changeset
    58
        except socket.error as err:
5654
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    59
            if err.args[0] in (111, 106):
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    60
                return port
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    61
        finally:
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    62
            s.close()
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    63
    raise RuntimeError('get_available_port([ports_range]) cannot find an available port')
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    64
6322
711e7e8c69e3 [windmill] initialize windmill registry if not already existing (possible error case in dev)
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 6315
diff changeset
    65
11158
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
    66
class _CubicWebServerTC(CubicWebTC):
11015
baf463175505 [devtools] Fix database creation issues for CubicWebServerTC tests
Rémi Cardona <remi.cardona@logilab.fr>
parents: 11014
diff changeset
    67
    """Class for running a Twisted-based test web server.
5654
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    68
    """
6424
f443a2b8a5c7 [devtools] refactor http server initialization in a much saner way
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 6322
diff changeset
    69
    ports_range = range(7000, 8000)
f443a2b8a5c7 [devtools] refactor http server initialization in a much saner way
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 6322
diff changeset
    70
5654
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    71
    def start_server(self):
11158
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
    72
        raise NotImplementedError
5654
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    73
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    74
    def stop_server(self, timeout=15):
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    75
        """Stop the webserver, waiting for the thread to return"""
11158
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
    76
        raise NotImplementedError
5654
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    77
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    78
    def web_login(self, user=None, passwd=None):
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    79
        """Log the current http session for the provided credential
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    80
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    81
        If no user is provided, admin connection are used.
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    82
        """
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    83
        if user is None:
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    84
            user  = self.admlogin
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    85
            passwd = self.admpassword
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    86
        if passwd is None:
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    87
            passwd = user
7434
17ef6f9efaa1 [test] fix tests broken by 7427:5338d895b891
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7075
diff changeset
    88
        response = self.web_get("login?__login=%s&__password=%s" %
5654
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    89
                                (user, passwd))
10604
d4bf85db41f2 [py3k] import HTTP client constants and exceptions using six.moves
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10603
diff changeset
    90
        assert response.status == http_client.SEE_OTHER, response.status
5654
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    91
        self._ident_cookie = response.getheader('Set-Cookie')
6315
8ca3ee849bee [test] fix broken tests and minor cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5700
diff changeset
    92
        assert self._ident_cookie
5654
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    93
        return True
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    94
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    95
    def web_logout(self, user='admin', pwd=None):
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    96
        """Log out current http user"""
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    97
        if self._ident_cookie is not None:
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    98
            response = self.web_get('logout')
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    99
        self._ident_cookie = None
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
   100
8727
5bca35901e9b [devtools/httptest] fix syntax error introduced by ce5ae7b80d2c
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8712
diff changeset
   101
    def web_request(self, path='', method='GET', body=None, headers=None):
10604
d4bf85db41f2 [py3k] import HTTP client constants and exceptions using six.moves
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10603
diff changeset
   102
        """Return an http_client.HTTPResponse object for the specified path
5654
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
   103
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
   104
        Use available credential if available.
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
   105
        """
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
   106
        if headers is None:
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
   107
            headers = {}
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
   108
        if self._ident_cookie is not None:
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
   109
            assert 'Cookie' not in headers
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
   110
            headers['Cookie'] = self._ident_cookie
8712
ce5ae7b80d2c [devtools/httptest] allow sending other types of requests besides GET
Julien Cristau <julien.cristau@logilab.fr>
parents: 8695
diff changeset
   111
        self._web_test_cnx.request(method, '/' + path, headers=headers, body=body)
5654
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
   112
        response = self._web_test_cnx.getresponse()
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
   113
        response.body = response.read() # to chain request
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
   114
        response.read = lambda : response.body
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
   115
        return response
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
   116
8712
ce5ae7b80d2c [devtools/httptest] allow sending other types of requests besides GET
Julien Cristau <julien.cristau@logilab.fr>
parents: 8695
diff changeset
   117
    def web_get(self, path='', body=None, headers=None):
ce5ae7b80d2c [devtools/httptest] allow sending other types of requests besides GET
Julien Cristau <julien.cristau@logilab.fr>
parents: 8695
diff changeset
   118
        return self.web_request(path=path, body=body, headers=headers)
ce5ae7b80d2c [devtools/httptest] allow sending other types of requests besides GET
Julien Cristau <julien.cristau@logilab.fr>
parents: 8695
diff changeset
   119
5654
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
   120
    def setUp(self):
11158
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   121
        super(_CubicWebServerTC, self).setUp()
11015
baf463175505 [devtools] Fix database creation issues for CubicWebServerTC tests
Rémi Cardona <remi.cardona@logilab.fr>
parents: 11014
diff changeset
   122
        port = self.config['port'] or get_available_port(self.ports_range)
baf463175505 [devtools] Fix database creation issues for CubicWebServerTC tests
Rémi Cardona <remi.cardona@logilab.fr>
parents: 11014
diff changeset
   123
        self.config.global_set_option('port', port) # force rewrite here
baf463175505 [devtools] Fix database creation issues for CubicWebServerTC tests
Rémi Cardona <remi.cardona@logilab.fr>
parents: 11014
diff changeset
   124
        self.config.global_set_option('base-url', 'http://127.0.0.1:%d/' % port)
baf463175505 [devtools] Fix database creation issues for CubicWebServerTC tests
Rémi Cardona <remi.cardona@logilab.fr>
parents: 11014
diff changeset
   125
        # call load_configuration again to let the config reset its datadir_url
baf463175505 [devtools] Fix database creation issues for CubicWebServerTC tests
Rémi Cardona <remi.cardona@logilab.fr>
parents: 11014
diff changeset
   126
        self.config.load_configuration()
5654
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
   127
        self.start_server()
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
   128
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
   129
    def tearDown(self):
11158
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   130
        self.stop_server()
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   131
        super(_CubicWebServerTC, self).tearDown()
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   132
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   133
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   134
class CubicWebServerTC(_CubicWebServerTC):
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   135
    def start_server(self):
11160
ed048bfd1b88 [devtools] skip CubicWebServerTC on python3
Julien Cristau <julien.cristau@logilab.fr>
parents: 11158
diff changeset
   136
        if PY3:
ed048bfd1b88 [devtools] skip CubicWebServerTC on python3
Julien Cristau <julien.cristau@logilab.fr>
parents: 11158
diff changeset
   137
            self.skipTest('not using twisted on python3')
11158
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   138
        from twisted.internet import reactor
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   139
        from cubicweb.etwist.server import run
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   140
        # use a semaphore to avoid starting test while the http server isn't
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   141
        # fully initilialized
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   142
        semaphore = threading.Semaphore(0)
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   143
        def safe_run(*args, **kwargs):
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   144
            try:
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   145
                run(*args, **kwargs)
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   146
            finally:
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   147
                semaphore.release()
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   148
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   149
        reactor.addSystemEventTrigger('after', 'startup', semaphore.release)
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   150
        t = threading.Thread(target=safe_run, name='cubicweb_test_web_server',
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   151
                args=(self.config, True), kwargs={'repo': self.repo})
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   152
        self.web_thread = t
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   153
        t.start()
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   154
        semaphore.acquire()
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   155
        if not self.web_thread.isAlive():
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   156
            # XXX race condition with actual thread death
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   157
            raise RuntimeError('Could not start the web server')
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   158
        #pre init utils connection
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   159
        parseurl = urlparse(self.config['base-url'])
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   160
        assert parseurl.port == self.config['port'], (self.config['base-url'], self.config['port'])
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   161
        self._web_test_cnx = http_client.HTTPConnection(parseurl.hostname,
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   162
                                                        parseurl.port)
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   163
        self._ident_cookie = None
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   164
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   165
    def stop_server(self, timeout=15):
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   166
        """Stop the webserver, waiting for the thread to return"""
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   167
        from twisted.internet import reactor
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   168
        if self._web_test_cnx is None:
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   169
            self.web_logout()
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   170
            self._web_test_cnx.close()
5654
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
   171
        try:
11158
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   172
            reactor.stop()
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   173
            self.web_thread.join(timeout)
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   174
            assert not self.web_thread.isAlive()
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   175
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   176
        finally:
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   177
            reactor.__init__()
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   178
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   179
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   180
class CubicWebWsgiTC(CubicWebServerTC):
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   181
    def start_server(self):
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   182
        from cubicweb.wsgi.handler import CubicWebWSGIApplication
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   183
        from wsgiref import simple_server
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   184
        from six.moves import queue
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   185
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   186
        config = self.config
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   187
        port = config['port'] or 8080
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   188
        interface = config['interface']
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   189
        handler_cls = simple_server.WSGIRequestHandler
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   190
        app = CubicWebWSGIApplication(config)
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   191
        start_flag = queue.Queue()
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   192
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   193
        def run(config, *args, **kwargs):
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   194
            try:
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   195
                self.httpd = simple_server.WSGIServer((interface, port), handler_cls)
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   196
                self.httpd.set_app(app)
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   197
            except Exception as exc:
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   198
                start_flag.put(False)
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   199
                start_flag.put(exc)
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   200
                raise
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   201
            else:
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   202
                start_flag.put(True)
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   203
            try:
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   204
                self.httpd.serve_forever()
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   205
            finally:
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   206
                self.httpd.server_close()
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   207
        t = threading.Thread(target=run, name='cubicweb_test_web_server',
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   208
                             args=(self.config, True), kwargs={'repo': self.repo})
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   209
        self.web_thread = t
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   210
        t.start()
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   211
        flag = start_flag.get()
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   212
        if not flag:
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   213
            t.join()
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   214
            self.fail(start_flag.get())
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   215
        parseurl = urlparse(self.config['base-url'])
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   216
        assert parseurl.port == self.config['port'], (self.config['base-url'], self.config['port'])
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   217
        self._web_test_cnx = http_client.HTTPConnection(parseurl.hostname,
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   218
                                                        parseurl.port)
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   219
        self._ident_cookie = None
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   220
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   221
    def stop_server(self, timeout=15):
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   222
        if self._web_test_cnx is None:
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   223
            self.web_logout()
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   224
            self._web_test_cnx.close()
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   225
        self.httpd.shutdown()
669eac69ea21 [devtools] add CubicWebWsgiTC
Julien Cristau <julien.cristau@logilab.fr>
parents: 11057
diff changeset
   226
        self.web_thread.join(timeout)