etwist/server.py
author Julien Cristau <julien.cristau@logilab.fr>
Wed, 07 Jan 2015 15:24:40 +0100
changeset 10138 43eef610ef11
parent 9919 e013c35067d7
child 10235 684215aca046
permissions -rw-r--r--
[pkg] 3.20.1
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
8349
fdb796435d7b backport stable
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8312 8341
diff changeset
     1
# copyright 2003-2012 LOGILAB S.A. (Paris, FRANCE), all rights reserved.
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5325
diff changeset
     2
# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5325
diff changeset
     3
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5325
diff changeset
     4
# This file is part of CubicWeb.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5325
diff changeset
     5
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5325
diff changeset
     6
# CubicWeb is free software: you can redistribute it and/or modify it under the
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5325
diff changeset
     7
# terms of the GNU Lesser General Public License as published by the Free
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5325
diff changeset
     8
# Software Foundation, either version 2.1 of the License, or (at your option)
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5325
diff changeset
     9
# any later version.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5325
diff changeset
    10
#
5424
8ecbcbff9777 replace logilab-common by CubicWeb in disclaimer
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5421
diff changeset
    11
# CubicWeb is distributed in the hope that it will be useful, but WITHOUT
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5325
diff changeset
    12
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5325
diff changeset
    13
# FOR A PARTICULAR PURPOSE.  See the GNU Lesser General Public License for more
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5325
diff changeset
    14
# details.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5325
diff changeset
    15
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5325
diff changeset
    16
# You should have received a copy of the GNU Lesser General Public License along
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5325
diff changeset
    17
# with CubicWeb.  If not, see <http://www.gnu.org/licenses/>.
5814
51cc4b61f9ae [repo] fix 'c-c start --loglevel LEVEL' for repository only configuration. Closes #1058269.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5762
diff changeset
    18
"""twisted server for CubicWeb web instances"""
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    19
__docformat__ = "restructuredtext en"
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    20
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    21
import sys
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    22
import select
5243
1ab7acb9abe9 [etwist] we must call all error not handled in _render_request, else the thread fail silently and nothing ends up on the ui
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5223
diff changeset
    23
import traceback
5652
904091dc4c7e [etwist] twisted won't install signal handler outside the main thread.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5553
diff changeset
    24
import threading
1520
b097057e629d provide an option to substitute the base-url (left-most part) subdomain by the one of the current http query to easy multiple subdomains website management
Florent <florent@secondweb.fr>
parents: 1420
diff changeset
    25
from urlparse import urlsplit, urlunsplit
5155
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
    26
from cgi import FieldStorage, parse_header
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    27
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    28
from twisted.internet import reactor, task, threads
5155
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
    29
from twisted.web import http, server
8930
6a02be304486 remove unused import
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8703
diff changeset
    30
from twisted.web import resource
5155
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
    31
from twisted.web.server import NOT_DONE_YET
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
    32
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    33
8703
db3164d09765 [etwist] Properly escape traceback output (closes #2712042)
Rémi Cardona <remi.cardona@logilab.fr>
parents: 8695
diff changeset
    34
from logilab.mtconverter import xml_escape
5155
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
    35
from logilab.common.decorators import monkeypatch
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    36
8930
6a02be304486 remove unused import
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8703
diff changeset
    37
from cubicweb import ConfigurationError, CW_EVENT_MANAGER
5940
0e3ae19b181a [uilib] refactor json_dumps code organization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5933
diff changeset
    38
from cubicweb.utils import json_dumps
8396
8d58fcf68539 [web] remove some unused imports
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8349
diff changeset
    39
from cubicweb.web import DirectResponse
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    40
from cubicweb.web.application import CubicWebPublisher
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    41
from cubicweb.etwist.request import CubicWebTwistedRequestAdapter
5155
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
    42
from cubicweb.etwist.http import HTTPResponse
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    43
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    44
def start_task(interval, func):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    45
    lc = task.LoopingCall(func)
3493
3e4603678d4e use default repo looping task so they are started in thread, else we get blocking task
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3198
diff changeset
    46
    # wait until interval has expired to actually start the task, else we have
8465
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8396
diff changeset
    47
    # to wait all tasks to be finished for the server to be actually started
3493
3e4603678d4e use default repo looping task so they are started in thread, else we get blocking task
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3198
diff changeset
    48
    lc.start(interval, now=False)
1420
25c13e5b12bd stop complaining about empty response, remove trailing spaces
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
    49
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    50
5155
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
    51
class CubicWebRootResource(resource.Resource):
9036
cf519fd876c6 [etwist] pass the repository to the root resource
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 9031
diff changeset
    52
    def __init__(self, config, repo):
7273
a949fc438029 [etwist] refactor child resources management
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6817
diff changeset
    53
        resource.Resource.__init__(self)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    54
        self.config = config
3828
335e40fdb5a5 when instance is started normally (eg not in debug mode), check versions before daemonization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3791
diff changeset
    55
        # instantiate publisher here and not in init_publisher to get some
335e40fdb5a5 when instance is started normally (eg not in debug mode), check versions before daemonization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3791
diff changeset
    56
        # checks done before daemonization (eg versions consistency)
9031
6ff29f2879da web/application: instantiate the repository outside of CubicWebPublisher
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 9030
diff changeset
    57
        self.appli = CubicWebPublisher(repo, config)
4909
a13c1d1ecc5b [web server] simplify base-url handling on startup. Ensure config['base-url'] is correctly set once started
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
    58
        self.base_url = config['base-url']
a13c1d1ecc5b [web server] simplify base-url handling on startup. Ensure config['base-url'] is correctly set once started
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
    59
        self.https_url = config['https-url']
5270
6297d5265572 [etwist] allow to specify max size for POST request in the configuration. We should deal with this nicely on the js form validation code.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5243
diff changeset
    60
        global MAX_POST_LENGTH
5330
19bc44f5f9d1 [web] no more needed with the bytes option type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5328
diff changeset
    61
        MAX_POST_LENGTH = config['max-post-length']
3606
8326aceecb46 fix startup as a daemon
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3558
diff changeset
    62
8326aceecb46 fix startup as a daemon
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3558
diff changeset
    63
    def init_publisher(self):
8326aceecb46 fix startup as a daemon
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3558
diff changeset
    64
        config = self.config
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    65
        # when we have an in-memory repository, clean unused sessions every XX
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    66
        # seconds and properly shutdown the server
8669
62213a34726e [db-api/configuration] simplify db-api and configuration so that all the connection information is in the repository url, closes #2521848
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8465
diff changeset
    67
        if config['repository-uri'] == 'inmemory://':
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    68
            if config.pyro_enabled():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    69
                # if pyro is enabled, we have to register to the pyro name
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    70
                # server, create a pyro daemon, and create a task to handle pyro
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    71
                # requests
9890
57e2cffa6329 Add warning messages when enabling remote pickle-based repository access
Julien Cristau <julien.cristau@logilab.fr>
parents: 9779
diff changeset
    72
                self.appli.repo.warning('remote repository access through pyro is deprecated')
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    73
                self.pyro_daemon = self.appli.repo.pyro_register()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    74
                self.pyro_listen_timeout = 0.02
3502
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3493
diff changeset
    75
                self.appli.repo.looping_task(1, self.pyro_loop_event)
5654
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5653
diff changeset
    76
            if config.mode != 'test':
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5653
diff changeset
    77
                reactor.addSystemEventTrigger('before', 'shutdown',
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5653
diff changeset
    78
                                              self.shutdown_event)
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5653
diff changeset
    79
                self.appli.repo.start_looping_tasks()
2685
0518ca8f63e3 [autoreload] recompute urlresolver / urlrewriter after autoreload
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2670
diff changeset
    80
        self.set_url_rewriter()
2705
30bcdbd92820 [events] renamed source-reload into registry-reload to avoid potential confusions with datasources
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2694
diff changeset
    81
        CW_EVENT_MANAGER.bind('after-registry-reload', self.set_url_rewriter)
3606
8326aceecb46 fix startup as a daemon
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3558
diff changeset
    82
8326aceecb46 fix startup as a daemon
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3558
diff changeset
    83
    def start_service(self):
5325
f1c660e1169e [web] consistent cleanup session interval time
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5279
diff changeset
    84
        start_task(self.appli.session_handler.clean_sessions_interval,
f1c660e1169e [web] consistent cleanup session interval time
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5279
diff changeset
    85
                   self.appli.session_handler.clean_sessions)
1420
25c13e5b12bd stop complaining about empty response, remove trailing spaces
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
    86
2685
0518ca8f63e3 [autoreload] recompute urlresolver / urlrewriter after autoreload
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2670
diff changeset
    87
    def set_url_rewriter(self):
2770
356e9d7c356d R propagate registry API changes
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2711
diff changeset
    88
        self.url_rewriter = self.appli.vreg['components'].select_or_none('urlrewriter')
2685
0518ca8f63e3 [autoreload] recompute urlresolver / urlrewriter after autoreload
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2670
diff changeset
    89
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    90
    def shutdown_event(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    91
        """callback fired when the server is shutting down to properly
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    92
        clean opened sessions
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    93
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    94
        self.appli.repo.shutdown()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    95
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    96
    def pyro_loop_event(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    97
        """listen for pyro events"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    98
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    99
            self.pyro_daemon.handleRequests(self.pyro_listen_timeout)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   100
        except select.error:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   101
            return
1420
25c13e5b12bd stop complaining about empty response, remove trailing spaces
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
   102
5155
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   103
    def getChild(self, path, request):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   104
        """Indicate which resource to use to process down the URL's path"""
5155
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   105
        return self
1420
25c13e5b12bd stop complaining about empty response, remove trailing spaces
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
   106
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   107
    def render(self, request):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   108
        """Render a page from the root resource"""
3638
648d6dbec630 system/user modes + CWDEV instead of installed/dev mixed modes. Fix behaviour when setting CW_MODE explicitly
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3618
diff changeset
   109
        # reload modified files in debug mode
5442
3ed8afbbdf70 [webconfig] refactor/cleanup debug mode management on startup: simply use config.debugmode instead of debug argument everywhere...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   110
        if self.config.debugmode:
5445
4467ed43d97d [web] use uiprops value to compile css transparently, handlig cache and reloading in debug mode
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5442
diff changeset
   111
            self.config.uiprops.reload_if_needed()
5467
57372dbfd114 [https] fix resource urls in https version of a site: should use the https version as well to avoid warnings from the nrowser
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5445
diff changeset
   112
            if self.https_url:
57372dbfd114 [https] fix resource urls in https version of a site: should use the https version as well to avoid warnings from the nrowser
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5445
diff changeset
   113
                self.config.https_uiprops.reload_if_needed()
5273
c4caef6f09c9 [vreg] refactor the [re]loading process so things get clearer and fix spurious errors on edge cases
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5252
diff changeset
   114
            self.appli.vreg.reload_if_needed()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   115
        if self.config['profile']: # default profiler don't trace threads
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   116
            return self.render_request(request)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   117
        else:
5155
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   118
            deferred = threads.deferToThread(self.render_request, request)
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   119
            return NOT_DONE_YET
1420
25c13e5b12bd stop complaining about empty response, remove trailing spaces
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
   120
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   121
    def render_request(self, request):
5243
1ab7acb9abe9 [etwist] we must call all error not handled in _render_request, else the thread fail silently and nothing ends up on the ui
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5223
diff changeset
   122
        try:
5759
5db529e0a3ee imported patch asynchronous multipart parsing.diff
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5660
diff changeset
   123
            # processing HUGE files (hundred of megabytes) in http.processReceived
5db529e0a3ee imported patch asynchronous multipart parsing.diff
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5660
diff changeset
   124
            # blocks other HTTP requests processing
5db529e0a3ee imported patch asynchronous multipart parsing.diff
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5660
diff changeset
   125
            # due to the clumsy & slow parsing algorithm of cgi.FieldStorage
5db529e0a3ee imported patch asynchronous multipart parsing.diff
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5660
diff changeset
   126
            # so we deferred that part to the cubicweb thread
5db529e0a3ee imported patch asynchronous multipart parsing.diff
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5660
diff changeset
   127
            request.process_multipart()
5243
1ab7acb9abe9 [etwist] we must call all error not handled in _render_request, else the thread fail silently and nothing ends up on the ui
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5223
diff changeset
   128
            return self._render_request(request)
7815
2a164a9cf81c [exceptions] stop catching any exception in various places (closes #1942716)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7764
diff changeset
   129
        except Exception:
8703
db3164d09765 [etwist] Properly escape traceback output (closes #2712042)
Rémi Cardona <remi.cardona@logilab.fr>
parents: 8695
diff changeset
   130
            trace = traceback.format_exc()
db3164d09765 [etwist] Properly escape traceback output (closes #2712042)
Rémi Cardona <remi.cardona@logilab.fr>
parents: 8695
diff changeset
   131
            return HTTPResponse(stream='<pre>%s</pre>' % xml_escape(trace),
5243
1ab7acb9abe9 [etwist] we must call all error not handled in _render_request, else the thread fail silently and nothing ends up on the ui
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5223
diff changeset
   132
                                code=500, twisted_request=request)
1ab7acb9abe9 [etwist] we must call all error not handled in _render_request, else the thread fail silently and nothing ends up on the ui
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5223
diff changeset
   133
1ab7acb9abe9 [etwist] we must call all error not handled in _render_request, else the thread fail silently and nothing ends up on the ui
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5223
diff changeset
   134
    def _render_request(self, request):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   135
        origpath = request.path
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   136
        host = request.host
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   137
        # dual http/https access handling: expect a rewrite rule to prepend
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   138
        # 'https' to the path to detect https access
8309
48ef505aa9f9 [request] gather all base_url logic in a single place (closes #2200756)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8308
diff changeset
   139
        https = False
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   140
        if origpath.split('/', 2)[1] == 'https':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   141
            origpath = origpath[6:]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   142
            request.uri = request.uri[6:]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   143
            https = True
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   144
        if self.url_rewriter is not None:
1115
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 151
diff changeset
   145
            # XXX should occur before authentication?
8312
6c2119509fac [web] Move request handling logic into cubicweb application. (closes #2200684)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8309
diff changeset
   146
            path = self.url_rewriter.rewrite(host, origpath, request)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   147
            request.uri.replace(origpath, path, 1)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   148
        else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   149
            path = origpath
8312
6c2119509fac [web] Move request handling logic into cubicweb application. (closes #2200684)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8309
diff changeset
   150
        req = CubicWebTwistedRequestAdapter(request, self.appli.vreg, https)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   151
        try:
8312
6c2119509fac [web] Move request handling logic into cubicweb application. (closes #2200684)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8309
diff changeset
   152
            ### Try to generate the actual request content
6c2119509fac [web] Move request handling logic into cubicweb application. (closes #2200684)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8309
diff changeset
   153
            content = self.appli.handle_request(req, path)
8695
358d8bed9626 [toward-py3k] rewrite to "except AnException as exc:" (part of #2711624)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 8694
diff changeset
   154
        except DirectResponse as ex:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   155
            return ex.response
8312
6c2119509fac [web] Move request handling logic into cubicweb application. (closes #2200684)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8309
diff changeset
   156
        # at last: create twisted object
6c2119509fac [web] Move request handling logic into cubicweb application. (closes #2200684)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8309
diff changeset
   157
        return HTTPResponse(code    = req.status_out,
6c2119509fac [web] Move request handling logic into cubicweb application. (closes #2200684)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8309
diff changeset
   158
                            headers = req.headers_out,
6c2119509fac [web] Move request handling logic into cubicweb application. (closes #2200684)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8309
diff changeset
   159
                            stream  = content,
6c2119509fac [web] Move request handling logic into cubicweb application. (closes #2200684)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8309
diff changeset
   160
                            twisted_request=req._twreq)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   161
7879
9aae456abab5 [pylint] fix pylint detected errors and tweak it so that pylint -E will be much less verbose next time (+ update some copyrights on the way)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7815
diff changeset
   162
    # these are overridden by set_log_methods below
9aae456abab5 [pylint] fix pylint detected errors and tweak it so that pylint -E will be much less verbose next time (+ update some copyrights on the way)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7815
diff changeset
   163
    # only defining here to prevent pylint from complaining
9aae456abab5 [pylint] fix pylint detected errors and tweak it so that pylint -E will be much less verbose next time (+ update some copyrights on the way)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7815
diff changeset
   164
    @classmethod
9aae456abab5 [pylint] fix pylint detected errors and tweak it so that pylint -E will be much less verbose next time (+ update some copyrights on the way)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7815
diff changeset
   165
    def debug(cls, msg, *a, **kw):
9aae456abab5 [pylint] fix pylint detected errors and tweak it so that pylint -E will be much less verbose next time (+ update some copyrights on the way)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7815
diff changeset
   166
        pass
9aae456abab5 [pylint] fix pylint detected errors and tweak it so that pylint -E will be much less verbose next time (+ update some copyrights on the way)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7815
diff changeset
   167
    info = warning = error = critical = exception = debug
9aae456abab5 [pylint] fix pylint detected errors and tweak it so that pylint -E will be much less verbose next time (+ update some copyrights on the way)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7815
diff changeset
   168
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   169
5335
9aed0b607130 [etwist] nicer, thouhg really not perferct) request max size exceeded support of json/frame posts
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5330
diff changeset
   170
JSON_PATHS = set(('json',))
9aed0b607130 [etwist] nicer, thouhg really not perferct) request max size exceeded support of json/frame posts
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5330
diff changeset
   171
FRAME_POST_PATHS = set(('validateform',))
1420
25c13e5b12bd stop complaining about empty response, remove trailing spaces
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
   172
5270
6297d5265572 [etwist] allow to specify max size for POST request in the configuration. We should deal with this nicely on the js form validation code.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5243
diff changeset
   173
orig_gotLength = http.Request.gotLength
6297d5265572 [etwist] allow to specify max size for POST request in the configuration. We should deal with this nicely on the js form validation code.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5243
diff changeset
   174
@monkeypatch(http.Request)
6297d5265572 [etwist] allow to specify max size for POST request in the configuration. We should deal with this nicely on the js form validation code.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5243
diff changeset
   175
def gotLength(self, length):
6297d5265572 [etwist] allow to specify max size for POST request in the configuration. We should deal with this nicely on the js form validation code.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5243
diff changeset
   176
    orig_gotLength(self, length)
6297d5265572 [etwist] allow to specify max size for POST request in the configuration. We should deal with this nicely on the js form validation code.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5243
diff changeset
   177
    if length > MAX_POST_LENGTH: # length is 0 on GET
5335
9aed0b607130 [etwist] nicer, thouhg really not perferct) request max size exceeded support of json/frame posts
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5330
diff changeset
   178
        path = self.channel._path.split('?', 1)[0].rstrip('/').rsplit('/', 1)[-1]
5270
6297d5265572 [etwist] allow to specify max size for POST request in the configuration. We should deal with this nicely on the js form validation code.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5243
diff changeset
   179
        self.clientproto = 'HTTP/1.1' # not yet initialized
6297d5265572 [etwist] allow to specify max size for POST request in the configuration. We should deal with this nicely on the js form validation code.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5243
diff changeset
   180
        self.channel.persistent = 0   # force connection close on cleanup
9919
e013c35067d7 [etwist] use more specific REQUEST_ENTITY_TOO_LARGE instead of BAD_REQUEST
Julien Cristau <julien.cristau@logilab.fr>
parents: 9890
diff changeset
   181
        self.setResponseCode(http.REQUEST_ENTITY_TOO_LARGE)
5335
9aed0b607130 [etwist] nicer, thouhg really not perferct) request max size exceeded support of json/frame posts
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5330
diff changeset
   182
        if path in JSON_PATHS: # XXX better json path detection
9aed0b607130 [etwist] nicer, thouhg really not perferct) request max size exceeded support of json/frame posts
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5330
diff changeset
   183
            self.setHeader('content-type',"application/json")
5940
0e3ae19b181a [uilib] refactor json_dumps code organization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5933
diff changeset
   184
            body = json_dumps({'reason': 'request max size exceeded'})
5335
9aed0b607130 [etwist] nicer, thouhg really not perferct) request max size exceeded support of json/frame posts
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5330
diff changeset
   185
        elif path in FRAME_POST_PATHS: # XXX better frame post path detection
9aed0b607130 [etwist] nicer, thouhg really not perferct) request max size exceeded support of json/frame posts
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5330
diff changeset
   186
            self.setHeader('content-type',"text/html")
9aed0b607130 [etwist] nicer, thouhg really not perferct) request max size exceeded support of json/frame posts
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5330
diff changeset
   187
            body = ('<script type="text/javascript">'
9aed0b607130 [etwist] nicer, thouhg really not perferct) request max size exceeded support of json/frame posts
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5330
diff changeset
   188
                    'window.parent.handleFormValidationResponse(null, null, null, %s, null);'
5940
0e3ae19b181a [uilib] refactor json_dumps code organization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5933
diff changeset
   189
                    '</script>' % json_dumps( (False, 'request max size exceeded', None) ))
5335
9aed0b607130 [etwist] nicer, thouhg really not perferct) request max size exceeded support of json/frame posts
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5330
diff changeset
   190
        else:
9aed0b607130 [etwist] nicer, thouhg really not perferct) request max size exceeded support of json/frame posts
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5330
diff changeset
   191
            self.setHeader('content-type',"text/html")
9aed0b607130 [etwist] nicer, thouhg really not perferct) request max size exceeded support of json/frame posts
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5330
diff changeset
   192
            body = ("<html><head><title>Processing Failed</title></head><body>"
9aed0b607130 [etwist] nicer, thouhg really not perferct) request max size exceeded support of json/frame posts
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5330
diff changeset
   193
                    "<b>request max size exceeded</b></body></html>")
5270
6297d5265572 [etwist] allow to specify max size for POST request in the configuration. We should deal with this nicely on the js form validation code.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5243
diff changeset
   194
        self.setHeader('content-length', str(len(body)))
6297d5265572 [etwist] allow to specify max size for POST request in the configuration. We should deal with this nicely on the js form validation code.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5243
diff changeset
   195
        self.write(body)
6297d5265572 [etwist] allow to specify max size for POST request in the configuration. We should deal with this nicely on the js form validation code.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5243
diff changeset
   196
        # see request.finish(). Done here since we get error due to not full
6297d5265572 [etwist] allow to specify max size for POST request in the configuration. We should deal with this nicely on the js form validation code.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5243
diff changeset
   197
        # initialized request
6297d5265572 [etwist] allow to specify max size for POST request in the configuration. We should deal with this nicely on the js form validation code.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5243
diff changeset
   198
        self.finished = 1
6297d5265572 [etwist] allow to specify max size for POST request in the configuration. We should deal with this nicely on the js form validation code.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5243
diff changeset
   199
        if not self.queued:
6297d5265572 [etwist] allow to specify max size for POST request in the configuration. We should deal with this nicely on the js form validation code.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5243
diff changeset
   200
            self._cleanup()
6297d5265572 [etwist] allow to specify max size for POST request in the configuration. We should deal with this nicely on the js form validation code.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5243
diff changeset
   201
        for d in self.notifications:
6297d5265572 [etwist] allow to specify max size for POST request in the configuration. We should deal with this nicely on the js form validation code.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5243
diff changeset
   202
            d.callback(None)
6297d5265572 [etwist] allow to specify max size for POST request in the configuration. We should deal with this nicely on the js form validation code.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5243
diff changeset
   203
        self.notifications = []
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   204
5155
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   205
@monkeypatch(http.Request)
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   206
def requestReceived(self, command, path, version):
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   207
    """Called by channel when all data has been received.
1420
25c13e5b12bd stop complaining about empty response, remove trailing spaces
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
   208
5155
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   209
    This method is not intended for users.
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   210
    """
5272
f7d2df59231a cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5270
diff changeset
   211
    self.content.seek(0, 0)
5155
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   212
    self.args = {}
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   213
    self.files = {}
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   214
    self.stack = []
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   215
    self.method, self.uri = command, path
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   216
    self.clientproto = version
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   217
    x = self.uri.split('?', 1)
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   218
    if len(x) == 1:
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   219
        self.path = self.uri
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   220
    else:
5155
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   221
        self.path, argstring = x
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   222
        self.args = http.parse_qs(argstring, 1)
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   223
    # cache the client and server information, we'll need this later to be
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   224
    # serialized and sent with the request so CGIs will work remotely
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   225
    self.client = self.channel.transport.getPeer()
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   226
    self.host = self.channel.transport.getHost()
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   227
    # Argument processing
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   228
    ctype = self.getHeader('content-type')
5759
5db529e0a3ee imported patch asynchronous multipart parsing.diff
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5660
diff changeset
   229
    self._do_process_multipart = False
5155
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   230
    if self.method == "POST" and ctype:
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   231
        key, pdict = parse_header(ctype)
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   232
        if key == 'application/x-www-form-urlencoded':
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   233
            self.args.update(http.parse_qs(self.content.read(), 1))
9533
fa4051e29fbe [etwist] Fix an empty request content after a Twisted processing (closes #3546795).
Damien Garaud <damien.garaud@logilab.fr>
parents: 9147
diff changeset
   234
            self.content.seek(0)
5155
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   235
        elif key == 'multipart/form-data':
5759
5db529e0a3ee imported patch asynchronous multipart parsing.diff
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5660
diff changeset
   236
            # defer this as it can be extremely time consumming
5db529e0a3ee imported patch asynchronous multipart parsing.diff
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5660
diff changeset
   237
            # with big files
5db529e0a3ee imported patch asynchronous multipart parsing.diff
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5660
diff changeset
   238
            self._do_process_multipart = True
5155
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   239
    self.process()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   240
5759
5db529e0a3ee imported patch asynchronous multipart parsing.diff
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5660
diff changeset
   241
@monkeypatch(http.Request)
5db529e0a3ee imported patch asynchronous multipart parsing.diff
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5660
diff changeset
   242
def process_multipart(self):
5db529e0a3ee imported patch asynchronous multipart parsing.diff
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5660
diff changeset
   243
    if not self._do_process_multipart:
5db529e0a3ee imported patch asynchronous multipart parsing.diff
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5660
diff changeset
   244
        return
5db529e0a3ee imported patch asynchronous multipart parsing.diff
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5660
diff changeset
   245
    form = FieldStorage(self.content, self.received_headers,
5db529e0a3ee imported patch asynchronous multipart parsing.diff
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5660
diff changeset
   246
                        environ={'REQUEST_METHOD': 'POST'},
5db529e0a3ee imported patch asynchronous multipart parsing.diff
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5660
diff changeset
   247
                        keep_blank_values=1,
5db529e0a3ee imported patch asynchronous multipart parsing.diff
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5660
diff changeset
   248
                        strict_parsing=1)
5db529e0a3ee imported patch asynchronous multipart parsing.diff
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5660
diff changeset
   249
    for key in form:
9147
01124cfd4b1f [etwist] fix handling of multiple files per field
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8930
diff changeset
   250
        values = form[key]
01124cfd4b1f [etwist] fix handling of multiple files per field
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8930
diff changeset
   251
        if not isinstance(values, list):
01124cfd4b1f [etwist] fix handling of multiple files per field
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8930
diff changeset
   252
            values = [values]
01124cfd4b1f [etwist] fix handling of multiple files per field
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8930
diff changeset
   253
        for value in values:
01124cfd4b1f [etwist] fix handling of multiple files per field
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8930
diff changeset
   254
            if value.filename:
01124cfd4b1f [etwist] fix handling of multiple files per field
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8930
diff changeset
   255
                if value.done != -1: # -1 is transfer has been interrupted
01124cfd4b1f [etwist] fix handling of multiple files per field
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8930
diff changeset
   256
                    self.files.setdefault(key, []).append((value.filename, value.file))
01124cfd4b1f [etwist] fix handling of multiple files per field
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8930
diff changeset
   257
                else:
01124cfd4b1f [etwist] fix handling of multiple files per field
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8930
diff changeset
   258
                    self.files.setdefault(key, []).append((None, None))
5759
5db529e0a3ee imported patch asynchronous multipart parsing.diff
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5660
diff changeset
   259
            else:
9147
01124cfd4b1f [etwist] fix handling of multiple files per field
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8930
diff changeset
   260
                self.args.setdefault(key, []).append(value.value)
5759
5db529e0a3ee imported patch asynchronous multipart parsing.diff
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5660
diff changeset
   261
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   262
from logging import getLogger
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   263
from cubicweb import set_log_methods
5155
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   264
LOGGER = getLogger('cubicweb.twisted')
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   265
set_log_methods(CubicWebRootResource, LOGGER)
2654
6512522860aa [twisted] don't use twistd anymore, all-in-one.py file is needed anymore
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2476
diff changeset
   266
9036
cf519fd876c6 [etwist] pass the repository to the root resource
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 9031
diff changeset
   267
def run(config, debug=None, repo=None):
cf519fd876c6 [etwist] pass the repository to the root resource
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 9031
diff changeset
   268
    # repo may by passed during test.
cf519fd876c6 [etwist] pass the repository to the root resource
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 9031
diff changeset
   269
    #
cf519fd876c6 [etwist] pass the repository to the root resource
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 9031
diff changeset
   270
    # Test has already created a repo object so we should not create a new one.
cf519fd876c6 [etwist] pass the repository to the root resource
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 9031
diff changeset
   271
    # Explicitly passing the repo object avoid relying on the fragile
cf519fd876c6 [etwist] pass the repository to the root resource
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 9031
diff changeset
   272
    # config.repository() cache. We could imagine making repo a mandatory
cf519fd876c6 [etwist] pass the repository to the root resource
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 9031
diff changeset
   273
    # argument and receives it from the starting command directly.
5653
c562791df9d2 [web server] vregistry may now be passed as named argument in CubicWebRootResource constructor
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5555
diff changeset
   274
    if debug is not None:
c562791df9d2 [web server] vregistry may now be passed as named argument in CubicWebRootResource constructor
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5555
diff changeset
   275
        config.debugmode = debug
5933
3d707b8f8a4d [web configuration] ensure data home directory / uicache file belong to daemon user and are writeable
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5907
diff changeset
   276
    config.check_writeable_uid_directory(config.appdatahome)
2654
6512522860aa [twisted] don't use twistd anymore, all-in-one.py file is needed anymore
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2476
diff changeset
   277
    # create the site
9036
cf519fd876c6 [etwist] pass the repository to the root resource
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 9031
diff changeset
   278
    if repo is None:
cf519fd876c6 [etwist] pass the repository to the root resource
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 9031
diff changeset
   279
        repo = config.repository()
cf519fd876c6 [etwist] pass the repository to the root resource
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 9031
diff changeset
   280
    root_resource = CubicWebRootResource(config, repo)
2654
6512522860aa [twisted] don't use twistd anymore, all-in-one.py file is needed anymore
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2476
diff changeset
   281
    website = server.Site(root_resource)
6512522860aa [twisted] don't use twistd anymore, all-in-one.py file is needed anymore
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2476
diff changeset
   282
    # serve it via standard HTTP on port set in the configuration
6512522860aa [twisted] don't use twistd anymore, all-in-one.py file is needed anymore
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2476
diff changeset
   283
    port = config['port'] or 8080
6817
1959d97ebf2e [etwist] add an 'interface' config option (passed to reactor.listenTCP)
david.douard@logilab.fr
parents: 6779
diff changeset
   284
    interface = config['interface']
7644
7a0914469618 [twisted] add an option to configure twisted's threadpool size
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 7620
diff changeset
   285
    reactor.suggestThreadPoolSize(config['webserver-threadpool-size'])
6817
1959d97ebf2e [etwist] add an 'interface' config option (passed to reactor.listenTCP)
david.douard@logilab.fr
parents: 6779
diff changeset
   286
    reactor.listenTCP(port, website, interface=interface)
5442
3ed8afbbdf70 [webconfig] refactor/cleanup debug mode management on startup: simply use config.debugmode instead of debug argument everywhere...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   287
    if not config.debugmode:
4219
87203a0a3a81 do not attempt to daemonize under windows
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 4212
diff changeset
   288
        if sys.platform == 'win32':
4221
da84ca26896d raising ConfigurationError is cleaner; remove some unused imports
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 4219
diff changeset
   289
            raise ConfigurationError("Under windows, you must use the service management "
da84ca26896d raising ConfigurationError is cleaner; remove some unused imports
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 4219
diff changeset
   290
                                     "commands (e.g : 'net start my_instance)'")
5659
755f56f01c9d [win32] defer import of logilab.common.daemon
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5553
diff changeset
   291
        from logilab.common.daemon import daemonize
5679
0f2ded880d01 backport stable
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5655 5660
diff changeset
   292
        LOGGER.info('instance started in the background on %s', root_resource.base_url)
6779
accf5978a440 [daemon] make process umask configurable
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6778
diff changeset
   293
        whichproc = daemonize(config['pid-file'], umask=config['umask'])
6778
7dd4835d5198 properly tell we've started an instance (close #1391262)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5940
diff changeset
   294
        if whichproc: # 1 = orig process, 2 = first fork, None = second fork (eg daemon process)
7dd4835d5198 properly tell we've started an instance (close #1391262)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5940
diff changeset
   295
            return whichproc # parent process
3606
8326aceecb46 fix startup as a daemon
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3558
diff changeset
   296
    root_resource.init_publisher() # before changing uid
3193
c1a8456cb5f9 should change process'user once pid file has been written
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3181
diff changeset
   297
    if config['uid'] is not None:
7764
c709f6c457ff Drop privileges properly (closes #1910561)
Julien Cristau <julien.cristau@logilab.fr>
parents: 7647
diff changeset
   298
        from logilab.common.daemon import setugid
c709f6c457ff Drop privileges properly (closes #1910561)
Julien Cristau <julien.cristau@logilab.fr>
parents: 7647
diff changeset
   299
        setugid(config['uid'])
3606
8326aceecb46 fix startup as a daemon
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3558
diff changeset
   300
    root_resource.start_service()
5679
0f2ded880d01 backport stable
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5655 5660
diff changeset
   301
    LOGGER.info('instance started on %s', root_resource.base_url)
5652
904091dc4c7e [etwist] twisted won't install signal handler outside the main thread.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5553
diff changeset
   302
    # avoid annoying warnign if not in Main Thread
904091dc4c7e [etwist] twisted won't install signal handler outside the main thread.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5553
diff changeset
   303
    signals = threading.currentThread().getName() == 'MainThread'
2654
6512522860aa [twisted] don't use twistd anymore, all-in-one.py file is needed anymore
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2476
diff changeset
   304
    if config['profile']:
4960
26b2468a1e73 [etwist] cleanup, use cProfile instead of hotshot
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4914
diff changeset
   305
        import cProfile
5652
904091dc4c7e [etwist] twisted won't install signal handler outside the main thread.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5553
diff changeset
   306
        cProfile.runctx('reactor.run(installSignalHandlers=%s)' % signals,
904091dc4c7e [etwist] twisted won't install signal handler outside the main thread.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5553
diff changeset
   307
                        globals(), locals(), config['profile'])
2654
6512522860aa [twisted] don't use twistd anymore, all-in-one.py file is needed anymore
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2476
diff changeset
   308
    else:
5652
904091dc4c7e [etwist] twisted won't install signal handler outside the main thread.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5553
diff changeset
   309
        reactor.run(installSignalHandlers=signals)