etwist/server.py
author Julien Cristau <julien.cristau@logilab.fr>
Fri, 20 Jul 2012 09:22:51 +0200
branchstable
changeset 8477 42e8755b3aba
parent 8465 12da4415aff0
child 8669 62213a34726e
permissions -rw-r--r--
[config] Set CWDEV, aka dev mode, according to presence of the i18n directory, not .hg
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
8349
fdb796435d7b backport stable
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8312 8341
diff changeset
     1
# copyright 2003-2012 LOGILAB S.A. (Paris, FRANCE), all rights reserved.
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5325
diff changeset
     2
# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5325
diff changeset
     3
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5325
diff changeset
     4
# This file is part of CubicWeb.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5325
diff changeset
     5
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5325
diff changeset
     6
# CubicWeb is free software: you can redistribute it and/or modify it under the
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5325
diff changeset
     7
# terms of the GNU Lesser General Public License as published by the Free
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5325
diff changeset
     8
# Software Foundation, either version 2.1 of the License, or (at your option)
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5325
diff changeset
     9
# any later version.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5325
diff changeset
    10
#
5424
8ecbcbff9777 replace logilab-common by CubicWeb in disclaimer
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5421
diff changeset
    11
# CubicWeb is distributed in the hope that it will be useful, but WITHOUT
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5325
diff changeset
    12
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5325
diff changeset
    13
# FOR A PARTICULAR PURPOSE.  See the GNU Lesser General Public License for more
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5325
diff changeset
    14
# details.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5325
diff changeset
    15
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5325
diff changeset
    16
# You should have received a copy of the GNU Lesser General Public License along
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5325
diff changeset
    17
# with CubicWeb.  If not, see <http://www.gnu.org/licenses/>.
5814
51cc4b61f9ae [repo] fix 'c-c start --loglevel LEVEL' for repository only configuration. Closes #1058269.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5762
diff changeset
    18
"""twisted server for CubicWeb web instances"""
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    19
7276
f9a68136eb87 handle js/ css file concatenation with twisted
Florent Cayré <florent.cayre@gmail.com>
parents: 7275
diff changeset
    20
from __future__ import with_statement
f9a68136eb87 handle js/ css file concatenation with twisted
Florent Cayré <florent.cayre@gmail.com>
parents: 7275
diff changeset
    21
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    22
__docformat__ = "restructuredtext en"
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    23
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    24
import sys
2654
6512522860aa [twisted] don't use twistd anymore, all-in-one.py file is needed anymore
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2476
diff changeset
    25
import os
7273
a949fc438029 [etwist] refactor child resources management
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6817
diff changeset
    26
import os.path as osp
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    27
import select
5243
1ab7acb9abe9 [etwist] we must call all error not handled in _render_request, else the thread fail silently and nothing ends up on the ui
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5223
diff changeset
    28
import traceback
5652
904091dc4c7e [etwist] twisted won't install signal handler outside the main thread.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5553
diff changeset
    29
import threading
7276
f9a68136eb87 handle js/ css file concatenation with twisted
Florent Cayré <florent.cayre@gmail.com>
parents: 7275
diff changeset
    30
import re
7879
9aae456abab5 [pylint] fix pylint detected errors and tweak it so that pylint -E will be much less verbose next time (+ update some copyrights on the way)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7815
diff changeset
    31
from hashlib import md5 # pylint: disable=E0611
5216
4f4369e63f5e [twisted] fix LongTimeExpiringFile and getChild implementation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5155
diff changeset
    32
from os.path import join
1016
26387b836099 use datetime instead of mx.DateTime
sylvain.thenault@logilab.fr
parents: 151
diff changeset
    33
from time import mktime
26387b836099 use datetime instead of mx.DateTime
sylvain.thenault@logilab.fr
parents: 151
diff changeset
    34
from datetime import date, timedelta
1520
b097057e629d provide an option to substitute the base-url (left-most part) subdomain by the one of the current http query to easy multiple subdomains website management
Florent <florent@secondweb.fr>
parents: 1420
diff changeset
    35
from urlparse import urlsplit, urlunsplit
5155
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
    36
from cgi import FieldStorage, parse_header
5243
1ab7acb9abe9 [etwist] we must call all error not handled in _render_request, else the thread fail silently and nothing ends up on the ui
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5223
diff changeset
    37
from cStringIO import StringIO
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    38
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    39
from twisted.internet import reactor, task, threads
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    40
from twisted.internet.defer import maybeDeferred
5155
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
    41
from twisted.web import http, server
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
    42
from twisted.web import static, resource
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
    43
from twisted.web.server import NOT_DONE_YET
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
    44
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    45
5155
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
    46
from logilab.common.decorators import monkeypatch
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    47
7276
f9a68136eb87 handle js/ css file concatenation with twisted
Florent Cayré <florent.cayre@gmail.com>
parents: 7275
diff changeset
    48
from cubicweb import (AuthenticationError, ConfigurationError,
f9a68136eb87 handle js/ css file concatenation with twisted
Florent Cayré <florent.cayre@gmail.com>
parents: 7275
diff changeset
    49
                      CW_EVENT_MANAGER, CubicWebException)
5940
0e3ae19b181a [uilib] refactor json_dumps code organization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5933
diff changeset
    50
from cubicweb.utils import json_dumps
8396
8d58fcf68539 [web] remove some unused imports
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8349
diff changeset
    51
from cubicweb.web import DirectResponse
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    52
from cubicweb.web.application import CubicWebPublisher
5216
4f4369e63f5e [twisted] fix LongTimeExpiringFile and getChild implementation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5155
diff changeset
    53
from cubicweb.web.http_headers import generateDateTime
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    54
from cubicweb.etwist.request import CubicWebTwistedRequestAdapter
5155
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
    55
from cubicweb.etwist.http import HTTPResponse
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    56
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    57
def start_task(interval, func):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    58
    lc = task.LoopingCall(func)
3493
3e4603678d4e use default repo looping task so they are started in thread, else we get blocking task
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3198
diff changeset
    59
    # wait until interval has expired to actually start the task, else we have
8465
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8396
diff changeset
    60
    # to wait all tasks to be finished for the server to be actually started
3493
3e4603678d4e use default repo looping task so they are started in thread, else we get blocking task
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3198
diff changeset
    61
    lc.start(interval, now=False)
1420
25c13e5b12bd stop complaining about empty response, remove trailing spaces
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
    62
1543
dca9817bb337 fix use-request-subdomain option behaviour and add tests
Florent <florent@secondweb.fr>
parents: 1542
diff changeset
    63
def host_prefixed_baseurl(baseurl, host):
dca9817bb337 fix use-request-subdomain option behaviour and add tests
Florent <florent@secondweb.fr>
parents: 1542
diff changeset
    64
    scheme, netloc, url, query, fragment = urlsplit(baseurl)
dca9817bb337 fix use-request-subdomain option behaviour and add tests
Florent <florent@secondweb.fr>
parents: 1542
diff changeset
    65
    netloc_domain = '.' + '.'.join(netloc.split('.')[-2:])
dca9817bb337 fix use-request-subdomain option behaviour and add tests
Florent <florent@secondweb.fr>
parents: 1542
diff changeset
    66
    if host.endswith(netloc_domain):
dca9817bb337 fix use-request-subdomain option behaviour and add tests
Florent <florent@secondweb.fr>
parents: 1542
diff changeset
    67
        netloc = host
dca9817bb337 fix use-request-subdomain option behaviour and add tests
Florent <florent@secondweb.fr>
parents: 1542
diff changeset
    68
    baseurl = urlunsplit((scheme, netloc, url, query, fragment))
dca9817bb337 fix use-request-subdomain option behaviour and add tests
Florent <florent@secondweb.fr>
parents: 1542
diff changeset
    69
    return baseurl
dca9817bb337 fix use-request-subdomain option behaviour and add tests
Florent <florent@secondweb.fr>
parents: 1542
diff changeset
    70
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    71
5155
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
    72
class CubicWebRootResource(resource.Resource):
5653
c562791df9d2 [web server] vregistry may now be passed as named argument in CubicWebRootResource constructor
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5555
diff changeset
    73
    def __init__(self, config, vreg=None):
7273
a949fc438029 [etwist] refactor child resources management
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6817
diff changeset
    74
        resource.Resource.__init__(self)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    75
        self.config = config
3828
335e40fdb5a5 when instance is started normally (eg not in debug mode), check versions before daemonization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3791
diff changeset
    76
        # instantiate publisher here and not in init_publisher to get some
335e40fdb5a5 when instance is started normally (eg not in debug mode), check versions before daemonization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3791
diff changeset
    77
        # checks done before daemonization (eg versions consistency)
5653
c562791df9d2 [web server] vregistry may now be passed as named argument in CubicWebRootResource constructor
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5555
diff changeset
    78
        self.appli = CubicWebPublisher(config, vreg=vreg)
4909
a13c1d1ecc5b [web server] simplify base-url handling on startup. Ensure config['base-url'] is correctly set once started
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
    79
        self.base_url = config['base-url']
a13c1d1ecc5b [web server] simplify base-url handling on startup. Ensure config['base-url'] is correctly set once started
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
    80
        self.https_url = config['https-url']
5270
6297d5265572 [etwist] allow to specify max size for POST request in the configuration. We should deal with this nicely on the js form validation code.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5243
diff changeset
    81
        global MAX_POST_LENGTH
5330
19bc44f5f9d1 [web] no more needed with the bytes option type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5328
diff changeset
    82
        MAX_POST_LENGTH = config['max-post-length']
3606
8326aceecb46 fix startup as a daemon
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3558
diff changeset
    83
8326aceecb46 fix startup as a daemon
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3558
diff changeset
    84
    def init_publisher(self):
8326aceecb46 fix startup as a daemon
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3558
diff changeset
    85
        config = self.config
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    86
        # when we have an in-memory repository, clean unused sessions every XX
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    87
        # seconds and properly shutdown the server
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    88
        if config.repo_method == 'inmemory':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    89
            if config.pyro_enabled():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    90
                # if pyro is enabled, we have to register to the pyro name
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    91
                # server, create a pyro daemon, and create a task to handle pyro
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    92
                # requests
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    93
                self.pyro_daemon = self.appli.repo.pyro_register()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    94
                self.pyro_listen_timeout = 0.02
3502
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3493
diff changeset
    95
                self.appli.repo.looping_task(1, self.pyro_loop_event)
5654
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5653
diff changeset
    96
            if config.mode != 'test':
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5653
diff changeset
    97
                reactor.addSystemEventTrigger('before', 'shutdown',
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5653
diff changeset
    98
                                              self.shutdown_event)
8bb34548be86 [web test] Add a CubicWebServerTC class to run test with a cw web serveur available.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5653
diff changeset
    99
                self.appli.repo.start_looping_tasks()
2685
0518ca8f63e3 [autoreload] recompute urlresolver / urlrewriter after autoreload
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2670
diff changeset
   100
        self.set_url_rewriter()
2705
30bcdbd92820 [events] renamed source-reload into registry-reload to avoid potential confusions with datasources
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2694
diff changeset
   101
        CW_EVENT_MANAGER.bind('after-registry-reload', self.set_url_rewriter)
3606
8326aceecb46 fix startup as a daemon
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3558
diff changeset
   102
8326aceecb46 fix startup as a daemon
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3558
diff changeset
   103
    def start_service(self):
5325
f1c660e1169e [web] consistent cleanup session interval time
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5279
diff changeset
   104
        start_task(self.appli.session_handler.clean_sessions_interval,
f1c660e1169e [web] consistent cleanup session interval time
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5279
diff changeset
   105
                   self.appli.session_handler.clean_sessions)
1420
25c13e5b12bd stop complaining about empty response, remove trailing spaces
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
   106
2685
0518ca8f63e3 [autoreload] recompute urlresolver / urlrewriter after autoreload
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2670
diff changeset
   107
    def set_url_rewriter(self):
2770
356e9d7c356d R propagate registry API changes
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2711
diff changeset
   108
        self.url_rewriter = self.appli.vreg['components'].select_or_none('urlrewriter')
2685
0518ca8f63e3 [autoreload] recompute urlresolver / urlrewriter after autoreload
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2670
diff changeset
   109
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   110
    def shutdown_event(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   111
        """callback fired when the server is shutting down to properly
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   112
        clean opened sessions
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   113
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   114
        self.appli.repo.shutdown()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   115
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   116
    def pyro_loop_event(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   117
        """listen for pyro events"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   118
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   119
            self.pyro_daemon.handleRequests(self.pyro_listen_timeout)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   120
        except select.error:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   121
            return
1420
25c13e5b12bd stop complaining about empty response, remove trailing spaces
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
   122
5155
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   123
    def getChild(self, path, request):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   124
        """Indicate which resource to use to process down the URL's path"""
5155
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   125
        return self
1420
25c13e5b12bd stop complaining about empty response, remove trailing spaces
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
   126
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   127
    def render(self, request):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   128
        """Render a page from the root resource"""
3638
648d6dbec630 system/user modes + CWDEV instead of installed/dev mixed modes. Fix behaviour when setting CW_MODE explicitly
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3618
diff changeset
   129
        # reload modified files in debug mode
5442
3ed8afbbdf70 [webconfig] refactor/cleanup debug mode management on startup: simply use config.debugmode instead of debug argument everywhere...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   130
        if self.config.debugmode:
5445
4467ed43d97d [web] use uiprops value to compile css transparently, handlig cache and reloading in debug mode
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5442
diff changeset
   131
            self.config.uiprops.reload_if_needed()
5467
57372dbfd114 [https] fix resource urls in https version of a site: should use the https version as well to avoid warnings from the nrowser
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5445
diff changeset
   132
            if self.https_url:
57372dbfd114 [https] fix resource urls in https version of a site: should use the https version as well to avoid warnings from the nrowser
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5445
diff changeset
   133
                self.config.https_uiprops.reload_if_needed()
5273
c4caef6f09c9 [vreg] refactor the [re]loading process so things get clearer and fix spurious errors on edge cases
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5252
diff changeset
   134
            self.appli.vreg.reload_if_needed()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   135
        if self.config['profile']: # default profiler don't trace threads
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   136
            return self.render_request(request)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   137
        else:
5155
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   138
            deferred = threads.deferToThread(self.render_request, request)
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   139
            return NOT_DONE_YET
1420
25c13e5b12bd stop complaining about empty response, remove trailing spaces
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
   140
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   141
    def render_request(self, request):
5243
1ab7acb9abe9 [etwist] we must call all error not handled in _render_request, else the thread fail silently and nothing ends up on the ui
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5223
diff changeset
   142
        try:
5759
5db529e0a3ee imported patch asynchronous multipart parsing.diff
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5660
diff changeset
   143
            # processing HUGE files (hundred of megabytes) in http.processReceived
5db529e0a3ee imported patch asynchronous multipart parsing.diff
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5660
diff changeset
   144
            # blocks other HTTP requests processing
5db529e0a3ee imported patch asynchronous multipart parsing.diff
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5660
diff changeset
   145
            # due to the clumsy & slow parsing algorithm of cgi.FieldStorage
5db529e0a3ee imported patch asynchronous multipart parsing.diff
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5660
diff changeset
   146
            # so we deferred that part to the cubicweb thread
5db529e0a3ee imported patch asynchronous multipart parsing.diff
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5660
diff changeset
   147
            request.process_multipart()
5243
1ab7acb9abe9 [etwist] we must call all error not handled in _render_request, else the thread fail silently and nothing ends up on the ui
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5223
diff changeset
   148
            return self._render_request(request)
7815
2a164a9cf81c [exceptions] stop catching any exception in various places (closes #1942716)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7764
diff changeset
   149
        except Exception:
5243
1ab7acb9abe9 [etwist] we must call all error not handled in _render_request, else the thread fail silently and nothing ends up on the ui
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5223
diff changeset
   150
            errorstream = StringIO()
1ab7acb9abe9 [etwist] we must call all error not handled in _render_request, else the thread fail silently and nothing ends up on the ui
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5223
diff changeset
   151
            traceback.print_exc(file=errorstream)
1ab7acb9abe9 [etwist] we must call all error not handled in _render_request, else the thread fail silently and nothing ends up on the ui
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5223
diff changeset
   152
            return HTTPResponse(stream='<pre>%s</pre>' % errorstream.getvalue(),
1ab7acb9abe9 [etwist] we must call all error not handled in _render_request, else the thread fail silently and nothing ends up on the ui
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5223
diff changeset
   153
                                code=500, twisted_request=request)
1ab7acb9abe9 [etwist] we must call all error not handled in _render_request, else the thread fail silently and nothing ends up on the ui
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5223
diff changeset
   154
1ab7acb9abe9 [etwist] we must call all error not handled in _render_request, else the thread fail silently and nothing ends up on the ui
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5223
diff changeset
   155
    def _render_request(self, request):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   156
        origpath = request.path
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   157
        host = request.host
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   158
        # dual http/https access handling: expect a rewrite rule to prepend
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   159
        # 'https' to the path to detect https access
8309
48ef505aa9f9 [request] gather all base_url logic in a single place (closes #2200756)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8308
diff changeset
   160
        https = False
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   161
        if origpath.split('/', 2)[1] == 'https':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   162
            origpath = origpath[6:]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   163
            request.uri = request.uri[6:]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   164
            https = True
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   165
        if self.url_rewriter is not None:
1115
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 151
diff changeset
   166
            # XXX should occur before authentication?
8312
6c2119509fac [web] Move request handling logic into cubicweb application. (closes #2200684)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8309
diff changeset
   167
            path = self.url_rewriter.rewrite(host, origpath, request)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   168
            request.uri.replace(origpath, path, 1)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   169
        else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   170
            path = origpath
8312
6c2119509fac [web] Move request handling logic into cubicweb application. (closes #2200684)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8309
diff changeset
   171
        req = CubicWebTwistedRequestAdapter(request, self.appli.vreg, https)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   172
        try:
8312
6c2119509fac [web] Move request handling logic into cubicweb application. (closes #2200684)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8309
diff changeset
   173
            ### Try to generate the actual request content
6c2119509fac [web] Move request handling logic into cubicweb application. (closes #2200684)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8309
diff changeset
   174
            content = self.appli.handle_request(req, path)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   175
        except DirectResponse, ex:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   176
            return ex.response
8312
6c2119509fac [web] Move request handling logic into cubicweb application. (closes #2200684)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8309
diff changeset
   177
        # at last: create twisted object
6c2119509fac [web] Move request handling logic into cubicweb application. (closes #2200684)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8309
diff changeset
   178
        return HTTPResponse(code    = req.status_out,
6c2119509fac [web] Move request handling logic into cubicweb application. (closes #2200684)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8309
diff changeset
   179
                            headers = req.headers_out,
6c2119509fac [web] Move request handling logic into cubicweb application. (closes #2200684)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8309
diff changeset
   180
                            stream  = content,
6c2119509fac [web] Move request handling logic into cubicweb application. (closes #2200684)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8309
diff changeset
   181
                            twisted_request=req._twreq)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   182
7879
9aae456abab5 [pylint] fix pylint detected errors and tweak it so that pylint -E will be much less verbose next time (+ update some copyrights on the way)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7815
diff changeset
   183
    # these are overridden by set_log_methods below
9aae456abab5 [pylint] fix pylint detected errors and tweak it so that pylint -E will be much less verbose next time (+ update some copyrights on the way)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7815
diff changeset
   184
    # only defining here to prevent pylint from complaining
9aae456abab5 [pylint] fix pylint detected errors and tweak it so that pylint -E will be much less verbose next time (+ update some copyrights on the way)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7815
diff changeset
   185
    @classmethod
9aae456abab5 [pylint] fix pylint detected errors and tweak it so that pylint -E will be much less verbose next time (+ update some copyrights on the way)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7815
diff changeset
   186
    def debug(cls, msg, *a, **kw):
9aae456abab5 [pylint] fix pylint detected errors and tweak it so that pylint -E will be much less verbose next time (+ update some copyrights on the way)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7815
diff changeset
   187
        pass
9aae456abab5 [pylint] fix pylint detected errors and tweak it so that pylint -E will be much less verbose next time (+ update some copyrights on the way)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7815
diff changeset
   188
    info = warning = error = critical = exception = debug
9aae456abab5 [pylint] fix pylint detected errors and tweak it so that pylint -E will be much less verbose next time (+ update some copyrights on the way)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7815
diff changeset
   189
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   190
5335
9aed0b607130 [etwist] nicer, thouhg really not perferct) request max size exceeded support of json/frame posts
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5330
diff changeset
   191
JSON_PATHS = set(('json',))
9aed0b607130 [etwist] nicer, thouhg really not perferct) request max size exceeded support of json/frame posts
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5330
diff changeset
   192
FRAME_POST_PATHS = set(('validateform',))
1420
25c13e5b12bd stop complaining about empty response, remove trailing spaces
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
   193
5270
6297d5265572 [etwist] allow to specify max size for POST request in the configuration. We should deal with this nicely on the js form validation code.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5243
diff changeset
   194
orig_gotLength = http.Request.gotLength
6297d5265572 [etwist] allow to specify max size for POST request in the configuration. We should deal with this nicely on the js form validation code.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5243
diff changeset
   195
@monkeypatch(http.Request)
6297d5265572 [etwist] allow to specify max size for POST request in the configuration. We should deal with this nicely on the js form validation code.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5243
diff changeset
   196
def gotLength(self, length):
6297d5265572 [etwist] allow to specify max size for POST request in the configuration. We should deal with this nicely on the js form validation code.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5243
diff changeset
   197
    orig_gotLength(self, length)
6297d5265572 [etwist] allow to specify max size for POST request in the configuration. We should deal with this nicely on the js form validation code.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5243
diff changeset
   198
    if length > MAX_POST_LENGTH: # length is 0 on GET
5335
9aed0b607130 [etwist] nicer, thouhg really not perferct) request max size exceeded support of json/frame posts
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5330
diff changeset
   199
        path = self.channel._path.split('?', 1)[0].rstrip('/').rsplit('/', 1)[-1]
5270
6297d5265572 [etwist] allow to specify max size for POST request in the configuration. We should deal with this nicely on the js form validation code.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5243
diff changeset
   200
        self.clientproto = 'HTTP/1.1' # not yet initialized
6297d5265572 [etwist] allow to specify max size for POST request in the configuration. We should deal with this nicely on the js form validation code.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5243
diff changeset
   201
        self.channel.persistent = 0   # force connection close on cleanup
6297d5265572 [etwist] allow to specify max size for POST request in the configuration. We should deal with this nicely on the js form validation code.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5243
diff changeset
   202
        self.setResponseCode(http.BAD_REQUEST)
5335
9aed0b607130 [etwist] nicer, thouhg really not perferct) request max size exceeded support of json/frame posts
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5330
diff changeset
   203
        if path in JSON_PATHS: # XXX better json path detection
9aed0b607130 [etwist] nicer, thouhg really not perferct) request max size exceeded support of json/frame posts
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5330
diff changeset
   204
            self.setHeader('content-type',"application/json")
5940
0e3ae19b181a [uilib] refactor json_dumps code organization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5933
diff changeset
   205
            body = json_dumps({'reason': 'request max size exceeded'})
5335
9aed0b607130 [etwist] nicer, thouhg really not perferct) request max size exceeded support of json/frame posts
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5330
diff changeset
   206
        elif path in FRAME_POST_PATHS: # XXX better frame post path detection
9aed0b607130 [etwist] nicer, thouhg really not perferct) request max size exceeded support of json/frame posts
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5330
diff changeset
   207
            self.setHeader('content-type',"text/html")
9aed0b607130 [etwist] nicer, thouhg really not perferct) request max size exceeded support of json/frame posts
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5330
diff changeset
   208
            body = ('<script type="text/javascript">'
9aed0b607130 [etwist] nicer, thouhg really not perferct) request max size exceeded support of json/frame posts
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5330
diff changeset
   209
                    'window.parent.handleFormValidationResponse(null, null, null, %s, null);'
5940
0e3ae19b181a [uilib] refactor json_dumps code organization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5933
diff changeset
   210
                    '</script>' % json_dumps( (False, 'request max size exceeded', None) ))
5335
9aed0b607130 [etwist] nicer, thouhg really not perferct) request max size exceeded support of json/frame posts
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5330
diff changeset
   211
        else:
9aed0b607130 [etwist] nicer, thouhg really not perferct) request max size exceeded support of json/frame posts
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5330
diff changeset
   212
            self.setHeader('content-type',"text/html")
9aed0b607130 [etwist] nicer, thouhg really not perferct) request max size exceeded support of json/frame posts
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5330
diff changeset
   213
            body = ("<html><head><title>Processing Failed</title></head><body>"
9aed0b607130 [etwist] nicer, thouhg really not perferct) request max size exceeded support of json/frame posts
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5330
diff changeset
   214
                    "<b>request max size exceeded</b></body></html>")
5270
6297d5265572 [etwist] allow to specify max size for POST request in the configuration. We should deal with this nicely on the js form validation code.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5243
diff changeset
   215
        self.setHeader('content-length', str(len(body)))
6297d5265572 [etwist] allow to specify max size for POST request in the configuration. We should deal with this nicely on the js form validation code.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5243
diff changeset
   216
        self.write(body)
6297d5265572 [etwist] allow to specify max size for POST request in the configuration. We should deal with this nicely on the js form validation code.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5243
diff changeset
   217
        # see request.finish(). Done here since we get error due to not full
6297d5265572 [etwist] allow to specify max size for POST request in the configuration. We should deal with this nicely on the js form validation code.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5243
diff changeset
   218
        # initialized request
6297d5265572 [etwist] allow to specify max size for POST request in the configuration. We should deal with this nicely on the js form validation code.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5243
diff changeset
   219
        self.finished = 1
6297d5265572 [etwist] allow to specify max size for POST request in the configuration. We should deal with this nicely on the js form validation code.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5243
diff changeset
   220
        if not self.queued:
6297d5265572 [etwist] allow to specify max size for POST request in the configuration. We should deal with this nicely on the js form validation code.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5243
diff changeset
   221
            self._cleanup()
6297d5265572 [etwist] allow to specify max size for POST request in the configuration. We should deal with this nicely on the js form validation code.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5243
diff changeset
   222
        for d in self.notifications:
6297d5265572 [etwist] allow to specify max size for POST request in the configuration. We should deal with this nicely on the js form validation code.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5243
diff changeset
   223
            d.callback(None)
6297d5265572 [etwist] allow to specify max size for POST request in the configuration. We should deal with this nicely on the js form validation code.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5243
diff changeset
   224
        self.notifications = []
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   225
5155
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   226
@monkeypatch(http.Request)
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   227
def requestReceived(self, command, path, version):
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   228
    """Called by channel when all data has been received.
1420
25c13e5b12bd stop complaining about empty response, remove trailing spaces
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
   229
5155
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   230
    This method is not intended for users.
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   231
    """
5272
f7d2df59231a cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5270
diff changeset
   232
    self.content.seek(0, 0)
5155
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   233
    self.args = {}
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   234
    self.files = {}
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   235
    self.stack = []
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   236
    self.method, self.uri = command, path
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   237
    self.clientproto = version
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   238
    x = self.uri.split('?', 1)
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   239
    if len(x) == 1:
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   240
        self.path = self.uri
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   241
    else:
5155
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   242
        self.path, argstring = x
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   243
        self.args = http.parse_qs(argstring, 1)
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   244
    # cache the client and server information, we'll need this later to be
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   245
    # serialized and sent with the request so CGIs will work remotely
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   246
    self.client = self.channel.transport.getPeer()
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   247
    self.host = self.channel.transport.getHost()
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   248
    # Argument processing
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   249
    ctype = self.getHeader('content-type')
5759
5db529e0a3ee imported patch asynchronous multipart parsing.diff
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5660
diff changeset
   250
    self._do_process_multipart = False
5155
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   251
    if self.method == "POST" and ctype:
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   252
        key, pdict = parse_header(ctype)
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   253
        if key == 'application/x-www-form-urlencoded':
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   254
            self.args.update(http.parse_qs(self.content.read(), 1))
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   255
        elif key == 'multipart/form-data':
5759
5db529e0a3ee imported patch asynchronous multipart parsing.diff
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5660
diff changeset
   256
            # defer this as it can be extremely time consumming
5db529e0a3ee imported patch asynchronous multipart parsing.diff
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5660
diff changeset
   257
            # with big files
5db529e0a3ee imported patch asynchronous multipart parsing.diff
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5660
diff changeset
   258
            self._do_process_multipart = True
5155
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   259
    self.process()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   260
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   261
5759
5db529e0a3ee imported patch asynchronous multipart parsing.diff
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5660
diff changeset
   262
@monkeypatch(http.Request)
5db529e0a3ee imported patch asynchronous multipart parsing.diff
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5660
diff changeset
   263
def process_multipart(self):
5db529e0a3ee imported patch asynchronous multipart parsing.diff
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5660
diff changeset
   264
    if not self._do_process_multipart:
5db529e0a3ee imported patch asynchronous multipart parsing.diff
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5660
diff changeset
   265
        return
5db529e0a3ee imported patch asynchronous multipart parsing.diff
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5660
diff changeset
   266
    form = FieldStorage(self.content, self.received_headers,
5db529e0a3ee imported patch asynchronous multipart parsing.diff
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5660
diff changeset
   267
                        environ={'REQUEST_METHOD': 'POST'},
5db529e0a3ee imported patch asynchronous multipart parsing.diff
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5660
diff changeset
   268
                        keep_blank_values=1,
5db529e0a3ee imported patch asynchronous multipart parsing.diff
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5660
diff changeset
   269
                        strict_parsing=1)
5db529e0a3ee imported patch asynchronous multipart parsing.diff
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5660
diff changeset
   270
    for key in form:
5db529e0a3ee imported patch asynchronous multipart parsing.diff
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5660
diff changeset
   271
        value = form[key]
5db529e0a3ee imported patch asynchronous multipart parsing.diff
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5660
diff changeset
   272
        if isinstance(value, list):
5db529e0a3ee imported patch asynchronous multipart parsing.diff
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5660
diff changeset
   273
            self.args[key] = [v.value for v in value]
5db529e0a3ee imported patch asynchronous multipart parsing.diff
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5660
diff changeset
   274
        elif value.filename:
5db529e0a3ee imported patch asynchronous multipart parsing.diff
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5660
diff changeset
   275
            if value.done != -1: # -1 is transfer has been interrupted
5db529e0a3ee imported patch asynchronous multipart parsing.diff
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5660
diff changeset
   276
                self.files[key] = (value.filename, value.file)
5db529e0a3ee imported patch asynchronous multipart parsing.diff
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5660
diff changeset
   277
            else:
5db529e0a3ee imported patch asynchronous multipart parsing.diff
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5660
diff changeset
   278
                self.files[key] = (None, None)
5db529e0a3ee imported patch asynchronous multipart parsing.diff
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5660
diff changeset
   279
        else:
5db529e0a3ee imported patch asynchronous multipart parsing.diff
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5660
diff changeset
   280
            self.args[key] = value.value
5db529e0a3ee imported patch asynchronous multipart parsing.diff
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5660
diff changeset
   281
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   282
from logging import getLogger
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   283
from cubicweb import set_log_methods
5155
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   284
LOGGER = getLogger('cubicweb.twisted')
1dea6e0fdfc1 Switched from TwistedWeb2 to TwistedWeb
Adrien Chauve <adrien.chauve@logilab.fr>
parents: 5062
diff changeset
   285
set_log_methods(CubicWebRootResource, LOGGER)
2654
6512522860aa [twisted] don't use twistd anymore, all-in-one.py file is needed anymore
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2476
diff changeset
   286
5653
c562791df9d2 [web server] vregistry may now be passed as named argument in CubicWebRootResource constructor
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5555
diff changeset
   287
def run(config, vreg=None, debug=None):
c562791df9d2 [web server] vregistry may now be passed as named argument in CubicWebRootResource constructor
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5555
diff changeset
   288
    if debug is not None:
c562791df9d2 [web server] vregistry may now be passed as named argument in CubicWebRootResource constructor
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5555
diff changeset
   289
        config.debugmode = debug
5933
3d707b8f8a4d [web configuration] ensure data home directory / uicache file belong to daemon user and are writeable
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5907
diff changeset
   290
    config.check_writeable_uid_directory(config.appdatahome)
2654
6512522860aa [twisted] don't use twistd anymore, all-in-one.py file is needed anymore
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2476
diff changeset
   291
    # create the site
5653
c562791df9d2 [web server] vregistry may now be passed as named argument in CubicWebRootResource constructor
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5555
diff changeset
   292
    root_resource = CubicWebRootResource(config, vreg=vreg)
2654
6512522860aa [twisted] don't use twistd anymore, all-in-one.py file is needed anymore
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2476
diff changeset
   293
    website = server.Site(root_resource)
6512522860aa [twisted] don't use twistd anymore, all-in-one.py file is needed anymore
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2476
diff changeset
   294
    # serve it via standard HTTP on port set in the configuration
6512522860aa [twisted] don't use twistd anymore, all-in-one.py file is needed anymore
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2476
diff changeset
   295
    port = config['port'] or 8080
6817
1959d97ebf2e [etwist] add an 'interface' config option (passed to reactor.listenTCP)
david.douard@logilab.fr
parents: 6779
diff changeset
   296
    interface = config['interface']
7644
7a0914469618 [twisted] add an option to configure twisted's threadpool size
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 7620
diff changeset
   297
    reactor.suggestThreadPoolSize(config['webserver-threadpool-size'])
6817
1959d97ebf2e [etwist] add an 'interface' config option (passed to reactor.listenTCP)
david.douard@logilab.fr
parents: 6779
diff changeset
   298
    reactor.listenTCP(port, website, interface=interface)
5442
3ed8afbbdf70 [webconfig] refactor/cleanup debug mode management on startup: simply use config.debugmode instead of debug argument everywhere...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   299
    if not config.debugmode:
4219
87203a0a3a81 do not attempt to daemonize under windows
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 4212
diff changeset
   300
        if sys.platform == 'win32':
4221
da84ca26896d raising ConfigurationError is cleaner; remove some unused imports
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 4219
diff changeset
   301
            raise ConfigurationError("Under windows, you must use the service management "
da84ca26896d raising ConfigurationError is cleaner; remove some unused imports
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 4219
diff changeset
   302
                                     "commands (e.g : 'net start my_instance)'")
5659
755f56f01c9d [win32] defer import of logilab.common.daemon
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5553
diff changeset
   303
        from logilab.common.daemon import daemonize
5679
0f2ded880d01 backport stable
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5655 5660
diff changeset
   304
        LOGGER.info('instance started in the background on %s', root_resource.base_url)
6779
accf5978a440 [daemon] make process umask configurable
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6778
diff changeset
   305
        whichproc = daemonize(config['pid-file'], umask=config['umask'])
6778
7dd4835d5198 properly tell we've started an instance (close #1391262)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5940
diff changeset
   306
        if whichproc: # 1 = orig process, 2 = first fork, None = second fork (eg daemon process)
7dd4835d5198 properly tell we've started an instance (close #1391262)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5940
diff changeset
   307
            return whichproc # parent process
3606
8326aceecb46 fix startup as a daemon
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3558
diff changeset
   308
    root_resource.init_publisher() # before changing uid
3193
c1a8456cb5f9 should change process'user once pid file has been written
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3181
diff changeset
   309
    if config['uid'] is not None:
7764
c709f6c457ff Drop privileges properly (closes #1910561)
Julien Cristau <julien.cristau@logilab.fr>
parents: 7647
diff changeset
   310
        from logilab.common.daemon import setugid
c709f6c457ff Drop privileges properly (closes #1910561)
Julien Cristau <julien.cristau@logilab.fr>
parents: 7647
diff changeset
   311
        setugid(config['uid'])
3606
8326aceecb46 fix startup as a daemon
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3558
diff changeset
   312
    root_resource.start_service()
5679
0f2ded880d01 backport stable
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5655 5660
diff changeset
   313
    LOGGER.info('instance started on %s', root_resource.base_url)
5652
904091dc4c7e [etwist] twisted won't install signal handler outside the main thread.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5553
diff changeset
   314
    # avoid annoying warnign if not in Main Thread
904091dc4c7e [etwist] twisted won't install signal handler outside the main thread.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5553
diff changeset
   315
    signals = threading.currentThread().getName() == 'MainThread'
2654
6512522860aa [twisted] don't use twistd anymore, all-in-one.py file is needed anymore
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2476
diff changeset
   316
    if config['profile']:
4960
26b2468a1e73 [etwist] cleanup, use cProfile instead of hotshot
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4914
diff changeset
   317
        import cProfile
5652
904091dc4c7e [etwist] twisted won't install signal handler outside the main thread.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5553
diff changeset
   318
        cProfile.runctx('reactor.run(installSignalHandlers=%s)' % signals,
904091dc4c7e [etwist] twisted won't install signal handler outside the main thread.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5553
diff changeset
   319
                        globals(), locals(), config['profile'])
2654
6512522860aa [twisted] don't use twistd anymore, all-in-one.py file is needed anymore
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2476
diff changeset
   320
    else:
5652
904091dc4c7e [etwist] twisted won't install signal handler outside the main thread.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 5553
diff changeset
   321
        reactor.run(installSignalHandlers=signals)