etwist/server.py
author Sylvain Thénault <sylvain.thenault@logilab.fr>
Tue, 30 Mar 2010 10:28:41 +0200
branchstable
changeset 5066 bf5cbc351e99
parent 5062 5691fd8697cd
child 5155 1dea6e0fdfc1
child 5252 9c765b1d8250
permissions -rw-r--r--
[repo] move eschema_eid function from hooks.metadata to server.utils
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
2476
1294a6bdf3bf application -> instance where it makes sense
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2181
diff changeset
     1
"""twisted server for CubicWeb web instances
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
     2
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
     3
:organization: Logilab
4212
ab6573088b4a update copyright: welcome 2010
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4194
diff changeset
     4
:copyright: 2001-2010 LOGILAB S.A. (Paris, FRANCE), license is LGPL v2.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
     5
:contact: http://www.logilab.fr/ -- mailto:contact@logilab.fr
1977
606923dff11b big bunch of copyright / docstring update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1936
diff changeset
     6
:license: GNU Lesser General Public License, v2.1 - http://www.gnu.org/licenses
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
     7
"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
     8
__docformat__ = "restructuredtext en"
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
     9
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    10
import sys
2654
6512522860aa [twisted] don't use twistd anymore, all-in-one.py file is needed anymore
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2476
diff changeset
    11
import os
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    12
import select
4398
4e7e74918631 add missing errno import in etwist/server.py
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4221
diff changeset
    13
import errno
1016
26387b836099 use datetime instead of mx.DateTime
sylvain.thenault@logilab.fr
parents: 151
diff changeset
    14
from time import mktime
26387b836099 use datetime instead of mx.DateTime
sylvain.thenault@logilab.fr
parents: 151
diff changeset
    15
from datetime import date, timedelta
1520
b097057e629d provide an option to substitute the base-url (left-most part) subdomain by the one of the current http query to easy multiple subdomains website management
Florent <florent@secondweb.fr>
parents: 1420
diff changeset
    16
from urlparse import urlsplit, urlunsplit
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    17
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    18
from twisted.internet import reactor, task, threads
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    19
from twisted.internet.defer import maybeDeferred
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    20
from twisted.web2 import channel, http, server, iweb
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    21
from twisted.web2 import static, resource, responsecode
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    22
4221
da84ca26896d raising ConfigurationError is cleaner; remove some unused imports
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 4219
diff changeset
    23
from cubicweb import ConfigurationError, CW_EVENT_MANAGER
1420
25c13e5b12bd stop complaining about empty response, remove trailing spaces
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
    24
from cubicweb.web import (AuthenticationError, NotFound, Redirect,
2685
0518ca8f63e3 [autoreload] recompute urlresolver / urlrewriter after autoreload
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2670
diff changeset
    25
                          RemoteCallFailed, DirectResponse, StatusResponse,
0518ca8f63e3 [autoreload] recompute urlresolver / urlrewriter after autoreload
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2670
diff changeset
    26
                          ExplicitLogin)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    27
from cubicweb.web.application import CubicWebPublisher
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    28
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    29
from cubicweb.etwist.request import CubicWebTwistedRequestAdapter
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    30
3193
c1a8456cb5f9 should change process'user once pid file has been written
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3181
diff changeset
    31
def daemonize():
3181
735c5f9fcded fix daemonize: set umask, move to /, consider uid
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2711
diff changeset
    32
    # XXX unix specific
735c5f9fcded fix daemonize: set umask, move to /, consider uid
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2711
diff changeset
    33
    # XXX factorize w/ code in cw.server.server and cw.server.serverctl
735c5f9fcded fix daemonize: set umask, move to /, consider uid
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2711
diff changeset
    34
    # (start-repository command)
735c5f9fcded fix daemonize: set umask, move to /, consider uid
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2711
diff changeset
    35
    # See http://www.erlenstar.demon.co.uk/unix/faq_toc.html#TOC16
735c5f9fcded fix daemonize: set umask, move to /, consider uid
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2711
diff changeset
    36
    if os.fork():   # launch child and...
3606
8326aceecb46 fix startup as a daemon
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3558
diff changeset
    37
        return 1
3181
735c5f9fcded fix daemonize: set umask, move to /, consider uid
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2711
diff changeset
    38
    os.setsid()
3606
8326aceecb46 fix startup as a daemon
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3558
diff changeset
    39
    if os.fork():   # launch child again.
8326aceecb46 fix startup as a daemon
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3558
diff changeset
    40
        return 1
3181
735c5f9fcded fix daemonize: set umask, move to /, consider uid
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2711
diff changeset
    41
    # move to the root to avoit mount pb
735c5f9fcded fix daemonize: set umask, move to /, consider uid
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2711
diff changeset
    42
    os.chdir('/')
735c5f9fcded fix daemonize: set umask, move to /, consider uid
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2711
diff changeset
    43
    # set paranoid umask
735c5f9fcded fix daemonize: set umask, move to /, consider uid
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2711
diff changeset
    44
    os.umask(077)
735c5f9fcded fix daemonize: set umask, move to /, consider uid
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2711
diff changeset
    45
    null = os.open('/dev/null', os.O_RDWR)
735c5f9fcded fix daemonize: set umask, move to /, consider uid
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2711
diff changeset
    46
    for i in range(3):
735c5f9fcded fix daemonize: set umask, move to /, consider uid
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2711
diff changeset
    47
        try:
735c5f9fcded fix daemonize: set umask, move to /, consider uid
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2711
diff changeset
    48
            os.dup2(null, i)
735c5f9fcded fix daemonize: set umask, move to /, consider uid
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2711
diff changeset
    49
        except OSError, e:
735c5f9fcded fix daemonize: set umask, move to /, consider uid
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2711
diff changeset
    50
            if e.errno != errno.EBADF:
735c5f9fcded fix daemonize: set umask, move to /, consider uid
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2711
diff changeset
    51
                raise
735c5f9fcded fix daemonize: set umask, move to /, consider uid
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2711
diff changeset
    52
    os.close(null)
735c5f9fcded fix daemonize: set umask, move to /, consider uid
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2711
diff changeset
    53
    return None
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    54
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    55
def start_task(interval, func):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    56
    lc = task.LoopingCall(func)
3493
3e4603678d4e use default repo looping task so they are started in thread, else we get blocking task
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3198
diff changeset
    57
    # wait until interval has expired to actually start the task, else we have
3e4603678d4e use default repo looping task so they are started in thread, else we get blocking task
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3198
diff changeset
    58
    # to wait all task to be finished for the server to be actually started
3e4603678d4e use default repo looping task so they are started in thread, else we get blocking task
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3198
diff changeset
    59
    lc.start(interval, now=False)
1420
25c13e5b12bd stop complaining about empty response, remove trailing spaces
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
    60
1543
dca9817bb337 fix use-request-subdomain option behaviour and add tests
Florent <florent@secondweb.fr>
parents: 1542
diff changeset
    61
def host_prefixed_baseurl(baseurl, host):
dca9817bb337 fix use-request-subdomain option behaviour and add tests
Florent <florent@secondweb.fr>
parents: 1542
diff changeset
    62
    scheme, netloc, url, query, fragment = urlsplit(baseurl)
dca9817bb337 fix use-request-subdomain option behaviour and add tests
Florent <florent@secondweb.fr>
parents: 1542
diff changeset
    63
    netloc_domain = '.' + '.'.join(netloc.split('.')[-2:])
dca9817bb337 fix use-request-subdomain option behaviour and add tests
Florent <florent@secondweb.fr>
parents: 1542
diff changeset
    64
    if host.endswith(netloc_domain):
dca9817bb337 fix use-request-subdomain option behaviour and add tests
Florent <florent@secondweb.fr>
parents: 1542
diff changeset
    65
        netloc = host
dca9817bb337 fix use-request-subdomain option behaviour and add tests
Florent <florent@secondweb.fr>
parents: 1542
diff changeset
    66
    baseurl = urlunsplit((scheme, netloc, url, query, fragment))
dca9817bb337 fix use-request-subdomain option behaviour and add tests
Florent <florent@secondweb.fr>
parents: 1542
diff changeset
    67
    return baseurl
dca9817bb337 fix use-request-subdomain option behaviour and add tests
Florent <florent@secondweb.fr>
parents: 1542
diff changeset
    68
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    69
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    70
class LongTimeExpiringFile(static.File):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    71
    """overrides static.File and sets a far futre ``Expires`` date
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    72
    on the resouce.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    73
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    74
    versions handling is done by serving static files by different
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    75
    URLs for each version. For instance::
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    76
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    77
      http://localhost:8080/data-2.48.2/cubicweb.css
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    78
      http://localhost:8080/data-2.49.0/cubicweb.css
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    79
      etc.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    80
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    81
    """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    82
    def renderHTTP(self, request):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    83
        def setExpireHeader(response):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    84
            response = iweb.IResponse(response)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    85
            # Don't provide additional resource information to error responses
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    86
            if response.code < 400:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    87
                # the HTTP RFC recommands not going further than 1 year ahead
1016
26387b836099 use datetime instead of mx.DateTime
sylvain.thenault@logilab.fr
parents: 151
diff changeset
    88
                expires = date.today() + timedelta(days=6*30)
26387b836099 use datetime instead of mx.DateTime
sylvain.thenault@logilab.fr
parents: 151
diff changeset
    89
                response.headers.setHeader('Expires', mktime(expires.timetuple()))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    90
            return response
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    91
        d = maybeDeferred(super(LongTimeExpiringFile, self).renderHTTP, request)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    92
        return d.addCallback(setExpireHeader)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    93
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    94
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    95
class CubicWebRootResource(resource.PostableResource):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    96
    addSlash = False
1420
25c13e5b12bd stop complaining about empty response, remove trailing spaces
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
    97
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    98
    def __init__(self, config, debug=None):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    99
        self.debugmode = debug
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   100
        self.config = config
3828
335e40fdb5a5 when instance is started normally (eg not in debug mode), check versions before daemonization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3791
diff changeset
   101
        # instantiate publisher here and not in init_publisher to get some
335e40fdb5a5 when instance is started normally (eg not in debug mode), check versions before daemonization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3791
diff changeset
   102
        # checks done before daemonization (eg versions consistency)
335e40fdb5a5 when instance is started normally (eg not in debug mode), check versions before daemonization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3791
diff changeset
   103
        self.appli = CubicWebPublisher(config, debug=self.debugmode)
4909
a13c1d1ecc5b [web server] simplify base-url handling on startup. Ensure config['base-url'] is correctly set once started
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   104
        self.base_url = config['base-url']
a13c1d1ecc5b [web server] simplify base-url handling on startup. Ensure config['base-url'] is correctly set once started
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   105
        self.https_url = config['https-url']
3828
335e40fdb5a5 when instance is started normally (eg not in debug mode), check versions before daemonization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3791
diff changeset
   106
        self.versioned_datadir = 'data%s' % config.instance_md5_version()
3606
8326aceecb46 fix startup as a daemon
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3558
diff changeset
   107
8326aceecb46 fix startup as a daemon
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3558
diff changeset
   108
    def init_publisher(self):
8326aceecb46 fix startup as a daemon
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3558
diff changeset
   109
        config = self.config
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   110
        # when we have an in-memory repository, clean unused sessions every XX
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   111
        # seconds and properly shutdown the server
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   112
        if config.repo_method == 'inmemory':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   113
            reactor.addSystemEventTrigger('before', 'shutdown',
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   114
                                          self.shutdown_event)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   115
            if config.pyro_enabled():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   116
                # if pyro is enabled, we have to register to the pyro name
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   117
                # server, create a pyro daemon, and create a task to handle pyro
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   118
                # requests
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   119
                self.pyro_daemon = self.appli.repo.pyro_register()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   120
                self.pyro_listen_timeout = 0.02
3502
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3493
diff changeset
   121
                self.appli.repo.looping_task(1, self.pyro_loop_event)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   122
            self.appli.repo.start_looping_tasks()
2685
0518ca8f63e3 [autoreload] recompute urlresolver / urlrewriter after autoreload
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2670
diff changeset
   123
        self.set_url_rewriter()
2705
30bcdbd92820 [events] renamed source-reload into registry-reload to avoid potential confusions with datasources
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2694
diff changeset
   124
        CW_EVENT_MANAGER.bind('after-registry-reload', self.set_url_rewriter)
3606
8326aceecb46 fix startup as a daemon
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3558
diff changeset
   125
8326aceecb46 fix startup as a daemon
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3558
diff changeset
   126
    def start_service(self):
8326aceecb46 fix startup as a daemon
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3558
diff changeset
   127
        config = self.config
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   128
        interval = min(config['cleanup-session-time'] or 120,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   129
                       config['cleanup-anonymous-session-time'] or 720) / 2.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   130
        start_task(interval, self.appli.session_handler.clean_sessions)
1420
25c13e5b12bd stop complaining about empty response, remove trailing spaces
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
   131
2685
0518ca8f63e3 [autoreload] recompute urlresolver / urlrewriter after autoreload
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2670
diff changeset
   132
    def set_url_rewriter(self):
2770
356e9d7c356d R propagate registry API changes
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2711
diff changeset
   133
        self.url_rewriter = self.appli.vreg['components'].select_or_none('urlrewriter')
2685
0518ca8f63e3 [autoreload] recompute urlresolver / urlrewriter after autoreload
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2670
diff changeset
   134
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   135
    def shutdown_event(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   136
        """callback fired when the server is shutting down to properly
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   137
        clean opened sessions
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   138
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   139
        self.appli.repo.shutdown()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   140
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   141
    def pyro_loop_event(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   142
        """listen for pyro events"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   143
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   144
            self.pyro_daemon.handleRequests(self.pyro_listen_timeout)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   145
        except select.error:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   146
            return
1420
25c13e5b12bd stop complaining about empty response, remove trailing spaces
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
   147
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   148
    def locateChild(self, request, segments):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   149
        """Indicate which resource to use to process down the URL's path"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   150
        if segments:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   151
            if segments[0] == 'https':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   152
                segments = segments[1:]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   153
            if len(segments) >= 2:
151
343e7a18675d static files support
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents: 0
diff changeset
   154
                if segments[0] in (self.versioned_datadir, 'data', 'static'):
343e7a18675d static files support
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents: 0
diff changeset
   155
                    # Anything in data/, static/ is treated as static files
343e7a18675d static files support
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents: 0
diff changeset
   156
                    if segments[0] == 'static':
343e7a18675d static files support
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents: 0
diff changeset
   157
                        # instance static directory
343e7a18675d static files support
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents: 0
diff changeset
   158
                        datadir = self.config.static_directory
2169
124fb0b9fa4b fckeditor may be a subdirectory of the data directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
   159
                    elif segments[1] == 'fckeditor':
124fb0b9fa4b fckeditor may be a subdirectory of the data directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
   160
                        fckeditordir = self.config.ext_resources['FCKEDITOR_PATH']
124fb0b9fa4b fckeditor may be a subdirectory of the data directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
   161
                        return static.File(fckeditordir), segments[2:]
151
343e7a18675d static files support
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents: 0
diff changeset
   162
                    else:
343e7a18675d static files support
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents: 0
diff changeset
   163
                        # cube static data file
343e7a18675d static files support
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents: 0
diff changeset
   164
                        datadir = self.config.locate_resource(segments[1])
343e7a18675d static files support
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents: 0
diff changeset
   165
                        if datadir is None:
343e7a18675d static files support
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents: 0
diff changeset
   166
                            return None, []
5062
5691fd8697cd [logging] demote static file served to debug
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 4960
diff changeset
   167
                    self.debug('static file %s from %s', segments[-1], datadir)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   168
                    if segments[0] == 'data':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   169
                        return static.File(str(datadir)), segments[1:]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   170
                    else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   171
                        return LongTimeExpiringFile(datadir), segments[1:]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   172
                elif segments[0] == 'fckeditor':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   173
                    fckeditordir = self.config.ext_resources['FCKEDITOR_PATH']
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   174
                    return static.File(fckeditordir), segments[1:]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   175
        # Otherwise we use this single resource
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   176
        return self, ()
1420
25c13e5b12bd stop complaining about empty response, remove trailing spaces
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
   177
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   178
    def render(self, request):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   179
        """Render a page from the root resource"""
3638
648d6dbec630 system/user modes + CWDEV instead of installed/dev mixed modes. Fix behaviour when setting CW_MODE explicitly
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3618
diff changeset
   180
        # reload modified files in debug mode
3641
cf30e4498674 fix debug attribute conflict on configuration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3638
diff changeset
   181
        if self.debugmode:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   182
            self.appli.vreg.register_objects(self.config.vregistry_path())
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   183
        if self.config['profile']: # default profiler don't trace threads
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   184
            return self.render_request(request)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   185
        else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   186
            return threads.deferToThread(self.render_request, request)
1420
25c13e5b12bd stop complaining about empty response, remove trailing spaces
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
   187
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   188
    def render_request(self, request):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   189
        origpath = request.path
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   190
        host = request.host
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   191
        # dual http/https access handling: expect a rewrite rule to prepend
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   192
        # 'https' to the path to detect https access
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   193
        if origpath.split('/', 2)[1] == 'https':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   194
            origpath = origpath[6:]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   195
            request.uri = request.uri[6:]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   196
            https = True
1420
25c13e5b12bd stop complaining about empty response, remove trailing spaces
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
   197
            baseurl = self.https_url or self.base_url
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   198
        else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   199
            https = False
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   200
            baseurl = self.base_url
1520
b097057e629d provide an option to substitute the base-url (left-most part) subdomain by the one of the current http query to easy multiple subdomains website management
Florent <florent@secondweb.fr>
parents: 1420
diff changeset
   201
        if self.config['use-request-subdomain']:
1543
dca9817bb337 fix use-request-subdomain option behaviour and add tests
Florent <florent@secondweb.fr>
parents: 1542
diff changeset
   202
            baseurl = host_prefixed_baseurl(baseurl, host)
dca9817bb337 fix use-request-subdomain option behaviour and add tests
Florent <florent@secondweb.fr>
parents: 1542
diff changeset
   203
            self.warning('used baseurl is %s for this request', baseurl)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   204
        req = CubicWebTwistedRequestAdapter(request, self.appli.vreg, https, baseurl)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   205
        if req.authmode == 'http':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   206
            # activate realm-based auth
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   207
            realm = self.config['realm']
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   208
            req.set_header('WWW-Authenticate', [('Basic', {'realm' : realm })], raw=False)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   209
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   210
            self.appli.connect(req)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   211
        except AuthenticationError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   212
            return self.request_auth(req)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   213
        except Redirect, ex:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   214
            return self.redirect(req, ex.location)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   215
        if https and req.cnx.anonymous_connection:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   216
            # don't allow anonymous on https connection
1420
25c13e5b12bd stop complaining about empty response, remove trailing spaces
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
   217
            return self.request_auth(req)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   218
        if self.url_rewriter is not None:
1115
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 151
diff changeset
   219
            # XXX should occur before authentication?
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   220
            try:
1936
c5af2fbda5b6 pass request to ApacheRewriter rewrite method
Florent <florent@secondweb.fr>
parents: 1543
diff changeset
   221
                path = self.url_rewriter.rewrite(host, origpath, req)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   222
            except Redirect, ex:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   223
                return self.redirect(req, ex.location)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   224
            request.uri.replace(origpath, path, 1)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   225
        else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   226
            path = origpath
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   227
        if not path or path == "/":
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   228
            path = 'view'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   229
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   230
            result = self.appli.publish(path, req)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   231
        except DirectResponse, ex:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   232
            return ex.response
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   233
        except StatusResponse, ex:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   234
            return http.Response(stream=ex.content, code=ex.status,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   235
                                 headers=req.headers_out or None)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   236
        except RemoteCallFailed, ex:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   237
            req.set_header('content-type', 'application/json')
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   238
            return http.Response(stream=ex.dumps(),
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   239
                                 code=responsecode.INTERNAL_SERVER_ERROR)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   240
        except NotFound:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   241
            result = self.appli.notfound_content(req)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   242
            return http.Response(stream=result, code=responsecode.NOT_FOUND,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   243
                                 headers=req.headers_out or None)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   244
        except ExplicitLogin:  # must be before AuthenticationError
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   245
            return self.request_auth(req)
4911
898c35be5873 #750055: make it easier to change post logout url
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4909
diff changeset
   246
        except AuthenticationError, ex:
898c35be5873 #750055: make it easier to change post logout url
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4909
diff changeset
   247
            if self.config['auth-mode'] == 'cookie' and getattr(ex, 'url', None):
898c35be5873 #750055: make it easier to change post logout url
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4909
diff changeset
   248
                return self.redirect(req, ex.url)
898c35be5873 #750055: make it easier to change post logout url
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4909
diff changeset
   249
            # in http we have to request auth to flush current http auth
898c35be5873 #750055: make it easier to change post logout url
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4909
diff changeset
   250
            # information
898c35be5873 #750055: make it easier to change post logout url
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4909
diff changeset
   251
            return self.request_auth(req, loggedout=True)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   252
        except Redirect, ex:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   253
            return self.redirect(req, ex.location)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   254
        # request may be referenced by "onetime callback", so clear its entity
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   255
        # cache to avoid memory usage
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   256
        req.drop_entity_cache()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   257
        return http.Response(stream=result, code=responsecode.OK,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   258
                             headers=req.headers_out or None)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   259
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   260
    def redirect(self, req, location):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   261
        req.headers_out.setHeader('location', str(location))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   262
        self.debug('redirecting to %s', location)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   263
        # 303 See other
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   264
        return http.Response(code=303, headers=req.headers_out)
1420
25c13e5b12bd stop complaining about empty response, remove trailing spaces
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
   265
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   266
    def request_auth(self, req, loggedout=False):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   267
        if self.https_url and req.base_url() != self.https_url:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   268
            req.headers_out.setHeader('location', self.https_url + 'login')
1420
25c13e5b12bd stop complaining about empty response, remove trailing spaces
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
   269
            return http.Response(code=303, headers=req.headers_out)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   270
        if self.config['auth-mode'] == 'http':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   271
            code = responsecode.UNAUTHORIZED
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   272
        else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   273
            code = responsecode.FORBIDDEN
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   274
        if loggedout:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   275
            if req.https:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   276
                req._base_url =  self.base_url
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   277
                req.https = False
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   278
            content = self.appli.loggedout_content(req)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   279
        else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   280
            content = self.appli.need_login_content(req)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   281
        return http.Response(code, req.headers_out, content)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   282
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   283
from twisted.internet import defer
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   284
from twisted.web2 import fileupload
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   285
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   286
# XXX set max file size to 100Mo: put max upload size in the configuration
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   287
# line below for twisted >= 8.0, default param value for earlier version
1420
25c13e5b12bd stop complaining about empty response, remove trailing spaces
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
   288
resource.PostableResource.maxSize = 100*1024*1024
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   289
def parsePOSTData(request, maxMem=100*1024, maxFields=1024,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   290
                  maxSize=100*1024*1024):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   291
    if request.stream.length == 0:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   292
        return defer.succeed(None)
1420
25c13e5b12bd stop complaining about empty response, remove trailing spaces
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
   293
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   294
    ctype = request.headers.getHeader('content-type')
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   295
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   296
    if ctype is None:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   297
        return defer.succeed(None)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   298
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   299
    def updateArgs(data):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   300
        args = data
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   301
        request.args.update(args)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   302
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   303
    def updateArgsAndFiles(data):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   304
        args, files = data
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   305
        request.args.update(args)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   306
        request.files.update(files)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   307
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   308
    def error(f):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   309
        f.trap(fileupload.MimeFormatError)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   310
        raise http.HTTPError(responsecode.BAD_REQUEST)
1420
25c13e5b12bd stop complaining about empty response, remove trailing spaces
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
   311
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   312
    if ctype.mediaType == 'application' and ctype.mediaSubtype == 'x-www-form-urlencoded':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   313
        d = fileupload.parse_urlencoded(request.stream, keep_blank_values=True)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   314
        d.addCallbacks(updateArgs, error)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   315
        return d
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   316
    elif ctype.mediaType == 'multipart' and ctype.mediaSubtype == 'form-data':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   317
        boundary = ctype.params.get('boundary')
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   318
        if boundary is None:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   319
            return defer.fail(http.HTTPError(
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   320
                http.StatusResponse(responsecode.BAD_REQUEST,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   321
                                    "Boundary not specified in Content-Type.")))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   322
        d = fileupload.parseMultipartFormData(request.stream, boundary,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   323
                                              maxMem, maxFields, maxSize)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   324
        d.addCallbacks(updateArgsAndFiles, error)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   325
        return d
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   326
    else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   327
        raise http.HTTPError(responsecode.BAD_REQUEST)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   328
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   329
server.parsePOSTData = parsePOSTData
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   330
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   331
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   332
from logging import getLogger
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   333
from cubicweb import set_log_methods
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   334
set_log_methods(CubicWebRootResource, getLogger('cubicweb.twisted'))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   335
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   336
4772
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   337
listiterator = type(iter([]))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   338
4772
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   339
def _gc_debug(all=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   340
    import gc
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   341
    from pprint import pprint
2657
de974465d381 [appobject] kill VObject class, move base selector classes to appobject
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2654
diff changeset
   342
    from cubicweb.appobject import AppObject
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   343
    gc.collect()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   344
    count = 0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   345
    acount = 0
4772
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   346
    fcount = 0
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   347
    rcount = 0
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   348
    ccount = 0
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   349
    scount = 0
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   350
    ocount = {}
4772
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   351
    from rql.stmts import Union
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   352
    from cubicweb.schema import CubicWebSchema
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   353
    from cubicweb.rset import ResultSet
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   354
    from cubicweb.dbapi import Connection, Cursor
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   355
    from cubicweb.req import RequestSessionBase
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   356
    from cubicweb.server.repository import Repository
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   357
    from cubicweb.server.sources.native import NativeSQLSource
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   358
    from cubicweb.server.session import Session
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   359
    from cubicweb.devtools.testlib import CubicWebTC
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   360
    from logilab.common.testlib import TestSuite
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   361
    from optparse import Values
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   362
    import types, weakref
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   363
    for obj in gc.get_objects():
4772
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   364
        if isinstance(obj, RequestSessionBase):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   365
            count += 1
4772
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   366
            if isinstance(obj, Session):
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   367
                print '   session', obj, referrers(obj, True)
2657
de974465d381 [appobject] kill VObject class, move base selector classes to appobject
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2654
diff changeset
   368
        elif isinstance(obj, AppObject):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   369
            acount += 1
4772
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   370
        elif isinstance(obj, ResultSet):
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   371
            rcount += 1
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   372
            #print '   rset', obj, referrers(obj)
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   373
        elif isinstance(obj, Repository):
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   374
            print '   REPO', obj, referrers(obj, True)
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   375
        #elif isinstance(obj, NativeSQLSource):
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   376
        #    print '   SOURCe', obj, referrers(obj)
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   377
        elif isinstance(obj, CubicWebTC):
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   378
            print '   TC', obj, referrers(obj)
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   379
        elif isinstance(obj, TestSuite):
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   380
            print '   SUITE', obj, referrers(obj)
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   381
        #elif isinstance(obj, Values):
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   382
        #    print '   values', '%#x' % id(obj), referrers(obj, True)
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   383
        elif isinstance(obj, Connection):
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   384
            ccount += 1
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   385
            #print '   cnx', obj, referrers(obj)
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   386
        #elif isinstance(obj, Cursor):
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   387
        #    ccount += 1
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   388
        #    print '   cursor', obj, referrers(obj)
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   389
        elif isinstance(obj, file):
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   390
            fcount += 1
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   391
        #    print '   open file', file.name, file.fileno
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   392
        elif isinstance(obj, CubicWebSchema):
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   393
            scount += 1
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   394
            print '   schema', obj, referrers(obj)
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   395
        elif not isinstance(obj, (type, tuple, dict, list, set, frozenset,
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   396
                                  weakref.ref, weakref.WeakKeyDictionary,
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   397
                                  listiterator,
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   398
                                  property, classmethod,
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   399
                                  types.ModuleType, types.MemberDescriptorType,
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   400
                                  types.FunctionType, types.MethodType)):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   401
            try:
1132
96752791c2b6 pylint cleanup
sylvain.thenault@logilab.fr
parents: 1016
diff changeset
   402
                ocount[obj.__class__] += 1
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   403
            except KeyError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   404
                ocount[obj.__class__] = 1
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   405
            except AttributeError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   406
                pass
4772
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   407
    if count:
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   408
        print ' NB REQUESTS/SESSIONS', count
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   409
    if acount:
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   410
        print ' NB APPOBJECTS', acount
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   411
    if ccount:
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   412
        print ' NB CONNECTIONS', ccount
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   413
    if rcount:
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   414
        print ' NB RSETS', rcount
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   415
    if scount:
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   416
        print ' NB SCHEMAS', scount
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   417
    if fcount:
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   418
        print ' NB FILES', fcount
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   419
    if all:
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   420
        ocount = sorted(ocount.items(), key=lambda x: x[1], reverse=True)[:20]
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   421
        pprint(ocount)
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   422
    if gc.garbage:
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   423
        print 'UNREACHABLE', gc.garbage
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   424
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   425
def referrers(obj, showobj=False):
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   426
    try:
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   427
        return sorted(set((type(x), showobj and x or getattr(x, '__name__', '%#x' % id(x)))
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   428
                          for x in _referrers(obj)))
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   429
    except TypeError:
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   430
        s = set()
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   431
        unhashable = []
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   432
        for x in _referrers(obj):
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   433
            try:
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   434
                s.add(x)
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   435
            except TypeError:
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   436
                unhashable.append(x)
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   437
        return sorted(s) + unhashable
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   438
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   439
def _referrers(obj, seen=None, level=0):
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   440
    import gc, types
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   441
    from cubicweb.schema import CubicWebRelationSchema, CubicWebEntitySchema
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   442
    interesting = []
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   443
    if seen is None:
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   444
        seen = set()
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   445
    for x in gc.get_referrers(obj):
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   446
        if id(x) in seen:
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   447
            continue
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   448
        seen.add(id(x))
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   449
        if isinstance(x, types.FrameType):
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   450
            continue
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   451
        if isinstance(x, (CubicWebRelationSchema, CubicWebEntitySchema)):
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   452
            continue
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   453
        if isinstance(x, (list, tuple, set, dict, listiterator)):
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   454
            if level >= 5:
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   455
                pass
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   456
                #interesting.append(x)
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   457
            else:
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   458
                interesting += _referrers(x, seen, level+1)
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   459
        else:
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   460
            interesting.append(x)
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   461
    return interesting
2654
6512522860aa [twisted] don't use twistd anymore, all-in-one.py file is needed anymore
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2476
diff changeset
   462
6512522860aa [twisted] don't use twistd anymore, all-in-one.py file is needed anymore
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2476
diff changeset
   463
def run(config, debug):
6512522860aa [twisted] don't use twistd anymore, all-in-one.py file is needed anymore
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2476
diff changeset
   464
    # create the site
6512522860aa [twisted] don't use twistd anymore, all-in-one.py file is needed anymore
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2476
diff changeset
   465
    root_resource = CubicWebRootResource(config, debug)
6512522860aa [twisted] don't use twistd anymore, all-in-one.py file is needed anymore
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2476
diff changeset
   466
    website = server.Site(root_resource)
6512522860aa [twisted] don't use twistd anymore, all-in-one.py file is needed anymore
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2476
diff changeset
   467
    # serve it via standard HTTP on port set in the configuration
6512522860aa [twisted] don't use twistd anymore, all-in-one.py file is needed anymore
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2476
diff changeset
   468
    port = config['port'] or 8080
6512522860aa [twisted] don't use twistd anymore, all-in-one.py file is needed anymore
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2476
diff changeset
   469
    reactor.listenTCP(port, channel.HTTPFactory(website))
2694
4303ff921a9b [twisted] use cubicweb.twisted.logger instead of print to notify when the instance is started
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2685
diff changeset
   470
    logger = getLogger('cubicweb.twisted')
2654
6512522860aa [twisted] don't use twistd anymore, all-in-one.py file is needed anymore
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2476
diff changeset
   471
    if not debug:
4219
87203a0a3a81 do not attempt to daemonize under windows
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 4212
diff changeset
   472
        if sys.platform == 'win32':
4221
da84ca26896d raising ConfigurationError is cleaner; remove some unused imports
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 4219
diff changeset
   473
            raise ConfigurationError("Under windows, you must use the service management "
da84ca26896d raising ConfigurationError is cleaner; remove some unused imports
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 4219
diff changeset
   474
                                     "commands (e.g : 'net start my_instance)'")
3558
0e478957a9d4 smallest possible change to fix #472806
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3502
diff changeset
   475
        print 'instance starting in the background'
3606
8326aceecb46 fix startup as a daemon
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3558
diff changeset
   476
        if daemonize():
8326aceecb46 fix startup as a daemon
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3558
diff changeset
   477
            return # child process
2654
6512522860aa [twisted] don't use twistd anymore, all-in-one.py file is needed anymore
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2476
diff changeset
   478
        if config['pid-file']:
2711
7aee3bd7a704 ensure runtime dir exists
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2708
diff changeset
   479
            # ensure the directory where the pid-file should be set exists (for
3791
62d6ab877694 put back url info started at the end of initialisations
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3641
diff changeset
   480
            # instance /var/run/cubicweb may be deleted on computer restart)
2711
7aee3bd7a704 ensure runtime dir exists
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2708
diff changeset
   481
            piddir = os.path.dirname(config['pid-file'])
7aee3bd7a704 ensure runtime dir exists
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2708
diff changeset
   482
            if not os.path.exists(piddir):
7aee3bd7a704 ensure runtime dir exists
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2708
diff changeset
   483
                os.makedirs(piddir)
2654
6512522860aa [twisted] don't use twistd anymore, all-in-one.py file is needed anymore
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2476
diff changeset
   484
            file(config['pid-file'], 'w').write(str(os.getpid()))
3606
8326aceecb46 fix startup as a daemon
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3558
diff changeset
   485
    root_resource.init_publisher() # before changing uid
3193
c1a8456cb5f9 should change process'user once pid file has been written
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3181
diff changeset
   486
    if config['uid'] is not None:
c1a8456cb5f9 should change process'user once pid file has been written
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3181
diff changeset
   487
        try:
c1a8456cb5f9 should change process'user once pid file has been written
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3181
diff changeset
   488
            uid = int(config['uid'])
c1a8456cb5f9 should change process'user once pid file has been written
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3181
diff changeset
   489
        except ValueError:
c1a8456cb5f9 should change process'user once pid file has been written
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3181
diff changeset
   490
            from pwd import getpwnam
c1a8456cb5f9 should change process'user once pid file has been written
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3181
diff changeset
   491
            uid = getpwnam(config['uid']).pw_uid
c1a8456cb5f9 should change process'user once pid file has been written
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3181
diff changeset
   492
        os.setuid(uid)
3606
8326aceecb46 fix startup as a daemon
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3558
diff changeset
   493
    root_resource.start_service()
3791
62d6ab877694 put back url info started at the end of initialisations
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3641
diff changeset
   494
    logger.info('instance started on %s', root_resource.base_url)
2654
6512522860aa [twisted] don't use twistd anymore, all-in-one.py file is needed anymore
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2476
diff changeset
   495
    if config['profile']:
4960
26b2468a1e73 [etwist] cleanup, use cProfile instead of hotshot
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4914
diff changeset
   496
        import cProfile
26b2468a1e73 [etwist] cleanup, use cProfile instead of hotshot
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4914
diff changeset
   497
        cProfile.runctx('reactor.run()', globals(), locals(), config['profile'])
2654
6512522860aa [twisted] don't use twistd anymore, all-in-one.py file is needed anymore
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2476
diff changeset
   498
    else:
6512522860aa [twisted] don't use twistd anymore, all-in-one.py file is needed anymore
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2476
diff changeset
   499
        reactor.run()