etwist/server.py
author Sylvain Thénault <sylvain.thenault@logilab.fr>
Fri, 12 Mar 2010 15:00:54 +0100
changeset 4895 7003912cec86
parent 4772 ae1b2a0c8e86
child 4914 dcb055f32d9b
permissions -rw-r--r--
[dbapi] return default argument instead of None when no connection set yet
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
2476
1294a6bdf3bf application -> instance where it makes sense
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2181
diff changeset
     1
"""twisted server for CubicWeb web instances
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
     2
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
     3
:organization: Logilab
4212
ab6573088b4a update copyright: welcome 2010
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4194
diff changeset
     4
:copyright: 2001-2010 LOGILAB S.A. (Paris, FRANCE), license is LGPL v2.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
     5
:contact: http://www.logilab.fr/ -- mailto:contact@logilab.fr
1977
606923dff11b big bunch of copyright / docstring update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1936
diff changeset
     6
:license: GNU Lesser General Public License, v2.1 - http://www.gnu.org/licenses
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
     7
"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
     8
__docformat__ = "restructuredtext en"
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
     9
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    10
import sys
2654
6512522860aa [twisted] don't use twistd anymore, all-in-one.py file is needed anymore
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2476
diff changeset
    11
import os
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    12
import select
4398
4e7e74918631 add missing errno import in etwist/server.py
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4221
diff changeset
    13
import errno
3493
3e4603678d4e use default repo looping task so they are started in thread, else we get blocking task
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3198
diff changeset
    14
import hotshot
1016
26387b836099 use datetime instead of mx.DateTime
sylvain.thenault@logilab.fr
parents: 151
diff changeset
    15
from time import mktime
26387b836099 use datetime instead of mx.DateTime
sylvain.thenault@logilab.fr
parents: 151
diff changeset
    16
from datetime import date, timedelta
1520
b097057e629d provide an option to substitute the base-url (left-most part) subdomain by the one of the current http query to easy multiple subdomains website management
Florent <florent@secondweb.fr>
parents: 1420
diff changeset
    17
from urlparse import urlsplit, urlunsplit
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    18
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    19
from twisted.internet import reactor, task, threads
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    20
from twisted.internet.defer import maybeDeferred
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    21
from twisted.web2 import channel, http, server, iweb
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    22
from twisted.web2 import static, resource, responsecode
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    23
4221
da84ca26896d raising ConfigurationError is cleaner; remove some unused imports
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 4219
diff changeset
    24
from cubicweb import ConfigurationError, CW_EVENT_MANAGER
1420
25c13e5b12bd stop complaining about empty response, remove trailing spaces
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
    25
from cubicweb.web import (AuthenticationError, NotFound, Redirect,
2685
0518ca8f63e3 [autoreload] recompute urlresolver / urlrewriter after autoreload
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2670
diff changeset
    26
                          RemoteCallFailed, DirectResponse, StatusResponse,
0518ca8f63e3 [autoreload] recompute urlresolver / urlrewriter after autoreload
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2670
diff changeset
    27
                          ExplicitLogin)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    28
from cubicweb.web.application import CubicWebPublisher
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    29
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    30
from cubicweb.etwist.request import CubicWebTwistedRequestAdapter
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    31
3193
c1a8456cb5f9 should change process'user once pid file has been written
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3181
diff changeset
    32
def daemonize():
3181
735c5f9fcded fix daemonize: set umask, move to /, consider uid
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2711
diff changeset
    33
    # XXX unix specific
735c5f9fcded fix daemonize: set umask, move to /, consider uid
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2711
diff changeset
    34
    # XXX factorize w/ code in cw.server.server and cw.server.serverctl
735c5f9fcded fix daemonize: set umask, move to /, consider uid
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2711
diff changeset
    35
    # (start-repository command)
735c5f9fcded fix daemonize: set umask, move to /, consider uid
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2711
diff changeset
    36
    # See http://www.erlenstar.demon.co.uk/unix/faq_toc.html#TOC16
735c5f9fcded fix daemonize: set umask, move to /, consider uid
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2711
diff changeset
    37
    if os.fork():   # launch child and...
3606
8326aceecb46 fix startup as a daemon
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3558
diff changeset
    38
        return 1
3181
735c5f9fcded fix daemonize: set umask, move to /, consider uid
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2711
diff changeset
    39
    os.setsid()
3606
8326aceecb46 fix startup as a daemon
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3558
diff changeset
    40
    if os.fork():   # launch child again.
8326aceecb46 fix startup as a daemon
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3558
diff changeset
    41
        return 1
3181
735c5f9fcded fix daemonize: set umask, move to /, consider uid
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2711
diff changeset
    42
    # move to the root to avoit mount pb
735c5f9fcded fix daemonize: set umask, move to /, consider uid
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2711
diff changeset
    43
    os.chdir('/')
735c5f9fcded fix daemonize: set umask, move to /, consider uid
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2711
diff changeset
    44
    # set paranoid umask
735c5f9fcded fix daemonize: set umask, move to /, consider uid
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2711
diff changeset
    45
    os.umask(077)
735c5f9fcded fix daemonize: set umask, move to /, consider uid
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2711
diff changeset
    46
    null = os.open('/dev/null', os.O_RDWR)
735c5f9fcded fix daemonize: set umask, move to /, consider uid
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2711
diff changeset
    47
    for i in range(3):
735c5f9fcded fix daemonize: set umask, move to /, consider uid
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2711
diff changeset
    48
        try:
735c5f9fcded fix daemonize: set umask, move to /, consider uid
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2711
diff changeset
    49
            os.dup2(null, i)
735c5f9fcded fix daemonize: set umask, move to /, consider uid
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2711
diff changeset
    50
        except OSError, e:
735c5f9fcded fix daemonize: set umask, move to /, consider uid
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2711
diff changeset
    51
            if e.errno != errno.EBADF:
735c5f9fcded fix daemonize: set umask, move to /, consider uid
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2711
diff changeset
    52
                raise
735c5f9fcded fix daemonize: set umask, move to /, consider uid
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2711
diff changeset
    53
    os.close(null)
735c5f9fcded fix daemonize: set umask, move to /, consider uid
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2711
diff changeset
    54
    return None
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    55
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    56
def start_task(interval, func):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    57
    lc = task.LoopingCall(func)
3493
3e4603678d4e use default repo looping task so they are started in thread, else we get blocking task
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3198
diff changeset
    58
    # wait until interval has expired to actually start the task, else we have
3e4603678d4e use default repo looping task so they are started in thread, else we get blocking task
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3198
diff changeset
    59
    # to wait all task to be finished for the server to be actually started
3e4603678d4e use default repo looping task so they are started in thread, else we get blocking task
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3198
diff changeset
    60
    lc.start(interval, now=False)
1420
25c13e5b12bd stop complaining about empty response, remove trailing spaces
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
    61
1543
dca9817bb337 fix use-request-subdomain option behaviour and add tests
Florent <florent@secondweb.fr>
parents: 1542
diff changeset
    62
def host_prefixed_baseurl(baseurl, host):
dca9817bb337 fix use-request-subdomain option behaviour and add tests
Florent <florent@secondweb.fr>
parents: 1542
diff changeset
    63
    scheme, netloc, url, query, fragment = urlsplit(baseurl)
dca9817bb337 fix use-request-subdomain option behaviour and add tests
Florent <florent@secondweb.fr>
parents: 1542
diff changeset
    64
    netloc_domain = '.' + '.'.join(netloc.split('.')[-2:])
dca9817bb337 fix use-request-subdomain option behaviour and add tests
Florent <florent@secondweb.fr>
parents: 1542
diff changeset
    65
    if host.endswith(netloc_domain):
dca9817bb337 fix use-request-subdomain option behaviour and add tests
Florent <florent@secondweb.fr>
parents: 1542
diff changeset
    66
        netloc = host
dca9817bb337 fix use-request-subdomain option behaviour and add tests
Florent <florent@secondweb.fr>
parents: 1542
diff changeset
    67
    baseurl = urlunsplit((scheme, netloc, url, query, fragment))
dca9817bb337 fix use-request-subdomain option behaviour and add tests
Florent <florent@secondweb.fr>
parents: 1542
diff changeset
    68
    return baseurl
dca9817bb337 fix use-request-subdomain option behaviour and add tests
Florent <florent@secondweb.fr>
parents: 1542
diff changeset
    69
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    70
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    71
class LongTimeExpiringFile(static.File):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    72
    """overrides static.File and sets a far futre ``Expires`` date
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    73
    on the resouce.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    74
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    75
    versions handling is done by serving static files by different
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    76
    URLs for each version. For instance::
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    77
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    78
      http://localhost:8080/data-2.48.2/cubicweb.css
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    79
      http://localhost:8080/data-2.49.0/cubicweb.css
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    80
      etc.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    81
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    82
    """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    83
    def renderHTTP(self, request):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    84
        def setExpireHeader(response):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    85
            response = iweb.IResponse(response)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    86
            # Don't provide additional resource information to error responses
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    87
            if response.code < 400:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    88
                # the HTTP RFC recommands not going further than 1 year ahead
1016
26387b836099 use datetime instead of mx.DateTime
sylvain.thenault@logilab.fr
parents: 151
diff changeset
    89
                expires = date.today() + timedelta(days=6*30)
26387b836099 use datetime instead of mx.DateTime
sylvain.thenault@logilab.fr
parents: 151
diff changeset
    90
                response.headers.setHeader('Expires', mktime(expires.timetuple()))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    91
            return response
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    92
        d = maybeDeferred(super(LongTimeExpiringFile, self).renderHTTP, request)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    93
        return d.addCallback(setExpireHeader)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    94
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    95
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    96
class CubicWebRootResource(resource.PostableResource):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    97
    addSlash = False
1420
25c13e5b12bd stop complaining about empty response, remove trailing spaces
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
    98
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    99
    def __init__(self, config, debug=None):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   100
        self.debugmode = debug
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   101
        self.config = config
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   102
        self.base_url = config['base-url'] or config.default_base_url()
4193
c3e7d0f85637 closes #607344 by simply appending '/' to base url if necessary instead of insulting the user
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3828
diff changeset
   103
        if self.base_url[-1] != '/':
c3e7d0f85637 closes #607344 by simply appending '/' to base url if necessary instead of insulting the user
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3828
diff changeset
   104
            self.base_url += '/'
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   105
        self.https_url = config['https-url']
4194
7d5d469f81e1 missing one litle char
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4193
diff changeset
   106
        if self.https_url and self.https_url[-1] != '/':
4193
c3e7d0f85637 closes #607344 by simply appending '/' to base url if necessary instead of insulting the user
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3828
diff changeset
   107
            self.https_url += '/'
3828
335e40fdb5a5 when instance is started normally (eg not in debug mode), check versions before daemonization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3791
diff changeset
   108
        # instantiate publisher here and not in init_publisher to get some
335e40fdb5a5 when instance is started normally (eg not in debug mode), check versions before daemonization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3791
diff changeset
   109
        # checks done before daemonization (eg versions consistency)
335e40fdb5a5 when instance is started normally (eg not in debug mode), check versions before daemonization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3791
diff changeset
   110
        self.appli = CubicWebPublisher(config, debug=self.debugmode)
335e40fdb5a5 when instance is started normally (eg not in debug mode), check versions before daemonization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3791
diff changeset
   111
        self.versioned_datadir = 'data%s' % config.instance_md5_version()
3606
8326aceecb46 fix startup as a daemon
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3558
diff changeset
   112
8326aceecb46 fix startup as a daemon
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3558
diff changeset
   113
    def init_publisher(self):
8326aceecb46 fix startup as a daemon
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3558
diff changeset
   114
        config = self.config
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   115
        # when we have an in-memory repository, clean unused sessions every XX
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   116
        # seconds and properly shutdown the server
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   117
        if config.repo_method == 'inmemory':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   118
            reactor.addSystemEventTrigger('before', 'shutdown',
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   119
                                          self.shutdown_event)
1115
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 151
diff changeset
   120
            # monkey patch start_looping_task to get proper reactor integration
3493
3e4603678d4e use default repo looping task so they are started in thread, else we get blocking task
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3198
diff changeset
   121
            #self.appli.repo.__class__.start_looping_tasks = start_looping_tasks
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   122
            if config.pyro_enabled():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   123
                # if pyro is enabled, we have to register to the pyro name
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   124
                # server, create a pyro daemon, and create a task to handle pyro
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   125
                # requests
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   126
                self.pyro_daemon = self.appli.repo.pyro_register()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   127
                self.pyro_listen_timeout = 0.02
3502
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3493
diff changeset
   128
                self.appli.repo.looping_task(1, self.pyro_loop_event)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   129
            self.appli.repo.start_looping_tasks()
2685
0518ca8f63e3 [autoreload] recompute urlresolver / urlrewriter after autoreload
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2670
diff changeset
   130
        self.set_url_rewriter()
2705
30bcdbd92820 [events] renamed source-reload into registry-reload to avoid potential confusions with datasources
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2694
diff changeset
   131
        CW_EVENT_MANAGER.bind('after-registry-reload', self.set_url_rewriter)
3606
8326aceecb46 fix startup as a daemon
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3558
diff changeset
   132
8326aceecb46 fix startup as a daemon
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3558
diff changeset
   133
    def start_service(self):
8326aceecb46 fix startup as a daemon
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3558
diff changeset
   134
        config = self.config
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   135
        interval = min(config['cleanup-session-time'] or 120,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   136
                       config['cleanup-anonymous-session-time'] or 720) / 2.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   137
        start_task(interval, self.appli.session_handler.clean_sessions)
1420
25c13e5b12bd stop complaining about empty response, remove trailing spaces
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
   138
2685
0518ca8f63e3 [autoreload] recompute urlresolver / urlrewriter after autoreload
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2670
diff changeset
   139
    def set_url_rewriter(self):
2770
356e9d7c356d R propagate registry API changes
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2711
diff changeset
   140
        self.url_rewriter = self.appli.vreg['components'].select_or_none('urlrewriter')
2685
0518ca8f63e3 [autoreload] recompute urlresolver / urlrewriter after autoreload
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2670
diff changeset
   141
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   142
    def shutdown_event(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   143
        """callback fired when the server is shutting down to properly
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   144
        clean opened sessions
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   145
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   146
        self.appli.repo.shutdown()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   147
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   148
    def pyro_loop_event(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   149
        """listen for pyro events"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   150
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   151
            self.pyro_daemon.handleRequests(self.pyro_listen_timeout)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   152
        except select.error:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   153
            return
1420
25c13e5b12bd stop complaining about empty response, remove trailing spaces
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
   154
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   155
    def locateChild(self, request, segments):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   156
        """Indicate which resource to use to process down the URL's path"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   157
        if segments:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   158
            if segments[0] == 'https':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   159
                segments = segments[1:]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   160
            if len(segments) >= 2:
151
343e7a18675d static files support
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents: 0
diff changeset
   161
                if segments[0] in (self.versioned_datadir, 'data', 'static'):
343e7a18675d static files support
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents: 0
diff changeset
   162
                    # Anything in data/, static/ is treated as static files
343e7a18675d static files support
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents: 0
diff changeset
   163
                    if segments[0] == 'static':
343e7a18675d static files support
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents: 0
diff changeset
   164
                        # instance static directory
343e7a18675d static files support
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents: 0
diff changeset
   165
                        datadir = self.config.static_directory
2169
124fb0b9fa4b fckeditor may be a subdirectory of the data directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
   166
                    elif segments[1] == 'fckeditor':
124fb0b9fa4b fckeditor may be a subdirectory of the data directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
   167
                        fckeditordir = self.config.ext_resources['FCKEDITOR_PATH']
124fb0b9fa4b fckeditor may be a subdirectory of the data directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
   168
                        return static.File(fckeditordir), segments[2:]
151
343e7a18675d static files support
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents: 0
diff changeset
   169
                    else:
343e7a18675d static files support
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents: 0
diff changeset
   170
                        # cube static data file
343e7a18675d static files support
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents: 0
diff changeset
   171
                        datadir = self.config.locate_resource(segments[1])
343e7a18675d static files support
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents: 0
diff changeset
   172
                        if datadir is None:
343e7a18675d static files support
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents: 0
diff changeset
   173
                            return None, []
3618
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3617
diff changeset
   174
                    self.info('static file %s from %s', segments[-1], datadir)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   175
                    if segments[0] == 'data':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   176
                        return static.File(str(datadir)), segments[1:]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   177
                    else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   178
                        return LongTimeExpiringFile(datadir), segments[1:]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   179
                elif segments[0] == 'fckeditor':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   180
                    fckeditordir = self.config.ext_resources['FCKEDITOR_PATH']
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   181
                    return static.File(fckeditordir), segments[1:]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   182
        # Otherwise we use this single resource
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   183
        return self, ()
1420
25c13e5b12bd stop complaining about empty response, remove trailing spaces
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
   184
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   185
    def render(self, request):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   186
        """Render a page from the root resource"""
3638
648d6dbec630 system/user modes + CWDEV instead of installed/dev mixed modes. Fix behaviour when setting CW_MODE explicitly
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3618
diff changeset
   187
        # reload modified files in debug mode
3641
cf30e4498674 fix debug attribute conflict on configuration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3638
diff changeset
   188
        if self.debugmode:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   189
            self.appli.vreg.register_objects(self.config.vregistry_path())
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   190
        if self.config['profile']: # default profiler don't trace threads
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   191
            return self.render_request(request)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   192
        else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   193
            return threads.deferToThread(self.render_request, request)
1420
25c13e5b12bd stop complaining about empty response, remove trailing spaces
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
   194
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   195
    def render_request(self, request):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   196
        origpath = request.path
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   197
        host = request.host
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   198
        # dual http/https access handling: expect a rewrite rule to prepend
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   199
        # 'https' to the path to detect https access
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   200
        if origpath.split('/', 2)[1] == 'https':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   201
            origpath = origpath[6:]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   202
            request.uri = request.uri[6:]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   203
            https = True
1420
25c13e5b12bd stop complaining about empty response, remove trailing spaces
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
   204
            baseurl = self.https_url or self.base_url
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   205
        else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   206
            https = False
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   207
            baseurl = self.base_url
1520
b097057e629d provide an option to substitute the base-url (left-most part) subdomain by the one of the current http query to easy multiple subdomains website management
Florent <florent@secondweb.fr>
parents: 1420
diff changeset
   208
        if self.config['use-request-subdomain']:
1543
dca9817bb337 fix use-request-subdomain option behaviour and add tests
Florent <florent@secondweb.fr>
parents: 1542
diff changeset
   209
            baseurl = host_prefixed_baseurl(baseurl, host)
dca9817bb337 fix use-request-subdomain option behaviour and add tests
Florent <florent@secondweb.fr>
parents: 1542
diff changeset
   210
            self.warning('used baseurl is %s for this request', baseurl)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   211
        req = CubicWebTwistedRequestAdapter(request, self.appli.vreg, https, baseurl)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   212
        if req.authmode == 'http':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   213
            # activate realm-based auth
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   214
            realm = self.config['realm']
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   215
            req.set_header('WWW-Authenticate', [('Basic', {'realm' : realm })], raw=False)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   216
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   217
            self.appli.connect(req)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   218
        except AuthenticationError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   219
            return self.request_auth(req)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   220
        except Redirect, ex:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   221
            return self.redirect(req, ex.location)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   222
        if https and req.cnx.anonymous_connection:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   223
            # don't allow anonymous on https connection
1420
25c13e5b12bd stop complaining about empty response, remove trailing spaces
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
   224
            return self.request_auth(req)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   225
        if self.url_rewriter is not None:
1115
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 151
diff changeset
   226
            # XXX should occur before authentication?
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   227
            try:
1936
c5af2fbda5b6 pass request to ApacheRewriter rewrite method
Florent <florent@secondweb.fr>
parents: 1543
diff changeset
   228
                path = self.url_rewriter.rewrite(host, origpath, req)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   229
            except Redirect, ex:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   230
                return self.redirect(req, ex.location)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   231
            request.uri.replace(origpath, path, 1)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   232
        else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   233
            path = origpath
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   234
        if not path or path == "/":
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   235
            path = 'view'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   236
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   237
            result = self.appli.publish(path, req)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   238
        except DirectResponse, ex:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   239
            return ex.response
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   240
        except StatusResponse, ex:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   241
            return http.Response(stream=ex.content, code=ex.status,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   242
                                 headers=req.headers_out or None)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   243
        except RemoteCallFailed, ex:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   244
            req.set_header('content-type', 'application/json')
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   245
            return http.Response(stream=ex.dumps(),
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   246
                                 code=responsecode.INTERNAL_SERVER_ERROR)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   247
        except NotFound:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   248
            result = self.appli.notfound_content(req)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   249
            return http.Response(stream=result, code=responsecode.NOT_FOUND,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   250
                                 headers=req.headers_out or None)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   251
        except ExplicitLogin:  # must be before AuthenticationError
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   252
            return self.request_auth(req)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   253
        except AuthenticationError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   254
            if self.config['auth-mode'] == 'cookie':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   255
                # in cookie mode redirecting to the index view is enough :
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   256
                # either anonymous connection is allowed and the page will
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   257
                # be displayed or we'll be redirected to the login form
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   258
                msg = req._('you have been logged out')
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   259
                if req.https:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   260
                    req._base_url =  self.base_url
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   261
                    req.https = False
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   262
                url = req.build_url('view', vid='index', __message=msg)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   263
                return self.redirect(req, url)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   264
            else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   265
                # in http we have to request auth to flush current http auth
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   266
                # information
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   267
                return self.request_auth(req, loggedout=True)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   268
        except Redirect, ex:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   269
            return self.redirect(req, ex.location)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   270
        # request may be referenced by "onetime callback", so clear its entity
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   271
        # cache to avoid memory usage
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   272
        req.drop_entity_cache()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   273
        return http.Response(stream=result, code=responsecode.OK,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   274
                             headers=req.headers_out or None)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   275
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   276
    def redirect(self, req, location):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   277
        req.headers_out.setHeader('location', str(location))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   278
        self.debug('redirecting to %s', location)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   279
        # 303 See other
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   280
        return http.Response(code=303, headers=req.headers_out)
1420
25c13e5b12bd stop complaining about empty response, remove trailing spaces
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
   281
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   282
    def request_auth(self, req, loggedout=False):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   283
        if self.https_url and req.base_url() != self.https_url:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   284
            req.headers_out.setHeader('location', self.https_url + 'login')
1420
25c13e5b12bd stop complaining about empty response, remove trailing spaces
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
   285
            return http.Response(code=303, headers=req.headers_out)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   286
        if self.config['auth-mode'] == 'http':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   287
            code = responsecode.UNAUTHORIZED
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   288
        else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   289
            code = responsecode.FORBIDDEN
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   290
        if loggedout:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   291
            if req.https:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   292
                req._base_url =  self.base_url
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   293
                req.https = False
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   294
            content = self.appli.loggedout_content(req)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   295
        else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   296
            content = self.appli.need_login_content(req)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   297
        return http.Response(code, req.headers_out, content)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   298
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   299
from twisted.internet import defer
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   300
from twisted.web2 import fileupload
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   301
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   302
# XXX set max file size to 100Mo: put max upload size in the configuration
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   303
# line below for twisted >= 8.0, default param value for earlier version
1420
25c13e5b12bd stop complaining about empty response, remove trailing spaces
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
   304
resource.PostableResource.maxSize = 100*1024*1024
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   305
def parsePOSTData(request, maxMem=100*1024, maxFields=1024,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   306
                  maxSize=100*1024*1024):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   307
    if request.stream.length == 0:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   308
        return defer.succeed(None)
1420
25c13e5b12bd stop complaining about empty response, remove trailing spaces
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
   309
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   310
    ctype = request.headers.getHeader('content-type')
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   311
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   312
    if ctype is None:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   313
        return defer.succeed(None)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   314
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   315
    def updateArgs(data):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   316
        args = data
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   317
        request.args.update(args)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   318
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   319
    def updateArgsAndFiles(data):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   320
        args, files = data
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   321
        request.args.update(args)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   322
        request.files.update(files)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   323
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   324
    def error(f):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   325
        f.trap(fileupload.MimeFormatError)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   326
        raise http.HTTPError(responsecode.BAD_REQUEST)
1420
25c13e5b12bd stop complaining about empty response, remove trailing spaces
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
   327
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   328
    if ctype.mediaType == 'application' and ctype.mediaSubtype == 'x-www-form-urlencoded':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   329
        d = fileupload.parse_urlencoded(request.stream, keep_blank_values=True)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   330
        d.addCallbacks(updateArgs, error)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   331
        return d
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   332
    elif ctype.mediaType == 'multipart' and ctype.mediaSubtype == 'form-data':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   333
        boundary = ctype.params.get('boundary')
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   334
        if boundary is None:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   335
            return defer.fail(http.HTTPError(
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   336
                http.StatusResponse(responsecode.BAD_REQUEST,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   337
                                    "Boundary not specified in Content-Type.")))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   338
        d = fileupload.parseMultipartFormData(request.stream, boundary,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   339
                                              maxMem, maxFields, maxSize)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   340
        d.addCallbacks(updateArgsAndFiles, error)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   341
        return d
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   342
    else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   343
        raise http.HTTPError(responsecode.BAD_REQUEST)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   344
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   345
server.parsePOSTData = parsePOSTData
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   346
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   347
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   348
from logging import getLogger
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   349
from cubicweb import set_log_methods
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   350
set_log_methods(CubicWebRootResource, getLogger('cubicweb.twisted'))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   351
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   352
4772
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   353
listiterator = type(iter([]))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   354
4772
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   355
def _gc_debug(all=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   356
    import gc
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   357
    from pprint import pprint
2657
de974465d381 [appobject] kill VObject class, move base selector classes to appobject
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2654
diff changeset
   358
    from cubicweb.appobject import AppObject
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   359
    gc.collect()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   360
    count = 0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   361
    acount = 0
4772
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   362
    fcount = 0
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   363
    rcount = 0
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   364
    ccount = 0
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   365
    scount = 0
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   366
    ocount = {}
4772
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   367
    from rql.stmts import Union
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   368
    from cubicweb.schema import CubicWebSchema
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   369
    from cubicweb.rset import ResultSet
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   370
    from cubicweb.dbapi import Connection, Cursor
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   371
    from cubicweb.req import RequestSessionBase
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   372
    from cubicweb.server.repository import Repository
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   373
    from cubicweb.server.sources.native import NativeSQLSource
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   374
    from cubicweb.server.session import Session
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   375
    from cubicweb.devtools.testlib import CubicWebTC
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   376
    from logilab.common.testlib import TestSuite
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   377
    from optparse import Values
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   378
    import types, weakref
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   379
    for obj in gc.get_objects():
4772
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   380
        if isinstance(obj, RequestSessionBase):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   381
            count += 1
4772
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   382
            if isinstance(obj, Session):
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   383
                print '   session', obj, referrers(obj, True)
2657
de974465d381 [appobject] kill VObject class, move base selector classes to appobject
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2654
diff changeset
   384
        elif isinstance(obj, AppObject):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   385
            acount += 1
4772
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   386
        elif isinstance(obj, ResultSet):
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   387
            rcount += 1
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   388
            #print '   rset', obj, referrers(obj)
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   389
        elif isinstance(obj, Repository):
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   390
            print '   REPO', obj, referrers(obj, True)
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   391
        #elif isinstance(obj, NativeSQLSource):
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   392
        #    print '   SOURCe', obj, referrers(obj)
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   393
        elif isinstance(obj, CubicWebTC):
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   394
            print '   TC', obj, referrers(obj)
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   395
        elif isinstance(obj, TestSuite):
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   396
            print '   SUITE', obj, referrers(obj)
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   397
        #elif isinstance(obj, Values):
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   398
        #    print '   values', '%#x' % id(obj), referrers(obj, True)
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   399
        elif isinstance(obj, Connection):
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   400
            ccount += 1
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   401
            #print '   cnx', obj, referrers(obj)
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   402
        #elif isinstance(obj, Cursor):
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   403
        #    ccount += 1
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   404
        #    print '   cursor', obj, referrers(obj)
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   405
        elif isinstance(obj, file):
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   406
            fcount += 1
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   407
        #    print '   open file', file.name, file.fileno
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   408
        elif isinstance(obj, CubicWebSchema):
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   409
            scount += 1
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   410
            print '   schema', obj, referrers(obj)
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   411
        elif not isinstance(obj, (type, tuple, dict, list, set, frozenset,
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   412
                                  weakref.ref, weakref.WeakKeyDictionary,
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   413
                                  listiterator,
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   414
                                  property, classmethod,
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   415
                                  types.ModuleType, types.MemberDescriptorType,
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   416
                                  types.FunctionType, types.MethodType)):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   417
            try:
1132
96752791c2b6 pylint cleanup
sylvain.thenault@logilab.fr
parents: 1016
diff changeset
   418
                ocount[obj.__class__] += 1
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   419
            except KeyError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   420
                ocount[obj.__class__] = 1
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   421
            except AttributeError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   422
                pass
4772
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   423
    if count:
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   424
        print ' NB REQUESTS/SESSIONS', count
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   425
    if acount:
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   426
        print ' NB APPOBJECTS', acount
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   427
    if ccount:
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   428
        print ' NB CONNECTIONS', ccount
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   429
    if rcount:
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   430
        print ' NB RSETS', rcount
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   431
    if scount:
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   432
        print ' NB SCHEMAS', scount
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   433
    if fcount:
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   434
        print ' NB FILES', fcount
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   435
    if all:
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   436
        ocount = sorted(ocount.items(), key=lambda x: x[1], reverse=True)[:20]
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   437
        pprint(ocount)
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   438
    if gc.garbage:
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   439
        print 'UNREACHABLE', gc.garbage
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   440
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   441
def referrers(obj, showobj=False):
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   442
    try:
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   443
        return sorted(set((type(x), showobj and x or getattr(x, '__name__', '%#x' % id(x)))
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   444
                          for x in _referrers(obj)))
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   445
    except TypeError:
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   446
        s = set()
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   447
        unhashable = []
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   448
        for x in _referrers(obj):
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   449
            try:
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   450
                s.add(x)
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   451
            except TypeError:
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   452
                unhashable.append(x)
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   453
        return sorted(s) + unhashable
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   454
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   455
def _referrers(obj, seen=None, level=0):
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   456
    import gc, types
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   457
    from cubicweb.schema import CubicWebRelationSchema, CubicWebEntitySchema
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   458
    interesting = []
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   459
    if seen is None:
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   460
        seen = set()
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   461
    for x in gc.get_referrers(obj):
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   462
        if id(x) in seen:
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   463
            continue
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   464
        seen.add(id(x))
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   465
        if isinstance(x, types.FrameType):
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   466
            continue
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   467
        if isinstance(x, (CubicWebRelationSchema, CubicWebEntitySchema)):
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   468
            continue
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   469
        if isinstance(x, (list, tuple, set, dict, listiterator)):
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   470
            if level >= 5:
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   471
                pass
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   472
                #interesting.append(x)
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   473
            else:
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   474
                interesting += _referrers(x, seen, level+1)
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   475
        else:
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   476
            interesting.append(x)
ae1b2a0c8e86 [debug] more (dirty) gc debuging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4436
diff changeset
   477
    return interesting
2654
6512522860aa [twisted] don't use twistd anymore, all-in-one.py file is needed anymore
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2476
diff changeset
   478
6512522860aa [twisted] don't use twistd anymore, all-in-one.py file is needed anymore
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2476
diff changeset
   479
def run(config, debug):
6512522860aa [twisted] don't use twistd anymore, all-in-one.py file is needed anymore
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2476
diff changeset
   480
    # create the site
6512522860aa [twisted] don't use twistd anymore, all-in-one.py file is needed anymore
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2476
diff changeset
   481
    root_resource = CubicWebRootResource(config, debug)
6512522860aa [twisted] don't use twistd anymore, all-in-one.py file is needed anymore
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2476
diff changeset
   482
    website = server.Site(root_resource)
6512522860aa [twisted] don't use twistd anymore, all-in-one.py file is needed anymore
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2476
diff changeset
   483
    # serve it via standard HTTP on port set in the configuration
6512522860aa [twisted] don't use twistd anymore, all-in-one.py file is needed anymore
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2476
diff changeset
   484
    port = config['port'] or 8080
6512522860aa [twisted] don't use twistd anymore, all-in-one.py file is needed anymore
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2476
diff changeset
   485
    reactor.listenTCP(port, channel.HTTPFactory(website))
2694
4303ff921a9b [twisted] use cubicweb.twisted.logger instead of print to notify when the instance is started
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2685
diff changeset
   486
    logger = getLogger('cubicweb.twisted')
2654
6512522860aa [twisted] don't use twistd anymore, all-in-one.py file is needed anymore
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2476
diff changeset
   487
    if not debug:
4219
87203a0a3a81 do not attempt to daemonize under windows
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 4212
diff changeset
   488
        if sys.platform == 'win32':
4221
da84ca26896d raising ConfigurationError is cleaner; remove some unused imports
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 4219
diff changeset
   489
            raise ConfigurationError("Under windows, you must use the service management "
da84ca26896d raising ConfigurationError is cleaner; remove some unused imports
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 4219
diff changeset
   490
                                     "commands (e.g : 'net start my_instance)'")
3558
0e478957a9d4 smallest possible change to fix #472806
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3502
diff changeset
   491
        print 'instance starting in the background'
3606
8326aceecb46 fix startup as a daemon
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3558
diff changeset
   492
        if daemonize():
8326aceecb46 fix startup as a daemon
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3558
diff changeset
   493
            return # child process
2654
6512522860aa [twisted] don't use twistd anymore, all-in-one.py file is needed anymore
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2476
diff changeset
   494
        if config['pid-file']:
2711
7aee3bd7a704 ensure runtime dir exists
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2708
diff changeset
   495
            # ensure the directory where the pid-file should be set exists (for
3791
62d6ab877694 put back url info started at the end of initialisations
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3641
diff changeset
   496
            # instance /var/run/cubicweb may be deleted on computer restart)
2711
7aee3bd7a704 ensure runtime dir exists
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2708
diff changeset
   497
            piddir = os.path.dirname(config['pid-file'])
7aee3bd7a704 ensure runtime dir exists
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2708
diff changeset
   498
            if not os.path.exists(piddir):
7aee3bd7a704 ensure runtime dir exists
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2708
diff changeset
   499
                os.makedirs(piddir)
2654
6512522860aa [twisted] don't use twistd anymore, all-in-one.py file is needed anymore
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2476
diff changeset
   500
            file(config['pid-file'], 'w').write(str(os.getpid()))
3606
8326aceecb46 fix startup as a daemon
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3558
diff changeset
   501
    root_resource.init_publisher() # before changing uid
3193
c1a8456cb5f9 should change process'user once pid file has been written
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3181
diff changeset
   502
    if config['uid'] is not None:
c1a8456cb5f9 should change process'user once pid file has been written
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3181
diff changeset
   503
        try:
c1a8456cb5f9 should change process'user once pid file has been written
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3181
diff changeset
   504
            uid = int(config['uid'])
c1a8456cb5f9 should change process'user once pid file has been written
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3181
diff changeset
   505
        except ValueError:
c1a8456cb5f9 should change process'user once pid file has been written
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3181
diff changeset
   506
            from pwd import getpwnam
c1a8456cb5f9 should change process'user once pid file has been written
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3181
diff changeset
   507
            uid = getpwnam(config['uid']).pw_uid
c1a8456cb5f9 should change process'user once pid file has been written
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3181
diff changeset
   508
        os.setuid(uid)
3606
8326aceecb46 fix startup as a daemon
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3558
diff changeset
   509
    root_resource.start_service()
3791
62d6ab877694 put back url info started at the end of initialisations
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3641
diff changeset
   510
    logger.info('instance started on %s', root_resource.base_url)
2654
6512522860aa [twisted] don't use twistd anymore, all-in-one.py file is needed anymore
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2476
diff changeset
   511
    if config['profile']:
6512522860aa [twisted] don't use twistd anymore, all-in-one.py file is needed anymore
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2476
diff changeset
   512
        prof = hotshot.Profile(config['profile'])
6512522860aa [twisted] don't use twistd anymore, all-in-one.py file is needed anymore
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2476
diff changeset
   513
        prof.runcall(reactor.run)
6512522860aa [twisted] don't use twistd anymore, all-in-one.py file is needed anymore
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2476
diff changeset
   514
    else:
6512522860aa [twisted] don't use twistd anymore, all-in-one.py file is needed anymore
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2476
diff changeset
   515
        reactor.run()