hooks/logstats.py
author Denis Laxalde <denis.laxalde@logilab.fr>
Mon, 29 Jun 2015 22:34:58 +0200
changeset 10511 84cb3299ec40
parent 10308 3f94034cc972
permissions -rw-r--r--
Add requirements.txt files in test directories These files, located in each test directory (when needed), list test dependencies supplementary to install requirements. Some requirements.txt files include psycopg2 because it won't get installed by logilab-database apparently. Related to #5447161.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
10308
3f94034cc972 [hooks] add a looping task that dumps the stats regularly in a file
David Douard <david.douard@logilab.fr>
parents:
diff changeset
     1
# copyright 2014 LOGILAB S.A. (Paris, FRANCE), all rights reserved.
3f94034cc972 [hooks] add a looping task that dumps the stats regularly in a file
David Douard <david.douard@logilab.fr>
parents:
diff changeset
     2
# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr
3f94034cc972 [hooks] add a looping task that dumps the stats regularly in a file
David Douard <david.douard@logilab.fr>
parents:
diff changeset
     3
#
3f94034cc972 [hooks] add a looping task that dumps the stats regularly in a file
David Douard <david.douard@logilab.fr>
parents:
diff changeset
     4
# This file is part of CubicWeb.
3f94034cc972 [hooks] add a looping task that dumps the stats regularly in a file
David Douard <david.douard@logilab.fr>
parents:
diff changeset
     5
#
3f94034cc972 [hooks] add a looping task that dumps the stats regularly in a file
David Douard <david.douard@logilab.fr>
parents:
diff changeset
     6
# CubicWeb is free software: you can redistribute it and/or modify it under the
3f94034cc972 [hooks] add a looping task that dumps the stats regularly in a file
David Douard <david.douard@logilab.fr>
parents:
diff changeset
     7
# terms of the GNU Lesser General Public License as published by the Free
3f94034cc972 [hooks] add a looping task that dumps the stats regularly in a file
David Douard <david.douard@logilab.fr>
parents:
diff changeset
     8
# Software Foundation, either version 2.1 of the License, or (at your option)
3f94034cc972 [hooks] add a looping task that dumps the stats regularly in a file
David Douard <david.douard@logilab.fr>
parents:
diff changeset
     9
# any later version.
3f94034cc972 [hooks] add a looping task that dumps the stats regularly in a file
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    10
#
3f94034cc972 [hooks] add a looping task that dumps the stats regularly in a file
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    11
# CubicWeb is distributed in the hope that it will be useful, but WITHOUT
3f94034cc972 [hooks] add a looping task that dumps the stats regularly in a file
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    12
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
3f94034cc972 [hooks] add a looping task that dumps the stats regularly in a file
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    13
# FOR A PARTICULAR PURPOSE.  See the GNU Lesser General Public License for more
3f94034cc972 [hooks] add a looping task that dumps the stats regularly in a file
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    14
# details.
3f94034cc972 [hooks] add a looping task that dumps the stats regularly in a file
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    15
#
3f94034cc972 [hooks] add a looping task that dumps the stats regularly in a file
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    16
# You should have received a copy of the GNU Lesser General Public License along
3f94034cc972 [hooks] add a looping task that dumps the stats regularly in a file
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    17
# with CubicWeb.  If not, see <http://www.gnu.org/licenses/>.
3f94034cc972 [hooks] add a looping task that dumps the stats regularly in a file
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    18
3f94034cc972 [hooks] add a looping task that dumps the stats regularly in a file
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    19
"""looping task for dumping instance's stats in a file
3f94034cc972 [hooks] add a looping task that dumps the stats regularly in a file
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    20
"""
3f94034cc972 [hooks] add a looping task that dumps the stats regularly in a file
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    21
3f94034cc972 [hooks] add a looping task that dumps the stats regularly in a file
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    22
__docformat__ = "restructuredtext en"
3f94034cc972 [hooks] add a looping task that dumps the stats regularly in a file
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    23
3f94034cc972 [hooks] add a looping task that dumps the stats regularly in a file
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    24
from datetime import datetime
3f94034cc972 [hooks] add a looping task that dumps the stats regularly in a file
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    25
import json
3f94034cc972 [hooks] add a looping task that dumps the stats regularly in a file
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    26
3f94034cc972 [hooks] add a looping task that dumps the stats regularly in a file
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    27
from cubicweb.server import hook
3f94034cc972 [hooks] add a looping task that dumps the stats regularly in a file
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    28
3f94034cc972 [hooks] add a looping task that dumps the stats regularly in a file
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    29
class LogStatsStartHook(hook.Hook):
3f94034cc972 [hooks] add a looping task that dumps the stats regularly in a file
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    30
    """register task to regularly dump instance's stats in a file
3f94034cc972 [hooks] add a looping task that dumps the stats regularly in a file
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    31
3f94034cc972 [hooks] add a looping task that dumps the stats regularly in a file
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    32
    data are stored as one json entry per row
3f94034cc972 [hooks] add a looping task that dumps the stats regularly in a file
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    33
    """
3f94034cc972 [hooks] add a looping task that dumps the stats regularly in a file
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    34
    __regid__ = 'cubicweb.hook.logstats.start'
3f94034cc972 [hooks] add a looping task that dumps the stats regularly in a file
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    35
    events = ('server_startup',)
3f94034cc972 [hooks] add a looping task that dumps the stats regularly in a file
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    36
3f94034cc972 [hooks] add a looping task that dumps the stats regularly in a file
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    37
    def __call__(self):
3f94034cc972 [hooks] add a looping task that dumps the stats regularly in a file
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    38
        interval = self.repo.config.get('logstat-interval', 0)
3f94034cc972 [hooks] add a looping task that dumps the stats regularly in a file
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    39
        if interval <= 0:
3f94034cc972 [hooks] add a looping task that dumps the stats regularly in a file
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    40
            return            
3f94034cc972 [hooks] add a looping task that dumps the stats regularly in a file
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    41
3f94034cc972 [hooks] add a looping task that dumps the stats regularly in a file
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    42
        def dump_stats(repo):
3f94034cc972 [hooks] add a looping task that dumps the stats regularly in a file
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    43
            statsfile = repo.config.get('logstat-file')
3f94034cc972 [hooks] add a looping task that dumps the stats regularly in a file
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    44
            with repo.internal_cnx() as cnx:
3f94034cc972 [hooks] add a looping task that dumps the stats regularly in a file
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    45
                stats = cnx.call_service('repo_stats')
3f94034cc972 [hooks] add a looping task that dumps the stats regularly in a file
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    46
                gcstats = cnx.call_service('repo_gc_stats', nmax=5)
3f94034cc972 [hooks] add a looping task that dumps the stats regularly in a file
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    47
                
3f94034cc972 [hooks] add a looping task that dumps the stats regularly in a file
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    48
            allstats = {'resources': stats,
3f94034cc972 [hooks] add a looping task that dumps the stats regularly in a file
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    49
                        'memory': gcstats,
3f94034cc972 [hooks] add a looping task that dumps the stats regularly in a file
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    50
                        'timestamp': datetime.utcnow().isoformat(),
3f94034cc972 [hooks] add a looping task that dumps the stats regularly in a file
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    51
                       }
3f94034cc972 [hooks] add a looping task that dumps the stats regularly in a file
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    52
            try:
3f94034cc972 [hooks] add a looping task that dumps the stats regularly in a file
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    53
                with open(statsfile, 'ab') as ofile:
3f94034cc972 [hooks] add a looping task that dumps the stats regularly in a file
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    54
                    json.dump(allstats, ofile)
3f94034cc972 [hooks] add a looping task that dumps the stats regularly in a file
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    55
                    ofile.write('\n')
3f94034cc972 [hooks] add a looping task that dumps the stats regularly in a file
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    56
            except IOError:
3f94034cc972 [hooks] add a looping task that dumps the stats regularly in a file
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    57
                repo.warning('Cannot open stats file for writing: %s', statsfile)
3f94034cc972 [hooks] add a looping task that dumps the stats regularly in a file
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    58
                    
3f94034cc972 [hooks] add a looping task that dumps the stats regularly in a file
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    59
        self.repo.looping_task(interval, dump_stats, self.repo)