statsd_logger.py
author Samuel Trégouët <samuel.tregouet@logilab.fr>
Tue, 01 Dec 2015 13:39:17 +0100
changeset 10942 b089858c49a2
parent 10650 28b3d39bcbc6
permissions -rw-r--r--
[dataimport] Fix method signature Since d4bd28d5fca8, _storage_handler expect 4 args.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
10477
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
     1
# copyright 2015 LOGILAB S.A. (Paris, FRANCE), all rights reserved.
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
     2
# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
     3
#
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
     4
# This file is part of CubicWeb.
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
     5
#
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
     6
# CubicWeb is free software: you can redistribute it and/or modify it under the
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
     7
# terms of the GNU Lesser General Public License as published by the Free
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
     8
# Software Foundation, either version 2.1 of the License, or (at your option)
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
     9
# any later version.
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    10
#
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    11
# CubicWeb is distributed in the hope that it will be useful, but WITHOUT
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    12
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    13
# FOR A PARTICULAR PURPOSE.  See the GNU Lesser General Public License for more
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    14
# details.
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    15
#
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    16
# You should have received a copy of the GNU Lesser General Public License along
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    17
# with CubicWeb.  If not, see <http://www.gnu.org/licenses/>.
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    18
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    19
"""Simple statsd_ logger for cubicweb.
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    20
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    21
This module is meant to be configured by setting a couple of global variables:
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    22
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    23
- ``bucket`` global variable will be used as statsd bucket in every
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    24
statsd_ UDP sent packet.
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    25
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    26
`- `address`` is a pair (IP, port) specifying the address of the
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    27
statsd_ server
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    28
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    29
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    30
There are 3 kinds of statds_ message::
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    31
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    32
- ``statsd_c(context, n)`` is a simple function to send statsd_
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    33
  counter-type of messages like::
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    34
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    35
    <bucket>.<context>:<n>|c\n
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    36
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    37
- ``statsd_g(context, value)`` to send statsd_ gauge-type of messages
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    38
  like::
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    39
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    40
    <bucket>.<context>:<n>|g\n
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    41
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    42
- ``statsd_t(context, ms)`` to send statsd_ time-type of messages
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    43
  like::
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    44
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    45
    <bucket>.<context>:<ms>|ms\n
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    46
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    47
There is also a decorator (``statsd_timeit``) that may be used to
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    48
measure and send to the statsd_ server the time passed in a function
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    49
or a method and the number of calls. It will send a message like::
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    50
   
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    51
    <bucket>.<funcname>:<ms>|ms\n<bucket>.<funcname>:1|c\n
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    52
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    53
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    54
.. _statsd: https://github.com/etsy/statsd
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    55
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    56
"""
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    57
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    58
__docformat__ = "restructuredtext en"
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    59
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    60
import time
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    61
import socket
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    62
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    63
_bucket = 'cubicweb'
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    64
_address = None
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    65
_socket = None
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    66
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    67
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    68
def setup(bucket, address):
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    69
    """Configure the statsd endpoint
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    70
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    71
    :param bucket: the name of the statsd bucket that will be used to
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    72
                   build messages.
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    73
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    74
    :param address: the UDP endpoint of the statsd server. Must a
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    75
                    couple (ip, port).
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    76
    """
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    77
    global _bucket, _address, _socket
10478
026b4ee032fb [statsd_logger] handle ipv6 addresses
Julien Cristau <julien.cristau@logilab.fr>
parents: 10477
diff changeset
    78
    packed = None
026b4ee032fb [statsd_logger] handle ipv6 addresses
Julien Cristau <julien.cristau@logilab.fr>
parents: 10477
diff changeset
    79
    for family in (socket.AF_INET6, socket.AF_INET):
026b4ee032fb [statsd_logger] handle ipv6 addresses
Julien Cristau <julien.cristau@logilab.fr>
parents: 10477
diff changeset
    80
        try:
10650
28b3d39bcbc6 [statsd] fix the statsd logger (closes #7558147)
David Douard <david.douard@logilab.fr>
parents: 10478
diff changeset
    81
            packed = socket.inet_pton(family, address[0])
10478
026b4ee032fb [statsd_logger] handle ipv6 addresses
Julien Cristau <julien.cristau@logilab.fr>
parents: 10477
diff changeset
    82
            break
026b4ee032fb [statsd_logger] handle ipv6 addresses
Julien Cristau <julien.cristau@logilab.fr>
parents: 10477
diff changeset
    83
        except socket.error:
026b4ee032fb [statsd_logger] handle ipv6 addresses
Julien Cristau <julien.cristau@logilab.fr>
parents: 10477
diff changeset
    84
            continue
026b4ee032fb [statsd_logger] handle ipv6 addresses
Julien Cristau <julien.cristau@logilab.fr>
parents: 10477
diff changeset
    85
    if packed is None:
026b4ee032fb [statsd_logger] handle ipv6 addresses
Julien Cristau <julien.cristau@logilab.fr>
parents: 10477
diff changeset
    86
        return
10477
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    87
    _bucket, _address = bucket, address
10478
026b4ee032fb [statsd_logger] handle ipv6 addresses
Julien Cristau <julien.cristau@logilab.fr>
parents: 10477
diff changeset
    88
    _socket = socket.socket(family, socket.SOCK_DGRAM)
10477
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    89
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    90
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    91
def statsd_c(context, n=1):
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    92
    if _address is not None:
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    93
        _socket.sendto('{0}.{1}:{2}|c\n'.format(_bucket, context, n), _address)
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    94
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    95
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    96
def statsd_g(context, value):
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    97
    if _address is not None:
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    98
        _socket.sendto('{0}.{1}:{2}|g\n'.format(_bucket, context, value), _address)
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    99
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   100
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   101
def statsd_t(context, value):
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   102
    if _address is not None:
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   103
        _socket.sendto('{0}.{1}:{2:.4f}|ms\n'.format(_bucket, context, value), _address)
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   104
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   105
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   106
class statsd_timeit(object):
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   107
    __slots__ = ('callable',)
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   108
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   109
    def __init__(self, callableobj):
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   110
        self.callable = callableobj
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   111
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   112
    @property
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   113
    def __doc__(self):
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   114
        return self.callable.__doc__
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   115
    @property
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   116
    def __name__(self):
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   117
        return self.callable.__name__
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   118
    
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   119
    def __call__(self, *args, **kw):
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   120
        if _address is None:
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   121
            return self.callable(*args, **kw)
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   122
        t0 = time.time()
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   123
        try:
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   124
            return self.callable(*args, **kw)
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   125
        finally:
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   126
            dt = 1000*(time.time()-t0)
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   127
            msg = '{0}.{1}:{2:.4f}|ms\n{0}.{1}:1|c\n'.format(_bucket, self.__name__, dt)
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   128
            _socket.sendto(msg, _address)
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   129
                
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   130
    def __get__(self, obj, objtype):
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   131
        """Support instance methods."""
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   132
        if obj is None: # class method or some already wrapped method
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   133
            return self
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   134
        import functools
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   135
        return functools.partial(self.__call__, obj)