cubicweb/statsd_logger.py
author Simon Chabot <simon.chabot@logilab.fr>
Thu, 05 Mar 2020 17:50:00 +0100
branch3.26
changeset 12909 a3d682abd5c3
parent 12291 b640ef6ad633
child 12302 0d474f888f4a
permissions -rw-r--r--
Added tag 3.26.17, debian/3.26.17-1 for changeset 5c6d242069b6
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
10477
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
     1
# copyright 2015 LOGILAB S.A. (Paris, FRANCE), all rights reserved.
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
     2
# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
     3
#
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
     4
# This file is part of CubicWeb.
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
     5
#
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
     6
# CubicWeb is free software: you can redistribute it and/or modify it under the
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
     7
# terms of the GNU Lesser General Public License as published by the Free
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
     8
# Software Foundation, either version 2.1 of the License, or (at your option)
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
     9
# any later version.
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    10
#
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    11
# CubicWeb is distributed in the hope that it will be useful, but WITHOUT
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    12
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    13
# FOR A PARTICULAR PURPOSE.  See the GNU Lesser General Public License for more
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    14
# details.
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    15
#
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    16
# You should have received a copy of the GNU Lesser General Public License along
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    17
# with CubicWeb.  If not, see <http://www.gnu.org/licenses/>.
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    18
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    19
"""Simple statsd_ logger for cubicweb.
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    20
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    21
This module is meant to be configured by setting a couple of global variables:
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    22
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    23
- ``bucket`` global variable will be used as statsd bucket in every
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    24
statsd_ UDP sent packet.
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    25
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    26
`- `address`` is a pair (IP, port) specifying the address of the
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    27
statsd_ server
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    28
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    29
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    30
There are 3 kinds of statds_ message::
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    31
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    32
- ``statsd_c(context, n)`` is a simple function to send statsd_
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    33
  counter-type of messages like::
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    34
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    35
    <bucket>.<context>:<n>|c\n
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    36
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    37
- ``statsd_g(context, value)`` to send statsd_ gauge-type of messages
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    38
  like::
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    39
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    40
    <bucket>.<context>:<n>|g\n
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    41
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    42
- ``statsd_t(context, ms)`` to send statsd_ time-type of messages
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    43
  like::
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    44
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    45
    <bucket>.<context>:<ms>|ms\n
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    46
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    47
There is also a decorator (``statsd_timeit``) that may be used to
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    48
measure and send to the statsd_ server the time passed in a function
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    49
or a method and the number of calls. It will send a message like::
12291
b640ef6ad633 [utils/statsd_logger] flake8
David Douard <david.douard@logilab.fr>
parents: 11767
diff changeset
    50
10477
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    51
    <bucket>.<funcname>:<ms>|ms\n<bucket>.<funcname>:1|c\n
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    52
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    53
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    54
.. _statsd: https://github.com/etsy/statsd
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    55
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    56
"""
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    57
11767
432f87a63057 flake8 and all
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11057
diff changeset
    58
10477
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    59
import time
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    60
import socket
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    61
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    62
_bucket = 'cubicweb'
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    63
_address = None
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    64
_socket = None
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    65
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    66
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    67
def setup(bucket, address):
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    68
    """Configure the statsd endpoint
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    69
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    70
    :param bucket: the name of the statsd bucket that will be used to
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    71
                   build messages.
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    72
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    73
    :param address: the UDP endpoint of the statsd server. Must a
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    74
                    couple (ip, port).
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    75
    """
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    76
    global _bucket, _address, _socket
10478
026b4ee032fb [statsd_logger] handle ipv6 addresses
Julien Cristau <julien.cristau@logilab.fr>
parents: 10477
diff changeset
    77
    packed = None
026b4ee032fb [statsd_logger] handle ipv6 addresses
Julien Cristau <julien.cristau@logilab.fr>
parents: 10477
diff changeset
    78
    for family in (socket.AF_INET6, socket.AF_INET):
026b4ee032fb [statsd_logger] handle ipv6 addresses
Julien Cristau <julien.cristau@logilab.fr>
parents: 10477
diff changeset
    79
        try:
10650
28b3d39bcbc6 [statsd] fix the statsd logger (closes #7558147)
David Douard <david.douard@logilab.fr>
parents: 10478
diff changeset
    80
            packed = socket.inet_pton(family, address[0])
10478
026b4ee032fb [statsd_logger] handle ipv6 addresses
Julien Cristau <julien.cristau@logilab.fr>
parents: 10477
diff changeset
    81
            break
026b4ee032fb [statsd_logger] handle ipv6 addresses
Julien Cristau <julien.cristau@logilab.fr>
parents: 10477
diff changeset
    82
        except socket.error:
026b4ee032fb [statsd_logger] handle ipv6 addresses
Julien Cristau <julien.cristau@logilab.fr>
parents: 10477
diff changeset
    83
            continue
026b4ee032fb [statsd_logger] handle ipv6 addresses
Julien Cristau <julien.cristau@logilab.fr>
parents: 10477
diff changeset
    84
    if packed is None:
026b4ee032fb [statsd_logger] handle ipv6 addresses
Julien Cristau <julien.cristau@logilab.fr>
parents: 10477
diff changeset
    85
        return
10477
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    86
    _bucket, _address = bucket, address
10478
026b4ee032fb [statsd_logger] handle ipv6 addresses
Julien Cristau <julien.cristau@logilab.fr>
parents: 10477
diff changeset
    87
    _socket = socket.socket(family, socket.SOCK_DGRAM)
10477
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    88
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    89
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    90
def statsd_c(context, n=1):
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    91
    if _address is not None:
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    92
        _socket.sendto('{0}.{1}:{2}|c\n'.format(_bucket, context, n), _address)
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    93
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    94
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    95
def statsd_g(context, value):
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    96
    if _address is not None:
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    97
        _socket.sendto('{0}.{1}:{2}|g\n'.format(_bucket, context, value), _address)
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    98
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
    99
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   100
def statsd_t(context, value):
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   101
    if _address is not None:
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   102
        _socket.sendto('{0}.{1}:{2:.4f}|ms\n'.format(_bucket, context, value), _address)
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   103
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   104
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   105
class statsd_timeit(object):
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   106
    __slots__ = ('callable',)
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   107
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   108
    def __init__(self, callableobj):
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   109
        self.callable = callableobj
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   110
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   111
    @property
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   112
    def __doc__(self):
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   113
        return self.callable.__doc__
12291
b640ef6ad633 [utils/statsd_logger] flake8
David Douard <david.douard@logilab.fr>
parents: 11767
diff changeset
   114
10477
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   115
    @property
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   116
    def __name__(self):
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   117
        return self.callable.__name__
12291
b640ef6ad633 [utils/statsd_logger] flake8
David Douard <david.douard@logilab.fr>
parents: 11767
diff changeset
   118
10477
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   119
    def __call__(self, *args, **kw):
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   120
        if _address is None:
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   121
            return self.callable(*args, **kw)
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   122
        t0 = time.time()
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   123
        try:
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   124
            return self.callable(*args, **kw)
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   125
        finally:
12291
b640ef6ad633 [utils/statsd_logger] flake8
David Douard <david.douard@logilab.fr>
parents: 11767
diff changeset
   126
            dt = 1000 * (time.time() - t0)
b640ef6ad633 [utils/statsd_logger] flake8
David Douard <david.douard@logilab.fr>
parents: 11767
diff changeset
   127
            msg = '{0}.{1}:{2:.4f}|ms\n{0}.{1}:1|c\n'.format(
b640ef6ad633 [utils/statsd_logger] flake8
David Douard <david.douard@logilab.fr>
parents: 11767
diff changeset
   128
                _bucket, self.__name__, dt)
10477
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   129
            _socket.sendto(msg, _address)
12291
b640ef6ad633 [utils/statsd_logger] flake8
David Douard <david.douard@logilab.fr>
parents: 11767
diff changeset
   130
10477
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   131
    def __get__(self, obj, objtype):
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   132
        """Support instance methods."""
12291
b640ef6ad633 [utils/statsd_logger] flake8
David Douard <david.douard@logilab.fr>
parents: 11767
diff changeset
   133
        if obj is None:  # class method or some already wrapped method
10477
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   134
            return self
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   135
        import functools
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents:
diff changeset
   136
        return functools.partial(self.__call__, obj)