devtools/exlog.py
author Nicolas Chauvat <nicolas.chauvat@logilab.fr>
Mon, 05 Jan 2009 22:30:25 +0100
changeset 331 1e12e8cd6901
permissions -rw-r--r--
exlog script to examine rql logs
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
331
1e12e8cd6901 exlog script to examine rql logs
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents:
diff changeset
     1
"""
1e12e8cd6901 exlog script to examine rql logs
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents:
diff changeset
     2
usage: python exlog.py < rql.log
1e12e8cd6901 exlog script to examine rql logs
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents:
diff changeset
     3
1e12e8cd6901 exlog script to examine rql logs
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents:
diff changeset
     4
will print out the following table
1e12e8cd6901 exlog script to examine rql logs
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents:
diff changeset
     5
1e12e8cd6901 exlog script to examine rql logs
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents:
diff changeset
     6
  total execution time || number of occurences || rql query
1e12e8cd6901 exlog script to examine rql logs
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents:
diff changeset
     7
1e12e8cd6901 exlog script to examine rql logs
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents:
diff changeset
     8
sorted by descending total execution time
1e12e8cd6901 exlog script to examine rql logs
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents:
diff changeset
     9
1e12e8cd6901 exlog script to examine rql logs
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents:
diff changeset
    10
chances are the lines at the top are the ones that will bring
1e12e8cd6901 exlog script to examine rql logs
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents:
diff changeset
    11
the higher benefit after optimisation. Start there.
1e12e8cd6901 exlog script to examine rql logs
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents:
diff changeset
    12
"""
1e12e8cd6901 exlog script to examine rql logs
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents:
diff changeset
    13
import sys, re
1e12e8cd6901 exlog script to examine rql logs
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents:
diff changeset
    14
1e12e8cd6901 exlog script to examine rql logs
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents:
diff changeset
    15
def run():
1e12e8cd6901 exlog script to examine rql logs
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents:
diff changeset
    16
    requests = {}
1e12e8cd6901 exlog script to examine rql logs
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents:
diff changeset
    17
    for line in sys.stdin:
1e12e8cd6901 exlog script to examine rql logs
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents:
diff changeset
    18
        if not ' WHERE ' in line:
1e12e8cd6901 exlog script to examine rql logs
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents:
diff changeset
    19
            continue
1e12e8cd6901 exlog script to examine rql logs
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents:
diff changeset
    20
        #sys.stderr.write( line )
1e12e8cd6901 exlog script to examine rql logs
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents:
diff changeset
    21
        rql, time = line.split('--')
1e12e8cd6901 exlog script to examine rql logs
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents:
diff changeset
    22
        rql = re.sub("(\'\w+': \d*)", '', rql)
1e12e8cd6901 exlog script to examine rql logs
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents:
diff changeset
    23
        req = requests.setdefault(rql, [])
1e12e8cd6901 exlog script to examine rql logs
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents:
diff changeset
    24
        time.strip()
1e12e8cd6901 exlog script to examine rql logs
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents:
diff changeset
    25
        chunks = time.split()
1e12e8cd6901 exlog script to examine rql logs
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents:
diff changeset
    26
        cputime = float(chunks[-3])
1e12e8cd6901 exlog script to examine rql logs
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents:
diff changeset
    27
        req.append( cputime )
1e12e8cd6901 exlog script to examine rql logs
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents:
diff changeset
    28
1e12e8cd6901 exlog script to examine rql logs
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents:
diff changeset
    29
    stat = []
1e12e8cd6901 exlog script to examine rql logs
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents:
diff changeset
    30
    for rql, times in requests.items():
1e12e8cd6901 exlog script to examine rql logs
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents:
diff changeset
    31
        stat.append( (sum(times), len(times), rql) )
1e12e8cd6901 exlog script to examine rql logs
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents:
diff changeset
    32
1e12e8cd6901 exlog script to examine rql logs
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents:
diff changeset
    33
    stat.sort()
1e12e8cd6901 exlog script to examine rql logs
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents:
diff changeset
    34
    stat.reverse()
1e12e8cd6901 exlog script to examine rql logs
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents:
diff changeset
    35
    for time, occ, rql in stat:
1e12e8cd6901 exlog script to examine rql logs
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents:
diff changeset
    36
        print time, occ, rql
1e12e8cd6901 exlog script to examine rql logs
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents:
diff changeset
    37
1e12e8cd6901 exlog script to examine rql logs
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents:
diff changeset
    38
if __name__ == '__main__':
1e12e8cd6901 exlog script to examine rql logs
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents:
diff changeset
    39
    run()