devtools/fix_po_encoding
author Sylvain Thénault <sylvain.thenault@logilab.fr>
Mon, 21 Jan 2013 18:01:25 +0100
changeset 8669 62213a34726e
parent 0 b97547f5f1fa
permissions -rwxr-xr-x
[db-api/configuration] simplify db-api and configuration so that all the connection information is in the repository url, closes #2521848 eg no more specific option of pyro ns host, group, etc. This also fixes broken ZMQ sources. Changes: * dropped pyro-ns-host, pyro-instance-id, pyro-ns-group from client side config, in favor of repository-uri. No migration done, supposing there is **no web-only config** in the wild. Also stop discovering the connection method through the repo_method class attribute of the configuration, varying according to the configuration class. This is a first step on the way to a simpler configuration handling. Notice those pyro options are still available for repository only / all-in-one configurations as they are needed to configure the pyro server. * stop telling connection method using ConnectionProperties, this is so boring. Also, drop _cnxtype from Connection and cnxtype from Session. The former is replaced by a is_repo_in_memory property and the later is totaly useless. * deprecate in_memory_cnx which becomes useless, use _repo_connect instead

#!/usr/bin/python

"""usage: fix-po-encodings [filename...]
change the encoding of the po files passed as arguments to utf-8
"""
import sys
import re
import codecs

def change_encoding(filename, target='UTF-8'):
    fdesc = open(filename)
    data = fdesc.read()
    fdesc.close()
    encoding = find_encoding(data)
    if encoding == target:
        return
    data = fix_encoding(data, target)
    data = unicode(data, encoding)
    fdesc = codecs.open(filename, 'wb', encoding=target)
    fdesc.write(data)
    fdesc.close()

def find_encoding(data):
    regexp = re.compile(r'"Content-Type:.* charset=([a-zA-Z0-9-]+)\\n"', re.M)
    mo = regexp.search(data)
    if mo is None:
        raise ValueError('No encoding declaration')
    return mo.group(1)

def fix_encoding(data, target_encoding):
    regexp = re.compile(r'("Content-Type:.* charset=)(.*)(\\n")', re.M)
    return regexp.sub(r'\1%s\3' % target_encoding, data)
    


for filename in sys.argv[1:]:
    print filename
    change_encoding(filename)