doc/tools/generate_modules.py
author sylvain.thenault@logilab.fr
Tue, 10 Mar 2009 19:35:38 +0100
branchtls-sprint
changeset 1036 593df4919845
parent 299 4761e0ca406d
child 1693 49075f57cf2c
child 1802 d628defebc17
permissions -rw-r--r--
when reading the schema while adding/removing cubes, read schema in non-strict mode
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
299
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
     1
import sys
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
     2
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
     3
"""
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
     4
Generates the chapter that list all the modules in CubicWeb
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
     5
in order to pull all the docstring.
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
     6
"""
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
     7
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
     8
class ModuleGenerator:
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
     9
    HEADER = """.. -*- coding: utf-8 -*-
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    10
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    11
============
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    12
CubicWeb API
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    13
============
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    14
"""
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    15
    EXCLUDE_DIRS = ('test', 'tests', 'examples', 'data', 'doc', '.hg', 'migration')  
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    16
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    17
    def __init__(self, output_fn, mod_names):
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    18
        self.mod_names =  mod_names
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    19
        self.fn = open(output_fn, 'w')
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    20
        self.fn.write(self.HEADER)
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    21
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    22
    def done(self):
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    23
        self.fn.close()
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    24
        
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    25
    def gen_module(self, mod_name):
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    26
        mod_entry = """
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    27
:mod:`%s`
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    28
%s
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    29
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    30
.. automodule:: %s
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    31
   :members:
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    32
""" % (mod_name, '='*(len(':mod:``'+mod_name)), mod_name)
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    33
        self.fn.write(mod_entry)
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    34
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    35
    def find_modules(self):
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    36
        import os
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    37
        modules = []
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    38
        for mod_name in self.mod_names:
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    39
            for root, dirs, files in os.walk(mod_name):
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    40
                if self.keep_module(root):
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    41
                    for name in files:
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    42
                        if name == "__init__.py":
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    43
                            if self.format_mod_name(root, mod_name) not in modules:
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    44
                                modules.append(self.format_mod_name(root, mod_name))
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    45
                        else:
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    46
                            if name.endswith(".py") and name != "__pkginfo__.py" and "__init__.py" in files:
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    47
                                filename = root + '/' + name.split('.py')[0]
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    48
                                if self.format_mod_name(filename, mod_name) not in modules:
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    49
                                    modules.append(self.format_mod_name(filename, mod_name))
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    50
        return modules
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    51
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    52
    def gen_modules(self):
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    53
        for module in self.find_modules():
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    54
            self.gen_module(module)
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    55
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    56
    def format_mod_name(self, path, mod_name):
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    57
        mod_root = mod_name.split('/')[-1]
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    58
        mod_end = path.split(mod_root)[-1]
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    59
        return mod_root + mod_end.replace('/', '.')
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    60
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    61
    def keep_module(self, mod_end):
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    62
        """
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    63
        Filter modules in order to exclude specific package directories.
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    64
        """
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    65
        for dir in self.EXCLUDE_DIRS:
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    66
            if mod_end.find(dir) != -1:
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    67
                return False
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    68
        return True
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    69
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    70
USAGE = """
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    71
Two arguments required:
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    72
    generate_modules [cubicweb-root] [file-out]
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    73
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    74
[cubicweb-root] : full path to cubicweb forest
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    75
[file-out] : rest file containing the list of modules for Sphinx
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    76
"""
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    77
def generate_modules_file(args):
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    78
    if len(args) != 2:
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    79
        print USAGE
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    80
        sys.exit()
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    81
    CW_ROOT = args[0]
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    82
    OUTPUT = args[1]
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    83
    modules = (CW_ROOT + '/cubicweb', \
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    84
               CW_ROOT + '/indexer', \
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    85
               CW_ROOT + '/logilab', \
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    86
               CW_ROOT + '/rql', \
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    87
               CW_ROOT + '/yams')
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    88
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    89
    mg = ModuleGenerator(CW_ROOT + '/cubicweb/doc/book/en/' + OUTPUT, modules)
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    90
    mg.find_modules()
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    91
    mg.gen_modules()
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    92
    mg.done()
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    93
    print args
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    94
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    95
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    96
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    97
if __name__ == '__main__':
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    98
    generate_modules_file(sys.argv[1:])
4761e0ca406d [doc] Adds scripts to auto-generate modules list to index and add to the documentation.
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents:
diff changeset
    99