web/views/csvexport.py
author Rémi Cardona <remi.cardona@logilab.fr>
Tue, 15 Sep 2015 16:56:57 +0200
changeset 10609 e2d8e81bfe68
parent 10046 96f5ca0a86e5
child 10663 54b8a1f249fb
permissions -rw-r--r--
[py3k] import range using six.moves
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
8035
f98012ec7c53 [export views] uniformize titles, much nicer when listing possible views. Closes #2060944
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6582
diff changeset
     1
# copyright 2003-2011 LOGILAB S.A. (Paris, FRANCE), all rights reserved.
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
     2
# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
     3
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
     4
# This file is part of CubicWeb.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
     5
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
     6
# CubicWeb is free software: you can redistribute it and/or modify it under the
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
     7
# terms of the GNU Lesser General Public License as published by the Free
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
     8
# Software Foundation, either version 2.1 of the License, or (at your option)
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
     9
# any later version.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    10
#
5424
8ecbcbff9777 replace logilab-common by CubicWeb in disclaimer
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5421
diff changeset
    11
# CubicWeb is distributed in the hope that it will be useful, but WITHOUT
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    12
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    13
# FOR A PARTICULAR PURPOSE.  See the GNU Lesser General Public License for more
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    14
# details.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    15
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    16
# You should have received a copy of the GNU Lesser General Public License along
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    17
# with CubicWeb.  If not, see <http://www.gnu.org/licenses/>.
6582
8eb7883b4223 [pylint] fix a bug of pylint detected errors and i18n pb (calling builtins._ instead of req._)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5424
diff changeset
    18
"""csv export views"""
824
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    19
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    20
__docformat__ = "restructuredtext en"
6582
8eb7883b4223 [pylint] fix a bug of pylint detected errors and i18n pb (calling builtins._ instead of req._)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5424
diff changeset
    21
_ = unicode
824
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    22
10609
e2d8e81bfe68 [py3k] import range using six.moves
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10046
diff changeset
    23
from six.moves import range
e2d8e81bfe68 [py3k] import range using six.moves
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10046
diff changeset
    24
2303
ea86d250cca9 import display_name
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
    25
from cubicweb.schema import display_name
10046
96f5ca0a86e5 [views] fix ecsvexport selector
David Douard <david.douard@logilab.fr>
parents: 9977
diff changeset
    26
from cubicweb.predicates import any_rset, empty_rset
4023
eae23c40627a drop common subpackage
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3720
diff changeset
    27
from cubicweb.uilib import UnicodeCSVWriter
824
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    28
from cubicweb.view import EntityView, AnyRsetView
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    29
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    30
class CSVMixIn(object):
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    31
    """mixin class for CSV views"""
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    32
    templatable = False
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 824
diff changeset
    33
    content_type = "text/comma-separated-values"
824
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    34
    binary = True # avoid unicode assertion
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    35
    csv_params = {'dialect': 'excel',
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    36
                  'quotechar': '"',
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    37
                  'delimiter': ';',
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    38
                  'lineterminator': '\n'}
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 824
diff changeset
    39
824
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    40
    def set_request_content_type(self):
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    41
        """overriden to set a .csv filename"""
3451
6b46d73823f5 [api] work in progress, use __regid__, cw_*, etc.
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 3377
diff changeset
    42
        self._cw.set_content_type(self.content_type, filename='cubicwebexport.csv')
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 824
diff changeset
    43
824
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    44
    def csvwriter(self, **kwargs):
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    45
        params = self.csv_params.copy()
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    46
        params.update(kwargs)
3451
6b46d73823f5 [api] work in progress, use __regid__, cw_*, etc.
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 3377
diff changeset
    47
        return UnicodeCSVWriter(self.w, self._cw.encoding, **params)
824
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    48
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 824
diff changeset
    49
824
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    50
class CSVRsetView(CSVMixIn, AnyRsetView):
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    51
    """dumps raw result set in CSV"""
3377
dd9d292b6a6d use __regid__ instead of id on appobject classes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3165
diff changeset
    52
    __regid__ = 'csvexport'
9977
e48e5a597ccc [views] csvexport accept an empty rset (closes #4236928)
Damien Garaud <damien.garaud@logilab.fr>
parents: 8035
diff changeset
    53
    __select__ = any_rset()
824
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    54
    title = _('csv export')
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 824
diff changeset
    55
824
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    56
    def call(self):
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    57
        writer = self.csvwriter()
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    58
        writer.writerow(self.columns_labels())
3451
6b46d73823f5 [api] work in progress, use __regid__, cw_*, etc.
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 3377
diff changeset
    59
        rset, descr = self.cw_rset, self.cw_rset.description
4045
f4a52abb6f4f cw 3.6 api update
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents: 4023
diff changeset
    60
        eschema = self._cw.vreg.schema.eschema
824
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    61
        for rowindex, row in enumerate(rset):
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    62
            csvrow = []
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    63
            for colindex, val in enumerate(row):
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    64
                etype = descr[rowindex][colindex]
3689
deb13e88e037 follow yams 0.25 api changes to improve performance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3165
diff changeset
    65
                if val is not None and not eschema(etype).final:
824
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    66
                    # csvrow.append(val) # val is eid in that case
3451
6b46d73823f5 [api] work in progress, use __regid__, cw_*, etc.
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 3377
diff changeset
    67
                    content = self._cw.view('textincontext', rset,
6b46d73823f5 [api] work in progress, use __regid__, cw_*, etc.
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 3377
diff changeset
    68
                                            row=rowindex, col=colindex)
824
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    69
                else:
3451
6b46d73823f5 [api] work in progress, use __regid__, cw_*, etc.
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 3377
diff changeset
    70
                    content = self._cw.view('final', rset,
6b46d73823f5 [api] work in progress, use __regid__, cw_*, etc.
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 3377
diff changeset
    71
                                            format='text/plain',
6b46d73823f5 [api] work in progress, use __regid__, cw_*, etc.
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 3377
diff changeset
    72
                                            row=rowindex, col=colindex)
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 824
diff changeset
    73
                csvrow.append(content)
824
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    74
            writer.writerow(csvrow)
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 824
diff changeset
    75
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 824
diff changeset
    76
824
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    77
class CSVEntityView(CSVMixIn, EntityView):
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    78
    """dumps rset's entities (with full set of attributes) in CSV
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    79
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    80
    the generated CSV file will have a table per entity type found in the
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    81
    resultset. ('table' here only means empty lines separation between table
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    82
    contents)
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    83
    """
3377
dd9d292b6a6d use __regid__ instead of id on appobject classes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3165
diff changeset
    84
    __regid__ = 'ecsvexport'
10046
96f5ca0a86e5 [views] fix ecsvexport selector
David Douard <david.douard@logilab.fr>
parents: 9977
diff changeset
    85
    __select__ = EntityView.__select__ | empty_rset()
8035
f98012ec7c53 [export views] uniformize titles, much nicer when listing possible views. Closes #2060944
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6582
diff changeset
    86
    title = _('csv export (entities)')
824
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    87
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    88
    def call(self):
3451
6b46d73823f5 [api] work in progress, use __regid__, cw_*, etc.
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 3377
diff changeset
    89
        req = self._cw
824
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    90
        rows_by_type = {}
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    91
        writer = self.csvwriter()
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    92
        rowdef_by_type = {}
10609
e2d8e81bfe68 [py3k] import range using six.moves
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10046
diff changeset
    93
        for index in range(len(self.cw_rset)):
3451
6b46d73823f5 [api] work in progress, use __regid__, cw_*, etc.
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 3377
diff changeset
    94
            entity = self.cw_rset.complete_entity(index)
824
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    95
            if entity.e_schema not in rows_by_type:
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    96
                rowdef_by_type[entity.e_schema] = [rs for rs, at in entity.e_schema.attribute_definitions()
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    97
                                                   if at != 'Bytes']
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    98
                rows_by_type[entity.e_schema] = [[display_name(req, rschema.type)
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    99
                                                  for rschema in rowdef_by_type[entity.e_schema]]]
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
   100
            rows = rows_by_type[entity.e_schema]
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
   101
            rows.append([entity.printable_value(rs.type, format='text/plain')
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
   102
                         for rs in rowdef_by_type[entity.e_schema]])
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
   103
        for rows in rows_by_type.itervalues():
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
   104
            writer.writerows(rows)
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
   105
            # use two empty lines as separator
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 824
diff changeset
   106
            writer.writerows([[], []])
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 824
diff changeset
   107