cubicweb/web/views/csvexport.py
author Denis Laxalde <denis.laxalde@logilab.fr>
Fri, 01 Dec 2017 11:19:19 +0100
changeset 12244 6a71cb23f827
parent 11767 432f87a63057
child 12567 26744ad37953
permissions -rw-r--r--
Disable pytest's logs reporting CubicWeb already prints captured logs in stderr upon test failure, so we get them twice. Until we fix our logging management, disabling pytest capture and restore brings the output level back to sanity.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
8035
f98012ec7c53 [export views] uniformize titles, much nicer when listing possible views. Closes #2060944
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6582
diff changeset
     1
# copyright 2003-2011 LOGILAB S.A. (Paris, FRANCE), all rights reserved.
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
     2
# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
     3
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
     4
# This file is part of CubicWeb.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
     5
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
     6
# CubicWeb is free software: you can redistribute it and/or modify it under the
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
     7
# terms of the GNU Lesser General Public License as published by the Free
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
     8
# Software Foundation, either version 2.1 of the License, or (at your option)
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
     9
# any later version.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    10
#
5424
8ecbcbff9777 replace logilab-common by CubicWeb in disclaimer
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5421
diff changeset
    11
# CubicWeb is distributed in the hope that it will be useful, but WITHOUT
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    12
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    13
# FOR A PARTICULAR PURPOSE.  See the GNU Lesser General Public License for more
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    14
# details.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    15
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    16
# You should have received a copy of the GNU Lesser General Public License along
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    17
# with CubicWeb.  If not, see <http://www.gnu.org/licenses/>.
6582
8eb7883b4223 [pylint] fix a bug of pylint detected errors and i18n pb (calling builtins._ instead of req._)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5424
diff changeset
    18
"""csv export views"""
824
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    19
11767
432f87a63057 flake8 and all
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11057
diff changeset
    20
10666
7f6b5f023884 [py3k] replace '_ = unicode' in global scope (closes #7589459)
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10663
diff changeset
    21
from cubicweb import _
824
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    22
10695
321b99973b69 [web/views] Port csvexport views to py3k
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10666
diff changeset
    23
from six import PY2
10609
e2d8e81bfe68 [py3k] import range using six.moves
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10046
diff changeset
    24
from six.moves import range
e2d8e81bfe68 [py3k] import range using six.moves
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10046
diff changeset
    25
2303
ea86d250cca9 import display_name
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
    26
from cubicweb.schema import display_name
10046
96f5ca0a86e5 [views] fix ecsvexport selector
David Douard <david.douard@logilab.fr>
parents: 9977
diff changeset
    27
from cubicweb.predicates import any_rset, empty_rset
4023
eae23c40627a drop common subpackage
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3720
diff changeset
    28
from cubicweb.uilib import UnicodeCSVWriter
824
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    29
from cubicweb.view import EntityView, AnyRsetView
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    30
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    31
class CSVMixIn(object):
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    32
    """mixin class for CSV views"""
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    33
    templatable = False
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 824
diff changeset
    34
    content_type = "text/comma-separated-values"
10695
321b99973b69 [web/views] Port csvexport views to py3k
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10666
diff changeset
    35
    binary = PY2 # python csv module is unicode aware in py3k
824
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    36
    csv_params = {'dialect': 'excel',
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    37
                  'quotechar': '"',
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    38
                  'delimiter': ';',
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    39
                  'lineterminator': '\n'}
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 824
diff changeset
    40
824
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    41
    def set_request_content_type(self):
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    42
        """overriden to set a .csv filename"""
3451
6b46d73823f5 [api] work in progress, use __regid__, cw_*, etc.
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 3377
diff changeset
    43
        self._cw.set_content_type(self.content_type, filename='cubicwebexport.csv')
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 824
diff changeset
    44
824
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    45
    def csvwriter(self, **kwargs):
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    46
        params = self.csv_params.copy()
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    47
        params.update(kwargs)
3451
6b46d73823f5 [api] work in progress, use __regid__, cw_*, etc.
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 3377
diff changeset
    48
        return UnicodeCSVWriter(self.w, self._cw.encoding, **params)
824
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    49
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 824
diff changeset
    50
824
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    51
class CSVRsetView(CSVMixIn, AnyRsetView):
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    52
    """dumps raw result set in CSV"""
3377
dd9d292b6a6d use __regid__ instead of id on appobject classes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3165
diff changeset
    53
    __regid__ = 'csvexport'
9977
e48e5a597ccc [views] csvexport accept an empty rset (closes #4236928)
Damien Garaud <damien.garaud@logilab.fr>
parents: 8035
diff changeset
    54
    __select__ = any_rset()
824
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    55
    title = _('csv export')
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 824
diff changeset
    56
824
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    57
    def call(self):
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    58
        writer = self.csvwriter()
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    59
        writer.writerow(self.columns_labels())
3451
6b46d73823f5 [api] work in progress, use __regid__, cw_*, etc.
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 3377
diff changeset
    60
        rset, descr = self.cw_rset, self.cw_rset.description
4045
f4a52abb6f4f cw 3.6 api update
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents: 4023
diff changeset
    61
        eschema = self._cw.vreg.schema.eschema
824
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    62
        for rowindex, row in enumerate(rset):
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    63
            csvrow = []
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    64
            for colindex, val in enumerate(row):
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    65
                etype = descr[rowindex][colindex]
3689
deb13e88e037 follow yams 0.25 api changes to improve performance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3165
diff changeset
    66
                if val is not None and not eschema(etype).final:
824
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    67
                    # csvrow.append(val) # val is eid in that case
3451
6b46d73823f5 [api] work in progress, use __regid__, cw_*, etc.
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 3377
diff changeset
    68
                    content = self._cw.view('textincontext', rset,
6b46d73823f5 [api] work in progress, use __regid__, cw_*, etc.
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 3377
diff changeset
    69
                                            row=rowindex, col=colindex)
824
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    70
                else:
3451
6b46d73823f5 [api] work in progress, use __regid__, cw_*, etc.
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 3377
diff changeset
    71
                    content = self._cw.view('final', rset,
6b46d73823f5 [api] work in progress, use __regid__, cw_*, etc.
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 3377
diff changeset
    72
                                            format='text/plain',
6b46d73823f5 [api] work in progress, use __regid__, cw_*, etc.
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 3377
diff changeset
    73
                                            row=rowindex, col=colindex)
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 824
diff changeset
    74
                csvrow.append(content)
824
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    75
            writer.writerow(csvrow)
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 824
diff changeset
    76
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 824
diff changeset
    77
824
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    78
class CSVEntityView(CSVMixIn, EntityView):
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    79
    """dumps rset's entities (with full set of attributes) in CSV
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    80
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    81
    the generated CSV file will have a table per entity type found in the
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    82
    resultset. ('table' here only means empty lines separation between table
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    83
    contents)
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    84
    """
3377
dd9d292b6a6d use __regid__ instead of id on appobject classes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3165
diff changeset
    85
    __regid__ = 'ecsvexport'
10046
96f5ca0a86e5 [views] fix ecsvexport selector
David Douard <david.douard@logilab.fr>
parents: 9977
diff changeset
    86
    __select__ = EntityView.__select__ | empty_rset()
8035
f98012ec7c53 [export views] uniformize titles, much nicer when listing possible views. Closes #2060944
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6582
diff changeset
    87
    title = _('csv export (entities)')
824
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    88
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    89
    def call(self):
3451
6b46d73823f5 [api] work in progress, use __regid__, cw_*, etc.
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 3377
diff changeset
    90
        req = self._cw
824
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    91
        rows_by_type = {}
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    92
        writer = self.csvwriter()
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    93
        rowdef_by_type = {}
10609
e2d8e81bfe68 [py3k] import range using six.moves
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10046
diff changeset
    94
        for index in range(len(self.cw_rset)):
3451
6b46d73823f5 [api] work in progress, use __regid__, cw_*, etc.
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 3377
diff changeset
    95
            entity = self.cw_rset.complete_entity(index)
824
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    96
            if entity.e_schema not in rows_by_type:
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    97
                rowdef_by_type[entity.e_schema] = [rs for rs, at in entity.e_schema.attribute_definitions()
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    98
                                                   if at != 'Bytes']
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
    99
                rows_by_type[entity.e_schema] = [[display_name(req, rschema.type)
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
   100
                                                  for rschema in rowdef_by_type[entity.e_schema]]]
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
   101
            rows = rows_by_type[entity.e_schema]
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
   102
            rows.append([entity.printable_value(rs.type, format='text/plain')
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
   103
                         for rs in rowdef_by_type[entity.e_schema]])
10663
54b8a1f249fb [py3k] dict.itervalues → dict.values
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10609
diff changeset
   104
        for rows in rows_by_type.values():
824
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
   105
            writer.writerows(rows)
a5e6acffde30 merge, split baseviews (new csvexport, xmlrss and editviews modules)
sylvain.thenault@logilab.fr
parents:
diff changeset
   106
            # use two empty lines as separator
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 824
diff changeset
   107
            writer.writerows([[], []])