sobjects/ldapparser.py
author Sylvain Thénault <sylvain.thenault@logilab.fr>
Tue, 24 Nov 2015 10:16:55 +0100
changeset 10915 cf800aa43f48
parent 10912 addc7ba36f69
child 11037 621ad47c7c86
permissions -rw-r--r--
[server/sources] fix name error in eid_type_source_pre_131 i.e. the eid_type_source implementation used when some pre 3.13.1 database is detected. Regression from 5fbdbbe51867.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
10909
53af91f77b9d [ldapfeed] simplify deactivation of unencountered users
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10907
diff changeset
     1
# copyright 2011-2015 LOGILAB S.A. (Paris, FRANCE), all rights reserved.
8188
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     2
# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     3
#
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     4
# This file is part of CubicWeb.
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     5
#
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     6
# CubicWeb is free software: you can redistribute it and/or modify it under the
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     7
# terms of the GNU Lesser General Public License as published by the Free
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     8
# Software Foundation, either version 2.1 of the License, or (at your option)
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     9
# any later version.
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    10
#
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    11
# CubicWeb is distributed in the hope that it will be useful, but WITHOUT
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    12
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    13
# FOR A PARTICULAR PURPOSE.  See the GNU Lesser General Public License for more
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    14
# details.
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    15
#
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    16
# You should have received a copy of the GNU Lesser General Public License along
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    17
# with CubicWeb.  If not, see <http://www.gnu.org/licenses/>.
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    18
"""cubicweb ldap feed source
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    19
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    20
unlike ldapuser source, this source is copy based and will import ldap content
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    21
(beside passwords for authentication) into the system source.
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    22
"""
10768
99689a5862ea [py3k] make ldapfeed tests pass
Julien Cristau <julien.cristau@logilab.fr>
parents: 10663
diff changeset
    23
from six.moves import map, filter
99689a5862ea [py3k] make ldapfeed tests pass
Julien Cristau <julien.cristau@logilab.fr>
parents: 10663
diff changeset
    24
8638
9f95c2368b8b [ldaputils,ldapparser] correctly compute "deleted" users (closes #2542083)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8633
diff changeset
    25
from logilab.common.decorators import cached, cachedproperty
8387
b59af20a868d [ldap] we may actually get back password from ldap
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8382
diff changeset
    26
from logilab.common.shellutils import generate_password
8188
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    27
8566
76bcfb3c483d [ldapparser] raise specific error if the configuration is wrong (closes #2498164)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8479
diff changeset
    28
from cubicweb import Binary, ConfigurationError
8387
b59af20a868d [ldap] we may actually get back password from ldap
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8382
diff changeset
    29
from cubicweb.server.utils import crypt_password
8188
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    30
from cubicweb.server.sources import datafeed
10912
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
    31
from cubicweb.dataimport import stores, importer
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
    32
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
    33
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
    34
class UserMetaGenerator(stores.MetaGenerator):
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
    35
    """Specific metadata generator, used to see newly created user into their initial state.
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
    36
    """
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
    37
    @cached
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
    38
    def base_etype_dicts(self, entity):
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
    39
        entity, rels = super(UserMetaGenerator, self).base_etype_dicts(entity)
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
    40
        if entity.cw_etype == 'CWUser':
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
    41
            wf_state = self._cnx.execute('Any S WHERE ET default_workflow WF, ET name %(etype)s, '
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
    42
                                         'WF initial_state S', {'etype': entity.cw_etype}).one()
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
    43
            rels['in_state'] = wf_state.eid
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
    44
        return entity, rels
8188
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    45
8387
b59af20a868d [ldap] we may actually get back password from ldap
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8382
diff changeset
    46
8430
5bee87a14bb1 fix ldap removal handling in ldapfeed (closes #2376625 and #2385133)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8427
diff changeset
    47
class DataFeedLDAPAdapter(datafeed.DataFeedParser):
8188
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    48
    __regid__ = 'ldapfeed'
8250
171a9d6bff8f [ldapfeed] fix synchronisation crash: ldap attributes are given while we want cw attributes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8188
diff changeset
    49
    # attributes that may appears in source user_attrs dict which are not
171a9d6bff8f [ldapfeed] fix synchronisation crash: ldap attributes are given while we want cw attributes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8188
diff changeset
    50
    # attributes of the cw user
8919
4cba95ef4738 [ldap] handle modification date
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8918
diff changeset
    51
    non_attribute_keys = set(('email', 'eid', 'member', 'modification_date'))
8188
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    52
8638
9f95c2368b8b [ldaputils,ldapparser] correctly compute "deleted" users (closes #2542083)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8633
diff changeset
    53
    @cachedproperty
9f95c2368b8b [ldaputils,ldapparser] correctly compute "deleted" users (closes #2542083)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8633
diff changeset
    54
    def searchfilterstr(self):
9f95c2368b8b [ldaputils,ldapparser] correctly compute "deleted" users (closes #2542083)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8633
diff changeset
    55
        """ ldap search string, including user-filter """
9f95c2368b8b [ldaputils,ldapparser] correctly compute "deleted" users (closes #2542083)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8633
diff changeset
    56
        return '(&%s)' % ''.join(self.source.base_filters)
9f95c2368b8b [ldaputils,ldapparser] correctly compute "deleted" users (closes #2542083)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8633
diff changeset
    57
9f95c2368b8b [ldaputils,ldapparser] correctly compute "deleted" users (closes #2542083)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8633
diff changeset
    58
    @cachedproperty
8922
715b9eec6da9 [ldapfeed] Add support for LDAP groups (closes #2528116)
David Douard <david.douard@logilab.fr>
parents: 8920
diff changeset
    59
    def searchgroupfilterstr(self):
715b9eec6da9 [ldapfeed] Add support for LDAP groups (closes #2528116)
David Douard <david.douard@logilab.fr>
parents: 8920
diff changeset
    60
        """ ldap search string, including user-filter """
715b9eec6da9 [ldapfeed] Add support for LDAP groups (closes #2528116)
David Douard <david.douard@logilab.fr>
parents: 8920
diff changeset
    61
        return '(&%s)' % ''.join(self.source.group_base_filters)
715b9eec6da9 [ldapfeed] Add support for LDAP groups (closes #2528116)
David Douard <david.douard@logilab.fr>
parents: 8920
diff changeset
    62
715b9eec6da9 [ldapfeed] Add support for LDAP groups (closes #2528116)
David Douard <david.douard@logilab.fr>
parents: 8920
diff changeset
    63
    @cachedproperty
8920
386049566ceb [ldap] prepare import of CWGroup
David Douard <david.douard@logilab.fr>
parents: 8919
diff changeset
    64
    def user_source_entities_by_extid(self):
8638
9f95c2368b8b [ldaputils,ldapparser] correctly compute "deleted" users (closes #2542083)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8633
diff changeset
    65
        source = self.source
8906
ed35d984ff28 [ldap] an empty 'user-base-dn' disable the user importation process,
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8900
diff changeset
    66
        if source.user_base_dn.strip():
10768
99689a5862ea [py3k] make ldapfeed tests pass
Julien Cristau <julien.cristau@logilab.fr>
parents: 10663
diff changeset
    67
            attrs = list(map(str, source.user_attrs.keys()))
99689a5862ea [py3k] make ldapfeed tests pass
Julien Cristau <julien.cristau@logilab.fr>
parents: 10663
diff changeset
    68
            return dict((userdict['dn'].encode('ascii'), userdict)
8906
ed35d984ff28 [ldap] an empty 'user-base-dn' disable the user importation process,
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8900
diff changeset
    69
                        for userdict in source._search(self._cw,
ed35d984ff28 [ldap] an empty 'user-base-dn' disable the user importation process,
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8900
diff changeset
    70
                                                       source.user_base_dn,
ed35d984ff28 [ldap] an empty 'user-base-dn' disable the user importation process,
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8900
diff changeset
    71
                                                       source.user_base_scope,
8918
43fd866e8f8a [ldap] refactor attributes mapping handling
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8917
diff changeset
    72
                                                       self.searchfilterstr,
43fd866e8f8a [ldap] refactor attributes mapping handling
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8917
diff changeset
    73
                                                       attrs))
8906
ed35d984ff28 [ldap] an empty 'user-base-dn' disable the user importation process,
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8900
diff changeset
    74
        return {}
8638
9f95c2368b8b [ldaputils,ldapparser] correctly compute "deleted" users (closes #2542083)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8633
diff changeset
    75
8922
715b9eec6da9 [ldapfeed] Add support for LDAP groups (closes #2528116)
David Douard <david.douard@logilab.fr>
parents: 8920
diff changeset
    76
    @cachedproperty
715b9eec6da9 [ldapfeed] Add support for LDAP groups (closes #2528116)
David Douard <david.douard@logilab.fr>
parents: 8920
diff changeset
    77
    def group_source_entities_by_extid(self):
715b9eec6da9 [ldapfeed] Add support for LDAP groups (closes #2528116)
David Douard <david.douard@logilab.fr>
parents: 8920
diff changeset
    78
        source = self.source
715b9eec6da9 [ldapfeed] Add support for LDAP groups (closes #2528116)
David Douard <david.douard@logilab.fr>
parents: 8920
diff changeset
    79
        if source.group_base_dn.strip():
10768
99689a5862ea [py3k] make ldapfeed tests pass
Julien Cristau <julien.cristau@logilab.fr>
parents: 10663
diff changeset
    80
            attrs = list(map(str, ['modifyTimestamp'] + list(source.group_attrs.keys())))
8922
715b9eec6da9 [ldapfeed] Add support for LDAP groups (closes #2528116)
David Douard <david.douard@logilab.fr>
parents: 8920
diff changeset
    81
            return dict((groupdict['dn'], groupdict)
715b9eec6da9 [ldapfeed] Add support for LDAP groups (closes #2528116)
David Douard <david.douard@logilab.fr>
parents: 8920
diff changeset
    82
                        for groupdict in source._search(self._cw,
715b9eec6da9 [ldapfeed] Add support for LDAP groups (closes #2528116)
David Douard <david.douard@logilab.fr>
parents: 8920
diff changeset
    83
                                                        source.group_base_dn,
715b9eec6da9 [ldapfeed] Add support for LDAP groups (closes #2528116)
David Douard <david.douard@logilab.fr>
parents: 8920
diff changeset
    84
                                                        source.group_base_scope,
715b9eec6da9 [ldapfeed] Add support for LDAP groups (closes #2528116)
David Douard <david.douard@logilab.fr>
parents: 8920
diff changeset
    85
                                                        self.searchgroupfilterstr,
715b9eec6da9 [ldapfeed] Add support for LDAP groups (closes #2528116)
David Douard <david.douard@logilab.fr>
parents: 8920
diff changeset
    86
                                                        attrs))
715b9eec6da9 [ldapfeed] Add support for LDAP groups (closes #2528116)
David Douard <david.douard@logilab.fr>
parents: 8920
diff changeset
    87
        return {}
715b9eec6da9 [ldapfeed] Add support for LDAP groups (closes #2528116)
David Douard <david.douard@logilab.fr>
parents: 8920
diff changeset
    88
8409
79534887943e [datafeed] fix/finish cleanup started by auc in 8393:c25b96ae4f8a: parser.process prototytpe is (url, raise_on_error=False). Drop partialcommit argument which were never specified
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8400
diff changeset
    89
    def process(self, url, raise_on_error=False):
8188
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    90
        """IDataFeedParser main entry point"""
8638
9f95c2368b8b [ldaputils,ldapparser] correctly compute "deleted" users (closes #2542083)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8633
diff changeset
    91
        self.debug('processing ldapfeed source %s %s', self.source, self.searchfilterstr)
10912
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
    92
        self._group_members = {}
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
    93
        eeimporter = self.build_importer(raise_on_error)
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
    94
        for name in self.source.user_default_groups:
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
    95
            geid = self._get_group(name)
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
    96
            eeimporter.extid2eid[geid] = geid
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
    97
        entities = self.extentities_generator()
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
    98
        set_cwuri = importer.use_extid_as_cwuri(eeimporter.extid2eid)
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
    99
        eeimporter.import_entities(set_cwuri(entities))
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   100
        self.stats['created'] = eeimporter.created
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   101
        self.stats['updated'] = eeimporter.updated
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   102
        # handle in_group relation
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   103
        for group, members in self._group_members.items():
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   104
            self._cw.execute('DELETE U in_group G WHERE G name %(g)s', {'g': group})
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   105
            if members:
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   106
                members = ["'%s'" % e for e in members]
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   107
                rql = 'SET U in_group G WHERE G name %%(g)s, U login IN (%s)' % ','.join(members)
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   108
                self._cw.execute(rql, {'g': group})
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   109
        # ensure updated users are activated
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   110
        for eid in eeimporter.updated:
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   111
            entity = self._cw.entity_from_eid(eid)
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   112
            if entity.cw_etype == 'CWUser':
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   113
                self.ensure_activated(entity)
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   114
        # manually set primary email if necessary, it's not handled automatically since hooks are
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   115
        # deactivated
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   116
        self._cw.execute('SET X primary_email E WHERE NOT X primary_email E, X use_email E, '
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   117
                         'X cw_source S, S eid %(s)s, X in_state ST, TS name "activated"',
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   118
                         {'s': self.source.eid})
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   119
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   120
    def build_importer(self, raise_on_error):
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   121
        """Instantiate and configure an importer"""
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   122
        etypes = ('CWUser', 'EmailAddress', 'CWGroup')
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   123
        extid2eid = importer.cwuri2eid(self._cw, etypes, source_eid=self.source.eid)
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   124
        existing_relations = {}
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   125
        for rtype in ('in_group', 'use_email', 'owned_by'):
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   126
            rql = 'Any S,O WHERE S {} O, S cw_source SO, SO eid %(s)s'.format(rtype)
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   127
            rset = self._cw.execute(rql, {'s': self.source.eid})
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   128
            existing_relations[rtype] = set(tuple(x) for x in rset)
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   129
        return importer.ExtEntitiesImporter(self._cw.vreg.schema, self.build_store(),
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   130
                                            extid2eid=extid2eid,
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   131
                                            existing_relations=existing_relations,
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   132
                                            etypes_order_hint=etypes,
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   133
                                            import_log=self.import_log,
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   134
                                            raise_on_error=raise_on_error)
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   135
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   136
    def build_store(self):
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   137
        """Instantiate and configure a store"""
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   138
        metagenerator = UserMetaGenerator(self._cw, source=self.source)
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   139
        return stores.NoHookRQLObjectStore(self._cw, metagenerator)
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   140
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   141
    def extentities_generator(self):
8922
715b9eec6da9 [ldapfeed] Add support for LDAP groups (closes #2528116)
David Douard <david.douard@logilab.fr>
parents: 8920
diff changeset
   142
        self.debug('processing ldapfeed source %s %s', self.source, self.searchgroupfilterstr)
10912
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   143
        # generate users and email addresses
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   144
        for userdict in self.user_source_entities_by_extid.values():
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   145
            attrs = self.ldap2cwattrs(userdict, 'CWUser')
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   146
            pwd = attrs.get('upassword')
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   147
            if not pwd:
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   148
                # generate a dumb password if not fetched from ldap (see
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   149
                # userPassword)
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   150
                pwd = crypt_password(generate_password())
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   151
                attrs['upassword'] = set([Binary(pwd)])
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   152
            extuser = importer.ExtEntity('CWUser', userdict['dn'], attrs)
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   153
            extuser.values['owned_by'] = set([extuser.extid])
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   154
            for extemail in self._process_email(extuser, userdict):
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   155
                yield extemail
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   156
            groups = list(filter(None, [self._get_group(name)
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   157
                                        for name in self.source.user_default_groups]))
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   158
            if groups:
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   159
                extuser.values['in_group'] = groups
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   160
            yield extuser
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   161
        # generate groups
10663
54b8a1f249fb [py3k] dict.itervalues → dict.values
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10662
diff changeset
   162
        for groupdict in self.group_source_entities_by_extid.values():
10912
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   163
            attrs = self.ldap2cwattrs(groupdict, 'CWGroup')
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   164
            extgroup = importer.ExtEntity('CWGroup', groupdict['dn'], attrs)
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   165
            yield extgroup
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   166
            # record group membership for later insertion
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   167
            members = groupdict.get(self.source.group_rev_attrs['member'], ())
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   168
            self._group_members[attrs['name']] = members
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   169
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   170
    def _process_email(self, extuser, userdict):
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   171
        try:
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   172
            emailaddrs = userdict.pop(self.source.user_rev_attrs['email'])
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   173
        except KeyError:
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   174
            return  # no email for that user, nothing to do
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   175
        if not isinstance(emailaddrs, list):
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   176
            emailaddrs = [emailaddrs]
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   177
        for emailaddr in emailaddrs:
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   178
            # search for existing email first, may be coming from another source
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   179
            rset = self._cw.execute('EmailAddress X WHERE X address %(addr)s',
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   180
                                    {'addr': emailaddr})
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   181
            if not rset:
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   182
                # not found, create it. first forge an external id
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   183
                emailextid = userdict['dn'] + '@@' + emailaddr
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   184
                extuser.values.setdefault('use_email', []).append(emailextid)
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   185
                yield importer.ExtEntity('EmailAddress', emailextid, dict(address=[emailaddr]))
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   186
            elif self.sourceuris:
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   187
                # pop from sourceuris anyway, else email may be removed by the
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   188
                # source once import is finished
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   189
                emailextid = userdict['dn'] + '@@' + emailaddr
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   190
                self.sourceuris.pop(emailextid, None)
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   191
            # XXX else check use_email relation?
8430
5bee87a14bb1 fix ldap removal handling in ldapfeed (closes #2376625 and #2385133)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8427
diff changeset
   192
9879
21278eb03bbf [datafeed sources] finish the session -> cnx switch
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9551
diff changeset
   193
    def handle_deletion(self, config, cnx, myuris):
8430
5bee87a14bb1 fix ldap removal handling in ldapfeed (closes #2376625 and #2385133)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8427
diff changeset
   194
        if config['delete-entities']:
9879
21278eb03bbf [datafeed sources] finish the session -> cnx switch
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9551
diff changeset
   195
            super(DataFeedLDAPAdapter, self).handle_deletion(config, cnx, myuris)
8432
96b4f7a35e6c [ldapparser] missing return
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8431
diff changeset
   196
            return
8430
5bee87a14bb1 fix ldap removal handling in ldapfeed (closes #2376625 and #2385133)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8427
diff changeset
   197
        if myuris:
10662
10942ed172de [py3k] dict.iteritems → dict.items
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10089
diff changeset
   198
            for extid, (eid, etype) in myuris.items():
10909
53af91f77b9d [ldapfeed] simplify deactivation of unencountered users
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10907
diff changeset
   199
                if etype != 'CWUser' or not self.is_deleted(extid, etype, eid):
8430
5bee87a14bb1 fix ldap removal handling in ldapfeed (closes #2376625 and #2385133)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8427
diff changeset
   200
                    continue
10909
53af91f77b9d [ldapfeed] simplify deactivation of unencountered users
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10907
diff changeset
   201
                self.info('deactivate user %s', eid)
53af91f77b9d [ldapfeed] simplify deactivation of unencountered users
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10907
diff changeset
   202
                wf = cnx.entity_from_eid(eid).cw_adapt_to('IWorkflowable')
53af91f77b9d [ldapfeed] simplify deactivation of unencountered users
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10907
diff changeset
   203
                wf.fire_transition_if_possible('deactivate')
9879
21278eb03bbf [datafeed sources] finish the session -> cnx switch
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9551
diff changeset
   204
        cnx.commit()
8430
5bee87a14bb1 fix ldap removal handling in ldapfeed (closes #2376625 and #2385133)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8427
diff changeset
   205
10912
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   206
    def ensure_activated(self, entity):
8900
010a59e12d89 use cw_etype instead of __regid__
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8888
diff changeset
   207
        if entity.cw_etype == 'CWUser':
8434
39c5bb4dcc59 [ldapfeed] do not crash on ldap user deletion + pull + already deactivated users, cleanups (closes #2392933)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8432
diff changeset
   208
            wf = entity.cw_adapt_to('IWorkflowable')
39c5bb4dcc59 [ldapfeed] do not crash on ldap user deletion + pull + already deactivated users, cleanups (closes #2392933)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8432
diff changeset
   209
            if wf.state == 'deactivated':
8639
2fddbe32ae8b [ldapfeed] if a deactivated user becomes available again in its source, reactivate it (closes #2542776)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8638
diff changeset
   210
                wf.fire_transition('activate')
9219
6afdeaabac74 [ldapparser] demote some logs from warning to debug (closes #2713671)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8922
diff changeset
   211
                self.info('user %s reactivated', entity.login)
8434
39c5bb4dcc59 [ldapfeed] do not crash on ldap user deletion + pull + already deactivated users, cleanups (closes #2392933)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8432
diff changeset
   212
10910
5ba4de264be4 [ldapfeed] simplify ldap2attrs
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10909
diff changeset
   213
    def ldap2cwattrs(self, sdict, etype):
5ba4de264be4 [ldapfeed] simplify ldap2attrs
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10909
diff changeset
   214
        """Transform dictionary of LDAP attributes to CW.
5ba4de264be4 [ldapfeed] simplify ldap2attrs
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10909
diff changeset
   215
5ba4de264be4 [ldapfeed] simplify ldap2attrs
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10909
diff changeset
   216
        etype must be CWUser or CWGroup
5ba4de264be4 [ldapfeed] simplify ldap2attrs
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10909
diff changeset
   217
        """
5ba4de264be4 [ldapfeed] simplify ldap2attrs
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10909
diff changeset
   218
        assert etype in ('CWUser', 'CWGroup'), etype
5ba4de264be4 [ldapfeed] simplify ldap2attrs
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10909
diff changeset
   219
        tdict = {}
8920
386049566ceb [ldap] prepare import of CWGroup
David Douard <david.douard@logilab.fr>
parents: 8919
diff changeset
   220
        if etype == 'CWUser':
10662
10942ed172de [py3k] dict.iteritems → dict.items
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10089
diff changeset
   221
            items = self.source.user_attrs.items()
8922
715b9eec6da9 [ldapfeed] Add support for LDAP groups (closes #2528116)
David Douard <david.douard@logilab.fr>
parents: 8920
diff changeset
   222
        elif etype == 'CWGroup':
10662
10942ed172de [py3k] dict.iteritems → dict.items
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10089
diff changeset
   223
            items = self.source.group_attrs.items()
8920
386049566ceb [ldap] prepare import of CWGroup
David Douard <david.douard@logilab.fr>
parents: 8919
diff changeset
   224
        for sattr, tattr in items:
8250
171a9d6bff8f [ldapfeed] fix synchronisation crash: ldap attributes are given while we want cw attributes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8188
diff changeset
   225
            if tattr not in self.non_attribute_keys:
8566
76bcfb3c483d [ldapparser] raise specific error if the configuration is wrong (closes #2498164)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8479
diff changeset
   226
                try:
10910
5ba4de264be4 [ldapfeed] simplify ldap2attrs
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10909
diff changeset
   227
                    value = sdict[sattr]
8566
76bcfb3c483d [ldapparser] raise specific error if the configuration is wrong (closes #2498164)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8479
diff changeset
   228
                except KeyError:
10910
5ba4de264be4 [ldapfeed] simplify ldap2attrs
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10909
diff changeset
   229
                    raise ConfigurationError(
5ba4de264be4 [ldapfeed] simplify ldap2attrs
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10909
diff changeset
   230
                        'source attribute %s has not been found in the source, '
5ba4de264be4 [ldapfeed] simplify ldap2attrs
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10909
diff changeset
   231
                        'please check the %s-attrs-map field and the permissions of '
5ba4de264be4 [ldapfeed] simplify ldap2attrs
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10909
diff changeset
   232
                        'the LDAP binding user' % (sattr, etype[2:].lower()))
10912
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   233
                if not isinstance(value, list):
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   234
                    value = [value]
10910
5ba4de264be4 [ldapfeed] simplify ldap2attrs
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10909
diff changeset
   235
                tdict[tattr] = value
8250
171a9d6bff8f [ldapfeed] fix synchronisation crash: ldap attributes are given while we want cw attributes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8188
diff changeset
   236
        return tdict
171a9d6bff8f [ldapfeed] fix synchronisation crash: ldap attributes are given while we want cw attributes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8188
diff changeset
   237
8638
9f95c2368b8b [ldaputils,ldapparser] correctly compute "deleted" users (closes #2542083)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8633
diff changeset
   238
    def is_deleted(self, extidplus, etype, eid):
8188
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   239
        try:
10906
e1ce0866afe9 [ldapparser] simplify extid splitting
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10768
diff changeset
   240
            extid = extidplus.rsplit(b'@@', 1)[0]
8188
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   241
        except ValueError:
8638
9f95c2368b8b [ldaputils,ldapparser] correctly compute "deleted" users (closes #2542083)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8633
diff changeset
   242
            # for some reason extids here tend to come in both forms, e.g:
9f95c2368b8b [ldaputils,ldapparser] correctly compute "deleted" users (closes #2542083)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8633
diff changeset
   243
            # dn, dn@@Babar
9f95c2368b8b [ldaputils,ldapparser] correctly compute "deleted" users (closes #2542083)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8633
diff changeset
   244
            extid = extidplus
8920
386049566ceb [ldap] prepare import of CWGroup
David Douard <david.douard@logilab.fr>
parents: 8919
diff changeset
   245
        return extid not in self.user_source_entities_by_extid
8188
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   246
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   247
    @cached
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   248
    def _get_group(self, name):
8679
cf4dacc80976 [ldapfeed] don't crash if one specify an unexisting group in the configuration. Closes #2538399
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8639
diff changeset
   249
        try:
cf4dacc80976 [ldapfeed] don't crash if one specify an unexisting group in the configuration. Closes #2538399
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8639
diff changeset
   250
            return self._cw.execute('Any X WHERE X is CWGroup, X name %(name)s',
10912
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   251
                                    {'name': name})[0][0]
8679
cf4dacc80976 [ldapfeed] don't crash if one specify an unexisting group in the configuration. Closes #2538399
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8639
diff changeset
   252
        except IndexError:
8888
738f97bc3e19 [ldap] Use correct API for logging message
Jérôme Roy <jerome.roy@logilab.fr>
parents: 8694
diff changeset
   253
            self.error('group %r referenced by source configuration %r does not exist',
738f97bc3e19 [ldap] Use correct API for logging message
Jérôme Roy <jerome.roy@logilab.fr>
parents: 8694
diff changeset
   254
                       name, self.source.uri)
8679
cf4dacc80976 [ldapfeed] don't crash if one specify an unexisting group in the configuration. Closes #2538399
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8639
diff changeset
   255
            return None