sobjects/ldapparser.py
author Sylvain Thénault <sylvain.thenault@logilab.fr>
Tue, 24 Nov 2015 14:36:41 +0100
changeset 10912 addc7ba36f69
parent 10910 5ba4de264be4
child 11037 621ad47c7c86
permissions -rw-r--r--
[ldapfeed / dataimport] port ldapfeed parser to dataimport API This makes the code easier to understand and will allow to deprecate the old multi-sources api, based on creation callback through repo.extid2eid. Currently with this patch, modification dates are not checked, hence entities will be systematically updated. We run the import with no hooks, because RQLObjectStore can only add entities to the system source.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
10909
53af91f77b9d [ldapfeed] simplify deactivation of unencountered users
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10907
diff changeset
     1
# copyright 2011-2015 LOGILAB S.A. (Paris, FRANCE), all rights reserved.
8188
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     2
# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     3
#
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     4
# This file is part of CubicWeb.
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     5
#
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     6
# CubicWeb is free software: you can redistribute it and/or modify it under the
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     7
# terms of the GNU Lesser General Public License as published by the Free
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     8
# Software Foundation, either version 2.1 of the License, or (at your option)
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     9
# any later version.
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    10
#
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    11
# CubicWeb is distributed in the hope that it will be useful, but WITHOUT
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    12
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    13
# FOR A PARTICULAR PURPOSE.  See the GNU Lesser General Public License for more
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    14
# details.
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    15
#
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    16
# You should have received a copy of the GNU Lesser General Public License along
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    17
# with CubicWeb.  If not, see <http://www.gnu.org/licenses/>.
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    18
"""cubicweb ldap feed source
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    19
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    20
unlike ldapuser source, this source is copy based and will import ldap content
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    21
(beside passwords for authentication) into the system source.
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    22
"""
10768
99689a5862ea [py3k] make ldapfeed tests pass
Julien Cristau <julien.cristau@logilab.fr>
parents: 10663
diff changeset
    23
from six.moves import map, filter
99689a5862ea [py3k] make ldapfeed tests pass
Julien Cristau <julien.cristau@logilab.fr>
parents: 10663
diff changeset
    24
8638
9f95c2368b8b [ldaputils,ldapparser] correctly compute "deleted" users (closes #2542083)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8633
diff changeset
    25
from logilab.common.decorators import cached, cachedproperty
8387
b59af20a868d [ldap] we may actually get back password from ldap
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8382
diff changeset
    26
from logilab.common.shellutils import generate_password
8188
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    27
8566
76bcfb3c483d [ldapparser] raise specific error if the configuration is wrong (closes #2498164)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8479
diff changeset
    28
from cubicweb import Binary, ConfigurationError
8387
b59af20a868d [ldap] we may actually get back password from ldap
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8382
diff changeset
    29
from cubicweb.server.utils import crypt_password
8188
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    30
from cubicweb.server.sources import datafeed
10912
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
    31
from cubicweb.dataimport import stores, importer
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
    32
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
    33
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
    34
class UserMetaGenerator(stores.MetaGenerator):
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
    35
    """Specific metadata generator, used to see newly created user into their initial state.
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
    36
    """
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
    37
    @cached
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
    38
    def base_etype_dicts(self, entity):
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
    39
        entity, rels = super(UserMetaGenerator, self).base_etype_dicts(entity)
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
    40
        if entity.cw_etype == 'CWUser':
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
    41
            wf_state = self._cnx.execute('Any S WHERE ET default_workflow WF, ET name %(etype)s, '
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
    42
                                         'WF initial_state S', {'etype': entity.cw_etype}).one()
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
    43
            rels['in_state'] = wf_state.eid
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
    44
        return entity, rels
8188
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    45
8387
b59af20a868d [ldap] we may actually get back password from ldap
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8382
diff changeset
    46
8430
5bee87a14bb1 fix ldap removal handling in ldapfeed (closes #2376625 and #2385133)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8427
diff changeset
    47
class DataFeedLDAPAdapter(datafeed.DataFeedParser):
8188
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    48
    __regid__ = 'ldapfeed'
8250
171a9d6bff8f [ldapfeed] fix synchronisation crash: ldap attributes are given while we want cw attributes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8188
diff changeset
    49
    # attributes that may appears in source user_attrs dict which are not
171a9d6bff8f [ldapfeed] fix synchronisation crash: ldap attributes are given while we want cw attributes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8188
diff changeset
    50
    # attributes of the cw user
8919
4cba95ef4738 [ldap] handle modification date
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8918
diff changeset
    51
    non_attribute_keys = set(('email', 'eid', 'member', 'modification_date'))
8188
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    52
8638
9f95c2368b8b [ldaputils,ldapparser] correctly compute "deleted" users (closes #2542083)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8633
diff changeset
    53
    @cachedproperty
9f95c2368b8b [ldaputils,ldapparser] correctly compute "deleted" users (closes #2542083)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8633
diff changeset
    54
    def searchfilterstr(self):
9f95c2368b8b [ldaputils,ldapparser] correctly compute "deleted" users (closes #2542083)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8633
diff changeset
    55
        """ ldap search string, including user-filter """
9f95c2368b8b [ldaputils,ldapparser] correctly compute "deleted" users (closes #2542083)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8633
diff changeset
    56
        return '(&%s)' % ''.join(self.source.base_filters)
9f95c2368b8b [ldaputils,ldapparser] correctly compute "deleted" users (closes #2542083)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8633
diff changeset
    57
9f95c2368b8b [ldaputils,ldapparser] correctly compute "deleted" users (closes #2542083)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8633
diff changeset
    58
    @cachedproperty
8922
715b9eec6da9 [ldapfeed] Add support for LDAP groups (closes #2528116)
David Douard <david.douard@logilab.fr>
parents: 8920
diff changeset
    59
    def searchgroupfilterstr(self):
715b9eec6da9 [ldapfeed] Add support for LDAP groups (closes #2528116)
David Douard <david.douard@logilab.fr>
parents: 8920
diff changeset
    60
        """ ldap search string, including user-filter """
715b9eec6da9 [ldapfeed] Add support for LDAP groups (closes #2528116)
David Douard <david.douard@logilab.fr>
parents: 8920
diff changeset
    61
        return '(&%s)' % ''.join(self.source.group_base_filters)
715b9eec6da9 [ldapfeed] Add support for LDAP groups (closes #2528116)
David Douard <david.douard@logilab.fr>
parents: 8920
diff changeset
    62
715b9eec6da9 [ldapfeed] Add support for LDAP groups (closes #2528116)
David Douard <david.douard@logilab.fr>
parents: 8920
diff changeset
    63
    @cachedproperty
8920
386049566ceb [ldap] prepare import of CWGroup
David Douard <david.douard@logilab.fr>
parents: 8919
diff changeset
    64
    def user_source_entities_by_extid(self):
8638
9f95c2368b8b [ldaputils,ldapparser] correctly compute "deleted" users (closes #2542083)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8633
diff changeset
    65
        source = self.source
8906
ed35d984ff28 [ldap] an empty 'user-base-dn' disable the user importation process,
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8900
diff changeset
    66
        if source.user_base_dn.strip():
10768
99689a5862ea [py3k] make ldapfeed tests pass
Julien Cristau <julien.cristau@logilab.fr>
parents: 10663
diff changeset
    67
            attrs = list(map(str, source.user_attrs.keys()))
99689a5862ea [py3k] make ldapfeed tests pass
Julien Cristau <julien.cristau@logilab.fr>
parents: 10663
diff changeset
    68
            return dict((userdict['dn'].encode('ascii'), userdict)
8906
ed35d984ff28 [ldap] an empty 'user-base-dn' disable the user importation process,
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8900
diff changeset
    69
                        for userdict in source._search(self._cw,
ed35d984ff28 [ldap] an empty 'user-base-dn' disable the user importation process,
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8900
diff changeset
    70
                                                       source.user_base_dn,
ed35d984ff28 [ldap] an empty 'user-base-dn' disable the user importation process,
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8900
diff changeset
    71
                                                       source.user_base_scope,
8918
43fd866e8f8a [ldap] refactor attributes mapping handling
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8917
diff changeset
    72
                                                       self.searchfilterstr,
43fd866e8f8a [ldap] refactor attributes mapping handling
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8917
diff changeset
    73
                                                       attrs))
8906
ed35d984ff28 [ldap] an empty 'user-base-dn' disable the user importation process,
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8900
diff changeset
    74
        return {}
8638
9f95c2368b8b [ldaputils,ldapparser] correctly compute "deleted" users (closes #2542083)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8633
diff changeset
    75
8922
715b9eec6da9 [ldapfeed] Add support for LDAP groups (closes #2528116)
David Douard <david.douard@logilab.fr>
parents: 8920
diff changeset
    76
    @cachedproperty
715b9eec6da9 [ldapfeed] Add support for LDAP groups (closes #2528116)
David Douard <david.douard@logilab.fr>
parents: 8920
diff changeset
    77
    def group_source_entities_by_extid(self):
715b9eec6da9 [ldapfeed] Add support for LDAP groups (closes #2528116)
David Douard <david.douard@logilab.fr>
parents: 8920
diff changeset
    78
        source = self.source
715b9eec6da9 [ldapfeed] Add support for LDAP groups (closes #2528116)
David Douard <david.douard@logilab.fr>
parents: 8920
diff changeset
    79
        if source.group_base_dn.strip():
10768
99689a5862ea [py3k] make ldapfeed tests pass
Julien Cristau <julien.cristau@logilab.fr>
parents: 10663
diff changeset
    80
            attrs = list(map(str, ['modifyTimestamp'] + list(source.group_attrs.keys())))
8922
715b9eec6da9 [ldapfeed] Add support for LDAP groups (closes #2528116)
David Douard <david.douard@logilab.fr>
parents: 8920
diff changeset
    81
            return dict((groupdict['dn'], groupdict)
715b9eec6da9 [ldapfeed] Add support for LDAP groups (closes #2528116)
David Douard <david.douard@logilab.fr>
parents: 8920
diff changeset
    82
                        for groupdict in source._search(self._cw,
715b9eec6da9 [ldapfeed] Add support for LDAP groups (closes #2528116)
David Douard <david.douard@logilab.fr>
parents: 8920
diff changeset
    83
                                                        source.group_base_dn,
715b9eec6da9 [ldapfeed] Add support for LDAP groups (closes #2528116)
David Douard <david.douard@logilab.fr>
parents: 8920
diff changeset
    84
                                                        source.group_base_scope,
715b9eec6da9 [ldapfeed] Add support for LDAP groups (closes #2528116)
David Douard <david.douard@logilab.fr>
parents: 8920
diff changeset
    85
                                                        self.searchgroupfilterstr,
715b9eec6da9 [ldapfeed] Add support for LDAP groups (closes #2528116)
David Douard <david.douard@logilab.fr>
parents: 8920
diff changeset
    86
                                                        attrs))
715b9eec6da9 [ldapfeed] Add support for LDAP groups (closes #2528116)
David Douard <david.douard@logilab.fr>
parents: 8920
diff changeset
    87
        return {}
715b9eec6da9 [ldapfeed] Add support for LDAP groups (closes #2528116)
David Douard <david.douard@logilab.fr>
parents: 8920
diff changeset
    88
8409
79534887943e [datafeed] fix/finish cleanup started by auc in 8393:c25b96ae4f8a: parser.process prototytpe is (url, raise_on_error=False). Drop partialcommit argument which were never specified
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8400
diff changeset
    89
    def process(self, url, raise_on_error=False):
8188
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    90
        """IDataFeedParser main entry point"""
8638
9f95c2368b8b [ldaputils,ldapparser] correctly compute "deleted" users (closes #2542083)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8633
diff changeset
    91
        self.debug('processing ldapfeed source %s %s', self.source, self.searchfilterstr)
10912
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
    92
        self._group_members = {}
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
    93
        eeimporter = self.build_importer(raise_on_error)
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
    94
        for name in self.source.user_default_groups:
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
    95
            geid = self._get_group(name)
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
    96
            eeimporter.extid2eid[geid] = geid
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
    97
        entities = self.extentities_generator()
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
    98
        set_cwuri = importer.use_extid_as_cwuri(eeimporter.extid2eid)
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
    99
        eeimporter.import_entities(set_cwuri(entities))
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   100
        self.stats['created'] = eeimporter.created
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   101
        self.stats['updated'] = eeimporter.updated
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   102
        # handle in_group relation
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   103
        for group, members in self._group_members.items():
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   104
            self._cw.execute('DELETE U in_group G WHERE G name %(g)s', {'g': group})
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   105
            if members:
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   106
                members = ["'%s'" % e for e in members]
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   107
                rql = 'SET U in_group G WHERE G name %%(g)s, U login IN (%s)' % ','.join(members)
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   108
                self._cw.execute(rql, {'g': group})
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   109
        # ensure updated users are activated
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   110
        for eid in eeimporter.updated:
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   111
            entity = self._cw.entity_from_eid(eid)
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   112
            if entity.cw_etype == 'CWUser':
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   113
                self.ensure_activated(entity)
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   114
        # manually set primary email if necessary, it's not handled automatically since hooks are
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   115
        # deactivated
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   116
        self._cw.execute('SET X primary_email E WHERE NOT X primary_email E, X use_email E, '
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   117
                         'X cw_source S, S eid %(s)s, X in_state ST, TS name "activated"',
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   118
                         {'s': self.source.eid})
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   119
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   120
    def build_importer(self, raise_on_error):
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   121
        """Instantiate and configure an importer"""
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   122
        etypes = ('CWUser', 'EmailAddress', 'CWGroup')
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   123
        extid2eid = importer.cwuri2eid(self._cw, etypes, source_eid=self.source.eid)
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   124
        existing_relations = {}
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   125
        for rtype in ('in_group', 'use_email', 'owned_by'):
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   126
            rql = 'Any S,O WHERE S {} O, S cw_source SO, SO eid %(s)s'.format(rtype)
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   127
            rset = self._cw.execute(rql, {'s': self.source.eid})
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   128
            existing_relations[rtype] = set(tuple(x) for x in rset)
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   129
        return importer.ExtEntitiesImporter(self._cw.vreg.schema, self.build_store(),
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   130
                                            extid2eid=extid2eid,
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   131
                                            existing_relations=existing_relations,
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   132
                                            etypes_order_hint=etypes,
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   133
                                            import_log=self.import_log,
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   134
                                            raise_on_error=raise_on_error)
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   135
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   136
    def build_store(self):
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   137
        """Instantiate and configure a store"""
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   138
        metagenerator = UserMetaGenerator(self._cw, source=self.source)
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   139
        return stores.NoHookRQLObjectStore(self._cw, metagenerator)
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   140
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   141
    def extentities_generator(self):
8922
715b9eec6da9 [ldapfeed] Add support for LDAP groups (closes #2528116)
David Douard <david.douard@logilab.fr>
parents: 8920
diff changeset
   142
        self.debug('processing ldapfeed source %s %s', self.source, self.searchgroupfilterstr)
10912
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   143
        # generate users and email addresses
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   144
        for userdict in self.user_source_entities_by_extid.values():
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   145
            attrs = self.ldap2cwattrs(userdict, 'CWUser')
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   146
            pwd = attrs.get('upassword')
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   147
            if not pwd:
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   148
                # generate a dumb password if not fetched from ldap (see
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   149
                # userPassword)
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   150
                pwd = crypt_password(generate_password())
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   151
                attrs['upassword'] = set([Binary(pwd)])
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   152
            extuser = importer.ExtEntity('CWUser', userdict['dn'], attrs)
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   153
            extuser.values['owned_by'] = set([extuser.extid])
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   154
            for extemail in self._process_email(extuser, userdict):
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   155
                yield extemail
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   156
            groups = list(filter(None, [self._get_group(name)
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   157
                                        for name in self.source.user_default_groups]))
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   158
            if groups:
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   159
                extuser.values['in_group'] = groups
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   160
            yield extuser
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   161
        # generate groups
10663
54b8a1f249fb [py3k] dict.itervalues → dict.values
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10662
diff changeset
   162
        for groupdict in self.group_source_entities_by_extid.values():
10912
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   163
            attrs = self.ldap2cwattrs(groupdict, 'CWGroup')
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   164
            extgroup = importer.ExtEntity('CWGroup', groupdict['dn'], attrs)
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   165
            yield extgroup
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   166
            # record group membership for later insertion
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   167
            members = groupdict.get(self.source.group_rev_attrs['member'], ())
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   168
            self._group_members[attrs['name']] = members
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   169
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   170
    def _process_email(self, extuser, userdict):
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   171
        try:
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   172
            emailaddrs = userdict.pop(self.source.user_rev_attrs['email'])
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   173
        except KeyError:
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   174
            return  # no email for that user, nothing to do
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   175
        if not isinstance(emailaddrs, list):
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   176
            emailaddrs = [emailaddrs]
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   177
        for emailaddr in emailaddrs:
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   178
            # search for existing email first, may be coming from another source
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   179
            rset = self._cw.execute('EmailAddress X WHERE X address %(addr)s',
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   180
                                    {'addr': emailaddr})
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   181
            if not rset:
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   182
                # not found, create it. first forge an external id
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   183
                emailextid = userdict['dn'] + '@@' + emailaddr
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   184
                extuser.values.setdefault('use_email', []).append(emailextid)
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   185
                yield importer.ExtEntity('EmailAddress', emailextid, dict(address=[emailaddr]))
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   186
            elif self.sourceuris:
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   187
                # pop from sourceuris anyway, else email may be removed by the
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   188
                # source once import is finished
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   189
                emailextid = userdict['dn'] + '@@' + emailaddr
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   190
                self.sourceuris.pop(emailextid, None)
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   191
            # XXX else check use_email relation?
8430
5bee87a14bb1 fix ldap removal handling in ldapfeed (closes #2376625 and #2385133)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8427
diff changeset
   192
9879
21278eb03bbf [datafeed sources] finish the session -> cnx switch
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9551
diff changeset
   193
    def handle_deletion(self, config, cnx, myuris):
8430
5bee87a14bb1 fix ldap removal handling in ldapfeed (closes #2376625 and #2385133)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8427
diff changeset
   194
        if config['delete-entities']:
9879
21278eb03bbf [datafeed sources] finish the session -> cnx switch
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9551
diff changeset
   195
            super(DataFeedLDAPAdapter, self).handle_deletion(config, cnx, myuris)
8432
96b4f7a35e6c [ldapparser] missing return
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8431
diff changeset
   196
            return
8430
5bee87a14bb1 fix ldap removal handling in ldapfeed (closes #2376625 and #2385133)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8427
diff changeset
   197
        if myuris:
10662
10942ed172de [py3k] dict.iteritems → dict.items
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10089
diff changeset
   198
            for extid, (eid, etype) in myuris.items():
10909
53af91f77b9d [ldapfeed] simplify deactivation of unencountered users
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10907
diff changeset
   199
                if etype != 'CWUser' or not self.is_deleted(extid, etype, eid):
8430
5bee87a14bb1 fix ldap removal handling in ldapfeed (closes #2376625 and #2385133)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8427
diff changeset
   200
                    continue
10909
53af91f77b9d [ldapfeed] simplify deactivation of unencountered users
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10907
diff changeset
   201
                self.info('deactivate user %s', eid)
53af91f77b9d [ldapfeed] simplify deactivation of unencountered users
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10907
diff changeset
   202
                wf = cnx.entity_from_eid(eid).cw_adapt_to('IWorkflowable')
53af91f77b9d [ldapfeed] simplify deactivation of unencountered users
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10907
diff changeset
   203
                wf.fire_transition_if_possible('deactivate')
9879
21278eb03bbf [datafeed sources] finish the session -> cnx switch
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9551
diff changeset
   204
        cnx.commit()
8430
5bee87a14bb1 fix ldap removal handling in ldapfeed (closes #2376625 and #2385133)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8427
diff changeset
   205
10912
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   206
    def ensure_activated(self, entity):
8900
010a59e12d89 use cw_etype instead of __regid__
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8888
diff changeset
   207
        if entity.cw_etype == 'CWUser':
8434
39c5bb4dcc59 [ldapfeed] do not crash on ldap user deletion + pull + already deactivated users, cleanups (closes #2392933)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8432
diff changeset
   208
            wf = entity.cw_adapt_to('IWorkflowable')
39c5bb4dcc59 [ldapfeed] do not crash on ldap user deletion + pull + already deactivated users, cleanups (closes #2392933)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8432
diff changeset
   209
            if wf.state == 'deactivated':
8639
2fddbe32ae8b [ldapfeed] if a deactivated user becomes available again in its source, reactivate it (closes #2542776)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8638
diff changeset
   210
                wf.fire_transition('activate')
9219
6afdeaabac74 [ldapparser] demote some logs from warning to debug (closes #2713671)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8922
diff changeset
   211
                self.info('user %s reactivated', entity.login)
8434
39c5bb4dcc59 [ldapfeed] do not crash on ldap user deletion + pull + already deactivated users, cleanups (closes #2392933)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8432
diff changeset
   212
10910
5ba4de264be4 [ldapfeed] simplify ldap2attrs
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10909
diff changeset
   213
    def ldap2cwattrs(self, sdict, etype):
5ba4de264be4 [ldapfeed] simplify ldap2attrs
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10909
diff changeset
   214
        """Transform dictionary of LDAP attributes to CW.
5ba4de264be4 [ldapfeed] simplify ldap2attrs
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10909
diff changeset
   215
5ba4de264be4 [ldapfeed] simplify ldap2attrs
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10909
diff changeset
   216
        etype must be CWUser or CWGroup
5ba4de264be4 [ldapfeed] simplify ldap2attrs
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10909
diff changeset
   217
        """
5ba4de264be4 [ldapfeed] simplify ldap2attrs
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10909
diff changeset
   218
        assert etype in ('CWUser', 'CWGroup'), etype
5ba4de264be4 [ldapfeed] simplify ldap2attrs
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10909
diff changeset
   219
        tdict = {}
8920
386049566ceb [ldap] prepare import of CWGroup
David Douard <david.douard@logilab.fr>
parents: 8919
diff changeset
   220
        if etype == 'CWUser':
10662
10942ed172de [py3k] dict.iteritems → dict.items
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10089
diff changeset
   221
            items = self.source.user_attrs.items()
8922
715b9eec6da9 [ldapfeed] Add support for LDAP groups (closes #2528116)
David Douard <david.douard@logilab.fr>
parents: 8920
diff changeset
   222
        elif etype == 'CWGroup':
10662
10942ed172de [py3k] dict.iteritems → dict.items
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10089
diff changeset
   223
            items = self.source.group_attrs.items()
8920
386049566ceb [ldap] prepare import of CWGroup
David Douard <david.douard@logilab.fr>
parents: 8919
diff changeset
   224
        for sattr, tattr in items:
8250
171a9d6bff8f [ldapfeed] fix synchronisation crash: ldap attributes are given while we want cw attributes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8188
diff changeset
   225
            if tattr not in self.non_attribute_keys:
8566
76bcfb3c483d [ldapparser] raise specific error if the configuration is wrong (closes #2498164)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8479
diff changeset
   226
                try:
10910
5ba4de264be4 [ldapfeed] simplify ldap2attrs
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10909
diff changeset
   227
                    value = sdict[sattr]
8566
76bcfb3c483d [ldapparser] raise specific error if the configuration is wrong (closes #2498164)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8479
diff changeset
   228
                except KeyError:
10910
5ba4de264be4 [ldapfeed] simplify ldap2attrs
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10909
diff changeset
   229
                    raise ConfigurationError(
5ba4de264be4 [ldapfeed] simplify ldap2attrs
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10909
diff changeset
   230
                        'source attribute %s has not been found in the source, '
5ba4de264be4 [ldapfeed] simplify ldap2attrs
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10909
diff changeset
   231
                        'please check the %s-attrs-map field and the permissions of '
5ba4de264be4 [ldapfeed] simplify ldap2attrs
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10909
diff changeset
   232
                        'the LDAP binding user' % (sattr, etype[2:].lower()))
10912
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   233
                if not isinstance(value, list):
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   234
                    value = [value]
10910
5ba4de264be4 [ldapfeed] simplify ldap2attrs
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10909
diff changeset
   235
                tdict[tattr] = value
8250
171a9d6bff8f [ldapfeed] fix synchronisation crash: ldap attributes are given while we want cw attributes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8188
diff changeset
   236
        return tdict
171a9d6bff8f [ldapfeed] fix synchronisation crash: ldap attributes are given while we want cw attributes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8188
diff changeset
   237
8638
9f95c2368b8b [ldaputils,ldapparser] correctly compute "deleted" users (closes #2542083)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8633
diff changeset
   238
    def is_deleted(self, extidplus, etype, eid):
8188
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   239
        try:
10906
e1ce0866afe9 [ldapparser] simplify extid splitting
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10768
diff changeset
   240
            extid = extidplus.rsplit(b'@@', 1)[0]
8188
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   241
        except ValueError:
8638
9f95c2368b8b [ldaputils,ldapparser] correctly compute "deleted" users (closes #2542083)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8633
diff changeset
   242
            # for some reason extids here tend to come in both forms, e.g:
9f95c2368b8b [ldaputils,ldapparser] correctly compute "deleted" users (closes #2542083)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8633
diff changeset
   243
            # dn, dn@@Babar
9f95c2368b8b [ldaputils,ldapparser] correctly compute "deleted" users (closes #2542083)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8633
diff changeset
   244
            extid = extidplus
8920
386049566ceb [ldap] prepare import of CWGroup
David Douard <david.douard@logilab.fr>
parents: 8919
diff changeset
   245
        return extid not in self.user_source_entities_by_extid
8188
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   246
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   247
    @cached
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   248
    def _get_group(self, name):
8679
cf4dacc80976 [ldapfeed] don't crash if one specify an unexisting group in the configuration. Closes #2538399
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8639
diff changeset
   249
        try:
cf4dacc80976 [ldapfeed] don't crash if one specify an unexisting group in the configuration. Closes #2538399
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8639
diff changeset
   250
            return self._cw.execute('Any X WHERE X is CWGroup, X name %(name)s',
10912
addc7ba36f69 [ldapfeed / dataimport] port ldapfeed parser to dataimport API
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10910
diff changeset
   251
                                    {'name': name})[0][0]
8679
cf4dacc80976 [ldapfeed] don't crash if one specify an unexisting group in the configuration. Closes #2538399
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8639
diff changeset
   252
        except IndexError:
8888
738f97bc3e19 [ldap] Use correct API for logging message
Jérôme Roy <jerome.roy@logilab.fr>
parents: 8694
diff changeset
   253
            self.error('group %r referenced by source configuration %r does not exist',
738f97bc3e19 [ldap] Use correct API for logging message
Jérôme Roy <jerome.roy@logilab.fr>
parents: 8694
diff changeset
   254
                       name, self.source.uri)
8679
cf4dacc80976 [ldapfeed] don't crash if one specify an unexisting group in the configuration. Closes #2538399
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8639
diff changeset
   255
            return None