misc/scripts/ldapuser2ldapfeed.py
author Aurelien Campeas <aurelien.campeas@logilab.fr>
Wed, 08 Jan 2014 14:00:31 +0100
changeset 9377 4e0d8f06efbc
parent 9267 24d9b86dfa54
child 9492 c7fc56eecd1a
permissions -rw-r--r--
[js/widgets] fix the InOut widget with modern jQuery versions Several things are done there: * reduction in size and complexity of the code * the unused defaultsettings are removed * the initial `unlinked` list is now correctly populated from python-side * the unit test is adjusted because it tested an irrelevant implementation detail which is no longer true (but the widget of course still handles correctly the linkto information) Tested with ie7, ie9, chromium, firefox. Tested with jQuery 1.6 (cw 3.17.x) and 1.10. Closes #3154531.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
8188
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     1
"""turn a pyro source into a datafeed source
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     2
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     3
Once this script is run, execute c-c db-check to cleanup relation tables.
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     4
"""
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     5
import sys
8468
f52bb4226020 [ldapuser2ldapfeed] fix confusing script structure and decode the extid to avoid an UnicodeDecodeError (closes #2413437)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8188
diff changeset
     6
from collections import defaultdict
8521
dfdffebce8a4 ldapuser2ldapfeed: create CWUsers with random passwords, not empty ones
Julien Cristau <julien.cristau@logilab.fr>
parents: 8468
diff changeset
     7
from logilab.common.shellutils import generate_password
8188
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     8
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     9
try:
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    10
    source_name, = __args__
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    11
    source = repo.sources_by_uri[source_name]
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    12
except ValueError:
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    13
    print('you should specify the source name as script argument (i.e. after --'
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    14
          ' on the command line)')
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    15
    sys.exit(1)
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    16
except KeyError:
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    17
    print '%s is not an active source' % source_name
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    18
    sys.exit(1)
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    19
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    20
# check source is reachable before doing anything
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    21
if not source.get_connection().cnx:
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    22
    print '%s is not reachable. Fix this before running this script' % source_name
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    23
    sys.exit(1)
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    24
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    25
raw_input('Ensure you have shutdown all instances of this application before continuing.'
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    26
          ' Type enter when ready.')
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    27
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    28
system_source = repo.system_source
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    29
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    30
from datetime import datetime
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    31
from cubicweb.server.edition import EditedEntity
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    32
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    33
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    34
session.mode = 'write' # hold on the connections set
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    35
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    36
print '******************** backport entity content ***************************'
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    37
8468
f52bb4226020 [ldapuser2ldapfeed] fix confusing script structure and decode the extid to avoid an UnicodeDecodeError (closes #2413437)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8188
diff changeset
    38
todelete = defaultdict(list)
f52bb4226020 [ldapuser2ldapfeed] fix confusing script structure and decode the extid to avoid an UnicodeDecodeError (closes #2413437)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8188
diff changeset
    39
extids = set()
f52bb4226020 [ldapuser2ldapfeed] fix confusing script structure and decode the extid to avoid an UnicodeDecodeError (closes #2413437)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8188
diff changeset
    40
duplicates = []
8188
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    41
for entity in rql('Any X WHERE X cw_source S, S eid %(s)s', {'s': source.eid}).entities():
8900
010a59e12d89 use cw_etype instead of __regid__
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8645
diff changeset
    42
    etype = entity.cw_etype
8468
f52bb4226020 [ldapuser2ldapfeed] fix confusing script structure and decode the extid to avoid an UnicodeDecodeError (closes #2413437)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8188
diff changeset
    43
    if not source.support_entity(etype):
f52bb4226020 [ldapuser2ldapfeed] fix confusing script structure and decode the extid to avoid an UnicodeDecodeError (closes #2413437)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8188
diff changeset
    44
        print "source doesn't support %s, delete %s" % (etype, entity.eid)
f52bb4226020 [ldapuser2ldapfeed] fix confusing script structure and decode the extid to avoid an UnicodeDecodeError (closes #2413437)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8188
diff changeset
    45
        todelete[etype].append(entity)
f52bb4226020 [ldapuser2ldapfeed] fix confusing script structure and decode the extid to avoid an UnicodeDecodeError (closes #2413437)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8188
diff changeset
    46
        continue
f52bb4226020 [ldapuser2ldapfeed] fix confusing script structure and decode the extid to avoid an UnicodeDecodeError (closes #2413437)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8188
diff changeset
    47
    try:
f52bb4226020 [ldapuser2ldapfeed] fix confusing script structure and decode the extid to avoid an UnicodeDecodeError (closes #2413437)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8188
diff changeset
    48
        entity.complete()
f52bb4226020 [ldapuser2ldapfeed] fix confusing script structure and decode the extid to avoid an UnicodeDecodeError (closes #2413437)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8188
diff changeset
    49
    except Exception:
f52bb4226020 [ldapuser2ldapfeed] fix confusing script structure and decode the extid to avoid an UnicodeDecodeError (closes #2413437)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8188
diff changeset
    50
        print '%s %s much probably deleted, delete it (extid %s)' % (
f52bb4226020 [ldapuser2ldapfeed] fix confusing script structure and decode the extid to avoid an UnicodeDecodeError (closes #2413437)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8188
diff changeset
    51
            etype, entity.eid, entity.cw_metainformation()['extid'])
f52bb4226020 [ldapuser2ldapfeed] fix confusing script structure and decode the extid to avoid an UnicodeDecodeError (closes #2413437)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8188
diff changeset
    52
        todelete[etype].append(entity)
f52bb4226020 [ldapuser2ldapfeed] fix confusing script structure and decode the extid to avoid an UnicodeDecodeError (closes #2413437)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8188
diff changeset
    53
        continue
f52bb4226020 [ldapuser2ldapfeed] fix confusing script structure and decode the extid to avoid an UnicodeDecodeError (closes #2413437)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8188
diff changeset
    54
    print 'get back', etype, entity.eid
f52bb4226020 [ldapuser2ldapfeed] fix confusing script structure and decode the extid to avoid an UnicodeDecodeError (closes #2413437)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8188
diff changeset
    55
    entity.cw_edited = EditedEntity(entity, **entity.cw_attr_cache)
f52bb4226020 [ldapuser2ldapfeed] fix confusing script structure and decode the extid to avoid an UnicodeDecodeError (closes #2413437)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8188
diff changeset
    56
    if not entity.creation_date:
f52bb4226020 [ldapuser2ldapfeed] fix confusing script structure and decode the extid to avoid an UnicodeDecodeError (closes #2413437)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8188
diff changeset
    57
        entity.cw_edited['creation_date'] = datetime.now()
f52bb4226020 [ldapuser2ldapfeed] fix confusing script structure and decode the extid to avoid an UnicodeDecodeError (closes #2413437)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8188
diff changeset
    58
    if not entity.modification_date:
f52bb4226020 [ldapuser2ldapfeed] fix confusing script structure and decode the extid to avoid an UnicodeDecodeError (closes #2413437)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8188
diff changeset
    59
        entity.cw_edited['modification_date'] = datetime.now()
f52bb4226020 [ldapuser2ldapfeed] fix confusing script structure and decode the extid to avoid an UnicodeDecodeError (closes #2413437)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8188
diff changeset
    60
    if not entity.upassword:
8521
dfdffebce8a4 ldapuser2ldapfeed: create CWUsers with random passwords, not empty ones
Julien Cristau <julien.cristau@logilab.fr>
parents: 8468
diff changeset
    61
        entity.cw_edited['upassword'] = generate_password()
8468
f52bb4226020 [ldapuser2ldapfeed] fix confusing script structure and decode the extid to avoid an UnicodeDecodeError (closes #2413437)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8188
diff changeset
    62
    extid = entity.cw_metainformation()['extid']
f52bb4226020 [ldapuser2ldapfeed] fix confusing script structure and decode the extid to avoid an UnicodeDecodeError (closes #2413437)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8188
diff changeset
    63
    if not entity.cwuri:
f52bb4226020 [ldapuser2ldapfeed] fix confusing script structure and decode the extid to avoid an UnicodeDecodeError (closes #2413437)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8188
diff changeset
    64
        entity.cw_edited['cwuri'] = '%s/?dn=%s' % (
f52bb4226020 [ldapuser2ldapfeed] fix confusing script structure and decode the extid to avoid an UnicodeDecodeError (closes #2413437)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8188
diff changeset
    65
            source.urls[0], extid.decode('utf-8', 'ignore'))
f52bb4226020 [ldapuser2ldapfeed] fix confusing script structure and decode the extid to avoid an UnicodeDecodeError (closes #2413437)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8188
diff changeset
    66
    print entity.cw_edited
f52bb4226020 [ldapuser2ldapfeed] fix confusing script structure and decode the extid to avoid an UnicodeDecodeError (closes #2413437)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8188
diff changeset
    67
    if extid in extids:
f52bb4226020 [ldapuser2ldapfeed] fix confusing script structure and decode the extid to avoid an UnicodeDecodeError (closes #2413437)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8188
diff changeset
    68
        duplicates.append(extid)
f52bb4226020 [ldapuser2ldapfeed] fix confusing script structure and decode the extid to avoid an UnicodeDecodeError (closes #2413437)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8188
diff changeset
    69
        continue
f52bb4226020 [ldapuser2ldapfeed] fix confusing script structure and decode the extid to avoid an UnicodeDecodeError (closes #2413437)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8188
diff changeset
    70
    extids.add(extid)
f52bb4226020 [ldapuser2ldapfeed] fix confusing script structure and decode the extid to avoid an UnicodeDecodeError (closes #2413437)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8188
diff changeset
    71
    system_source.add_entity(session, entity)
f52bb4226020 [ldapuser2ldapfeed] fix confusing script structure and decode the extid to avoid an UnicodeDecodeError (closes #2413437)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8188
diff changeset
    72
    sql("UPDATE entities SET source='system' "
f52bb4226020 [ldapuser2ldapfeed] fix confusing script structure and decode the extid to avoid an UnicodeDecodeError (closes #2413437)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8188
diff changeset
    73
        "WHERE eid=%(eid)s", {'eid': entity.eid})
8188
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    74
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    75
# only cleanup entities table, remaining stuff should be cleaned by a c-c
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    76
# db-check to be run after this script
8468
f52bb4226020 [ldapuser2ldapfeed] fix confusing script structure and decode the extid to avoid an UnicodeDecodeError (closes #2413437)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8188
diff changeset
    77
if duplicates:
f52bb4226020 [ldapuser2ldapfeed] fix confusing script structure and decode the extid to avoid an UnicodeDecodeError (closes #2413437)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8188
diff changeset
    78
    print 'found %s duplicate entries' % len(duplicates)
f52bb4226020 [ldapuser2ldapfeed] fix confusing script structure and decode the extid to avoid an UnicodeDecodeError (closes #2413437)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8188
diff changeset
    79
    from pprint import pprint
f52bb4226020 [ldapuser2ldapfeed] fix confusing script structure and decode the extid to avoid an UnicodeDecodeError (closes #2413437)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8188
diff changeset
    80
    pprint(duplicates)
f52bb4226020 [ldapuser2ldapfeed] fix confusing script structure and decode the extid to avoid an UnicodeDecodeError (closes #2413437)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8188
diff changeset
    81
f52bb4226020 [ldapuser2ldapfeed] fix confusing script structure and decode the extid to avoid an UnicodeDecodeError (closes #2413437)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8188
diff changeset
    82
print len(todelete), 'entities will be deleted'
8635
c261d9465e65 [ldapuser2ldapfeed] .values -> .iteritems (closes #2542872)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8521
diff changeset
    83
for etype, entities in todelete.iteritems():
8468
f52bb4226020 [ldapuser2ldapfeed] fix confusing script structure and decode the extid to avoid an UnicodeDecodeError (closes #2413437)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8188
diff changeset
    84
    print 'deleting', etype, [e.login for e in entities]
8188
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    85
    system_source.delete_info_multi(session, entities, source_name)
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    86
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    87
8468
f52bb4226020 [ldapuser2ldapfeed] fix confusing script structure and decode the extid to avoid an UnicodeDecodeError (closes #2413437)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8188
diff changeset
    88
8188
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    89
source_ent = rql('CWSource S WHERE S eid %(s)s', {'s': source.eid}).get_entity(0, 0)
8483
4ba11607d84a [entity api] unify set_attributes / set_relations into a cw_set method. Closes #2423719
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8188
diff changeset
    90
source_ent.cw_set(type=u"ldapfeed", parser=u"ldapfeed")
8188
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    91
1867e252e487 [repository] ldap-feed source. Closes #2086984
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    92
8468
f52bb4226020 [ldapuser2ldapfeed] fix confusing script structure and decode the extid to avoid an UnicodeDecodeError (closes #2413437)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8188
diff changeset
    93
if raw_input('Commit ?') in 'yY':
f52bb4226020 [ldapuser2ldapfeed] fix confusing script structure and decode the extid to avoid an UnicodeDecodeError (closes #2413437)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8188
diff changeset
    94
    print 'committing'
f52bb4226020 [ldapuser2ldapfeed] fix confusing script structure and decode the extid to avoid an UnicodeDecodeError (closes #2413437)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8188
diff changeset
    95
    commit()
f52bb4226020 [ldapuser2ldapfeed] fix confusing script structure and decode the extid to avoid an UnicodeDecodeError (closes #2413437)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8188
diff changeset
    96
else:
f52bb4226020 [ldapuser2ldapfeed] fix confusing script structure and decode the extid to avoid an UnicodeDecodeError (closes #2413437)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8188
diff changeset
    97
    rollback()
9267
24d9b86dfa54 spelling: rollbacked -> rolled back
Rémi Cardona <remi.cardona@logilab.fr>
parents: 8900
diff changeset
    98
    print 'rolled back'
8468
f52bb4226020 [ldapuser2ldapfeed] fix confusing script structure and decode the extid to avoid an UnicodeDecodeError (closes #2413437)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8188
diff changeset
    99