# HG changeset patch # User Sylvain Thénault # Date 1390481248 -3600 # Node ID 032825bbacab756f2ceb4cbaa7c4545f05f47036 # Parent 39b7a91a3f4c4bcc2ca216a979ec6659f38c25b7 [multi-sources-removal] Drop entities.source column Since we remove federated multi-sources support, the same value ('system') is always stored in this column and so could be removed. Also, cleanup repository caches and a few api where the very same useless information could be removed. Closes #2919300, at last [jcr: restore sanity check of etype against schema in checkintegrity.has_eid] diff -r 39b7a91a3f4c -r 032825bbacab cwvreg.py --- a/cwvreg.py Thu Jan 23 12:32:16 2014 +0100 +++ b/cwvreg.py Thu Jan 23 13:47:28 2014 +0100 @@ -701,7 +701,7 @@ def solutions(self, req, rqlst, args): def type_from_eid(eid, req=req): - return req.describe(eid)[0] + return req.entity_metas(eid)['type'] return self.rqlhelper.compute_solutions(rqlst, {'eid': type_from_eid}, args) def parse(self, req, rql, args=None): diff -r 39b7a91a3f4c -r 032825bbacab dbapi.py --- a/dbapi.py Thu Jan 23 12:32:16 2014 +0100 +++ b/dbapi.py Thu Jan 23 13:47:28 2014 +0100 @@ -91,8 +91,6 @@ self.close_on_del = close - - @deprecated('[3.19] the dbapi is deprecated. Have a look at the new repoapi.') def _repo_connect(repo, login, **kwargs): """Constructor to create a new connection to the given CubicWeb repository. @@ -360,14 +358,19 @@ # server session compat layer ############################################# - def describe(self, eid, asdict=False): + def entity_metas(self, eid): """return a tuple (type, sourceuri, extid) for the entity with id """ - return self.cnx.describe(eid, asdict) + return self.cnx.entity_metas(eid) def source_defs(self): """return the definition of sources used by the repository.""" return self.cnx.source_defs() + @deprecated('[3.19] use .entity_metas(eid) instead') + def describe(self, eid, asdict=False): + """return a tuple (type, sourceuri, extid) for the entity with id """ + return self.cnx.describe(eid, asdict) + # these are overridden by set_log_methods below # only defining here to prevent pylint from complaining info = warning = error = critical = exception = debug = lambda msg,*a,**kw: None @@ -671,16 +674,31 @@ stacklevel=2) return self._repo.get_option_value(option) + + @check_not_closed + def entity_metas(self, eid): + """return a tuple (type, sourceuri, extid) for the entity with id """ + return self._repo.entity_metas(self.sessionid, eid, **self._txid()) + + @deprecated('[3.19] use .entity_metas(eid) instead') @check_not_closed def describe(self, eid, asdict=False): - metas = self._repo.describe(self.sessionid, eid, **self._txid()) - if len(metas) == 3: # backward compat - metas = list(metas) - metas.append(metas[1]) + try: + metas = self._repo.entity_metas(self.sessionid, eid, **self._txid()) + except AttributeError: + metas = self._repo.describe(self.sessionid, eid, **self._txid()) + # talking to pre 3.19 repository + if len(metas) == 3: # even older backward compat + metas = list(metas) + metas.append(metas[1]) + if asdict: + return dict(zip(('type', 'source', 'extid', 'asource'), metas)) + return metas[:-1] if asdict: - return dict(zip(('type', 'source', 'extid', 'asource'), metas)) - # XXX :-1 for cw compat, use asdict=True for full information - return metas[:-1] + metas['asource'] = meta['source'] # XXX pre 3.19 client compat + return metas + return metas['type'], metas['source'], metas['extid'] + # db-api like interface #################################################### diff -r 39b7a91a3f4c -r 032825bbacab devtools/fake.py --- a/devtools/fake.py Thu Jan 23 12:32:16 2014 +0100 +++ b/devtools/fake.py Thu Jan 23 13:47:28 2014 +0100 @@ -191,9 +191,6 @@ source.after_entity_insertion(session, extid, entity) return eid - def eid2extid(self, source, eid, session=None): - return self.eids[eid] - class FakeSource(object): dbhelper = get_db_helper('sqlite') diff -r 39b7a91a3f4c -r 032825bbacab entity.py --- a/entity.py Thu Jan 23 12:32:16 2014 +0100 +++ b/entity.py Thu Jan 23 13:47:28 2014 +0100 @@ -633,11 +633,9 @@ @cached def cw_metainformation(self): - res = self._cw.describe(self.eid, asdict=True) - # use 'asource' and not 'source' since this is the actual source, - # while 'source' is the physical source (where it's stored) - res['source'] = self._cw.source_defs()[res.pop('asource')] - return res + metas = self._cw.entity_metas(self.eid) + metas['source'] = self._cw.source_defs()[metas['source']] + return metas def cw_check_perm(self, action): self.e_schema.check_perm(self._cw, action, eid=self.eid) diff -r 39b7a91a3f4c -r 032825bbacab hooks/integrity.py --- a/hooks/integrity.py Thu Jan 23 12:32:16 2014 +0100 +++ b/hooks/integrity.py Thu Jan 23 13:47:28 2014 +0100 @@ -85,7 +85,7 @@ if rtype in pendingrtypes: continue if not session.execute(self.base_rql % rtype, {'x': eid}): - etype = session.describe(eid)[0] + etype = session.entity_metas(eid)['type'] msg = _('at least one relation %(rtype)s is required on ' '%(etype)s (%(eid)s)') raise validation_error(eid, {(rtype, self.role): msg}, @@ -325,7 +325,7 @@ for eid, rtype in self.get_data(): # don't do anything if the entity is being deleted if eid not in pendingeids: - etype = session.describe(eid)[0] + etype = session.entity_metas(eid)['type'] key = (etype, rtype) if key not in eids_by_etype_rtype: eids_by_etype_rtype[key] = [str(eid)] diff -r 39b7a91a3f4c -r 032825bbacab hooks/metadata.py --- a/hooks/metadata.py Thu Jan 23 12:32:16 2014 +0100 +++ b/hooks/metadata.py Thu Jan 23 13:47:28 2014 +0100 @@ -1,4 +1,4 @@ -# copyright 2003-2012 LOGILAB S.A. (Paris, FRANCE), all rights reserved. +# copyright 2003-2013 LOGILAB S.A. (Paris, FRANCE), all rights reserved. # contact http://www.logilab.fr/ -- mailto:contact@logilab.fr # # This file is part of CubicWeb. @@ -158,8 +158,8 @@ entity = self.entity extid = entity.cw_metainformation()['extid'] repo._type_source_cache[entity.eid] = ( - entity.cw_etype, self.newsource.uri, None, self.newsource.uri) - repo._extid_cache[(extid, 'system')] = -entity.eid + entity.cw_etype, None, self.newsource.uri) + repo._extid_cache[extid] = -entity.eid class ChangeEntitySourceDeleteHook(MetaDataHook): @@ -204,7 +204,7 @@ self._cw.system_sql('UPDATE entities SET eid=-eid WHERE eid=%(eid)s', {'eid': self.eidfrom}) attrs = {'type': entity.cw_etype, 'eid': entity.eid, 'extid': None, - 'source': 'system', 'asource': 'system'} + 'asource': 'system'} self._cw.system_sql(syssource.sqlgen.insert('entities', attrs), attrs) # register an operation to update repository/sources caches ChangeEntitySourceUpdateCaches(self._cw, entity=entity, diff -r 39b7a91a3f4c -r 032825bbacab hooks/security.py --- a/hooks/security.py Thu Jan 23 12:32:16 2014 +0100 +++ b/hooks/security.py Thu Jan 23 13:47:28 2014 +0100 @@ -78,8 +78,8 @@ def precommit_event(self): session = self.session for action, rschema, eidfrom, eidto in self.get_data(): - rdef = rschema.rdef(session.describe(eidfrom)[0], - session.describe(eidto)[0]) + rdef = rschema.rdef(session.entity_metas(eidfrom)['type'], + session.entity_metas(eidto)['type']) rdef.check_perm(session, action, fromeid=eidfrom, toeid=eidto) @@ -134,8 +134,8 @@ if (self.eidfrom, self.rtype, self.eidto) in nocheck: return rschema = self._cw.repo.schema[self.rtype] - rdef = rschema.rdef(self._cw.describe(self.eidfrom)[0], - self._cw.describe(self.eidto)[0]) + rdef = rschema.rdef(self._cw.entity_metas(self.eidfrom)['type'], + self._cw.entity_metas(self.eidto)['type']) rdef.check_perm(self._cw, 'add', fromeid=self.eidfrom, toeid=self.eidto) @@ -153,8 +153,8 @@ CheckRelationPermissionOp.get_instance(self._cw).add_data( ('add', rschema, self.eidfrom, self.eidto) ) else: - rdef = rschema.rdef(self._cw.describe(self.eidfrom)[0], - self._cw.describe(self.eidto)[0]) + rdef = rschema.rdef(self._cw.entity_metas(self.eidfrom)['type'], + self._cw.entity_metas(self.eidto)['type']) rdef.check_perm(self._cw, 'add', fromeid=self.eidfrom, toeid=self.eidto) @@ -167,7 +167,7 @@ if (self.eidfrom, self.rtype, self.eidto) in nocheck: return rschema = self._cw.repo.schema[self.rtype] - rdef = rschema.rdef(self._cw.describe(self.eidfrom)[0], - self._cw.describe(self.eidto)[0]) + rdef = rschema.rdef(self._cw.entity_metas(self.eidfrom)['type'], + self._cw.entity_metas(self.eidto)['type']) rdef.check_perm(self._cw, 'delete', fromeid=self.eidfrom, toeid=self.eidto) diff -r 39b7a91a3f4c -r 032825bbacab hooks/syncschema.py --- a/hooks/syncschema.py Thu Jan 23 12:32:16 2014 +0100 +++ b/hooks/syncschema.py Thu Jan 23 13:47:28 2014 +0100 @@ -294,9 +294,9 @@ self.info('renamed table %s to %s', oldname, newname) sqlexec('UPDATE entities SET type=%(newname)s WHERE type=%(oldname)s', {'newname': newname, 'oldname': oldname}) - for eid, (etype, uri, extid, auri) in self.session.repo._type_source_cache.items(): + for eid, (etype, extid, auri) in self.session.repo._type_source_cache.items(): if etype == oldname: - self.session.repo._type_source_cache[eid] = (newname, uri, extid, auri) + self.session.repo._type_source_cache[eid] = (newname, extid, auri) # XXX transaction records def precommit_event(self): @@ -1180,7 +1180,7 @@ def __call__(self): action = self.rtype.split('_', 1)[0] - if self._cw.describe(self.eidto)[0] == 'CWGroup': + if self._cw.entity_metas(self.eidto)['type'] == 'CWGroup': MemSchemaPermissionAdd(self._cw, action=action, eid=self.eidfrom, group_eid=self.eidto) else: # RQLExpression @@ -1201,7 +1201,7 @@ if self._cw.deleted_in_transaction(self.eidfrom): return action = self.rtype.split('_', 1)[0] - if self._cw.describe(self.eidto)[0] == 'CWGroup': + if self._cw.entity_metas(self.eidto)['type'] == 'CWGroup': MemSchemaPermissionDel(self._cw, action=action, eid=self.eidfrom, group_eid=self.eidto) else: # RQLExpression diff -r 39b7a91a3f4c -r 032825bbacab hooks/syncsession.py --- a/hooks/syncsession.py Thu Jan 23 12:32:16 2014 +0100 +++ b/hooks/syncsession.py Thu Jan 23 13:47:28 2014 +0100 @@ -229,7 +229,7 @@ def __call__(self): session = self._cw eidfrom = self.eidfrom - if not session.describe(eidfrom)[0] == 'CWProperty': + if not session.entity_metas(eidfrom)['type'] == 'CWProperty': return key, value = session.execute('Any K,V WHERE P eid %(x)s,P pkey K,P value V', {'x': eidfrom})[0] diff -r 39b7a91a3f4c -r 032825bbacab hooks/workflow.py --- a/hooks/workflow.py Thu Jan 23 12:32:16 2014 +0100 +++ b/hooks/workflow.py Thu Jan 23 13:47:28 2014 +0100 @@ -32,12 +32,8 @@ nocheck = session.transaction_data.setdefault('skip-security', set()) nocheck.add((x, 'in_state', oldstate)) nocheck.add((x, 'in_state', newstate)) - # delete previous state first unless in_state isn't stored in the system - # source - fromsource = session.describe(x)[1] - if fromsource == 'system' or \ - not session.repo.sources_by_uri[fromsource].support_relation('in_state'): - session.delete_relation(x, 'in_state', oldstate) + # delete previous state first + session.delete_relation(x, 'in_state', oldstate) session.add_relation(x, 'in_state', newstate) diff -r 39b7a91a3f4c -r 032825bbacab misc/migration/3.19.0_Any.py --- a/misc/migration/3.19.0_Any.py Thu Jan 23 12:32:16 2014 +0100 +++ b/misc/migration/3.19.0_Any.py Thu Jan 23 13:47:28 2014 +0100 @@ -1,2 +1,4 @@ sql('DROP TABLE "deleted_entities"') sql('ALTER TABLE "entities" DROP COLUMN "mtime"') +sql('ALTER TABLE "entities" DROP COLUMN "source"') + diff -r 39b7a91a3f4c -r 032825bbacab repoapi.py --- a/repoapi.py Thu Jan 23 12:32:16 2014 +0100 +++ b/repoapi.py Thu Jan 23 13:47:28 2014 +0100 @@ -241,7 +241,8 @@ """Return the value for `option` in the configuration.""" return self._session.repo.get_option_value(option) - describe = _srv_cnx_func('describe') + entity_metas = _srv_cnx_func('entity_metas') + describe = _srv_cnx_func('describe') # XXX deprecated in 3.19 # undo support ############################################################ diff -r 39b7a91a3f4c -r 032825bbacab req.py --- a/req.py Thu Jan 23 12:32:16 2014 +0100 +++ b/req.py Thu Jan 23 13:47:28 2014 +0100 @@ -131,7 +131,7 @@ """ eid = int(eid) if etype is None: - etype = self.describe(eid)[0] + etype = self.entity_metas(eid)['type'] rset = ResultSet([(eid,)], 'Any X WHERE X eid %(x)s', {'x': eid}, [(etype,)]) rset.req = self diff -r 39b7a91a3f4c -r 032825bbacab schema.py --- a/schema.py Thu Jan 23 12:32:16 2014 +0100 +++ b/schema.py Thu Jan 23 13:47:28 2014 +0100 @@ -812,20 +812,20 @@ assert not ('fromeid' in kwargs or 'toeid' in kwargs), kwargs assert action in ('read', 'update') if 'eid' in kwargs: - subjtype = _cw.describe(kwargs['eid'])[0] + subjtype = _cw.entity_metas(kwargs['eid'])['type'] else: subjtype = objtype = None else: assert not 'eid' in kwargs, kwargs assert action in ('read', 'add', 'delete') if 'fromeid' in kwargs: - subjtype = _cw.describe(kwargs['fromeid'])[0] + subjtype = _cw.entity_metas(kwargs['fromeid'])['type'] elif 'frometype' in kwargs: subjtype = kwargs.pop('frometype') else: subjtype = None if 'toeid' in kwargs: - objtype = _cw.describe(kwargs['toeid'])[0] + objtype = _cw.entity_metas(kwargs['toeid'])['type'] elif 'toetype' in kwargs: objtype = kwargs.pop('toetype') else: diff -r 39b7a91a3f4c -r 032825bbacab server/checkintegrity.py --- a/server/checkintegrity.py Thu Jan 23 12:32:16 2014 +0100 +++ b/server/checkintegrity.py Thu Jan 23 13:47:28 2014 +0100 @@ -39,24 +39,12 @@ """return true if the eid is a valid eid""" if eid in eids: return eids[eid] - sqlcursor.execute('SELECT type, source FROM entities WHERE eid=%s' % eid) + sqlcursor.execute('SELECT type FROM entities WHERE eid=%s' % eid) try: - etype, source = sqlcursor.fetchone() + etype = sqlcursor.fetchone()[0] except Exception: eids[eid] = False return False - if source and source != 'system': - try: - # insert eid *and* etype to attempt checking entity has not been - # replaced by another subsquently to a restore of an old dump - if session.execute('Any X WHERE X is %s, X eid %%(x)s' % etype, - {'x': eid}): - eids[eid] = True - return True - except Exception: # TypeResolverError, Unauthorized... - pass - eids[eid] = False - return False if etype not in session.vreg.schema: eids[eid] = False return False diff -r 39b7a91a3f4c -r 032825bbacab server/hook.py --- a/server/hook.py Thu Jan 23 12:32:16 2014 +0100 +++ b/server/hook.py Thu Jan 23 13:47:28 2014 +0100 @@ -460,10 +460,10 @@ if kwargs.get('rtype') not in self.expected: return 0 if self.frometypes is not None and \ - req.describe(kwargs['eidfrom'])[0] not in self.frometypes: + req.entity_metas(kwargs['eidfrom'])['type'] not in self.frometypes: return 0 if self.toetypes is not None and \ - req.describe(kwargs['eidto'])[0] not in self.toetypes: + req.entity_metas(kwargs['eidto'])['type'] not in self.toetypes: return 0 return 1 @@ -604,7 +604,7 @@ def __call__(self): assert self.main_rtype for eid in (self.eidfrom, self.eidto): - etype = self._cw.describe(eid)[0] + etype = self._cw.entity_metas(eid)['type'] if self.main_rtype not in self._cw.vreg.schema.eschema(etype).subjrels: return if self.rtype in self.subject_relations: @@ -640,7 +640,7 @@ skip_object_relations = () def __call__(self): - eschema = self._cw.vreg.schema.eschema(self._cw.describe(self.eidfrom)[0]) + eschema = self._cw.vreg.schema.eschema(self._cw.entity_metas(self.eidfrom)['type']) execute = self._cw.execute for rel in self.subject_relations: if rel in eschema.subjrels and not rel in self.skip_subject_relations: @@ -664,7 +664,7 @@ events = ('after_delete_relation',) def __call__(self): - eschema = self._cw.vreg.schema.eschema(self._cw.describe(self.eidfrom)[0]) + eschema = self._cw.vreg.schema.eschema(self._cw.entity_metas(self.eidfrom)['type']) execute = self._cw.execute for rel in self.subject_relations: if rel in eschema.subjrels and not rel in self.skip_subject_relations: diff -r 39b7a91a3f4c -r 032825bbacab server/querier.py --- a/server/querier.py Thu Jan 23 12:32:16 2014 +0100 +++ b/server/querier.py Thu Jan 23 13:47:28 2014 +0100 @@ -71,7 +71,7 @@ try: return solution[term.name] except AttributeError: - return session.describe(term.eval(args))[0] + return session.entity_metas(term.eval(args))['type'] def check_read_access(session, rqlst, solution, args): """Check that the given user has credentials to access data read by the @@ -667,7 +667,7 @@ def _build_descr(tx, result, basedescription, todetermine): description = [] - etype_from_eid = tx.describe + entity_metas = tx.entity_metas todel = [] for i, row in enumerate(result): row_descr = basedescription[:] @@ -681,7 +681,7 @@ row_descr[index] = etype_from_pyobj(value) else: try: - row_descr[index] = etype_from_eid(value)[0] + row_descr[index] = entity_metas(value)['type'] except UnknownEid: tx.error('wrong eid %s in repository, you should ' 'db-check the database' % value) diff -r 39b7a91a3f4c -r 032825bbacab server/repository.py --- a/server/repository.py Thu Jan 23 12:32:16 2014 +0100 +++ b/server/repository.py Thu Jan 23 13:47:28 2014 +0100 @@ -188,9 +188,9 @@ self.sources_by_uri = {'system': self.system_source} # querier helper, need to be created after sources initialization self.querier = querier.QuerierHelper(self, self.schema) - # cache eid -> (type, physical source, extid, actual source) + # cache eid -> (type, extid, actual source) self._type_source_cache = {} - # cache (extid, source uri) -> eid + # cache extid -> eid self._extid_cache = {} # open some connection sets if config.init_cnxset_pool: @@ -728,13 +728,35 @@ finally: session.free_cnxset() + @deprecated('[3.19] use .entity_metas(sessionid, eid, txid) instead') def describe(self, sessionid, eid, txid=None): """return a tuple `(type, physical source uri, extid, actual source uri)` for the entity of the given `eid` + + As of 3.19, physical source uri is always the system source. """ session = self._get_session(sessionid, setcnxset=True, txid=txid) try: - return self.type_and_source_from_eid(eid, session) + etype, extid, source = self.type_and_source_from_eid(eid, session) + return etype, source, extid, source + finally: + session.free_cnxset() + + def entity_metas(self, sessionid, eid, txid=None): + """return a dictionary containing meta-datas for the entity of the given + `eid`. Available keys are: + + * 'type', the entity's type name, + + * 'source', the name of the source from which this entity's coming from, + + * 'extid', the identifierfor this entity in its originating source, as + an encoded string or `None` for entities from the 'system' source. + """ + session = self._get_session(sessionid, setcnxset=True, txid=txid) + try: + etype, extid, source = self.type_and_source_from_eid(eid, session) + return {'type': etype, 'source': source, 'extid': extid} finally: session.free_cnxset() @@ -934,8 +956,8 @@ # * correspondance between eid and local id (i.e. specific to a given source) def type_and_source_from_eid(self, eid, session=None): - """return a tuple `(type, physical source uri, extid, actual source - uri)` for the entity of the given `eid` + """return a tuple `(type, extid, actual source uri)` for the entity of + the given `eid` """ try: eid = int(eid) @@ -950,15 +972,13 @@ else: free_cnxset = False try: - etype, uri, extid, auri = self.system_source.eid_type_source( + etype, extid, auri = self.system_source.eid_type_source( session, eid) finally: if free_cnxset: session.free_cnxset() - self._type_source_cache[eid] = (etype, uri, extid, auri) - if uri != 'system': - self._extid_cache[(extid, uri)] = eid - return etype, uri, extid, auri + self._type_source_cache[eid] = (etype, extid, auri) + return etype, extid, auri def clear_caches(self, eids): etcache = self._type_source_cache @@ -966,9 +986,9 @@ rqlcache = self.querier._rql_cache for eid in eids: try: - etype, uri, extid, auri = etcache.pop(int(eid)) # may be a string in some cases + etype, extid, auri = etcache.pop(int(eid)) # may be a string in some cases rqlcache.pop( ('%s X WHERE X eid %s' % (etype, eid),), None) - extidcache.pop((extid, uri), None) + extidcache.pop(extid, None) except KeyError: etype = None rqlcache.pop( ('Any X WHERE X eid %s' % eid,), None) @@ -978,10 +998,6 @@ """return the type of the entity with id """ return self.type_and_source_from_eid(eid, session)[0] - def source_from_eid(self, eid, session=None): - """return the source for the given entity's eid""" - return self.sources_by_uri[self.type_and_source_from_eid(eid, session)[1]] - def querier_cache_key(self, session, rql, args, eidkeys): cachekey = [rql] for key in sorted(eidkeys): @@ -997,14 +1013,6 @@ args[key] = int(args[key]) return tuple(cachekey) - def eid2extid(self, source, eid, session=None): - """get local id from an eid""" - etype, uri, extid, _ = self.type_and_source_from_eid(eid, session) - if source.uri != uri: - # eid not from the given source - raise UnknownEid(eid) - return extid - def extid2eid(self, source, extid, etype, session=None, insert=True, complete=True, commit=True, sourceparams=None): """Return eid from a local id. If the eid is a negative integer, that @@ -1029,20 +1037,18 @@ 6. unless source's :attr:`should_call_hooks` tell otherwise, 'before_add_entity' hooks are called """ - uri = 'system' - cachekey = (extid, uri) try: - return self._extid_cache[cachekey] + return self._extid_cache[extid] except KeyError: pass free_cnxset = False if session is None: session = self.internal_session() free_cnxset = True - eid = self.system_source.extid2eid(session, uri, extid) + eid = self.system_source.extid2eid(session, extid) if eid is not None: - self._extid_cache[cachekey] = eid - self._type_source_cache[eid] = (etype, uri, extid, source.uri) + self._extid_cache[extid] = eid + self._type_source_cache[eid] = (etype, extid, source.uri) if free_cnxset: session.free_cnxset() return eid @@ -1059,8 +1065,8 @@ free_cnxset = True try: eid = self.system_source.create_eid(session) - self._extid_cache[cachekey] = eid - self._type_source_cache[eid] = (etype, uri, extid, source.uri) + self._extid_cache[extid] = eid + self._type_source_cache[eid] = (etype, extid, source.uri) entity = source.before_entity_insertion( session, extid, etype, eid, sourceparams) if source.should_call_hooks: @@ -1081,7 +1087,7 @@ else: # XXX do some cleanup manually so that the transaction has a # chance to be commited, with simply this entity discarded - self._extid_cache.pop(cachekey, None) + self._extid_cache.pop(extid, None) self._type_source_cache.pop(eid, None) if 'entity' in locals(): hook.CleanupDeletedEidsCacheOp.get_instance(session).add_data(entity.eid) @@ -1177,15 +1183,12 @@ Return entity's ext id if the source isn't the system source. """ session.set_entity_cache(entity) - suri = source.uri - if suri == 'system': + if source.uri == 'system': extid = None else: - suri = 'system' extid = source.get_extid(entity) - self._extid_cache[(str(extid), suri)] = entity.eid - self._type_source_cache[entity.eid] = (entity.cw_etype, suri, extid, - source.uri) + self._extid_cache[str(extid)] = entity.eid + self._type_source_cache[entity.eid] = (entity.cw_etype, extid, source.uri) return extid def glob_add_entity(self, session, edited): diff -r 39b7a91a3f4c -r 032825bbacab server/session.py --- a/server/session.py Thu Jan 23 12:32:16 2014 +0100 +++ b/server/session.py Thu Jan 23 13:47:28 2014 +0100 @@ -452,6 +452,7 @@ if cnxid is None: cnxid = '%s-%s' % (session.id, uuid4().hex) self.connectionid = cnxid + self.sessionid = session.id #: self._session_handled #: are the life cycle of this Connection automatically controlled by the #: Session This is the old backward compatibility mode @@ -788,7 +789,7 @@ rset.rows.append([targeteid]) if not isinstance(rset.description, list): # else description not set rset.description = list(rset.description) - rset.description.append([self.describe(targeteid)[0]]) + rset.description.append([self.entity_metas(targeteid)['type']]) targetentity = self.entity_from_eid(targeteid) if targetentity.cw_rset is None: targetentity.cw_rset = rset @@ -981,21 +982,24 @@ def source_defs(self): return self.repo.source_defs() + @deprecated('[3.19] use .entity_metas(eid) instead') @_with_cnx_set @_open_only def describe(self, eid, asdict=False): """return a tuple (type, sourceuri, extid) for the entity with id """ - metas = self.repo.type_and_source_from_eid(eid, self) + etype, extid, source = self.repo.type_and_source_from_eid(eid, self) + metas = {'type': etype, 'source': source, 'extid': extid} if asdict: - return dict(zip(('type', 'source', 'extid', 'asource'), metas)) - # XXX :-1 for cw compat, use asdict=True for full information - return metas[:-1] + metas['asource'] = meta['source'] # XXX pre 3.19 client compat + return meta + return etype, source, extid @_with_cnx_set @_open_only - def source_from_eid(self, eid): - """return the source where the entity with id is located""" - return self.repo.source_from_eid(eid, self) + def entity_metas(self, eid): + """return a tuple (type, sourceuri, extid) for the entity with id """ + etype, extid, source = self.repo.type_and_source_from_eid(eid, self) + return {'type': etype, 'source': source, 'extid': extid} # core method ############################################################# @@ -1589,8 +1593,8 @@ drop_entity_cache = cnx_meth('drop_entity_cache') source_defs = cnx_meth('source_defs') - describe = cnx_meth('describe') - source_from_eid = cnx_meth('source_from_eid') + entity_metas = cnx_meth('entity_metas') + describe = cnx_meth('describe') # XXX deprecated in 3.19 @deprecated('[3.19] use a Connection object instead') diff -r 39b7a91a3f4c -r 032825bbacab server/sources/__init__.py --- a/server/sources/__init__.py Thu Jan 23 12:32:16 2014 +0100 +++ b/server/sources/__init__.py Thu Jan 23 13:47:28 2014 +0100 @@ -450,10 +450,6 @@ raise NotImplementedError(self) - @deprecated('[3.13] use repo.eid2extid(source, eid, session)') - def eid2extid(self, eid, session=None): - return self.repo.eid2extid(self, eid, session) - @deprecated('[3.13] use extid2eid(source, value, etype, session, **kwargs)') def extid2eid(self, value, etype, session=None, **kwargs): return self.repo.extid2eid(self, value, etype, session, **kwargs) diff -r 39b7a91a3f4c -r 032825bbacab server/sources/native.py --- a/server/sources/native.py Thu Jan 23 12:32:16 2014 +0100 +++ b/server/sources/native.py Thu Jan 23 13:47:28 2014 +0100 @@ -592,7 +592,7 @@ else: # used by data import etypes = {} for subject, object in subj_obj_list: - etype = session.describe(subject)[0] + etype = session.entity_metas(subject)['type'] if etype in etypes: etypes[etype].append((subject, object)) else: @@ -617,7 +617,7 @@ def _delete_relation(self, session, subject, rtype, object, inlined=False): """delete a relation from the source""" if inlined: - table = SQL_PREFIX + session.describe(subject)[0] + table = SQL_PREFIX + session.entity_metas(subject)['type'] column = SQL_PREFIX + rtype sql = 'UPDATE %s SET %s=NULL WHERE %seid=%%(eid)s' % (table, column, SQL_PREFIX) @@ -764,7 +764,7 @@ def eid_type_source(self, session, eid): # pylint: disable=E0202 """return a tuple (type, source, extid) for the entity with id """ - sql = 'SELECT type, source, extid, asource FROM entities WHERE eid=%s' % eid + sql = 'SELECT type, extid, asource FROM entities WHERE eid=%s' % eid res = self._eid_type_source(session, eid, sql) if res[-2] is not None: if not isinstance(res, list): @@ -774,7 +774,7 @@ def eid_type_source_pre_131(self, session, eid): """return a tuple (type, source, extid) for the entity with id """ - sql = 'SELECT type, source, extid FROM entities WHERE eid=%s' % eid + sql = 'SELECT type, extid FROM entities WHERE eid=%s' % eid res = self._eid_type_source(session, eid, sql) if not isinstance(res, list): res = list(res) @@ -783,13 +783,12 @@ res.append(res[1]) return res - def extid2eid(self, session, source_uri, extid): + def extid2eid(self, session, extid): """get eid from an external id. Return None if no record found.""" assert isinstance(extid, str) cursor = self.doexec(session, - 'SELECT eid FROM entities ' - 'WHERE extid=%(x)s AND source=%(s)s', - {'x': b64encode(extid), 's': source_uri}) + 'SELECT eid FROM entities WHERE extid=%(x)s', + {'x': b64encode(extid)}) # XXX testing rowcount cause strange bug with sqlite, results are there # but rowcount is 0 #if cursor.rowcount > 0: @@ -863,7 +862,7 @@ assert isinstance(extid, str) extid = b64encode(extid) attrs = {'type': entity.cw_etype, 'eid': entity.eid, 'extid': extid, - 'source': 'system', 'asource': source.uri} + 'asource': source.uri} self._handle_insert_entity_sql(session, self.sqlgen.insert('entities', attrs), attrs) # insert core relations: is, is_instance_of and cw_source try: @@ -1349,7 +1348,6 @@ CREATE TABLE entities ( eid INTEGER PRIMARY KEY NOT NULL, type VARCHAR(64) NOT NULL, - source VARCHAR(128) NOT NULL, asource VARCHAR(128) NOT NULL, extid VARCHAR(256) );; diff -r 39b7a91a3f4c -r 032825bbacab server/ssplanner.py --- a/server/ssplanner.py Thu Jan 23 12:32:16 2014 +0100 +++ b/server/ssplanner.py Thu Jan 23 13:47:28 2014 +0100 @@ -90,7 +90,7 @@ # to be selected) if checkread and eid not in neweids: with session.security_enabled(read=False): - eschema(session.describe(eid)[0]).check_perm( + eschema(session.entity_metas(eid)['type']).check_perm( session, 'read', eid=eid) eidconsts[lhs.variable] = eid return eidconsts diff -r 39b7a91a3f4c -r 032825bbacab server/test/unittest_datafeed.py --- a/server/test/unittest_datafeed.py Thu Jan 23 12:32:16 2014 +0100 +++ b/server/test/unittest_datafeed.py Thu Jan 23 13:47:28 2014 +0100 @@ -1,4 +1,4 @@ -# copyright 2011-2012 LOGILAB S.A. (Paris, FRANCE), all rights reserved. +# copyright 2011-2013 LOGILAB S.A. (Paris, FRANCE), all rights reserved. # contact http://www.logilab.fr/ -- mailto:contact@logilab.fr # # This file is part of CubicWeb. @@ -70,8 +70,8 @@ self.assertEqual(entity.absolute_url(), 'http://www.cubicweb.org/') # test repo cache keys self.assertEqual(self.repo._type_source_cache[entity.eid], - ('Card', 'system', 'http://www.cubicweb.org/', 'myfeed')) - self.assertEqual(self.repo._extid_cache[('http://www.cubicweb.org/', 'system')], + ('Card', 'http://www.cubicweb.org/', 'myfeed')) + self.assertEqual(self.repo._extid_cache['http://www.cubicweb.org/'], entity.eid) # test repull session.set_cnxset() @@ -86,8 +86,8 @@ self.assertEqual(stats['created'], set()) self.assertEqual(stats['updated'], set((entity.eid,))) self.assertEqual(self.repo._type_source_cache[entity.eid], - ('Card', 'system', 'http://www.cubicweb.org/', 'myfeed')) - self.assertEqual(self.repo._extid_cache[('http://www.cubicweb.org/', 'system')], + ('Card', 'http://www.cubicweb.org/', 'myfeed')) + self.assertEqual(self.repo._extid_cache['http://www.cubicweb.org/'], entity.eid) self.assertEqual(dfsource.source_cwuris(self.session), @@ -109,8 +109,8 @@ 'extid': 'http://www.cubicweb.org/'} ) self.assertEqual(self.repo._type_source_cache[entity.eid], - ('Card', 'system', 'http://www.cubicweb.org/', 'myrenamedfeed')) - self.assertEqual(self.repo._extid_cache[('http://www.cubicweb.org/', 'system')], + ('Card', 'http://www.cubicweb.org/', 'myrenamedfeed')) + self.assertEqual(self.repo._extid_cache['http://www.cubicweb.org/'], entity.eid) # test_delete_source diff -r 39b7a91a3f4c -r 032825bbacab server/test/unittest_repository.py --- a/server/test/unittest_repository.py Thu Jan 23 12:32:16 2014 +0100 +++ b/server/test/unittest_repository.py Thu Jan 23 13:47:28 2014 +0100 @@ -424,12 +424,8 @@ cnxid = repo.connect(self.admlogin, password=self.admpassword) session = repo._get_session(cnxid, setcnxset=True) self.assertEqual(repo.type_and_source_from_eid(2, session), - ('CWGroup', 'system', None, 'system')) + ('CWGroup', None, 'system')) self.assertEqual(repo.type_from_eid(2, session), 'CWGroup') - self.assertEqual(repo.source_from_eid(2, session).uri, 'system') - self.assertEqual(repo.eid2extid(repo.system_source, 2, session), None) - class dummysource: uri = 'toto' - self.assertRaises(UnknownEid, repo.eid2extid, dummysource, 2, session) repo.close(cnxid) def test_public_api(self): @@ -445,7 +441,9 @@ repo = self.repo cnxid = repo.connect(self.admlogin, password=self.admpassword) self.assertEqual(repo.user_info(cnxid), (6, 'admin', set([u'managers']), {})) - self.assertEqual(repo.describe(cnxid, 2), (u'CWGroup', u'system', None, 'system')) + self.assertEqual({'type': u'CWGroup', 'extid': None, 'source': 'system'}, + repo.entity_metas(cnxid, 2)) + self.assertEqual(repo.describe(cnxid, 2), (u'CWGroup', 'system', None, 'system')) repo.close(cnxid) self.assertRaises(BadConnectionId, repo.user_info, cnxid) self.assertRaises(BadConnectionId, repo.describe, cnxid, 1) @@ -670,15 +668,6 @@ self.session.set_cnxset() self.assert_(self.repo.system_source.create_eid(self.session)) - def test_source_from_eid(self): - self.session.set_cnxset() - self.assertEqual(self.repo.source_from_eid(1, self.session), - self.repo.sources_by_uri['system']) - - def test_source_from_eid_raise(self): - self.session.set_cnxset() - self.assertRaises(UnknownEid, self.repo.source_from_eid, -2, self.session) - def test_type_from_eid(self): self.session.set_cnxset() self.assertEqual(self.repo.type_from_eid(2, self.session), 'CWGroup') @@ -695,8 +684,7 @@ self.repo.add_info(self.session, entity, self.repo.system_source) cu = self.session.system_sql('SELECT * FROM entities WHERE eid = -1') data = cu.fetchall() - self.assertEqual(tuplify(data), [(-1, 'Personne', 'system', 'system', - None)]) + self.assertEqual(tuplify(data), [(-1, 'Personne', 'system', None)]) self.repo.delete_info(self.session, entity, 'system') #self.repo.commit() cu = self.session.system_sql('SELECT * FROM entities WHERE eid = -1') diff -r 39b7a91a3f4c -r 032825bbacab sobjects/supervising.py --- a/sobjects/supervising.py Thu Jan 23 12:32:16 2014 +0100 +++ b/sobjects/supervising.py Thu Jan 23 13:47:28 2014 +0100 @@ -145,7 +145,7 @@ session = self._cw def describe(eid): try: - return session._(session.describe(eid)[0]).lower() + return session._(session.entity_metas(eid)['type']).lower() except UnknownEid: # may occurs when an entity has been deleted from an external # source and we're cleaning its relation diff -r 39b7a91a3f4c -r 032825bbacab test/unittest_dbapi.py --- a/test/unittest_dbapi.py Thu Jan 23 12:32:16 2014 +0100 +++ b/test/unittest_dbapi.py Thu Jan 23 13:47:28 2014 +0100 @@ -49,9 +49,12 @@ def test_api(self): cnx = _repo_connect(self.repo, login='anon', password='anon') self.assertEqual(cnx.user(None).login, 'anon') + self.assertEqual({'type': u'CWSource', 'source': u'system', 'extid': None}, + cnx.entity_metas(1)) self.assertEqual(cnx.describe(1), (u'CWSource', u'system', None)) cnx.close() self.assertRaises(ProgrammingError, cnx.user, None) + self.assertRaises(ProgrammingError, cnx.entity_metas, 1) self.assertRaises(ProgrammingError, cnx.describe, 1) def test_shared_data_api(self): diff -r 39b7a91a3f4c -r 032825bbacab web/request.py --- a/web/request.py Thu Jan 23 12:32:16 2014 +0100 +++ b/web/request.py Thu Jan 23 13:47:28 2014 +0100 @@ -1052,10 +1052,11 @@ self._ = self.__ = unicode self.pgettext = lambda x, y: unicode(y) - describe = _cnx_func('describe') + entity_metas = _cnx_func('entity_metas') source_defs = _cnx_func('source_defs') get_shared_data = _cnx_func('get_shared_data') set_shared_data = _cnx_func('set_shared_data') + describe = _cnx_func('describe') # deprecated XXX # server-side service call #################################################