97 :meth:`~cubicweb.entities.AnyEntity.cw_fti_index_rql_queries` class

    97 :meth:`~cubicweb.entities.AnyEntity.cw_fti_index_rql_limit` class

   100 .. automethod:: cubicweb.entities.AnyEntity.cw_fti_index_rql_queries

   100 .. automethod:: cubicweb.entities.AnyEntity.cw_fti_index_rql_limit

   101 

   102 Now, suppose you've got a _huge_ table to index, you probably don't want to

   103 get all entities at once. So here's a simple customized example that will

   104 process block of 10000 entities:

   105 

   106 .. sourcecode:: python

   107 

   108 

   109     class MyEntityClass(AnyEntity):

   110         __regid__ = 'MyEntityClass'

   111 

   112     @classmethod

   113     def cw_fti_index_rql_queries(cls, req):

   114         # get the default RQL method and insert LIMIT / OFFSET instructions

   115         base_rql = super(SearchIndex, cls).cw_fti_index_rql_queries(req)[0]

   116         selected, restrictions = base_rql.split(' WHERE ')

   117         rql_template = '%s ORDERBY X LIMIT %%(limit)s OFFSET %%(offset)s WHERE %s' % (

   118             selected, restrictions)

   119         # count how many entities you'll have to index

   120         count = req.execute('Any COUNT(X) WHERE X is MyEntityClass')[0][0]

   121         # iterate by blocks of 10000 entities

   122         chunksize = 10000

   123         for offset in xrange(0, count, chunksize):

   124             print 'SENDING', rql_template % {'limit': chunksize, 'offset': offset}

   125             yield rql_template % {'limit': chunksize, 'offset': offset}

   126 

   127 Since you have access to ``req``, you can more or less fetch whatever you want.