# HG changeset patch # User Adrien Di Mascio # Date 1249456556 -7200 # Node ID a357d4147eee2817dd5b4f1736c56f8bfdd7bb6e # Parent ab797c5374b7e7435c44ab59b8344a01e6441e88 [forms] work-in-progress, big editcontroller refactoring: let fields/widgets process posted data diff -r ab797c5374b7 -r a357d4147eee web/_exceptions.py --- a/web/_exceptions.py Wed Aug 05 09:14:34 2009 +0200 +++ b/web/_exceptions.py Wed Aug 05 09:15:56 2009 +0200 @@ -19,6 +19,10 @@ class NothingToEdit(RequestError): """raised when an edit request doesn't specify any eid to edit""" +class ProcessFormError(RequestError): + """raised when posted data can't be processed by the corresponding field + """ + class NotFound(RequestError): """raised when a 404 error should be returned""" diff -r ab797c5374b7 -r a357d4147eee web/formfields.py --- a/web/formfields.py Wed Aug 05 09:14:34 2009 +0200 +++ b/web/formfields.py Wed Aug 05 09:15:56 2009 +0200 @@ -198,6 +198,21 @@ """ pass + def process_form_value(self, form): + """process posted form and return correctly typed value""" + widget = self.get_widget(form) + return widget.process_field_data(form, self) + + def process_posted(self, form): + for field in self.actual_fields(form): + if field is self: + yield field.name, field.process_form_value(form) + else: + # recursive function: we might have compound fields + # of compound fields (of compound fields of ...) + for fieldname, value in field.process_posted(form): + yield fieldname, value + class StringField(Field): widget = TextArea @@ -345,6 +360,25 @@ + renderer.render_help(form, field) + u'
') + def process_form_value(self, form): + posted = form.req.form + value = posted.get(form.form_field_name(self)) + formkey = form.form_field_name(self) + if ('%s__detach' % form.context[self]['name']) in posted: + # drop current file value + value = None + # no need to check value when nor explicit detach nor new file + # submitted, since it will think the attribute is not modified + elif value: + filename, _, stream = value + # value is a 3-uple (filename, mimetype, stream) + value = Binary(stream.read()) + if not val.getvalue(): # usually an unexistant file + value = None + else: + value.filename = filename + return value + class EditableFileField(FileField): editable_formats = ('text/plain', 'text/html', 'text/rest') @@ -375,6 +409,14 @@ # XXX restore form context? return '\n'.join(wdgs) + def process_form_value(self, form): + value = form.req.form.get(form.form_field_name(self)) + if isinstance(value, unicode): + # file modified using a text widget + encoding = form.form_field_encoding(self) + return Binary(value.encode(encoding)) + return super(EditableFileField, self).process_form_value(form) + class IntField(Field): def __init__(self, min=None, max=None, **kwargs): @@ -385,6 +427,8 @@ self.widget.attrs.setdefault('size', 5) self.widget.attrs.setdefault('maxlength', 15) + def process_form_value(self, form): + return int(Field.process_form_value(self, form)) class BooleanField(Field): widget = Radio @@ -394,6 +438,8 @@ return self.choices return [(form.req._('yes'), '1'), (form.req._('no'), '')] + def process_form_value(self, form): + return bool(Field.process_form_value(self, form)) class FloatField(IntField): def format_single_value(self, req, value): @@ -405,6 +451,8 @@ def render_example(self, req): return self.format_single_value(req, 1.234) + def process_form_value(self, form): + return float(Field.process_form_value(self, form)) class DateField(StringField): format_prop = 'ui.date-format' @@ -416,15 +464,39 @@ def render_example(self, req): return self.format_single_value(req, datetime.now()) + def process_form_value(self, form): + # widget is supposed to return a date as a correctly formatted string + date = Field.process_form_value(self, form) + # but for some widgets, it might be simpler to return date objects + # directly, so handle that case : + if isinstance(date, basestring): + date = form.parse_date(wdgdate, 'Date') + return date class DateTimeField(DateField): format_prop = 'ui.datetime-format' + def process_form_value(self, form): + # widget is supposed to return a date as a correctly formatted string + date = Field.process_form_value(self, form) + # but for some widgets, it might be simpler to return date objects + # directly, so handle that case : + if isinstance(date, basestring): + date = form.parse_datetime(wdgdate, 'Datetime') + return date class TimeField(DateField): format_prop = 'ui.time-format' widget = TextInput + def process_form_value(self, form): + # widget is supposed to return a date as a correctly formatted string + time = Field.process_form_value(self, form) + # but for some widgets, it might be simpler to return time objects + # directly, so handle that case : + if isinstance(time, basestring): + time = form.parse_time(wdgdate, 'Time') + return time class RelationField(Field): def __init__(self, **kwargs): diff -r ab797c5374b7 -r a357d4147eee web/formwidgets.py --- a/web/formwidgets.py Wed Aug 05 09:14:34 2009 +0200 +++ b/web/formwidgets.py Wed Aug 05 09:15:56 2009 +0200 @@ -11,7 +11,7 @@ from warnings import warn from cubicweb.common import tags, uilib -from cubicweb.web import stdmsgs, INTERNAL_FIELD_VALUE +from cubicweb.web import stdmsgs, INTERNAL_FIELD_VALUE, ProcessFormError class FieldWidget(object): @@ -63,6 +63,10 @@ attrs['tabindex'] = form.req.next_tabindex() return name, values, attrs + def process_field_data(self, form, field): + formkey = form.form_field_name(field) + posted = form.req.form + return posted.get(formkey) class Input(FieldWidget): """abstract widget class for tag based widgets""" @@ -114,11 +118,25 @@ **{'class': 'emphasis'})] return u'\n'.join(inputs) + def process_field_data(self, form, field): + passwd1 = super(PasswordInput, self).process_field_data(form, field) + fieldname = form.form_field_name(field) + passwd2 = form.req.form[fieldname+'-confirm'] + if passwd1 == passwd2: + if passwd1 is None: + return None + return passwd1.encode('utf-8') + raise ProcessFormError(form.req._("password and confirmation don't match")) class PasswordSingleInput(Input): """ without a confirmation field""" type = 'password' + def process_field_data(self, form, field): + value = super(PasswordSingleInput, self).process_field_data(form, field) + if value is not None: + return value.encode('utf-8') + return value class FileInput(Input): """""" diff -r ab797c5374b7 -r a357d4147eee web/views/editcontroller.py --- a/web/views/editcontroller.py Wed Aug 05 09:14:34 2009 +0200 +++ b/web/views/editcontroller.py Wed Aug 05 09:15:56 2009 +0200 @@ -12,7 +12,7 @@ from rql.utils import rqlvar_maker from cubicweb import Binary, ValidationError, typed_eid -from cubicweb.web import INTERNAL_FIELD_VALUE, RequestError, NothingToEdit +from cubicweb.web import INTERNAL_FIELD_VALUE, RequestError, NothingToEdit, ProcessFormError from cubicweb.web.controller import parse_relations_descr from cubicweb.web.views.basecontrollers import ViewController @@ -22,6 +22,32 @@ can't be handled right now and have to be handled later """ +class RqlQuery(object): + def __init__(self): + self.edited = [] + self.restrictions = [] + self.kwargs = {} + + def insert_query(self, etype): + if self.edited: + rql = 'INSERT %s X: %s' % (etype, ','.join(self.edited)) + else: + rql = 'INSERT %s X' % etype + if self.restrictions: + rql += ' WHERE %s' % ','.join(self.restrictions) + return rql + + def update_query(self, eid): + varmaker = rqlvar_maker() + var = varmaker.next() + while var in self.kwargs: + var = varmaker.next() + rql = 'SET %s WHERE X eid %%(%s)s' % (','.join(self.edited), var) + if self.restrictions: + rql += ', %s' % ','.join(self.restrictions) + self.kwargs[var] = eid + return rql + class EditController(ViewController): id = 'edit' @@ -42,14 +68,15 @@ def _default_publish(self): req = self.req - form = req.form + self.errors = [] + self.relations_rql = [] # no specific action, generic edition self._to_create = req.data['eidmap'] = {} self._pending_relations = [] todelete = self.req.get_pending_deletes() toinsert = self.req.get_pending_inserts() try: - methodname = form.pop('__method', None) + methodname = req.form.pop('__method', None) for eid in req.edited_eids(): formparams = req.extract_entity_params(eid) if methodname is not None: @@ -58,83 +85,71 @@ method(formparams) eid = self.edit_entity(formparams) except (RequestError, NothingToEdit): - if '__linkto' in form and 'eid' in form: + if '__linkto' in req.form and 'eid' in req.form: self.execute_linkto() - elif not ('__delete' in form or '__insert' in form or todelete or toinsert): + elif not ('__delete' in req.form or '__insert' in req.form or todelete or toinsert): raise ValidationError(None, {None: req._('nothing to edit')}) + for querydef in self.relations_rql: + self.req.execute(*querydef) # handle relations in newly created entities + # XXX find a way to merge _pending_relations and relations_rql if self._pending_relations: - for rschema, formparams, x, entity in self._pending_relations: - self.handle_relation(rschema, formparams, x, entity, True) - + for form, field, entity in self._pending_relations: + for querydef in self.handle_relation(form, field, entity, True): + self.req.execute(*querydef) # XXX this processes *all* pending operations of *all* entities if req.form.has_key('__delete'): todelete += req.list_form_param('__delete', req.form, pop=True) if todelete: self.delete_relations(parse_relations_descr(todelete)) - if form.has_key('__insert'): - toinsert = req.list_form_param('__insert', form, pop=True) + if req.form.has_key('__insert'): + toinsert = req.list_form_param('__insert', req.form, pop=True) if toinsert: self.insert_relations(parse_relations_descr(toinsert)) self.req.remove_pending_operations() + def _insert_entity(self, etype, eid, rqlquery): + rql = rqlquery.insert_query(etype) + try: + # get the new entity (in some cases, the type might have + # changed as for the File --> Image mutation) + entity = self.req.execute(rql, rqlquery.kwargs).get_entity(0, 0) + neweid = entity.eid + except ValidationError, ex: + self._to_create[eid] = ex.entity + if self.req.json_request: # XXX (syt) why? + ex.entity = eid + raise + self._to_create[eid] = neweid + return neweid + + def _update_entity(self, eid, rqlquery): + rql = rqlquery.update_query(eid) + self.req.execute(rql, rqlquery.kwargs) + def edit_entity(self, formparams, multiple=False): """edit / create / copy an entity and return its eid""" etype = formparams['__type'] entity = self.vreg['etypes'].etype_class(etype)(self.req) - entity.eid = eid = self._get_eid(formparams['eid']) - # let a chance to do some entity specific stuff. + entity.eid = formparams['eid'] + eid = self._get_eid(entity.eid) is_main_entity = self.req.form.get('__maineid') == formparams['eid'] + # let a chance to do some entity specific stuff.tn entity.pre_web_edit() # create a rql query from parameters - self.relations = [] - self.restrictions = [] + rqlquery = RqlQuery() # process inlined relations at the same time as attributes - # this is required by some external source such as the svn source which - # needs some information provided by those inlined relation. Moreover - # this will generate less write queries. - for rschema in entity.e_schema.subject_relations(): - if rschema.is_final(): - self.handle_attribute(entity, rschema, formparams) - elif rschema.inlined: - self.handle_inlined_relation(rschema, formparams, entity) - execute = self.req.execute + # this will generate less rql queries and might be useful in + # a few dark corners + formid = self.req.form.get('__form_id', 'edition') + form = self.vreg['forms'].select(formid, self.req, entity=entity) + for field in form.fields: + if form.form_field_modified(field): + self.handle_formfield(form, field, entity, rqlquery) if eid is None: # creation or copy - if self.relations: - rql = 'INSERT %s X: %s' % (etype, ','.join(self.relations)) - else: - rql = 'INSERT %s X' % etype - if self.restrictions: - rql += ' WHERE %s' % ','.join(self.restrictions) - try: - # get the new entity (in some cases, the type might have - # changed as for the File --> Image mutation) - entity = execute(rql, formparams).get_entity(0, 0) - eid = entity.eid - except ValidationError, ex: - self._to_create[formparams['eid']] = ex.entity - if self.req.json_request: # XXX (syt) why? - ex.entity = formparams['eid'] - raise - self._to_create[formparams['eid']] = eid - elif self.relations: # edition of an existant entity - varmaker = rqlvar_maker() - var = varmaker.next() - while var in formparams: - var = varmaker.next() - rql = 'SET %s WHERE X eid %%(%s)s' % (','.join(self.relations), var) - if self.restrictions: - rql += ', %s' % ','.join(self.restrictions) - formparams[var] = eid - execute(rql, formparams) - for rschema in entity.e_schema.subject_relations(): - if rschema.is_final() or rschema.inlined: - continue - self.handle_relation(rschema, formparams, 'subject', entity) - for rschema in entity.e_schema.object_relations(): - if rschema.is_final(): - continue - self.handle_relation(rschema, formparams, 'object', entity) + entity.eid = self._insert_entity(etype, formparams['eid'], rqlquery) + elif rqlquery.edited: # edition of an existant entity + self._update_entity(eid, rqlquery) if is_main_entity: self.notify_edited(entity) if formparams.has_key('__delete'): @@ -149,6 +164,27 @@ self.execute_linkto(eid) return eid + def handle_formfield(self, form, field, entity, rqlquery): + eschema = entity.e_schema + try: + for attr, value in field.process_posted(form): + if not ( + (field.role == 'subject' and eschema.has_subject_relation(field.name)) + or + (field.role == 'object' and eschema.has_object_relation(field.name))): + continue + rschema = self.schema.rschema(field.name) + if rschema.is_final(): + rqlquery.kwargs[attr] = value + rqlquery.edited.append('X %s %%(%s)s' % (attr, attr)) + elif rschema.inlined: + self.handle_inlined_relation(form, field, entity, rqlquery) + else: + self.relations_rql += self.handle_relation( + form, field, entity) + except ProcessFormError, exc: + self.errors.append((field, exc)) + def _action_apply(self): self._default_publish() self.reset() @@ -163,143 +199,49 @@ self.delete_entities(self.req.edited_eids(withtype=True)) return self.reset() - def _needs_edition(self, rtype, formparams, entity): - """returns True and and the new value if `rtype` was edited""" - editkey = 'edits-%s' % rtype - if not editkey in formparams: - return False, None # not edited - value = formparams.get(rtype) or None - if entity.has_eid() and (formparams.get(editkey) or None) == value: - return False, None # not modified - if value == INTERNAL_FIELD_VALUE: - value = None - return True, value - - def handle_attribute(self, entity, rschema, formparams): - """append to `relations` part of the rql query to edit the - attribute described by the given schema if necessary + def _relation_values(self, form, field, entity, late=False): + """handle edition for the (rschema, x) relation of the given entity """ - attr = rschema.type - edition_needed, value = self._needs_edition(attr, formparams, entity) - if not edition_needed: - return - # test if entity class defines a special handler for this attribute - custom_edit = getattr(entity, 'custom_%s_edit' % attr, None) - if custom_edit: - custom_edit(formparams, value, self.relations) - return - attrtype = rschema.objects(entity.e_schema)[0].type - # on checkbox or selection, the field may not be in params - if attrtype == 'Boolean': - value = bool(value) - elif attrtype == 'Decimal': - value = Decimal(value) - elif attrtype == 'Bytes': - # if it is a file, transport it using a Binary (StringIO) - # XXX later __detach is for the new widget system, the former is to - # be removed once web/widgets.py has been dropped - if formparams.has_key('__%s_detach' % attr) or formparams.has_key('%s__detach' % attr): - # drop current file value - value = None - # no need to check value when nor explicit detach nor new file - # submitted, since it will think the attribute is not modified - elif isinstance(value, unicode): - # file modified using a text widget - encoding = entity.attr_metadata(attr, 'encoding') - value = Binary(value.encode(encoding)) - elif value: - # value is a 3-uple (filename, mimetype, stream) - val = Binary(value[2].read()) - if not val.getvalue(): # usually an unexistant file - value = None - else: - val.filename = value[0] - # ignore browser submitted MIME type since it may be buggy - # XXX add a config option to tell if we should consider it - # or not? - #if entity.e_schema.has_metadata(attr, 'format'): - # key = '%s_format' % attr - # formparams[key] = value[1] - # self.relations.append('X %s_format %%(%s)s' - # % (attr, key)) - # XXX suppose a File compatible schema - if entity.e_schema.has_subject_relation('name') \ - and not formparams.get('name'): - formparams['name'] = value[0] - self.relations.append('X name %(name)s') - value = val - else: - # no specified value, skip - return - elif value is not None: - if attrtype in ('Date', 'Datetime', 'Time'): - try: - value = self.parse_datetime(value, attrtype) - except ValueError: - raise ValidationError(entity.eid, - {attr: self.req._("invalid date")}) - elif attrtype == 'Password': - # check confirmation (see PasswordWidget for confirmation field name) - confirmval = formparams.get(attr + '-confirm') - if confirmval != value: - raise ValidationError(entity.eid, - {attr: self.req._("password and confirmation don't match")}) - # password should *always* be utf8 encoded - value = value.encode('UTF8') - else: - # strip strings - value = value.strip() - elif attrtype == 'Password': - # skip None password - return # unset password - formparams[attr] = value - self.relations.append('X %s %%(%s)s' % (attr, attr)) + values = set() + for eid in field.process_form_data(form): + if not eid: # AutoCompletionWidget + continue + typed_eid = self._get_eid(eid) + if typed_eid is None: + if late: + # eid is still None while it's already a late call + # this mean that the associated entity has not been created + raise Exception("eid %s is still not created" % eid) + self._pending_relations.append( (form, field, entity) ) + return None + values.add(typed_eid) + return values - def _relation_values(self, rschema, formparams, x, entity, late=False): + def handle_inlined_relation(self, form, field, entity, rqlquery): """handle edition for the (rschema, x) relation of the given entity """ - rtype = rschema.type - editkey = 'edit%s-%s' % (x[0], rtype) - if not editkey in formparams: - return # not edited - try: - values = self._linked_eids(self.req.list_form_param(rtype, formparams), late) - except ToDoLater: - self._pending_relations.append((rschema, formparams, x, entity)) - return - origvalues = set(typed_eid(eid) for eid in self.req.list_form_param(editkey, formparams)) - return values, origvalues + origvalues = set(row[0] for row in entity.related(field.name, field.role)) + values = self._relation_values(form, field, entity) + if values is None or values == origvalues: + return # not edited / not modified / to do later + attr = field.name + if values: + rqlquery.kwargs[attr] = iter(values).next() + rqlquery.edition.append('X %s %s' % (attr, attr.upper())) + rqlquery.restrictions.append('%s eid %%(%s)s' % (attr.upper(), attr)) + elif entity.has_eid(): + self.relations_rql += self.handle_relation(form, field, entity) - def handle_inlined_relation(self, rschema, formparams, entity, late=False): + def handle_relation(self, form, field, entity, late=False): """handle edition for the (rschema, x) relation of the given entity """ - try: - values, origvalues = self._relation_values(rschema, formparams, - 'subject', entity, late) - except TypeError: - return # not edited / to do later - if values == origvalues: - return # not modified - attr = str(rschema) - if values: - formparams[attr] = iter(values).next() - self.relations.append('X %s %s' % (attr, attr.upper())) - self.restrictions.append('%s eid %%(%s)s' % (attr.upper(), attr)) - elif entity.has_eid(): - self.handle_relation(rschema, formparams, 'subject', entity, late) - - def handle_relation(self, rschema, formparams, x, entity, late=False): - """handle edition for the (rschema, x) relation of the given entity - """ - try: - values, origvalues = self._relation_values(rschema, formparams, x, - entity, late) - except TypeError: - return # not edited / to do later + origvalues = set(row[0] for row in entity.related(field.name, field.role)) + values = self._relation_values(form, field, entity, late) + if values is None or values == origvalues: + return # not edited / not modified / to do later etype = entity.e_schema - if values == origvalues: - return # not modified - if x == 'subject': + rschema = self.schema.rschema(field.name) + if field.role == 'subject': desttype = rschema.objects(etype)[0] card = rschema.rproperty(etype, desttype, 'cardinality')[0] subjvar, objvar = 'X', 'Y' @@ -308,19 +250,19 @@ card = rschema.rproperty(desttype, etype, 'cardinality')[1] subjvar, objvar = 'Y', 'X' eid = entity.eid - if x == 'object' or not rschema.inlined or not values: + if field.role == 'object' or not rschema.inlined or not values: # this is not an inlined relation or no values specified, # explicty remove relations rql = 'DELETE %s %s %s WHERE X eid %%(x)s, Y eid %%(y)s' % ( subjvar, rschema, objvar) for reid in origvalues.difference(values): - self.req.execute(rql, {'x': eid, 'y': reid}, ('x', 'y')) + yield (rql, {'x': eid, 'y': reid}, ('x', 'y')) seteids = values.difference(origvalues) if seteids: rql = 'SET %s %s %s WHERE X eid %%(x)s, Y eid %%(y)s' % ( subjvar, rschema, objvar) for reid in seteids: - self.req.execute(rql, {'x': eid, 'y': reid}, ('x', 'y')) + yield (rql, {'x': eid, 'y': reid}, ('x', 'y')) def _get_eid(self, eid): # should be either an int (existant entity) or a variable (to be @@ -338,18 +280,5 @@ def _linked_eids(self, eids, late=False): """return a list of eids if they are all known, else raise ToDoLater """ - result = set() - for eid in eids: - if not eid: # AutoCompletionWidget - continue - eid = self._get_eid(eid) - if eid is None: - if not late: - raise ToDoLater() - # eid is still None while it's already a late call - # this mean that the associated entity has not been created - raise Exception('duh') - result.add(eid) - return result diff -r ab797c5374b7 -r a357d4147eee web/views/forms.py --- a/web/views/forms.py Wed Aug 05 09:14:34 2009 +0200 +++ b/web/views/forms.py Wed Aug 05 09:15:56 2009 +0200 @@ -16,7 +16,7 @@ from cubicweb.web import INTERNAL_FIELD_VALUE, eid_param from cubicweb.web import form, formwidgets as fwdgs from cubicweb.web.controller import NAV_FORM_PARAMETERS -from cubicweb.web.formfields import HiddenInitialValueField, StringField +from cubicweb.web.formfields import StringField class FieldsForm(form.Form): @@ -280,6 +280,9 @@ """ raise NotImplementedError + def form_field_modified(self, field): + return field.is_visible() + def _field_has_error(self, field): """return true if the field has some error in given validation exception """ @@ -474,6 +477,32 @@ # cases, it doesn't make sense to sort results afterwards. return vocabfunc(rtype, limit) + def form_field_modified(self, field): + if field.is_visible(): + # fields not corresponding to an entity attribute / relations + # are considered modified + if not field.eidparam: + return True # XXX + try: + if field.role == 'subject': + previous_value = getattr(self.edited_entity, field.name) + else: + previous_value = getattr(self.edited_entity, + 'reverse_%s' % field.name) + except AttributeError: + # fields with eidparam=True but not corresponding to an actual + # attribute or relation + return True + # if it's a non final relation, we need the eids + if isinstance(previous_value, list): + # widget should return untyped eids + previous_value = set(unicode(e.eid) for e in previous_value) + new_value = field.process_form_value(self) + if self.edited_entity.has_eid() and (previous_value == new_value): + return False # not modified + return True + return False + def subject_relation_vocabulary(self, rtype, limit=None): """defaut vocabulary method for the given relation, looking for relation's object entities (i.e. self is the subject)