cubicweb: cubicweb/devtools/htmlparser.py@d464495452aa (annotated)

8937 8a1809c9a043 [htmlparser] add missing deprecation message Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8695 diff changeset	1	# copyright 2003-2013 LOGILAB S.A. (Paris, FRANCE), all rights reserved.
5421 8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time. Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 5276 diff changeset	2	# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time. Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 5276 diff changeset	3	#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time. Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 5276 diff changeset	4	# This file is part of CubicWeb.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time. Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 5276 diff changeset	5	#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time. Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 5276 diff changeset	6	# CubicWeb is free software: you can redistribute it and/or modify it under the
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time. Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 5276 diff changeset	7	# terms of the GNU Lesser General Public License as published by the Free
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time. Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 5276 diff changeset	8	# Software Foundation, either version 2.1 of the License, or (at your option)
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time. Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 5276 diff changeset	9	# any later version.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time. Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 5276 diff changeset	10	#
5424 8ecbcbff9777 replace logilab-common by CubicWeb in disclaimer Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 5421 diff changeset	11	# CubicWeb is distributed in the hope that it will be useful, but WITHOUT
5421 8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time. Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 5276 diff changeset	12	# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time. Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 5276 diff changeset	13	# FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time. Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 5276 diff changeset	14	# details.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time. Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 5276 diff changeset	15	#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time. Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 5276 diff changeset	16	# You should have received a copy of the GNU Lesser General Public License along
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time. Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 5276 diff changeset	17	# with CubicWeb. If not, see <http://www.gnu.org/licenses/>.
6771 da71f1ad1721 minor code cleanup Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 5424 diff changeset	18	"""defines a validating HTML parser used in web application tests"""
0 b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	19
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	20	import re
3325 44caeccd2db9 fix sys import Julien Jehannet <julien.jehannet@logilab.fr> parents: 3151 diff changeset	21	import sys
8977 57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	22	from xml import sax
10618 3274a1648c7e [py3k] io.BytesIO Rémi Cardona <remi.cardona@logilab.fr> parents: 10588 diff changeset	23	from io import BytesIO
0 b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	24
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	25	from lxml import etree
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	26
1421 77ee26df178f doc type handling refactoring: do the ext substitution at the module level sylvain.thenault@logilab.fr parents: 1132 diff changeset	27	from cubicweb.view import STRICT_DOCTYPE, TRANSITIONAL_DOCTYPE
6771 da71f1ad1721 minor code cleanup Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 5424 diff changeset	28
1485 4d532f3c012e nicer fix sylvain.thenault@logilab.fr parents: 1480 diff changeset	29	STRICT_DOCTYPE = str(STRICT_DOCTYPE)
4d532f3c012e nicer fix sylvain.thenault@logilab.fr parents: 1480 diff changeset	30	TRANSITIONAL_DOCTYPE = str(TRANSITIONAL_DOCTYPE)
0 b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	31
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	32	ERR_COUNT = 0
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	33
10696 4ba4be5553cf [py3k] unicode vs str vs bytes vs the world Rémi Cardona <remi.cardona@logilab.fr> parents: 10662 diff changeset	34	_REM_SCRIPT_RGX = re.compile(br"<script[^>]>.?</script>", re.M\|re.I\|re.S)
8977 57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	35	def _remove_script_tags(data):
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	36	"""Remove the script (usually javascript) tags to help the lxml
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	37	XMLParser / HTMLParser do their job. Without that, they choke on
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	38	tags embedded in JS strings.
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	39	"""
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	40	# Notice we may want to use lxml cleaner, but it's far too intrusive:
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	41	#
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	42	# cleaner = Cleaner(scripts=True,
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	43	# javascript=False,
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	44	# comments=False,
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	45	# style=False,
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	46	# links=False,
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	47	# meta=False,
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	48	# page_structure=False,
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	49	# processing_instructions=False,
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	50	# embedded=False,
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	51	# frames=False,
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	52	# forms=False,
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	53	# annoying_tags=False,
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	54	# remove_tags=(),
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	55	# remove_unknown_tags=False,
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	56	# safe_attrs_only=False,
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	57	# add_nofollow=False)
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	58	# >>> cleaner.clean_html('<body></body>')
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	59	# '<span></span>'
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	60	# >>> cleaner.clean_html('<!DOCTYPE html><body></body>')
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	61	# '<html><body></body></html>'
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	62	# >>> cleaner.clean_html('<body><div/></body>')
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	63	# '<div></div>'
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	64	# >>> cleaner.clean_html('<html><body><div/><br></body><html>')
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	65	# '<html><body><div></div><br></body></html>'
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	66	# >>> cleaner.clean_html('<html><body><div/><br><span></body><html>')
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	67	# '<html><body><div></div><br><span></span></body></html>'
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	68	#
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	69	# using that, we'll miss most actual validation error we want to
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	70	# catch. For now, use dumb regexp
10696 4ba4be5553cf [py3k] unicode vs str vs bytes vs the world Rémi Cardona <remi.cardona@logilab.fr> parents: 10662 diff changeset	71	return _REM_SCRIPT_RGX.sub(b'', data)
8977 57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	72
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	73
0 b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	74	class Validator(object):
8977 57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	75	""" base validator API """
8973 6711f78c18be [testlib] unspaghettify Validator / PageInfo api Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8972 diff changeset	76	parser = None
1485 4d532f3c012e nicer fix sylvain.thenault@logilab.fr parents: 1480 diff changeset	77
8973 6711f78c18be [testlib] unspaghettify Validator / PageInfo api Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8972 diff changeset	78	def parse_string(self, source):
6711f78c18be [testlib] unspaghettify Validator / PageInfo api Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8972 diff changeset	79	etree = self._parse(self.preprocess_data(source))
6711f78c18be [testlib] unspaghettify Validator / PageInfo api Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8972 diff changeset	80	return PageInfo(source, etree)
6711f78c18be [testlib] unspaghettify Validator / PageInfo api Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8972 diff changeset	81
6711f78c18be [testlib] unspaghettify Validator / PageInfo api Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8972 diff changeset	82	def preprocess_data(self, data):
6711f78c18be [testlib] unspaghettify Validator / PageInfo api Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8972 diff changeset	83	return data
6711f78c18be [testlib] unspaghettify Validator / PageInfo api Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8972 diff changeset	84
6711f78c18be [testlib] unspaghettify Validator / PageInfo api Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8972 diff changeset	85	def _parse(self, pdata):
0 b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	86	try:
8973 6711f78c18be [testlib] unspaghettify Validator / PageInfo api Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8972 diff changeset	87	return etree.fromstring(pdata, self.parser)
8695 358d8bed9626 [toward-py3k] rewrite to "except AnException as exc:" (part of #2711624) Nicolas Chauvat <nicolas.chauvat@logilab.fr> parents: 7014 diff changeset	88	except etree.XMLSyntaxError as exc:
8941 7b26fe71404f drop xhtml content-type support (closes #2065651) Aurelien Campeas <aurelien.campeas@logilab.fr> parents: 8940 diff changeset	89	new_exc = AssertionError(u'invalid document: %s' % exc)
0 b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	90	new_exc.position = exc.position
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	91	raise new_exc
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	92
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	93
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	94	class DTDValidator(Validator):
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	95	def __init__(self):
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	96	Validator.__init__(self)
3151 5d45c0945bd3 note about this test under windows Aurelien Campeas parents: 1977 diff changeset	97	# XXX understand what's happening under windows
6771 da71f1ad1721 minor code cleanup Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 5424 diff changeset	98	self.parser = etree.XMLParser(dtd_validation=sys.platform != 'win32')
0 b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	99
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	100	def preprocess_data(self, data):
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	101	"""used to fix potential blockquote mess generated by docutils"""
1485 4d532f3c012e nicer fix sylvain.thenault@logilab.fr parents: 1480 diff changeset	102	if STRICT_DOCTYPE not in data:
0 b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	103	return data
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	104	# parse using transitional DTD
1485 4d532f3c012e nicer fix sylvain.thenault@logilab.fr parents: 1480 diff changeset	105	data = data.replace(STRICT_DOCTYPE, TRANSITIONAL_DOCTYPE)
8979 8f5416b1562a [devtools] use self._parse so AssertionError is properly raised instead of lxml error (test failure introduced in 6711f78c18be) Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8977 diff changeset	106	tree = self._parse(data)
0 b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	107	namespace = tree.nsmap.get(None)
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	108	# this is the list of authorized child tags for <blockquote> nodes
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	109	expected = 'p h1 h2 h3 h4 h5 h6 div ul ol dl pre hr blockquote address ' \
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	110	'fieldset table form noscript ins del script'.split()
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	111	if namespace:
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	112	blockquotes = tree.findall('.//{%s}blockquote' % namespace)
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	113	expected = ['{%s}%s' % (namespace, tag) for tag in expected]
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	114	else:
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	115	blockquotes = tree.findall('.//blockquote')
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	116	# quick and dirty approach: remove all blockquotes
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	117	for blockquote in blockquotes:
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	118	parent = blockquote.getparent()
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	119	parent.remove(blockquote)
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	120	data = etree.tostring(tree)
1485 4d532f3c012e nicer fix sylvain.thenault@logilab.fr parents: 1480 diff changeset	121	return '<?xml version="1.0" encoding="UTF-8"?>%s\n%s' % (
4d532f3c012e nicer fix sylvain.thenault@logilab.fr parents: 1480 diff changeset	122	STRICT_DOCTYPE, data)
0 b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	123
1485 4d532f3c012e nicer fix sylvain.thenault@logilab.fr parents: 1480 diff changeset	124
8938 198fdadafed6 [htmlparser] rename SaxOnlyValidator to XMLValidator Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8937 diff changeset	125	class XMLValidator(Validator):
8977 57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	126	"""XML validator, checks that XML is well-formed and used XMLNS are defined"""
0 b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	127
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	128	def __init__(self):
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	129	Validator.__init__(self)
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	130	self.parser = etree.XMLParser()
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	131
8977 57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	132
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	133	class XMLSyntaxValidator(Validator):
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	134	"""XML syntax validator, check XML is well-formed"""
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	135
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	136	class MySaxErrorHandler(sax.ErrorHandler):
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	137	"""override default handler to avoid choking because of unknown entity"""
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	138	def fatalError(self, exception):
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	139	# XXX check entity in htmlentitydefs
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	140	if not str(exception).endswith('undefined entity'):
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	141	raise exception
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	142	_parser = sax.make_parser()
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	143	_parser.setContentHandler(sax.handler.ContentHandler())
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	144	_parser.setErrorHandler(MySaxErrorHandler())
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	145
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	146	def __init__(self):
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	147	super(XMLSyntaxValidator, self).__init__()
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	148	# XMLParser() wants xml namespaces defined
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	149	# XMLParser(recover=True) will accept almost anything
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	150	#
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	151	# -> use the later but preprocess will check xml well-formness using a
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	152	# dumb SAX parser
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	153	self.parser = etree.XMLParser(recover=True)
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	154
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	155	def preprocess_data(self, data):
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	156	return _remove_script_tags(data)
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	157
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	158	def _parse(self, data):
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	159	inpsrc = sax.InputSource()
10618 3274a1648c7e [py3k] io.BytesIO Rémi Cardona <remi.cardona@logilab.fr> parents: 10588 diff changeset	160	inpsrc.setByteStream(BytesIO(data))
8977 57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	161	try:
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	162	self._parser.parse(inpsrc)
10588 fdaa0e4b7eaf [py3k] except as Samuel Trégouët <samuel.tregouet@logilab.fr> parents: 10066 diff changeset	163	except sax.SAXParseException as exc:
8977 57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	164	new_exc = AssertionError(u'invalid document: %s' % exc)
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	165	new_exc.position = (exc._linenum, exc._colnum)
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	166	raise new_exc
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	167	return super(XMLSyntaxValidator, self)._parse(data)
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	168
57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	169
0 b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	170	class HTMLValidator(Validator):
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	171
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	172	def __init__(self):
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	173	Validator.__init__(self)
8940 ae898a084da2 [htmlparser] exclude <script> tag from html source Pierre-Yves David <pierre-yves.david@logilab.fr> parents: 8939 diff changeset	174	self.parser = etree.HTMLParser(recover=False)
0 b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	175
8940 ae898a084da2 [htmlparser] exclude <script> tag from html source Pierre-Yves David <pierre-yves.david@logilab.fr> parents: 8939 diff changeset	176	def preprocess_data(self, data):
8977 57e564c0118e [testlib] introduce a validator that check xml-well formness Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8973 diff changeset	177	return _remove_script_tags(data)
1485 4d532f3c012e nicer fix sylvain.thenault@logilab.fr parents: 1480 diff changeset	178
0 b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	179
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	180	class PageInfo(object):
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	181	"""holds various informations on the view's output"""
8973 6711f78c18be [testlib] unspaghettify Validator / PageInfo api Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8972 diff changeset	182	def __init__(self, source, root):
0 b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	183	self.source = source
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	184	self.etree = root
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	185	self.raw_text = u''.join(root.xpath('//text()'))
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	186	self.namespace = self.etree.nsmap
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	187	self.default_ns = self.namespace.get(None)
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	188	self.a_tags = self.find_tag('a')
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	189	self.h1_tags = self.find_tag('h1')
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	190	self.h2_tags = self.find_tag('h2')
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	191	self.h3_tags = self.find_tag('h3')
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	192	self.h4_tags = self.find_tag('h4')
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	193	self.input_tags = self.find_tag('input')
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	194	self.title_tags = [self.h1_tags, self.h2_tags, self.h3_tags, self.h4_tags]
1485 4d532f3c012e nicer fix sylvain.thenault@logilab.fr parents: 1480 diff changeset	195
7014 7e3e80f4179a [testlib pageinfo] extract matching_node method from has_tag to ease looking for a node with a given set of attributes Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 6977 diff changeset	196	def _iterstr(self, tag):
6977 cb78108bf603 [testlib] new method on page info object to ensure some tag with arbitrary attributes is found Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 6772 diff changeset	197	if self.default_ns is None:
cb78108bf603 [testlib] new method on page info object to ensure some tag with arbitrary attributes is found Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 6772 diff changeset	198	return ".//%s" % tag
cb78108bf603 [testlib] new method on page info object to ensure some tag with arbitrary attributes is found Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 6772 diff changeset	199	else:
cb78108bf603 [testlib] new method on page info object to ensure some tag with arbitrary attributes is found Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 6772 diff changeset	200	return ".//{%s}%s" % (self.default_ns, tag)
cb78108bf603 [testlib] new method on page info object to ensure some tag with arbitrary attributes is found Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 6772 diff changeset	201
7014 7e3e80f4179a [testlib pageinfo] extract matching_node method from has_tag to ease looking for a node with a given set of attributes Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 6977 diff changeset	202	def matching_nodes(self, tag, **attrs):
7e3e80f4179a [testlib pageinfo] extract matching_node method from has_tag to ease looking for a node with a given set of attributes Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 6977 diff changeset	203	for elt in self.etree.iterfind(self._iterstr(tag)):
7e3e80f4179a [testlib pageinfo] extract matching_node method from has_tag to ease looking for a node with a given set of attributes Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 6977 diff changeset	204	eltattrs = elt.attrib
10662 10942ed172de [py3k] dict.iteritems → dict.items Rémi Cardona <remi.cardona@logilab.fr> parents: 10618 diff changeset	205	for attr, value in attrs.items():
7014 7e3e80f4179a [testlib pageinfo] extract matching_node method from has_tag to ease looking for a node with a given set of attributes Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 6977 diff changeset	206	try:
7e3e80f4179a [testlib pageinfo] extract matching_node method from has_tag to ease looking for a node with a given set of attributes Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 6977 diff changeset	207	if eltattrs[attr] != value:
7e3e80f4179a [testlib pageinfo] extract matching_node method from has_tag to ease looking for a node with a given set of attributes Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 6977 diff changeset	208	break
7e3e80f4179a [testlib pageinfo] extract matching_node method from has_tag to ease looking for a node with a given set of attributes Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 6977 diff changeset	209	except KeyError:
7e3e80f4179a [testlib pageinfo] extract matching_node method from has_tag to ease looking for a node with a given set of attributes Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 6977 diff changeset	210	break
7e3e80f4179a [testlib pageinfo] extract matching_node method from has_tag to ease looking for a node with a given set of attributes Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 6977 diff changeset	211	else: # all attributes match
7e3e80f4179a [testlib pageinfo] extract matching_node method from has_tag to ease looking for a node with a given set of attributes Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 6977 diff changeset	212	yield elt
7e3e80f4179a [testlib pageinfo] extract matching_node method from has_tag to ease looking for a node with a given set of attributes Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 6977 diff changeset	213
7e3e80f4179a [testlib pageinfo] extract matching_node method from has_tag to ease looking for a node with a given set of attributes Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 6977 diff changeset	214	def has_tag(self, tag, nboccurs=1, **attrs):
7e3e80f4179a [testlib pageinfo] extract matching_node method from has_tag to ease looking for a node with a given set of attributes Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 6977 diff changeset	215	"""returns True if tag with given attributes appears in the page
7e3e80f4179a [testlib pageinfo] extract matching_node method from has_tag to ease looking for a node with a given set of attributes Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 6977 diff changeset	216	`nbtimes` (any if None)
7e3e80f4179a [testlib pageinfo] extract matching_node method from has_tag to ease looking for a node with a given set of attributes Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 6977 diff changeset	217	"""
7e3e80f4179a [testlib pageinfo] extract matching_node method from has_tag to ease looking for a node with a given set of attributes Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 6977 diff changeset	218	for elt in self.matching_nodes(tag, **attrs):
7e3e80f4179a [testlib pageinfo] extract matching_node method from has_tag to ease looking for a node with a given set of attributes Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 6977 diff changeset	219	if nboccurs is None: # no need to check number of occurences
7e3e80f4179a [testlib pageinfo] extract matching_node method from has_tag to ease looking for a node with a given set of attributes Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 6977 diff changeset	220	return True
7e3e80f4179a [testlib pageinfo] extract matching_node method from has_tag to ease looking for a node with a given set of attributes Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 6977 diff changeset	221	if not nboccurs: # too much occurences
7e3e80f4179a [testlib pageinfo] extract matching_node method from has_tag to ease looking for a node with a given set of attributes Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 6977 diff changeset	222	return False
7e3e80f4179a [testlib pageinfo] extract matching_node method from has_tag to ease looking for a node with a given set of attributes Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 6977 diff changeset	223	nboccurs -= 1
7e3e80f4179a [testlib pageinfo] extract matching_node method from has_tag to ease looking for a node with a given set of attributes Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 6977 diff changeset	224	if nboccurs == 0: # correct number of occurences
7e3e80f4179a [testlib pageinfo] extract matching_node method from has_tag to ease looking for a node with a given set of attributes Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 6977 diff changeset	225	return True
7e3e80f4179a [testlib pageinfo] extract matching_node method from has_tag to ease looking for a node with a given set of attributes Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 6977 diff changeset	226	return False # no matching tag/attrs
7e3e80f4179a [testlib pageinfo] extract matching_node method from has_tag to ease looking for a node with a given set of attributes Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 6977 diff changeset	227
1945 2b59d9ae17ae new argument telling if we want text or (text / attrs), keeping bw compat Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 1485 diff changeset	228	def find_tag(self, tag, gettext=True):
0 b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	229	"""return a list which contains text of all "tag" elements """
7014 7e3e80f4179a [testlib pageinfo] extract matching_node method from has_tag to ease looking for a node with a given set of attributes Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 6977 diff changeset	230	iterstr = self._iterstr(tag)
1945 2b59d9ae17ae new argument telling if we want text or (text / attrs), keeping bw compat Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 1485 diff changeset	231	if not gettext or tag in ('a', 'input'):
6977 cb78108bf603 [testlib] new method on page info object to ensure some tag with arbitrary attributes is found Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 6772 diff changeset	232	return [(elt.text, elt.attrib)
cb78108bf603 [testlib] new method on page info object to ensure some tag with arbitrary attributes is found Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 6772 diff changeset	233	for elt in self.etree.iterfind(iterstr)]
cb78108bf603 [testlib] new method on page info object to ensure some tag with arbitrary attributes is found Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 6772 diff changeset	234	return [u''.join(elt.xpath('.//text()'))
cb78108bf603 [testlib] new method on page info object to ensure some tag with arbitrary attributes is found Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 6772 diff changeset	235	for elt in self.etree.iterfind(iterstr)]
1485 4d532f3c012e nicer fix sylvain.thenault@logilab.fr parents: 1480 diff changeset	236
0 b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	237	def appears(self, text):
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	238	"""returns True if <text> appears in the page"""
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	239	return text in self.raw_text
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	240
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	241	def __contains__(self, text):
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	242	return text in self.source
1485 4d532f3c012e nicer fix sylvain.thenault@logilab.fr parents: 1480 diff changeset	243
0 b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	244	def has_title(self, text, level=None):
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	245	"""returns True if <h?>text</h?>
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	246
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	247	:param level: the title's level (1 for h1, 2 for h2, etc.)
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	248	"""
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	249	if level is None:
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	250	for hlist in self.title_tags:
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	251	if text in hlist:
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	252	return True
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	253	return False
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	254	else:
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	255	hlist = self.title_tags[level - 1]
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	256	return text in hlist
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	257
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	258	def has_title_regexp(self, pattern, level=None):
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	259	"""returns True if <h?>pattern</h?>"""
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	260	sre = re.compile(pattern)
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	261	if level is None:
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	262	for hlist in self.title_tags:
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	263	for title in hlist:
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	264	if sre.match(title):
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	265	return True
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	266	return False
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	267	else:
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	268	hlist = self.title_tags[level - 1]
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	269	for title in hlist:
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	270	if sre.match(title):
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	271	return True
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	272	return False
1485 4d532f3c012e nicer fix sylvain.thenault@logilab.fr parents: 1480 diff changeset	273
0 b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	274	def has_link(self, text, url=None):
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	275	"""returns True if <a href=url>text</a> was found in the page"""
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	276	for link_text, attrs in self.a_tags:
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	277	if text == link_text:
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	278	if url is None:
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	279	return True
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	280	try:
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	281	href = attrs['href']
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	282	if href == url:
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	283	return True
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	284	except KeyError:
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	285	continue
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	286	return False
1485 4d532f3c012e nicer fix sylvain.thenault@logilab.fr parents: 1480 diff changeset	287
0 b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	288	def has_link_regexp(self, pattern, url=None):
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	289	"""returns True if <a href=url>pattern</a> was found in the page"""
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	290	sre = re.compile(pattern)
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	291	for link_text, attrs in self.a_tags:
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	292	if sre.match(link_text):
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	293	if url is None:
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	294	return True
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	295	try:
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	296	href = attrs['href']
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	297	if href == url:
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	298	return True
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	299	except KeyError:
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	300	continue
b97547f5f1fa Showtime ! Adrien Di Mascio <Adrien.DiMascio@logilab.fr> parents: diff changeset	301	return False
2773 b2530e3e0afb [testlib] #345052 and #344207: major test lib refactoring/cleanup + update usage Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 1977 diff changeset	302
8972 771337c3a754 [testlib] update htmlparsers.VALMAP: stop using SaxOnlyValidator and add an entry for html Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8951 diff changeset	303	VALMAP = {None: None,
771337c3a754 [testlib] update htmlparsers.VALMAP: stop using SaxOnlyValidator and add an entry for html Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8951 diff changeset	304	'dtd': DTDValidator,
771337c3a754 [testlib] update htmlparsers.VALMAP: stop using SaxOnlyValidator and add an entry for html Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8951 diff changeset	305	'xml': XMLValidator,
771337c3a754 [testlib] update htmlparsers.VALMAP: stop using SaxOnlyValidator and add an entry for html Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8951 diff changeset	306	'html': HTMLValidator,
771337c3a754 [testlib] update htmlparsers.VALMAP: stop using SaxOnlyValidator and add an entry for html Sylvain Thénault <sylvain.thenault@logilab.fr> parents: 8951 diff changeset	307	}

author	Jérémy Bobbio <jeremy.bobbio@irq7.fr>
	Tue, 09 Jul 2019 11:13:08 +0200
changeset 12684	d464495452aa
parent 12505	55014a79b2a5
permissions	-rw-r--r--