html5lib is a pure-python library for parsing HTML. It is designed to conform to the WHATWG HTML specification, as is implemented by all major web browsers.
Simple usage follows this pattern:
import html5lib
with open("mydocument.html", "rb") as f:
document = html5lib.parse(f)or:
import html5lib
document = html5lib.parse("<p>Hello World!")By default, the document will be an xml.etree element instance.
Whenever possible, html5lib chooses the accelerated ElementTree
implementation (i.e. xml.etree.cElementTree on Python 2.x).
Two other tree types are supported: xml.dom.minidom and
lxml.etree. To use an alternative format, specify the name of
a treebuilder:
import html5lib
with open("mydocument.html", "rb") as f:
lxml_etree_document = html5lib.parse(f, treebuilder="lxml")When using with urllib2 (Python 2), the charset from HTTP should be
pass into html5lib as follows:
from contextlib import closing
from urllib2 import urlopen
import html5lib
with closing(urlopen("http://example.com/")) as f:
document = html5lib.parse(f, encoding=f.info().getparam("charset"))When using with urllib.request (Python 3), the charset from HTTP
should be pass into html5lib as follows:
from urllib.request import urlopen
import html5lib
with urlopen("http://example.com/") as f:
document = html5lib.parse(f, encoding=f.info().get_content_charset())To have more control over the parser, create a parser object explicitly. For instance, to make the parser raise exceptions on parse errors, use:
import html5lib
with open("mydocument.html", "rb") as f:
parser = html5lib.HTMLParser(strict=True)
document = parser.parse(f)When you're instantiating parser objects explicitly, pass a treebuilder
class as the tree keyword argument to use an alternative document
format:
import html5lib
parser = html5lib.HTMLParser(tree=html5lib.getTreeBuilder("dom"))
minidom_document = parser.parse("<p>Hello World!")More documentation is available at http://html5lib.readthedocs.org/.