We introduce
landmark grammars
, a new family of context-free grammars aimed at describing the HTML source code of pages published by large and templated websites and therefore at effectively tackling Web data extraction problems. Indeed, they address the inherent
ambiguity
of HTML, one of the main challenges of Web data extraction, which, despite over twenty years of research, has been largely neglected by the approaches presented in literature.
We then formalize the
Smallest Extraction Problem
(SEP), an optimization problem for finding the grammar of a family that best describes a set of pages and contextually extract their data.
Finally, we present an unsupervised learning algorithm to induce a landmark grammar from a set of pages sharing a common HTML template, and we present an automatic Web data extraction system. The experiments on consolidated benchmarks show that the approach can substantially contribute to improve the state-of-the-art.
Web Data Extraction using Hybrid Program Synthesis: A Combination of Top-down and Bottom-up Inference