Trinity: On Using Trinary Trees for Unsupervised Web Data Extraction

• Published in: IEEE transactions on knowledge and data engineering Vol. 26; no. 6; pp. 1544 - 1556
• Main Authors: Sleiman, Hassan A., Corchuelo, Rafael
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.06.2014; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithm design and analysis; Computing Methodologies; Data mining; HTML; Information extraction; Java; Knowledge and data engineering tools and techniques; Machine learning; Particle separators; Partitioning algorithms; Pattern Recognition; Proposals; Software; Web data extraction; automatic wrapper generation; wrappers; unsupervised learning
• ISSN: 1041-4347; 1558-2191
• DOI: 10.1109/TKDE.2013.161

Web data extractors are used to extract data from web documents in order to feed automated processes. In this article, we propose a technique that works on two or more web documents generated by the same server-side template and learns a regular expression that models it and can later be used to extract data from similar documents. The technique builds on the hypothesis that the template introduces some shared patterns that do not provide any relevant data and can thus be ignored. We have evaluated and compared our technique to others in the literature on a large collection of web documents; our results demonstrate that our proposal performs better than the others and that input errors do not have a negative impact on its effectiveness; furthermore, its efficiency can be easily boosted by means of a couple of parameters, without sacrificing its effectiveness.