The DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index.
A Survey of Web Information Extraction Systems
Abstract
The Internet presents a huge amount of useful information which is usually formatted for its users, which makes it difficult to extract relevant data from various sources. Therefore, the availability of robust, flexible Information Extraction (IE) systems that transform the Web pages into program-friendly structures such as a relational database will become a great necessity. Although many approaches for data extraction from Web pages have been developed, there has been limited effort to compare such tools. Unfortunately, in only a few cases can the results generated by distinct tools be directly compared since the addressed extraction tasks are different. This paper surveys the major Web data extraction approaches and compares them in three dimensions: the task domain, the automation degree, and the techniques used. The criteria of the first dimension explain why an IE system fails to handle some Web sites of particular structures. The criteria of the second dimension classify IE systems based on the techniques used. The criteria of the third dimension measure the degree of automation for IE systems. We believe these criteria provide qualitatively measures to evaluate various IE approaches.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Fundamentals of Database Systems
Elmasri, R. A., & Navathe, S. B. (1989) · Cited by 3,141
Learning Information Extraction Rules for Semi-Structured and Free Text
Soderland, S. (1999) · Machine Learning · doi:10.1023/a:1007562322031 · Cited by 934
RoadRunner: Towards Automatic Data Extraction from Large Web Sites
Valter Crescenzi, A., Mecca, G., & Paolo Merialdo, A. (2001) · Cited by 928
Extracting structured data from Web pages
Arasu, A., & García-Molina, H. (2003) · Proceedings - ACM-SIGMOD International Conference on Management of Data · doi:10.1145/872757.872799 · Cited by 698
Relational Learning of Pattern-Match Rules for Information Extraction.
Califf, M. E., & Mooney, R. J. (1997) · Cited by 507
Web data extraction based on partial tree alignment
Zhai, Y., & Liu, B. (2005) · Proceedings of the 14th international conference on World Wide Web - WWW '05 · doi:10.1145/1060745.1060761 · Cited by 496
Automatically constructing a dictionary for information extraction tasks
Riloff, E. (1993) · Cited by 453
XWRAP: an XML-enabled wrapper construction system for Web information sources
Liu, L., Pu, C., & Han, W. (2002) · Proceedings of 16th International Conference on Data Engineering (Cat. No.00CB37073) · doi:10.1109/icde.2000.839475 · Cited by 422
Generating finite-state transducers for semi-structured data extraction from the Web
Hsu, C.-N., & Dung, M.-T. (1998) · Information Systems · doi:10.1016/s0306-4379(98)00027-1 · Cited by 411
IEPAD: information extraction based on pattern discovery
Chang, C.-H., & Lui, S.-C. (2001) · Proceedings of the 10th international conference on World Wide Web · doi:10.1145/371920.372182 · Cited by 411
Self-citationA hierarchical approach to wrapper induction
Muslea, I., Minton, S., & Knoblock, C. (1999) · Proceedings of the third annual conference on Autonomous Agents · doi:10.1145/301136.301191 · Cited by 371
Table extraction using conditional random fields
Pinto, D., McCallum, A., Wei, X., & Croft, W. B. (2003) · Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval · doi:10.1145/860435.860479 · Cited by 355
Eliminating noisy information in Web pages for data mining
Yi, L., Liu, B., & Li, X. (2003) · Proceedings of the ninth ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/956750.956785 · Cited by 332
Adelberg, B. (1998) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/276304.276330 · Cited by 331
An interactive clustering-based approach to integrating source query interfaces on the deep Web
Wu, W., Yu, C., Doan, A., & Meng, W. (2004) · Proceedings of the 2004 ACM SIGMOD international conference on Management of data · doi:10.1145/1007568.1007582 · Cited by 273
CRYSTAL: Inducing a Conceptual Dictionary
Soderland, S. G., Fisher, D., Aseltine, J., & Lehnert, W. G. (1995) · arXiv (Cornell University) · doi:10.48550/arxiv.cmp-lg/9505020 · Cited by 269
Information extraction from HTML: application of a general machine learning approach
Freitag, D. (1998) · Cited by 236
Extraction Patterns for Information Extraction Tasks: A Survey
Muslea, I. (1999) · Cited by 226
Learning information extraction patterns from examples
Huffman, S. B. (1996) · Lecture notes in computer science · doi:10.1007/3-540-60925-3_51 · Cited by 149
Discovering complex matchings across web query interfaces: a correlation mining approach
He, B., Chang, K. C., & Han, J. (2004) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/1014052.1014071 · Cited by 141
(1998) · doi:10.1016/s0306-4379(98)00028-3 · Cited by 119
Adaptive Information Extraction: Core Technologies for Information Agents
Kushmerick, N., & Thomas, B. (2003) · Lecture notes in computer science · doi:10.1007/3-540-36561-3_4 · Cited by 68
Extracting semi-structured data through examples
Ribeiro-Neto, B. A., Laender, A. H. F., & da Silva, A. S. (1999) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/319950.319962 · Cited by 58
Cited by
Exploring entrepreneurial legitimacy in reward-based crowdfunding
Frydrych, D., Bock, A. J., Kinder, T., & Koeck, B. (2014) · Venture Capital · doi:10.1080/13691066.2014.916512 · Cited by 448
Web data extraction, applications and techniques: A survey
Ferrara, E., De Meo, P., Fiumara, G., & Baumgartner, R. (2014) · Knowledge-Based Systems · doi:10.1016/j.knosys.2014.07.007 · Cited by 396
Data-Driven Techniques in Disaster Information Management
Li, T., Xie, N., Zeng, C., Zhou, W., Zheng, L., Jiang, Y., Yang, Y., Ha, H.-Y., Xue, W., Huang, Y., et al. (2018) · ACM Computing Surveys · doi:10.1145/3017678 · Cited by 360
Big data challenge: a data management perspective
Chen, J., Chen, Y., Du, X., Li, C., Lu, J., Zhao, S., & Zhou, X. (2013) · Frontiers of Computer Science · doi:10.1007/s11704-013-3903-7 · Cited by 339
Attention please!: learning analytics for visualization and recommendation
Duval, E. (2011) · Proceedings of the 1st International Conference on Learning Analytics and Knowledge · doi:10.1145/2090116.2090118 · Cited by 325
Metadata
- Authors
- Chang, C.-H., Kayed, M., Girgis, M. R., & Shaalan, K. F.
- Venue
- IEEE Transactions on Knowledge and Data Engineering
- Published
- 2006-10
- Volume
- 18
- Issue
- 10
- Pages
- 1411-1428
- ISSN
- 1041-4347 (ISSN-L)
- OpenAlex
- W2134150392
- Cited by
- 793
- References
- 23
- FWCI
- 119.67
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.