The DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.
Web data extraction based on partial tree alignment
Abstract
This paper studies the problem of extracting data from a Web page that contains several structured data records. The objective is to segment these data records, extract data items/fields from them and put the data in a database table. This problem has been studied by several researchers. However, existing methods still have some serious limitations. The first class of methods is based on machine learning, which requires human labeling of many examples from each Web site that one is interested in extracting data from. The process is time consuming due to the large number of sites and pages on the Web. The second class of algorithms is based on automatic pattern discovery. These methods are either inaccurate or make many assumptions. This paper proposes a new method to perform the task automatically. It consists of two steps, (1) identifying individual data records in a page, and (2) aligning and extracting data items from the identified data records. For step 1, we propose a method based on visual information to segment data records, which is more accurate than existing methods. For step 2, we propose a novel partial alignment technique based on tree matching. Partial alignment means that we align only those data fields in a pair of data records that can be aligned (or matched) with certainty, and make no commitment on the rest of the data fields. This approach enables very accurate alignment of multiple data records. Experimental results using a large number of Web pages from diverse domains show that the proposed two-step technique is able to segment data records, align and extract data from them very accurately.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
RoadRunner: Towards Automatic Data Extraction from Large Web Sites
Crescenzi, V., Mecca, G., & Merialdo, P. (2001) · Iris (Roma Tre University) · Cited by 1,043
The Tree-to-Tree Correction Problem
Tai, K.-C. (1979) · Journal of the ACM · doi:10.1145/322139.322143 · Cited by 865
Extracting structured data from Web pages
Arasu, A., & García-Molina, H. (2003) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/872757.872799 · Cited by 756
Mining the Web: Discovering Knowledge from Hypertext Data
Chakrabarti, S. (2002) · Thammasat University Digital Collections · Cited by 696
Wrapper induction: Efficiency and expressiveness
Kushmerick, N. (2000) · Artificial Intelligence · doi:10.1016/s0004-3702(99)00100-9 · Cited by 553
The Multiple Sequence Alignment Problem in Biology
Carrillo-Calvet, H., & Lipman, D. J. (1988) · SIAM Journal on Applied Mathematics · doi:10.1137/0148063 · Cited by 526
A strategy for the rapid multiple alignment of protein sequences
Barton, G. J., & Sternberg, M. J. E. (1987) · Journal of Molecular Biology · doi:10.1016/0022-2836(87)90316-0 · Cited by 478
Generating finite-state transducers for semi-structured data extraction from the Web
Hsu, C.-N., & Dung, M.-T. (1998) · Information Systems · doi:10.1016/s0306-4379(98)00027-1 · Cited by 411
IEPAD: information extraction based on pattern discovery
Chang, C.-H., & Lui, S.-C. (2001) · Proceedings of the 10th international conference on World Wide Web · doi:10.1145/371920.372182 · Cited by 411
A hierarchical approach to wrapper induction
Muslea, I., Minton, S., & Knoblock, C. (1999) · Proceedings of the third annual conference on Autonomous Agents · doi:10.1145/301136.301191 · Cited by 371
On the editing distance between unordered labeled trees
Zhang, K., Statman, R., & Shasha, D. E. (1992) · Information Processing Letters · doi:10.1016/0020-0190(92)90136-j · Cited by 359
Table extraction using conditional random fields
Pinto, D., McCallum, A., Wei, X., & Croft, W. B. (2003) · Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval · doi:10.1145/860435.860479 · Cited by 355
Recent progress in multiple sequence alignment: a survey
Notredame, C. (2002) · Pharmacogenomics · doi:10.1517/14622416.3.1.131 · Cited by 353
Identifying syntactic differences between two programs
Yang, W. (1991) · Software: Practice and Experience · doi:10.1002/spe.4380210706 · Cited by 346
Automatic web news extraction using tree edit distance
Reis, D. C., Golgher, P. B., Silva, A. S., & Laender, A. F. (2004) · Proceedings of the 13th international conference on World Wide Web · doi:10.1145/988672.988740 · Cited by 342
Learning block importance models for web pages
Song, R., Liu, H., Wen, J.-R., & Ma, W.-Y. (2004) · Proceedings of the 13th international conference on World Wide Web · doi:10.1145/988672.988700 · Cited by 286
Template detection via data mining and its applications
Bar-Yossef, Z., & Rajagopalan, S. (2002) · Proceedings of the 11th international conference on World Wide Web · doi:10.1145/511446.511522 · Cited by 271
The alignment of sets of sequences and the construction of phyletic trees: An integrated method
Hogeweg, P., & Hesper, B. (1984) · Journal of Molecular Evolution · doi:10.1007/bf02257378 · Cited by 263
A flexible learning system for wrapping tables and lists in HTML documents
Cohen, W. W., Hurst, M. F., & Jensen, L. S. (2002) · International World Wide Web Conference (WWW) · doi:10.1145/511446.511477 · Cited by 235
A machine learning based approach for table detection on the web
Wang, Y., & Hu, J. (2002) · Proceedings of the 11th international conference on World Wide Web · doi:10.1145/511446.511478 · Cited by 199
Mining tables from large scale HTML texts
Chen, H.-H., Tsai, S.-C., & Tsai, J.-H. (2000) · Proceedings of the 18th conference on Computational linguistics - · doi:10.3115/990820.990845 · Cited by 166
New Algorithm for Ordered Tree-to-Tree Correction Problem
Chen, W. (2001) · Journal of Algorithms · doi:10.1006/jagm.2001.1170 · Cited by 93
Automatic detection of fragments in dynamically generated web pages
Ramaswamy, L. M., Iyengar, A., LIU, L., & Douglis, F. (2004) · International World Wide Web Conference (WWW) · doi:10.1145/988672.988732 · Cited by 77
Algorithms for string searching
Baeza-Yates, R. A. (1989) · ACM SIGIR Forum · doi:10.1145/74697.74700 · Cited by 48
Tree edit distance and common subtrees
Feruglio, V., & Alejandro, G. (2002) · Cited by 10
(1997)
Cited by
Opinion observer: analyzing and comparing opinions on the Web
Liu, B., Hu, M., & Cheng, J. (2005) · Proceedings of the 14th international conference on World Wide Web - WWW '05 · doi:10.1145/1060745.1060797 · Cited by 1,597
Self-citationA Survey of Web Information Extraction Systems
Chang, C.-H., Kayed, M., Girgis, M. R., & Shaalan, K. F. (2006) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2006.152 · Cited by 793
A survey on sentiment detection of reviews
Tang, H., Tan, S., & Cheng, X. (2009) · Expert Systems with Applications · doi:10.1016/j.eswa.2009.02.063 · Cited by 476
Web data extraction, applications and techniques: A survey
Ferrara, E., De Meo, P., Fiumara, G., & Baumgartner, R. (2014) · Knowledge-Based Systems · doi:10.1016/j.knosys.2014.07.007 · Cited by 396
Metadata
- Authors
- Zhai, Y., & Liu, B.
- Venue
- Proceedings of the 14th international conference on World Wide Web - WWW '05
- Publisher
- ACM Press
- Published
- 2005
- Pages
- 76
- OpenAlex
- W2143309843
- Cited by
- 496
- References
- 26
- Type
- conference abstract
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.