The DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.
Robust and efficient fuzzy match for online data cleaning
Abstract
To ensure high data quality, data warehouses must validate and cleanse incoming data tuples from external sources. In many situations, clean tuples must match acceptable tuples in reference tables. For example, product name and description fields in a sales record from a distributor must match the pre-recorded name and description fields in a product reference relation.A significant challenge in such a scenario is to implement an efficient and accurate fuzzy match operation that can effectively clean an incoming tuple if it fails to match exactly with any tuple in the reference relation. In this paper, we propose a new similarity function which overcomes limitations of commonly used similarity functions, and develop an efficient fuzzy match algorithm. We demonstrate the effectiveness of our techniques by evaluating them on real datasets.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Baeza-Yates, R. A., & Ribeiro-Neto, B. A. (1999) · Cited by 11,505
Identification of common molecular subsequences
Smith, T. F., & Waterman, M. S. (1981) · Journal of Molecular Biology · doi:10.1016/0022-2836(81)90087-5 · Cited by 9,940
Motwani, R., & Raghavan, P. (1995) · Cambridge University Press eBooks · doi:10.1017/cbo9780511814075 · Cited by 5,123
Self-citationApproximate nearest neighbors: towards removing the curse of dimensionality
Indyk, P., & Motwani, R. (1998) · Proceedings of the thirtieth annual ACM symposium on Theory of computing - STOC '98 · doi:10.1145/276698.276876 · Cited by 4,188
Self-citationOn the resemblance and containment of documents
Broder, A. Z. (2002) · Proceedings. Compression and Complexity of SEQUENCES 1997 (Cat. No.97TB100171) · doi:10.1109/sequen.1997.666900 · Cited by 1,698
M-tree: An Efficient Access Method for Similarity Search in Metric Spaces
Ciaccia, P., Patella, M., & Zezula, P. (1997) · Cited by 1,527
The State of Record Linkage and Current Research Problems
Winkler, W. E. (1999) · Cited by 1,015
The merge/purge problem for large databases
Hernández, M. A., & Stolfo, S. J. (1995) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/223784.223807 · Cited by 809
Interactive deduplication using active learning
Sarawagi, S., & Bhamidipaty, A. (2002) · Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/775047.775087 · Cited by 692
Size-Estimation Framework with Applications to Transitive Closure and Reachability
Cohen, E. (1997) · Journal of Computer and System Sciences · doi:10.1006/jcss.1997.1534 · Cited by 416
Eliminating Fuzzy Duplicates in Data Warehouses
Ananthakrishna, R., Chaudhuri, S., & Ganti, V. (2002) · VLDB '02: Proceedings of the 28th International Conference on Very Large Databases · doi:10.1016/b978-155860869-6/50058-5 · Cited by 387
Self-citationCohen, W. W. (1998) · Proceedings of the 1998 ACM SIGMOD international conference on Management of data · doi:10.1145/276304.276323 · Cited by 379
Searching in metric spaces by spatial approximation
Navarro, G. (2002) · The VLDB Journal The International Journal on Very Large Data Bases · doi:10.1007/s007780200060 · Cited by 272
Data integration using similarity joins and a word-based information representation language
Cohen, W. W. (2000) · ACM Transactions on Information Systems · doi:10.1145/352595.352598 · Cited by 223
Indexing methods for approximate string matching
Navarro, G., Baeza-Yates, R. A., Sutinen, E., et al. (2001) · Cited by 189
Two algorithms for approxmate string matching in static texts
Jokinen, P., & Ukkonen, E. (1991) · Lecture notes in computer science · doi:10.1007/3-540-54345-7_67 · Cited by 167
Cited by
Duplicate Record Detection: A Survey
Elmagarmid, A. K., Ipeirotis, P. G., & Verykios, V. S. (2007) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2007.250581 · Cited by 1,267
Sarawagi, S. (2008) · Foundations and Trends in Databases · doi:10.1561/1900000003 · Cited by 707
Collective entity resolution in relational data
Bhattacharya, I., & Getoor, L. (2007) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/1217299.1217304 · Cited by 593
Reference reconciliation in complex information spaces
Dong, X., Halevy, A., & Madhavan, J. (2005) · Proceedings of the 2005 ACM SIGMOD international conference on Management of data · doi:10.1145/1066157.1066168 · Cited by 518
Enterprise Data Analysis and Visualization: An Interview Study
Kandel, S., Paepcke, A., Hellerstein, J. M., & Heer, J. (2012) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2012.219 · Cited by 439
Metadata
- Authors
- Chaudhuri, S., Ganjam, K., Ganti, V., & Motwani, R.
- Venue
- Proceedings of the 2003 ACM SIGMOD international conference on Management of data
- Publisher
- ACM
- Published
- 2003-06-09
- Pages
- 313-324
- ISSN
- 0730-8078 (ISSN-L)
- OpenAlex
- W2105423800
- Cited by
- 457
- References
- 16
- FWCI
- 39.73
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.