The DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.
TAILOR: a record linkage toolbox
Abstract
Data cleaning is a vital process that ensures the quality of data stored in real-world databases. Data cleaning problems are frequently encountered in many research areas, such as knowledge discovery in databases, data warehousing, system integration and e-services. The process of identifying the record pairs that represent the same entity (duplicate records), commonly known as record linkage, is one of the essential elements of data cleaning. In this paper, we address the record linkage problem by adopting a machine learning approach. Three models are proposed and are analyzed empirically. Since no existing model, including those proposed in this paper, has been proved to be superior, we have developed an interactive record linkage toolbox named TAILOR (backwards acronym for "RecOrd LInkAge Toolbox"). Users of TAILOR can build their own record linkage models by tuning system parameters and by plugging in in-house-developed and public-domain tools. The proposed toolbox serves as a framework for the record linkage process, and is designed in an extensible way to interface with existing and future record linkage models. We have conducted an extensive experimental study to evaluate our proposed models using not only synthetic but also real data. The results show that the proposed machine-learning record linkage models outperform the existing ones both in accuracy and in performance.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Maximum Likelihood from Incomplete Data Via the EM Algorithm
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1977.tb01600.x · Cited by 50,444
Quinlan, J. R. (1986) · Machine Learning · doi:10.1007/bf00116251 · Cited by 21,439
Identification of common molecular subsequences
Smith, T. F., & Waterman, M. S. (1981) · Journal of Molecular Biology · doi:10.1016/0022-2836(81)90087-5 · Cited by 9,940
The Art of Computer Programming
Whitesides, G. E. (1970) · Nuclear Science and Engineering · doi:10.13182/nse70-a19705 · Cited by 6,008
(1996) · Cited by 4,738
Maximum likelihood estimation from incomplete data via the EM algorithm
Dempster, A. P. (1977) · Journal of the Royal Statistical Society Series A (Statistics in Society) · Cited by 4,543
Maximum Likelihood from Incomplete Data via the EM Algorithm
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1976) · Cited by 4,517
Fellegi, I. P., & Sunter, A. B. (1969) · Journal of the American Statistical Association · doi:10.1080/01621459.1969.10501049 · Cited by 2,475
Information Retrieval: Data Structures and Algorithms
Frakes, W. B., & Baeza-Yates, R. A. (1992) · Cited by 2,393
Advances in Record-Linkage Methodology as Applied to Matching the 1985 Census of Tampa, Florida
Jaro, M. A. (1989) · Journal of the American Statistical Association · doi:10.1080/01621459.1989.10478785 · Cited by 1,401
Refining Initial Points for K-Means Clustering
Bradley, P. S., & Fayyad, U. M. (1998) · Cited by 1,031
Quinlan (2003) · Cited by 1,000
Real-world Data is Dirty: Data Cleansing and The Merge/Purge Problem
Hernández, M. A., & Stolfo, S. J. (1998) · Data Mining and Knowledge Discovery · doi:10.1023/a:1009761603038 · Cited by 812
The State of Record Linkage and Current Research Problems
Winkler, W. E. (1999) · Cited by 810
The merge/purge problem for large databases
Hernández, M. A., & Stolfo, S. J. (1995) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/223784.223807 · Cited by 809
Newcombe, H. B., Kennedy, J. M., Axford, S. J., & James, A. P. (1959) · Science · doi:10.1126/science.130.3381.954 · Cited by 770
Data mining and knowledge discovery in databases
Fayyad, U., & Uthurusamy, R. (1996) · Communications of the ACM · doi:10.1145/240455.240463 · Cited by 590
Tolerating noisy, irrelevant and novel attributes in instance-based learning algorithms
Aha, D. W. (1992) · International Journal of Man-Machine Studies · doi:10.1016/0020-7373(92)90018-g · Cited by 405
Declarative Data Cleaning: Language, Model, and Algorithms
Galhardas, H., Florescu, D., Shasha, D. E., Simon, E., & Saita, C.-A. (2001) · Cited by 320
Data Mining Using MLC a Machine Learning Library in C
Kohavi, R., Sommerfield, D., & Dougherty, J. (1997) · Cited by 298
Duplicate record elimination in large data files
Bitton, D., & DeWitt, D. J. (1983) · ACM Transactions on Database Systems · doi:10.1145/319983.319987 · Cited by 179
Data mining using /spl Mscr/ℒ/spl Cscr/++ a machine learning library in C++
Kohavi, R., Sommerfield, D. A., & Dougherty, J. (2005) · IEEE International Conference on Tools with Artificial Intelligence (ICTAI) · doi:10.1109/tai.1996.560457 · Cited by 146
Automating the approximate record-matching process
Verykios, V. S., Elmagarmid, A. K., & Houstis, E. N. (2000) · Information Sciences · doi:10.1016/s0020-0255(00)00013-x · Cited by 102
Self-citationUsing the EM Algorithm for Weight Computation in the Fellegi-Sunter Model of Record Linkage
Winkler, W. E., & Winkler, W. E. (2000) · Cited by 94
A Probabilistic Decision Model for Entity Matching in Heterogeneous Databases
Dey, D., Sarkar, S., & De, P. (1998) · Management Science · doi:10.1287/mnsc.44.10.1379 · Cited by 62
On the Accuracy and Completeness of the Record Matching Process.
Verykios, V. S., Elfeky, M. G., Elmagarmid, A. K., Cochinwala, M., & Dalal, S. R. (2000) · Cited by 16
Self-citation
Cited by
Duplicate Record Detection: A Survey
Elmagarmid, A. K., Ipeirotis, P. G., & Verykios, V. S. (2007) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2007.250581 · Cited by 1,267
Self-citationA Survey of Indexing Techniques for Scalable Record Linkage and Deduplication
Christen, P. (2012) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2011.127 · Cited by 679
Evaluation of entity resolution approaches on real-world match problems
Köpcke, H., Thor, A., & Rahm, E. (2010) · Proceedings of the VLDB Endowment · doi:10.14778/1920841.1920904 · Cited by 443
A cost-based model and effective heuristic for repairing constraints by value modification
Bohannon, P., Fan, W., Flaster, M., & Rastogi, R. (2005) · Proceedings of the 2005 ACM SIGMOD international conference on Management of data · doi:10.1145/1066157.1066175 · Cited by 424
NADEEF: a commodity data cleaning system
Dallachiesa, M., Ebaid, A., Eldawy, A., Elmagarmid, A., Ilyas, I. F., Ouzzani, M., & Tang, N. (2013) · Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data · doi:10.1145/2463676.2465327 · Cited by 313
Self-citation
Metadata
- Authors
- Elfeky, M. G., Verykios, V. S., & Elmagarmid, A. K.
- Venue
- Proceedings 18th International Conference on Data Engineering
- Publisher
- IEEE Comput. Soc
- Published
- 2003-06-25
- Pages
- 17-28
- ISSN
- 1084-4627 (ISSN-L)
- OpenAlex
- W2161600801
- Cited by
- 289
- References
- 26
- FWCI
- 25.68
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.