HoloClean: holistic data repairs with probabilistic inference
The DOI matches its Crossref record; 49 of the 49 references it cites were found in a registry or an index.
Abstract
We introduce HoloClean, a framework for holistic data repairing driven by probabilistic inference. HoloClean unifies qualitative data repairing, which relies on integrity constraints or external data sources, with quantitative data repairing methods, which leverage statistical properties of the input data. Given an inconsistent dataset as input, HoloClean automatically generates a probabilistic program that performs data repairing. Inspired by recent theoretical advances in probabilistic inference, we introduce a series of optimizations which ensure that inference over HoloClean's probabilistic model scales to instances with millions of tuples. We show that HoloClean finds data repairs with an average precision of ∼ 90% and an average recall of above ∼ 76% across a diverse array of datasets exhibiting different types of errors. This yields an average F1 improvement of more than 2× against state-of-the-art methods.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Methods to automatically identify the appropriate data cleaning activities for an incomplete or biased dataset are still emerging, but could be achieved with the use of frameworks such as MLClean (that removes data poisoning to help training accurate and fair models [100]) and tools such as like HoloClean, Activeclean [98,99] and Universal Cleanser [160].”
From Operationalising ethics in artificial intelligence for healthcare: a framework for AI developers (2023) · cites this paper as [98]
Citations
Cites
Koller, D., & Friedman, N. (2009) · The MIT Press eBooks · Cited by 7,555
(2000) · Cited by 1,467
Wrangler: interactive visual specification of data transformation scripts
Kandel, S., Paepcke, A., Hellerstein, J., & Heer, J. (2011) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1978942.1979444 · Cited by 581
A cost-based model and effective heuristic for repairing constraints by value modification
Bohannon, P., Fan, W., Flaster, M., & Rastogi, R. (2005) · Proceedings of the 2005 ACM SIGMOD international conference on Management of data · doi:10.1145/1066157.1066175 · Cited by 424
Conditional Functional Dependencies for Data Cleaning
Bohannon, P., Fan, W., Geerts, F., Jia, X., & Kementsietsidis, A. (2007) · 2007 IEEE 23rd International Conference on Data Engineering · doi:10.1109/icde.2007.367920 · Cited by 375
Data Programming: Creating Large Training Sets, Quickly
Ratner, A., De, C., Wu, S., Selsam, D., & Ré, C. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.07723 · Cited by 350
Self-citationEntity resolution: theory, practice & open challenges
Getoor, L., & Machanavajjhala, A. (2012) · Proceedings of the VLDB Endowment · doi:10.14778/2367502.2367564 · Cited by 348
Minimal-change integrity maintenance using tuple deletions
Chomicki, J., & Marcinkowski, J. (2005) · Information and Computation · doi:10.1016/j.ic.2004.04.007 · Cited by 347
Holistic data cleaning: Putting violations into context
Chu, X., Ilyas, I. F., & Papotti, P. (2013) · 2013 IEEE 29th International Conference on Data Engineering (ICDE) · doi:10.1109/icde.2013.6544847 · Cited by 326
Self-citationNADEEF: a commodity data cleaning system
Dallachiesa, M., Ebaid, A., Eldawy, A., Elmagarmid, A., Ilyas, I. F., Ouzzani, M., & Tang, N. (2013) · Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data · doi:10.1145/2463676.2465327 · Cited by 313
Self-citationImproving data quality: consistency and accuracy
Cong, G., Fan, W., Geerts, F., et al. (2007) · Edinburgh Research Explorer (University of Edinburgh) · Cited by 313
Record linkage: similarity measures and algorithms
Koudas, N., Sarawagi, S., & Srivastava, D. (2006) · Proceedings of the 2006 ACM SIGMOD international conference on Management of data · doi:10.1145/1142473.1142599 · Cited by 289
KATARA: A Data Cleaning System Powered by Knowledge Bases and Crowdsourcing
Chu, X., Morcos, J., Ilyas, I. F., Ouzzani, M., Papotti, P., Tang, N., & Ye, Y. (2015) · Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data · doi:10.1145/2723372.2749431 · Cited by 279
Self-citationHinge-Loss Markov Random Fields and Probabilistic Soft LogicPreprint
Bach, S. H., Broecheler, M., Huang, B., & Getoor, L. C. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.04406 · Cited by 270
Discovering denial constraints
Chu, X., Ilyas, I. F., & Papotti, P. (2013) · Proceedings of the VLDB Endowment · doi:10.14778/2536258.2536262 · Cited by 262
Self-citationFoundations of Data Quality Management
Fan, W., & Geerts, F. (2012) · Synthesis Lectures on Data Management · doi:10.1007/978-3-031-01892-3 · Cited by 251
Tuffy: scaling up statistical inference in Markov logic networks using an RDBMS
Niu, F., Ré, C., Doan, A., & Shavlik, J. (2011) · Proceedings of the VLDB Endowment · doi:10.14778/1978665.1978669 · Cited by 251
Self-citationDetecting data errors: where are we and what needs to be done?
Abedjan, Z., Chu, X., Deng, D., Fernandez, R. C., Ilyas, I. F., Ouzzani, M., Papotti, P., Stonebraker, M., & Tang, N. (2016) · Proceedings of the VLDB Endowment · doi:10.14778/2994509.2994518 · Cited by 251
Self-citationTruth finding on the deep web: is the problem solved?
Li, X., Dong, X. L., Lyons, K., Meng, W., & Srivastava, D. (2012) · Proceedings of the VLDB Endowment · doi:10.14778/2535568.2448943 · Cited by 247
An Introduction to Duplicate Detection
Naumann, F., & Herschel, M. (2010) · Synthesis lectures on data management · doi:10.2200/s00262ed1v01y201003dtm003 · Cited by 245
Dependencies revisited for improving data quality
Fan, W. (2008) · Proceedings of the twenty-seventh ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems · doi:10.1145/1376916.1376940 · Cited by 240
Incremental knowledge base construction using DeepDive
Shin, J., Wu, S., Wang, F., De Sa, C., Zhang, C., & Ré, C. (2015) · Proceedings of the VLDB Endowment · doi:10.14778/2809974.2809991 · Cited by 232
Self-citationReasoning about record matching rules
Fan, W., Jia, X., Li, J., & Ma, S. (2009) · Proceedings of the VLDB Endowment · doi:10.14778/1687627.1687674 · Cited by 226
On approximating optimum repairs for functional dependency violations
Kolahi, S., & Lakshmanan, L. V. S. (2009) · Proceedings of the 12th International Conference on Database Theory · doi:10.1145/1514894.1514901 · Cited by 206
Towards certain fixes with editing rules and master data
Fan, W., Li, J., Ma, S., Tang, N., & Yu, W. (2010) · Proceedings of the VLDB Endowment · doi:10.14778/1920841.1920867 · Cited by 202
Quantitative Data Cleaning for Large Databases
Hellerstein, J. M. (2008) · Cited by 201
Data Curation at Scale: The Data Tamer System
Stonebraker, M. R., Beskales, G., Pagan, A., Bruckner, D., Cherniack, M., Xu, S., Analytics, V., Ilyas, I. F., & Zdonik, S. (2013) · Cited by 191
Self-citationDetecting anomalous records in categorical datasets
Das, K., & Schneider, J. (2007) · Proceedings of the 13th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1281192.1281219 · Cited by 180
The LLUNATIC data-cleaning framework
Geerts, F., Mecca, G., Papotti, P., & Santoro, D. (2013) · Proceedings of the VLDB Endowment · doi:10.14778/2536360.2536363 · Cited by 179
Computational and statistical tradeoffs via convex relaxation
Chandrasekaran, V., & Jordan, M. I. (2013) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1302293110 · Cited by 179
ERACER: a database approach for statistical inference and data cleaning
Mayfield, C., Neville, J., & Prabhakar, S. (2010) · Proceedings of the 2010 ACM SIGMOD International Conference on Management of data · doi:10.1145/1807167.1807178 · Cited by 173
Yakout, M., Berti‐Équille, L., & Elmagarmid, A. K. (2013) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/2463676.2463706 · Cited by 150
Repair checking in inconsistent databases: algorithms and complexity
Afrati, F., & Kolaitis, P. G. (2009) · International Conference on Database Theory · doi:10.1145/1514894.1514899 · Cited by 150
DimmWitted: a study of main-memory statistical analytics
Zhang, C., & Ré, C. (2014) · Proceedings of the VLDB Endowment · doi:10.14778/2732977.2733001 · Cited by 131
Self-citationSampling the repairs of functional dependency violations under hard constraints
Beskales, G., Ilyas, I. F., & Golab, L. (2010) · Proceedings of the VLDB Endowment · doi:10.14778/1920841.1920870 · Cited by 127
Self-citationTrends in Cleaning Relational Data: Consistency and Deduplication
Ilyas, I. F., & Chu, X. (2015) · Foundations and Trends in Databases · doi:10.1561/1900000045 · Cited by 127
Self-citationTowards dependable data repairing with fixing rules
Wang, J., & Tang, N. (2014) · Proceedings - ACM-SIGMOD International Conference on Management of Data · doi:10.1145/2588555.2610494 · Cited by 104
On the relative trust between inconsistent data and inaccurate constraints
Beskales, G., Ilyas, I. F., Golab, L., & Galiullin, A. A. (2013) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde.2013.6544854 · Cited by 103
Self-citationMetric Functional Dependencies
Koudas, N., Saha, A., Srivastava, D., et al. (2009) · Proceedings - International Conference on Data Engineering · doi:10.1109/icde.2009.219 · Cited by 95
Trends in Cleaning Relational Data: Consistency and Deduplication
Ilyas, I. F., & Chu, X. (2015) · now publishers, Inc. eBooks · doi:10.1561/9781680830231 · Cited by 83
Self-citationData cleaning and query answering with matching dependencies and matching functions
Bertossi, L., Kolahi, S., & Lakshmanan, L. V. S. (2011) · International Conference on Database Theory · doi:10.1145/1938551.1938585 · Cited by 70
Rekatsinas, T., Joglekar, M., García-Molina, H., Parameswaran, A., & Ré, C. (2017) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/3035918.3035951 · Cited by 60
Self-citationUniform Solution Sampling Using a Constraint Solver As an OraclePreprint
Ermon, S., Gomes, C. P., & Selman, B. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1210.4861 · Cited by 40
Proof positive and negative in data cleaning
Interlandi, M., & Tang, N. (2015) · Proceedings - International Conference on Data Engineering · doi:10.1109/icde.2015.7113269 · Cited by 38
A general lower bound for mixing of single-site dynamics on graphs
Hayes, T. P., & Sinclair, A. (2007) · The Annals of Applied Probability · doi:10.1214/105051607000000104 · Cited by 35
Dichotomies in the Complexity of Preferred Repairs
Fagin, R., Kimelfeld, B., & Kolaitis, P. G. (2015) · ACM SIGMOD-SIGACT-SIGAI Symposium on Principles of Database Systems (PODS) · doi:10.1145/2745754.2745762 · Cited by 31
Rapidly Mixing Gibbs Sampling for a Class of Factor Graphs Using Hierarchy Width
De, C., Zhang, C., Olukotun, K., & Ré, C. (2015) · PubMed · doi:10.48550/arxiv.1510.00756 · Cited by 23
Self-citationInteractive Data Exploration via Machine Learning Models.
Papaemmanouil, O., Diao, Y., Dimitriadou, K., et al. (2016) · IEEE Data(base) Engineering Bulletin · Cited by 21
United Nations Economic Commission for Europe (UNECE)
United Nations Economic Commission for Europe (2022) · ATP · doi:10.18356/9789210014175c001 · Cited by 3
Research works printed but not linked: 1
- I. F. Ilyas . Effective data cleaning with continuous evaluation . IEEE Data Eng. Bull. , 39 : 38 -- 46 , 2016 . I. F. Ilyas. Effective data cleaning with continuous evaluation. IEEE Data Eng. Bull., 39:38--46, 2016.
Cited by
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
Self-citationA Survey on Data Collection for Machine Learning: A Big Data - AI Integration Perspective
Roh, Y., Heo, G., & Whang, S. E. (2021) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2019.2946162 · Cited by 968
Snorkel: rapid training data creation with weak supervision
Ratner, A., Bach, S. H., Ehrenberg, H., Fries, J., Wu, S., & Ré, C. (2017) · Proceedings of the VLDB Endowment · doi:10.14778/3157794.3157797 · Cited by 777
Self-citationBenchmark and Survey of Automated Machine Learning Frameworks
Zöller, M.-A., & Huber, M. F. (2021) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.11854 · Cited by 448
Enhancing transportation systems via deep learning: A survey
Wang, Y., Zhang, D., Liu, Y., Dai, B., & Lee, L. H. (2019) · Transportation Research Part C: Emerging Technologies · doi:10.1016/j.trc.2018.12.004 · Cited by 312
Metadata
- Authors
- Rekatsinas, T., Chu, X., Ilyas, I. F., & Ré, C.
- Affiliations
- Stanford University (US)
- University of Waterloo (CA)
- Venue
- Proceedings of the VLDB Endowment
- Published
- 2017-08
- Volume
- 10
- Issue
- 11
- Pages
- 1190-1201
- ISSN
- 2150-8097 (ISSN-L)
- OpenAlex
- W2591700809
- Cited by
- 480
- References
- 49
- Citations by year
- 2026: 48 · 2025: 59 · 2024: 68 · 2023: 41 · 2022: 51 · 2021: 55 · 2020: 57 · 2019: 64 · 2018: 33 · 2017: 4
- FWCI
- 38.32
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Proceedings of the VLDB Endowment (publishedVersion)
- Funders
- Defense Advanced Research Projects Agency
- Natural Sciences and Engineering Research Council of Canada
- Topics
- Physical Sciences › Computer Science › Information Systems › Data Quality and Management
- Physical Sciences › Computer Science › Information Systems › Advanced Database Systems and Queries
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Keywords
- data cleaning, data repair, probabilistic inference, probabilistic programs, data inconsistency, integrity constraints
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.