ActiveClean: interactive data cleaning for statistical modeling
The DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index.
Abstract
Analysts often clean dirty data iteratively--cleaning some data, executing the analysis, and then cleaning more data based on the results. We explore the iterative cleaning process in the context of statistical model training, which is an increasingly popular form of data analytics. We propose ActiveClean, which allows for progressive and iterative cleaning in statistical modeling problems while preserving convergence guarantees. ActiveClean supports an important class of models called convex loss models (e.g., linear regression and SVMs), and prioritizes cleaning those records likely to affect the results. We evaluate ActiveClean on five real-world datasets UCI Adult, UCI EEG, MNIST, IMDB, and Dollars For Docs with both real and synthetic errors. The results show that our proposed optimizations can improve model accuracy by up-to 2.5x for the same amount of data cleaned. Furthermore for a fixed cleaning budget and on all real dirty datasets, ActiveClean returns more accurate models than uniform sampling and Active Learning.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Methods to automatically identify the appropriate data cleaning activities for an incomplete or biased dataset are still emerging, but could be achieved with the use of frameworks such as MLClean (that removes data poisoning to help training accurate and fair models [100]) and tools such as like HoloClean, Activeclean [98,99] and Universal Cleanser [160].”
From Operationalising ethics in artificial intelligence for healthcare: a framework for AI developers (2023) · cites this paper as [99]
Citations
Cites
The Elements of Statistical Learning
Hastie, T., Friedman, J., & Tibshirani, R. (2001) · Springer Series in Statistics · doi:10.1007/978-0-387-21606-5 · Cited by 24,162
Pan, S. J., & Yang, Q. (2010) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2009.191 · Cited by 23,387 · cited 1×
“Transfer Learning and Bias Mitigation: ActiveClean has a strong link to a field called Transfer Learning and Domain Adaptation [27].”
BackgroundCausality: Models, Reasoning, and Inference
Pearl, J. (2009) · doi:10.1017/cbo9780511803161 · Cited by 11,653
(2009) · Minds at UW (University of Wisconsin) · Cited by 4,791
(1996) · Cited by 4,738
The interpretation of interaction in contingency tables
Simpson, E. H. (1951) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1951.tb00088.x · Cited by 2,301 · cited 2×
“It is known that aggregates over mixtures of different populations of data can result in spurious relationships due to the well-known phenomenon called Simpson’s paradox [32].”
BackgroundMethodStochastic Gradient Descent Tricks
Bottou, L. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-35289-8_25 · Cited by 1,929
(2000) · Cited by 1,467
Model-Driven Data Acquisition in Sensor Networks
Deshpande, A., Guestrin, C., Madden, S. R., Hellerstein, J. M., & Hong, W. (2004) · Proceedings 2004 VLDB Conference · doi:10.1016/b978-012088469-8.50053-x · Cited by 1,036
Exploratory Data Mining and Data Cleaning
Dasu, T., & Johnson, T. (2003) · Wiley Series in Probability and Statistics · doi:10.1002/0471448354 · Cited by 618
Optimal Distributed Online Prediction using Mini-Batches
Dekel, O., Gilad-Bachrach, R., Shamir, O., & Xiao, L. (2010) · arXiv (Cornell University) · doi:10.48550/arxiv.1012.1367 · Cited by 512
Enterprise Data Analysis and Visualization: An Interview Study
Kandel, S., Paepcke, A., Hellerstein, J. M., & Heer, J. (2012) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2012.219 · Cited by 439
Incremental Gradient, Subgradient, and Proximal Methods for Convex Optimization: A Survey
Bertsekas, D. P. (2011) · Optimization for Machine Learning · doi:10.7551/mitpress/8996.003.0006 · Cited by 381
Is feature selection secure against training data poisoning?
Huang, X. X., Biggio, B., Brown, G., Fumera, G., Eckert, C. M., & Roli, F. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.07933 · Cited by 269
Foundations of Data Quality Management
Fan, W., & Geerts, F. (2012) · Synthesis Lectures on Data Management · doi:10.1007/978-3-031-01892-3 · Cited by 251
Declarative Support for Sensor Data Cleaning
Jeffery, S. R., Alonso, G., Franklin, M. J., Hong, W., & Widom, J. (2006) · Lecture Notes in Computer Science · doi:10.1007/11748625_6 · Cited by 249
Self-citationStochastic Optimization with Importance Sampling for Regularized Loss Minimization
Zhao, P., & Zhang, T. (2015) · Rare & Special e-Zone (The Hong Kong University of Science and Technology) · Cited by 228
Corleone: hands-off crowdsourcing for entity matching
Gokhale, C., Das, S., Doan, A., Naughton, J. F., Rampalli, N., Shavlik, J., & Zhu, X. (2014) · Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data · doi:10.1145/2588555.2588576 · Cited by 219
Yakout, M., Elmagarmid, A. K., Neville, J., Ouzzani, M., & Ilyas, I. F. (2011) · Proceedings of the VLDB Endowment · doi:10.14778/1952376.1952378 · Cited by 216
Scaling up crowd-sourcing to very large datasets: a case for active learning
Mozafari, B., Sarkar, P., Franklin, M., Jordan, M., & Madden, S. (2014) · Proceedings of the VLDB Endowment · doi:10.14778/2735471.2735474 · Cited by 165
Self-citationFast approximation of matrix coherence and statistical leverage
Petros Drineas, B., Magdon‐Ismail, M., Mahoney, M. W., & Woodruff, D. P. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1109.3843 · Cited by 164
Don't be SCAREd: use SCalable Automatic REpairing with maximal likelihood and bounded changes
Yakout, M., Berti-Équille, L., & Elmagarmid, A. K. (2013) · Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data · doi:10.1145/2463676.2463706 · Cited by 150
Progressive Duplicate Detection
Papenbrock, T., Heise, A., & Naumann, F. (2014) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2014.2359666 · Cited by 125
A sample-and-clean framework for fast and accurate query processing on dirty data
Wang, J., Krishnan, S., Franklin, M. J., Goldberg, K., Kraska, T., & Milo, T. (2014) · Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data · doi:10.1145/2588555.2610505 · Cited by 118
Self-citationRobust Logistic Regression and Classification
Feng, J., Xu, H., Mannor, S., & Yan, S. (2014) · neural information processing systems · Cited by 118
Query Strategies for Evading Convex-Inducing Classifiers
Nelson, B. A., Rubinstein, B. I. P., Huang, L., Joseph, A. D., Lee, S. J., Rao, S. B., & Tygar, J. D. (2010) · arXiv (Cornell University) · doi:10.48550/arxiv.1007.0484 · Cited by 94
A methodology for learning, analyzing, and mitigating social influence bias in recommender systems
Krishnan, S., Patel, J., Franklin, M. J., & Goldberg, K. (2014) · Proceedings of the 8th ACM Conference on Recommender systems · doi:10.1145/2645710.2645740 · Cited by 89
Self-citationTowards reliable interactive data cleaning: a user survey and recommendations
Krishnan, S., Haas, D., Franklin, M. J. M., & Wu, E. (2016) · Workshop on Human-In-the-Loop Data Analytics (HILDA) · doi:10.1145/2939502.2939511 · Cited by 81
Self-citationActiveClean: An Interactive Data Cleaning Framework For Modern Machine Learning
Krishnan, S., Franklin, M. J., Goldberg, K., Wang, J., & Wu, E. (2016) · Proceedings of the 2016 International Conference on Management of Data · doi:10.1145/2882903.2899409 · Cited by 72
Self-citationProgressive approach to relational entity resolution
Altowim, Y., Kalashnikov, D. V., & Mehrotra, S. (2014) · Proceedings of the VLDB Endowment · doi:10.14778/2732967.2732975 · Cited by 64
Declarative Support for Sensor Data Cleaning
Shawn, R. J., Alonso, G., Michael, J. F., Hong, W., & Widom, J. (2006) · Lecture notes in computer science · Cited by 61
Query-Oriented Data Cleaning with Oracles
Bergman, M., Milo, T., Novgorodov, S., & Tan, W.-C. (2015) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/2723372.2737786 · Cited by 53
COMMENTS ON NEUBERG'S REVIEW OF CAUSALITY
Pearl, J. (2003) · Econometric Theory · doi:10.1017/s0266466603004110 · Cited by 37
ActiveClean: Interactive Data Cleaning While Learning Convex Loss ModelsPreprint
Krishnan, S., Wang, J., Wu, E., Franklin, M. J. M., & Goldberg, K. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1601.03797 · Cited by 23
Self-citationEstimating the Number and Sizes of Fuzzy-Duplicate Clusters
Heise, A., Kasneci, G., & Naumann, F. (2014) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/2661829.2661885 · Cited by 19
Active Learning as Non-Convex Optimization
Guillory, A., Chastain, E. J., & Bilmes, J. (2009) · International Conference on Artificial Intelligence and Statistics · Cited by 19
Research works printed but not linked: 2
- 10.5555/861869 · doi:10.5555/861869
- 10.5555/1316689.1316741 · doi:10.5555/1316689.1316741
Cited by
A Survey on Data Collection for Machine Learning: A Big Data - AI Integration Perspective
Roh, Y., Heo, G., & Whang, S. E. (2021) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2019.2946162 · Cited by 968
Machine Learning Testing: Survey, Landscapes and Horizons
Zhang, J. M., Harman, M., Ma, L., & Liu, Y. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2019.2962027 · Cited by 897
“Everyone wants to do the model work, not the data work”: Data Cascades in High-Stakes AI
Sambasivan, N., Kapania, S., Highfill, H., Akrong, D., Paritosh, P., & Aroyo, L. M. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445518 · Cited by 781
“and clean data include ActiveClean and BoostClean [70, 72].”
BackgroundInfluentialData collection and quality challenges in deep learning: a data-centric AI perspective
Whang, S. E., Roh, Y., Song, H., & Lee, J.-G. (2023) · The VLDB Journal · doi:10.1007/s00778-022-00775-9 · Cited by 573
Advances, challenges and opportunities in creating data for trustworthy AI
Liang, W., Tadesse, G. A., Ho, D., Fei-Fei, L., Zaharia, M., Zhang, C., & Zou, J. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-022-00516-1 · Cited by 559
Metadata
- Authors
- Krishnan, S., Wang, J., Wu, E., Franklin, M. J., & Goldberg, K.
- Affiliations
- University of California, Berkeley (US)
- Simon Fraser University (CA)
- Columbia University (US)
- Venue
- Proceedings of the VLDB Endowment
- Published
- 2016-08
- Volume
- 9
- Issue
- 12
- Pages
- 948-959
- ISSN
- 2150-8097 (ISSN-L)
- OpenAlex
- W2548122763
- Cited by
- 264
- References
- 36
- Citations by year
- 2026: 31 · 2025: 30 · 2024: 28 · 2023: 27 · 2022: 24 · 2021: 39 · 2020: 25 · 2019: 36 · 2018: 14 · 2017: 9 · 2016: 1
- FWCI
- 16.27
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Proceedings of the VLDB Endowment (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Physical Sciences › Computer Science › Information Systems › Data Quality and Management
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
- Keywords
- data cleaning, linear regression, support vector machine, convergence guarantees, noisy data
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.