The reusable holdout: Preserving validity in adaptive data analysis
Abstract
Testing hypotheses privately: Large data sets offer a vast scope for testing already-formulated ideas and exploring new ones. Unfortunately, researchers who attempt to do both on the same data set run the risk of making false discoveries, even when testing and exploration are carried out on distinct subsets of data. Based on ideas drawn from differential privacy, Dwork et al. now provide a theoretical solution. Ideas are tested against aggregate information, whereas individual data set components remain confidential. Preserving that privacy also preserves statistical inference validity. Science , this issue p. 636
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The performance of a segmentation method on clinical data is usually lower than what is found through cross-validation on annotated data.
“The problem is that the performance of a method might be substantially different on clinical data, and is usually lower than what is found through cross-validation on annotated data carried out beforehand due to several reasons.”
From Reverse Classification Accuracy: Predicting Segmentation Performance in the Absence of Ground Truth (2017) · cites this paper as [14]
Using the annotated database during incremental method development for training, model selection and hyper-parameter tuning can lead to overfitting.
“First, the annotated database is normally used during incremental method development for training, model selection and fine tuning of hyper-parameters. This can lead to overfitting [14] which is a potential cause for lower performance on new data.”
From Reverse Classification Accuracy: Predicting Segmentation Performance in the Absence of Ground Truth (2017) · cites this paper as [14]
Cites
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505 · see citing works
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Cross-validation failure: Small sample sizes lead to large error bars
Varoquaux, G. (2018) · NeuroImage · doi:10.1016/j.neuroimage.2017.06.061 · Cited by 897 · see citing works
“Platforms to share neuroimaging data are rapidly growing, as with OpenfMRI (Poldrack et al., 2013) that now hosts 63 studies comprising 2 200 subjects, or Neurovault (Gorgolewski et al., 2015) with 26 000 brain maps in 1 100 collection.”
BackgroundIntegrating explanation and prediction in computational social science
Hofman, J. M., Watts, D. J., Athey, S., Garip, F., Griffiths, T. L., Kleinberg, J., Margetts, H., Mullainathan, S., Salganik, M. J., Vazire, S., et al. (2021) · Nature · doi:10.1038/s41586-021-03659-0 · Cited by 371 · see citing works
Causality matters in medical imaging
Castro, D. C., Walker, I., & Glocker, B. (2020) · Nature Communications · doi:10.1038/s41467-020-17478-w · Cited by 364 · see citing works
Digital medicine and the curse of dimensionality
Berisha, V., Krantsevich, C., Hahn, P. R., Hahn, S., Dasarathy, G., Turaga, P., & Liss, J. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00521-5 · Cited by 344 · see citing works
Flesia, L., Monaro, M., Mazza, C., Fietta, V., Colicino, E., Segatto, B., & Roma, P. (2020) · Journal of Clinical Medicine · doi:10.3390/jcm9103350 · Cited by 291 · see citing works
Exposed! A Survey of Attacks on Private Data
Dwork, C., Smith, A., Steinke, T., & Ullman, J. (2017) · Annual Review of Statistics and Its Application · doi:10.1146/annurev-statistics-060116-054123 · Cited by 259 · see citing works
Self-citationWhen Machine Learning Meets Privacy in 6G: A Survey
Sun, Y., Liu, J., Wang, J., Cao, Y., & Kato, N. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.3011561 · Cited by 211 · see citing works
Multiaccuracy: Black-Box Post-Processing for Fairness in Classification
Kim, M. P., Ghorbani, A., & Zou, J. (2019) · Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3306618.3314287 · Cited by 193 · see citing works
More Than Privacy: Applying Differential Privacy in Key Areas of Artificial Intelligence
Zhu, T., Ye, D., Wang, W., Zhou, W., & Yu, P. S. (2022) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2020.3014246 · Cited by 219 · see citing works
FairTest: Discovering Unwarranted Associations in Data-Driven Applications
Tramèr, F., Atlidakis, V., Geambasu, R., Hsu, D., Hubaux, J.-P., Humbert, M., Juels, A., & Lin, H. (2015) · arXiv · doi:10.48550/arxiv.1510.02377 · Cited by 171 · see citing works
I tried a bunch of things: The dangers of unexpected overfitting in classification of brain data
Hosseini, M., Powell, M., Collins, J., Callahan-Flintoft, C., Jones, W., Bowman, H., & Wyble, B. (2020) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2020.09.036 · Cited by 165 · see citing works
FairTest: Discovering Unwarranted Associations in Data-Driven Applications
Tramer, F., Atlidakis, V., Geambasu, R., Hsu, D., Hubaux, J.-P., Humbert, M., Juels, A., & Lin, H. (2017) · 2017 IEEE European Symposium on Security and Privacy (EuroS&P) · doi:10.1109/eurosp.2017.29 · Cited by 171 · see citing works
Reverse Classification Accuracy: Predicting Segmentation Performance in the Absence of Ground Truth
Valindria, V. V., Lavdas, I., Bai, W., Kamnitsas, K., Aboagye, E. O., Rockall, A. G., Rueckert, D., & Glocker, B. (2017) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2017.2665165 · Cited by 152 · see citing works
Study preregistration: An evaluation of a method for transparent reporting
Toth, A. A., Banks, G. C., Mellor, D., O’Boyle, E. H., Dickson, A., Davis, D. J., DeHaven, A., Bochantin, J., & Borns, J. (2021) · Journal of Business and Psychology · doi:10.1007/s10869-020-09695-3 · Cited by 73 · see citing works
In Search of Lost Domain Generalization
Gulrajani, I., & Lopez-Paz, D. (2020) · arXiv · doi:10.48550/arxiv.2007.01434 · Cited by 440 · see citing works
“As an alternative to limiting the number of queries, we could borrow tools from differential privacy, previously applied to enable multiple re-uses of validation sets in standard supervised learning [Dwork et al., 2015].”
MethodWu, T., Weld, D. S., & Heer, J. (2019) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3319616 · Cited by 39 · see citing works
Building Continuous Integration Services for Machine Learning
Karlaš, B., Interlandi, M., Renggli, C., Wu, W., Zhang, C., Mukunthu Iyappan Babu, D., Edwards, J., Lauren, C., Xu, A., & Weimer, M. (2020) · Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3394486.3403290 · Cited by 30 · see citing works
- Authors
- Dwork, C., Feldman, V., Hardt, M., Pitassi, T., Reingold, O., & Roth, A.
- Affiliations
- Microsoft (United States) (US)
- IBM Research - Almaden (US)
- Google (United States) (US)
- University of Toronto (CA)
- Samsung Research America, Mountain View, CA 94043, USA.
- University of Pennsylvania (US)
- Corresponding author
- Dwork, Cynthia
- Feldman, Vitaly
- Hardt, Moritz
- Pitassi, Toniann
- Reingold, Omer
- Roth, Aaron
- Venue
- Science
- Published
- 2015-08-07
- Volume
- 349
- Issue
- 6248
- Pages
- 636-638
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- OpenAlex
- W1555759181
- Cited by
- 307
- References
- 2
- Citations by year
- 2026: 21 · 2025: 12 · 2024: 22 · 2023: 20 · 2022: 23 · 2021: 29 · 2020: 50 · 2019: 38 · 2018: 35 · 2017: 29 · 2016: 17 · 2015: 9 · 2014: 1 · 2013: 1
- FWCI
- 29.41
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Science (publishedVersion)
- PubMed (publishedVersion)
- Funders
- NSF (CCF 1101389)
- Alfred P. Sloan Foundation
- Natural Sciences and Engineering Research Council of Canada
- NSF CAREER (CNS 1253345)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Inference
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Applications
- Keywords
- adaptive data analysis, data reuse, validity of inference, privacy-preserving data analysis, exploratory data analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
