50 Years of Data Science
Open accessAbstract
More than 50 years ago, John Tukey called for a reformation of academic statistics. In “The Future of Data Analysis,” he pointed to the existence of an as-yet unrecognized science, whose subject of interest was learning from data, or “data analysis.” Ten to 20 years ago, John Chambers, Jeff Wu, Bill Cleveland, and Leo Breiman independently once again urged academic statistics to expand its boundaries beyond the classical domain of theoretical statistics; Chambers called for more emphasis on data preparation and presentation rather than statistical modeling; and Breiman called for emphasis on prediction rather than inference. Cleveland and Wu even suggested the catchy name “data science” for this envisioned field. A recent and growing phenomenon has been the emergence of “data science” programs at major universities, including UC Berkeley, NYU, MIT, and most prominently, the University of Michigan, which in September 2015 announced a $100M “Data Science Initiative” that aims to hire 35 new faculty. Teaching in these new programs has significant overlap in curricular subject matter with traditional statistics courses; yet many academic statisticians perceive the new programs as “cultural appropriation.” This article reviews some ingredients of the current “data science moment,” including recent commentary about data science in the popular media, and about how/whether data science is really different from statistics. The now-contemplated field of data science amounts to a superset of the fields of statistics and machine learning, which adds some technology for “scaling up” to “big data.” This chosen superset is motivated by commercial rather than intellectual developments. Choosing in this way is likely to miss out on the really important intellectual event of the next 50 years. Because all of science itself will soon become data that can be mined, the imminent revolution in data science is not about mere “scaling up,” but instead the emergence of scientific studies of data analysis science-wide. In the future, we will be able to predict how a proposal to change data analysis workflows would impact the validity of data analysis across all of science, even predicting the impacts field-by-field. Drawing on work by Tukey, Cleveland, Chambers, and Breiman, I present a vision of data science based on the activities of people who are “learning from data,” and I describe an academic field dedicated to improving that activity in an evidence-based manner. This new field is a better academic enlargement of statistics and machine learning than today’s data science initiatives, while being able to accommodate the same short-term goals. Based on a presentation at the Tukey Centennial Workshop, Princeton, NJ, September 18, 2015.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Benchmarks that create a level playing field are the core reason for the rapid progress of ML over the last half century.
“Benchmarks that create a level playing field are the core reason for the rapid progress of ML over the last half century [12, 46].”
From AI Agents That Matter (2024) · cites this paper as [12]
Clinical FMs lack the shared evaluation frameworks and datasets that have underpinned progress in other fields such as natural language processing and computer vision.
“Clinical FMs lack the shared evaluation frameworks and datasets20 that have underpinned progress in other fields, such as natural language processing (NLP) and computer vision21.”
From The shaky foundations of large language models and foundation models for electronic health records (2023) · cites this paper as [21]
Cites
Estimating the reproducibility of psychological science
Open Science Collaboration (2015) · Science · doi:10.1126/science.aac4716 · Cited by 8,515 · see citing works
Power failure: why small sample size undermines the reliability of neuroscience
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3475 · Cited by 7,641 · see citing works
Believe it or not: how much can we rely on published data on potential drug targets?
Prinz, F., Schlange, T., & Asadullah, K. (2011) · Nature Reviews Drug Discovery · doi:10.1038/nrd3439-c1 · Cited by 1,888 · see citing works
Evolution of Reporting <i>P</i> Values in the Biomedical Literature, 1990-2015
Chavalarias, D., Wallach, J. D., Li, A. H. T., & Ioannidis, J. P. A. (2016) · JAMA · doi:10.1001/jama.2016.1952 · Cited by 370 · see citing works
Implementing Reproducible Research
Stodden, V. (2014) · doi:10.1201/b16868 · Cited by 314 · see citing works
McNutt, M. (2014) · Science · doi:10.1126/science.1250475 · Cited by 291 · see citing works
Liberman, M. (2010) · Computational Linguistics · doi:10.1162/coli_a_00032 · Cited by 29 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Habitat: A Platform for Embodied AI Research
Savva, M., Kadian, A., Maksymets, O., Zhao, Y., Wijmans, E., Jain, B., Straub, J., Liu, J., Koltun, V., Malik, J., et al. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00943 · Cited by 1,237 · see citing works
Leakage and the reproducibility crisis in machine-learning-based science
Kapoor, S., & Narayanan, A. (2023) · Patterns · doi:10.1016/j.patter.2023.100804 · Cited by 1,387 · see citing works
Sarker, I. H. (2021) · SN Computer Science · doi:10.1007/s42979-021-00765-8 · Cited by 570 · see citing works
Data Science: A Comprehensive Overview
Cao, L. (2018) · ACM Computing Surveys · doi:10.1145/3076253 · Cited by 407 · see citing works
“Focusing on data science (rather than big data), it is clear that only a few articles and references have discussed its history and contents, such as in Press [Press 2013], Donoho [Donoho 2015], and Galetto [Galetto 2016].”
BackgroundMethodInfluentialIntegrating explanation and prediction in computational social science
Hofman, J. M., Watts, D. J., Athey, S., Garip, F., Griffiths, T. L., Kleinberg, J., Margetts, H., Mullainathan, S., Salganik, M. J., Vazire, S., et al. (2021) · Nature · doi:10.1038/s41586-021-03659-0 · Cited by 371 · see citing works
Measuring the predictability of life outcomes with a scientific mass collaboration
Salganik, M. J., Lundberg, I., Kindel, A. T., Ahearn, C. E., Al-Ghoneim, K., Almaatouq, A., Altschul, D. M., Brand, J. E., Carnegie, N. B., Compton, R. J., et al. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1915006117 · Cited by 322 · see citing works
The shaky foundations of large language models and foundation models for electronic health records
Wornow, M., Xu, Y., Thapa, R., Patel, B., Steinberg, E., Fleming, S., Pfeffer, M. A., Fries, J., & Shah, N. H. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00879-8 · Cited by 324 · see citing works
Meng, X.-L. (2018) · The Annals of Applied Statistics · doi:10.1214/18-aoas1161sf · Cited by 316 · see citing works
The growing ubiquity of algorithms in society: implications, impacts and innovations
Olhede, S. C., & Wolfe, P. J. (2018) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2017.0364 · Cited by 120 · see citing works
A Human-Centered Review of Algorithms used within the U.S. Child Welfare System
Saxena, D., Badillo-Urquiola, K., Wisniewski, P. J., & Guha, S. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376229 · Cited by 116 · see citing works
Assessing Generalization in Deep Reinforcement Learning
Packer, C., Gao, K., Kos, J., Krähenbühl, P., Koltun, V., & Song, D. (2018) · arXiv · doi:10.48550/arxiv.1810.12282 · Cited by 114 · see citing works
Rearrangement: A Challenge for Embodied AI
Batra, D., Chang, A. X., Chernova, S., Davison, A. J., Deng, J., Koltun, V., Levine, S., Malik, J., Mordatch, I., Mottaghi, R., et al. (2020) · arXiv · doi:10.48550/arxiv.2011.01975 · Cited by 109 · see citing works
Multimodal single cell data integration challenge: results and lessons learned
Lance, C., Luecken, M. D., Burkhardt, D. B., Cannoodt, R., Rautenstrauch, P., Laddach, A., Ubingazhibov, A., Cao, Z.-J., Deng, K., Khan, S., et al. (2022) · doi:10.1101/2022.04.11.487796 · Cited by 81 · see citing works
Computational Reproducibility via Containers in Psychology
Clyburne-Sherin, A., Fei, X., & Green, S. A. (2019) · Meta-Psychology · doi:10.15626/mp.2018.892 · Cited by 69 · see citing works
Liu, D. M., & Salganik, M. J. (2019) · Socius Sociological Research for a Dynamic World · doi:10.1177/2378023119849803 · Cited by 47 · see citing works
Lee, A., Inceoglu, I., Hauser, O., & Greene, M. (2022) · The Leadership Quarterly · doi:10.1016/j.leaqua.2020.101426 · Cited by 48 · see citing works
The Foundation Model Transparency Index
Bommasani, R., Klyman, K., Longpre, S., Kapoor, S., Maslej, N., Xiong, B., Zhang, D., & Liang, P. (2023) · arXiv · doi:10.48550/arxiv.2310.12941 · Cited by 46 · see citing works
Solving medicine's data bottleneck: Nightingale Open Science
Mullainathan, S., & Obermeyer, Z. (2022) · Nature Medicine · doi:10.1038/s41591-022-01804-4 · Cited by 43 · see citing works
Kapoor, S., Stroebl, B., Siegel, Z. S., Nadgir, N., & Narayanan, A. (2024) · arXiv · doi:10.48550/arxiv.2407.01502 · Cited by 13 · see citing works
“Benchmarks that create a level playing field are the core reason for the rapid progress of ML over the last half century [12, 46].”
- Authors
- Donoho, D.
- Affiliations
- Stanford University (US)
- Corresponding author
- Donoho, David
- ORCID
- Donoho, David
- Venue
- Journal of Computational and Graphical Statistics
- Publisher
- Informa UK Limited
- Published
- 2017-10-02
- Volume
- 26
- Issue
- 4
- Pages
- 745-766
- ISSN
- 1061-8600 (ISSN-L)
- 1537-2715
- OpenAlex
- W2557007068
- Cited by
- 695
- References
- 7
- Citations by year
- 2026: 59 · 2025: 67 · 2024: 73 · 2023: 78 · 2022: 83 · 2021: 105 · 2020: 80 · 2019: 66 · 2018: 55 · 2017: 18 · 2016: 8 · 2015: 2 · 2014: 1
- FWCI
- 41.65
- Type
- journal article
- Language
- en
- Update notices
- correction (2018-12-12)
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Journal of Computational and Graphical Statistics (publishedVersion, cc-by)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistics Education and Methodologies
- Social Sciences › Decision Sciences › Information Systems and Management › Big Data Technologies and Applications
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Applications
- Keywords
- academic statistics, John Tukey, big data, machine learning, data analysis pipeline, scientific validity, cultural appropriation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
