The DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
Is AI leading to a reproducibility crisis in science?
Open accessAbstract
Scientists worry that ill-informed use of artificial intelligence is driving a deluge of unreliable or useless research. Scientists worry that ill-informed use of artificial intelligence is driving a deluge of unreliable or useless research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Transparency about potential limitations and AI-related errors is important for making research trustworthy and reproducible.
“Transparency about these is important for making research trustworthy and reproducible [16].”
From The ethics of using artificial intelligence in scientific research: new guidance needed for a new tool (2025) · cites this paper as [16]
AI can reproduce and amplify biases inherent in the data and generate results that lend support to policies that are discriminatory, unfair, harmful, or ineffective.
“AI can reproduce and amplify biases inherent in the data and generate results that lend support to policies that are discriminatory, unfair, harmful, or ineffective [16, 202].”
From The ethics of using artificial intelligence in scientific research: new guidance needed for a new tool (2025) · cites this paper as [16]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
SMOTE synthetic minority over-sampling technique
Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002) · Journal of Artificial Intelligence Research · doi:10.1613/jair.953 · Cited by 32,865
International evaluation of an AI system for breast cancer screening
McKinney, S. M., Sieniek, M., Godbole, V., Godwin, J., Antropova, N., Ashrafian, H., Back, T., Chesus, M., Corrado, G. S., Darzi, A., et al. (2020) · Nature · doi:10.1038/s41586-019-1799-6 · Cited by 3,287
Leakage and the reproducibility crisis in machine-learning-based science
Kapoor, S., & Narayanan, A. (2023) · Patterns · doi:10.1016/j.patter.2023.100804 · Cited by 1,387
CoroNet: A deep neural network for detection and diagnosis of COVID-19 from chest x-ray images
Khan, A. I., Shah, J. L., & Bhat, M. M. (2020) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2020.105581 · Cited by 1,351
Roberts, M., Driggs, D., Thorpe, M., Gilbey, J., Yeung, M., Ursprung, S., Aviles-Rivero, A. I., Etmann, C., McCague, C., Beer, L., et al. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00307-0 · Cited by 1,044
Roberts, M. T., Driggs, D., Thorpe, M., Gilbey, J., Yeung, M., Ursprung, S., Avilés-Rivero, A. I., Etmann, C., McCague, C., Beer, L., et al. (2020) · Apollo (University of Cambridge) · doi:10.17863/cam.64000 · Cited by 902
Machine learning for medical imaging: methodological failures and recommendations for the future
Varoquaux, G. P., & Cheplygina, V. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00592-y · Cited by 794
Beede, E., Baylor, E., Hersch, F., Iurchenko, A., Wilcox, L., Ruamviboonsuk, P., & Vardoulakis, L. M. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376718 · Cited by 614
Transparency and reproducibility in artificial intelligence
Haibe-Kains, B., Adam, G. A., Hosny, A., Khodakarami, F., Massive Analysis Quality Control (MAQC) Society Board of Directors, Shraddha, T., Kusko, R., Sansone, S.-A., Tong, W., Wolfinger, R. D., et al. (2020) · Nature · doi:10.1038/s41586-020-2766-y · Cited by 540
Best practices in machine learning for chemistry
Artrith, N., Butler, K. T., Coudert, F., Han, S., Isayev, O., Jain, A., & Walsh, A. (2021) · Nature Chemistry · doi:10.1038/s41557-021-00716-z · Cited by 538
Reproducibility standards for machine learning in the life sciences
Heil, B. J., Hoffman, M. M., Markowetz, F. M., Lee, S., Greene, C. S., & Hicks, S. C. (2021) · Nature Methods · doi:10.1038/s41592-021-01256-7 · Cited by 261
Insights from an autism imaging biomarker challenge: Promises and threats to biomarker discovery
Traut, N., Heuer, K., Lemaître, G., Beggiato, A., Germanaud, D., Elmaleh, M., Bethegnies, A., Bonnasse‐Gahot, L., Cai, W., Chambon, S., et al. (2022) · NeuroImage · doi:10.1016/j.neuroimage.2022.119171 · Cited by 69
Evaluation of the benchmark datasets for testing the efficacy of deep convolutional neural networks
Dhar, S., & Shamir, L. (2021) · Visual Informatics · doi:10.1016/j.visinf.2021.10.001 · Cited by 36
Reply to: Transparency and reproducibility in artificial intelligence
McKinney, S. M., Karthikesalingam, A., Tse, D., Kelly, C. J., Liu, Y., Corrado, G. S., & Shetty, S. (2020) · Nature · doi:10.1038/s41586-020-2767-x · Cited by 32
Öner, M. Ü., Cheng, Y.-C., Lee, H. K., & Sung, W.-K. K. (2020) · medRxiv · doi:10.1101/2020.04.23.20076406 · Cited by 10
Flagged first, then by citations
Cited by
Sorted by citations
Artificial Intelligence and the Transformation of Higher Education Institutions: A Systems Approach
Katsamakas, E., Pavlov, O. V., & Saklad, R. (2024) · Sustainability · doi:10.3390/su16146118 · Cited by 195
Resnik, D. B., & Hosseini, M. (2025) · AI and Ethics · doi:10.1007/s43681-024-00493-8 · Cited by 228
“…essential to good scientific methodology and practice [79, 89], they become crucial when AI is employed in research because AI can reproduce and amplify biases inherent in the data and generate results that lend support to policies that are discriminatory, unfair, harmful, or ineffective [16, 202].”
Metadata
- Authors
- Ball, P.
- Corresponding author
- Ball, Philip
- ORCID
- Ball, Philip
- Venue
- Nature
- Published
- 2023-12-05
- Volume
- 624
- Issue
- 7990
- Pages
- 22-25
- ISSN
- 0028-0836 (ISSN-L)
- 1476-4687
- OpenAlex
- W4389338955
- Cited by
- 106
- References
- 15
- Citations by year
- 2026: 27 · 2025: 43 · 2024: 31 · 2023: 4
- FWCI
- 26.75
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Nature (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Topics
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Education › Academic integrity and plagiarism
- Keywords
- artificial intelligence in science, replication crisis, research reliability, AI misuse, research quality
- MeSH terms
- Machine Learning, Artificial Intelligence, Research, Reproducibility of Results
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
