The DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index.
Transparency and reproducibility in artificial intelligence
Open accessAbstract
Open Access articles citing this article. Deep learning in the stock market—a systematic survey of practice, backtesting, and applications Kenniy Olorunnimbe & Herna Viktor Artificial Intelligence Review Open Access 30 June 2022 Machine learning generalizability across healthcare settings: insights from multi-site COVID-19 screening Jenny Yang , Andrew A. S. Soltan & David A. Clifton npj Digital Medicine Open Access 07 June 2022 Bringing machine learning to research on intellectual and developmental disabilities: taking inspiration from neurological diseases Chirag Gupta , Pramod Chandrashekar … Daifeng Wang Journal of Neurodevelopmental Disorders Open Access 02 May 2022
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In addition, AI's transparency and interpretability are challenged by factors such as patient privacy, algorithm interpretability, publication bias, etc. 39,40.”
From Artificial intelligence in pancreatic cancer (2022) · cites this paper as [40]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“highlighting the risk of using corporate models in a sensitive area such as medicine, where patient privacy, transparency and reliability are essential74,76”
From Evaluation and mitigation of the limitations of large language models in clinical decision-making (2024) · cites this paper as [76]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“When generative AI follows the release-by-blogpost model, it is reaping the benefits of mimicking scientific communication -including associations of reproducibility and rigour [21, 23]- without actually doing the work.”
From Rethinking open source generative AI: open washing and the EU AI Act (2024) · cites this paper as [23]
In the citing paper
“One of the most effective methods of mitigating AI biases is to ensure the use of diverse and representative datasets during AI development and training [67, 87].”
From Fairness of artificial intelligence in healthcare: review and recommendations (2024) · cites this paper as [87]
In the citing paper
“A critical component of AI evaluation and development more generally is ensuring transparency and reproducibility, allowing independent research groups to review and build on prior scientific progress (196).”
From Artificial Intelligence in Oncology: Current Landscape, Challenges, and Future Directions (2024) · cites this paper as [196]
In the citing paper
“In addition, the lack of transparency and reproducibility of AI computational algorithm methods and code in many published studies threatens scientific progress and discovery, which require that independent scientists are able to scrutinize and reproduce results98.”
From Smart wearable devices in cardiovascular care: Where we are and how to move forward (2021) · cites this paper as [98]
In the citing paper
“Because of the particular significance of addressing the data-related concerns, different countries have developed policies and laws [160].”
From Explainable, trustworthy, and ethical machine learning for healthcare: A survey (2022) · cites this paper as [160]
In the citing paper
“Unfortunately, though, the majority of existing AI/ML models remain in specialized journals, with poorly maintained code and variable rigor in validation.283”
From Advancing Targeted Protein Degradation via Multiomics Profiling and Artificial Intelligence (2023) · cites this paper as [283]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A survey on Image Data Augmentation for Deep Learning
Shorten, C., & Khoshgoftaar, T. M. (2019) · Journal of Big Data · doi:10.1186/s40537-019-0197-0 · Cited by 13,203
International evaluation of an AI system for breast cancer screening
McKinney, S. M., Sieniek, M., Godbole, V., Godwin, J., Antropova, N., Ashrafian, H., Back, T., Chesus, M., Corrado, G. S., Darzi, A., et al. (2020) · Nature · doi:10.1038/s41586-019-1799-6 · Cited by 3,287
Text Data Augmentation for Deep Learning
Shorten, C., Khoshgoftaar, T. M., & Furht, B. (2021) · Journal Of Big Data · doi:10.1186/s40537-021-00492-0 · Cited by 1,852
(2015) · neural information processing systems · Cited by 912
A curated mammography data set for use in computer-aided detection and diagnosis research
Lee, R. S., Gimenez, F., Hoogi, A., Miyake, K. K., Gorovoy, M., & Rubin, D. L. (2017) · Scientific Data · doi:10.1038/sdata.2017.177 · Cited by 859
Artificial intelligence faces reproducibility crisis
Hutson, M. (2018) · Science · doi:10.1126/science.359.6377.725 · Cited by 573
Enhancing reproducibility for computational methods
Stodden, V., Mcnutt, M. K., Bailey, D. H., Deelman, E., Gil, Y., Hanson, B., Heroux, M. A., Ioannidis, J. P. A., & Taufer, M. (2016) · Science · doi:10.1126/science.aah6168 · Cited by 464
Self-citationBluemke, D. A., Moy, L., Bredella, M. A., Ertl-Wagner, B. B., Fowler, K. J., Goh, V. J., Halpern, E. F., Hess, C. P., Schiebler, M. L., & Weiss, C. R. (2020) · Radiology · doi:10.1148/radiol.2019192515 · Cited by 343
Reproducible Research Practices and Transparency across the Biomedical Literature
Iqbal, S. A., Wallach, J. D., Khoury, M. J., Schully, S. D., & Ioannidis, J. P. A. (2016) · PLOS Biology · doi:10.1371/journal.pbio.1002333 · Cited by 310
Self-citationState of the Art: Reproducibility in Artificial Intelligence
Gundersen, O. E., & Kjensmo, S. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11503 · Cited by 293 · cited 1×
“Even with extensive description, reproducing complex computational pipelines based purely on text is a subjective and challenging task 9 .”
Background
Research works printed but not linked: 1
- Sculley, D. et al. in Advances in Neural Information Processing Systems 28 (eds Cortes, C. et al.) 2503–2511 (Curran Associates, Inc., 2015).
Cited by
Sorted by citations
Items 501–541 of 541
Marsilla, J., Welch, M. L., Siraj, J., McElcheran, C., Kim, J. W., Tkachuk, D., Kim, S., Cho, J., Hahn, E., Hosni, A., et al. (2026) · Physics and Imaging in Radiation Oncology · doi:10.1016/j.phro.2026.101015
“To isolate architectural effects on performance, all models used a consistent loss function tailored for semantic segmentation with high class imbalance [19,25 – 27]: a weighted TopK Cross-Entropy loss [28] selecting the top 10% hardest voxels combined with Focal Tversky Loss.”
Self-citationCobb, M., & COGHLAN, S. (2026) · Frontiers in Animal Science · doi:10.3389/fanim.2026.1868726 · Cited by 1
Alam, M. S., Singh, P., & Bazilinskyy, P. (2026) · Machine Vision and Applications · doi:10.1007/s00138-026-01843-8
Sakib, A. N., Chisty, M. M. O., & Zhang, Z. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3812342
Chen, L., Song, D., Ding, J., Wei, H., & Qu, Z. (2026) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.70105
“Such studies, which often emphasize experimental transparency, data openness, and reproducible procedures, tend to enhance their accessibility, usability, and reproducibility in scholarly communication (Haibe-Kains et al., 2020).”
Inter-Observer Reproducibility of [18F]FDG PET/CT Radiomic Features in Primary Breast Carcinoma
Mitoi, A., Mititelu, R., Medar, C., Bolocan, V.-O., Constantin, C., & Mateș, I. N. (2026) · Journal of Imaging · doi:10.3390/jimaging12070300 · Cited by 1
“This initiative is complemented by the CheckList for EvaluAtion of Radiomics (CLEAR) guideline, which aims for transparent and complete reporting of radiomic studies [31], as well as broader initiatives for transparency and reproducibility in medical artificial intelligence [32].”
Wang, Y., Hu, L., Xue, Z., Li, Q., Li, J., & Kong, X. (2026) · Frontiers in Medicine · doi:10.3389/fmed.2026.1878351
Han, D., Shu, X., Hu, Y., Zou, H., Wang, C., & Peng, X. (2026) · Computational and Structural Biotechnology Journal · doi:10.34133/csbj.0185
Yazdi, S. M., Motevaselian, M., Khatami, S., Radfar, N., Jourahmad, Z., & Perez, H. A. (2026) · medRxiv · doi:10.64898/2026.07.15.26358113
Federated Learning vs. Centralized Approaches for Lesion Detection in Mammography Images
Petrović, N., Jakovljevic, N. M., Bajović, D., Lazić, I., Rapaić, M. R., & Lončar-Turukalo, T. (2026) · International Balkan Conference on Communications and Networking (BalkanCom) · doi:10.1109/balkancom71095.2026.11605029
“However, neither the trained model, nor the datasets used were made available, which spurred reactions of the scientific community on reproducibility and transparency of the AI research [10].”
Makieu, P., Sahr, F., & Turay, A. (2026) · Journal of Artificial Intelligence for Medical Sciences · doi:10.55578/joaims.260713.001
“To support reproducibility, all analysis code, random seeds, and exact software versions have been made available, in line with recommended standards for machine learning research in the life sciences [12,32].”
Zhao, M., Ma, X., Shi, W., & Sun, Y. (2026) · Life Sciences · doi:10.1016/j.lfs.2026.124600 · Cited by 1
Cosma, C., Butiurcă, V. O., Botoncea, M., Molnar, D., & Molnar, C.-D. (2026) · Emergency Care and Medicine · doi:10.3390/ecm3030024
meng, K., Wang, L., Gong, X., Tang, W., Jia, G., Wang, Y., & He, C. (2026) · Functional & Integrative Genomics · doi:10.1007/s10142-026-01980-3
“…given prior machine-learning studies reporting high-performing molecular classifiers in IPF (Ding et al. 2024 , Guo et al. 2025, Li et al. 2023 ) and broader concerns regarding transparency and reproducibility in artificial intelligence-based biomedical prediction (Haibe-Kains et al. 2020 ).”
Haggam, R. A., Jayabal, R., Mehar, K., Jangid, S. K., Nanda, J., Shahul Hameed, M., Nagappan, B., Vaghela, D. R., & Ismoilov, M. (2026) · Chemometrics and Intelligent Laboratory Systems · doi:10.1016/j.chemolab.2026.105848
Exploring the Intricate Relationship Between Artificial Intelligence and Healthcare Organizations
Cannavacciuolo, L., Mancuso, P., & Scarpa, A. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-23684-5_27
Ali, U., Imam, A. A., & Apong, R. A. A. H. M. (2026) · Diagnostics · doi:10.3390/diagnostics16162500
“TCII values that are near to 1 indicate high-quality technical and high clinical applicability studies [16,47].”
Sosa-Santos, R. F., Arce-Diego, J. L., & Fanjul-Vélez, F. (2026) · Sensors · doi:10.3390/s26165020
Explainable AI and Transparency in Clinical Decision Support
Gupta, D., Jumaniyazova, N., Mamura, A., Akhmedov, J., Masharipova, G., Masharipova, D., & Aminqulov, S. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-8904-2.ch007
Fischer, D., Meier, L., Koch, S., & Braun, T. (2026) · Archives of Pharmacy Practice · doi:10.51847/whawvgp2pm
Shechter, S., & Shomron, N. (2026) · Journal of Scientific Innovation in Medicine · doi:10.29024/jsim.283
Meyer, L., Schmid, A., Braun, S., & Keller, L. (2026) · International Journal Of Pharmaceutical Research And Allied Sciences · doi:10.51847/ugyerdbsld
Artificial intelligence in oncology: linking biological discovery to clinical utility
Kuchana, S. K., Alahari, N. V., Kode, R., Jalapu, K. R., & Alahari, S. K. (2026) · Molecular Cancer · doi:10.1186/s12943-026-02771-x
“It examines AI from molecular discovery to implementation, with particular emphasis on why some applications have crossed into prospective care while others remain biologically interesting but clinically unproven [13–15].”
Kama, M. O., Ajah, I. A., Ikegwu, A. C., Obayi, A. A., Uloko, F. O., & Ogbuagu, I. C. (2026) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-026-03796-4
“Moreover, many studies used retrospective clinical datasets with poor external validity, reproducibility, and insufficiently documented preprocessing procedures and modeling approaches [52].”
Artificial intelligence development and urban resilience: Evidence from Chinese cities
Wang, Y., Han, Y., & Lyu, X. (2026) · Economic Analysis and Policy · doi:10.1016/j.eap.2026.09.005
Singhera, F. U., Overhulse, J. M., Lee, T. M., Katz, J. G., Lee, J. S. H., & McKenna, C. E. (2026) · Cancers · doi:10.3390/cancers18182939
Artificial Intelligence in Medical Imaging: A Review of Radiodiagnosis in Indian Perspectives
Das, E., Mondal, R., Dhor, A., & Basu, T. (2026) · Indian journal of radiology and imaging - new series/Indian journal of radiology and imaging/Indian Journal of Radiology & Imaging · doi:10.1055/s-0046-1827803
Hassani, H., Mashhad, L. M., & Komendantova, N. (2026) · Data · doi:10.3390/data11090239
Editorial: Dark Data and the Cost to Open Science
Makonin, S. (2026) · IEEE data descriptions. · doi:10.1109/ieeedata.2026.3730296
Fernandes, A., Rodriguez, J. P. M., Moleirinho, S., Trofimenko, I., Guseva, E., Martinovich, A., Morais, I. M. D. O., Bisolo, L., Gomes, L. M., & Machado, J. (2026) · Frontiers in Radiology · doi:10.3389/fradi.2026.1722312
“This approach is particularly relevant in the field of deep learning for medical imaging, where variations in data or training parameters can introduce significant biases in performance evaluation (20).”
Kejriwal, M. (2026) · ACM Conference on Reproducibility and Replicability (ACM REP) · doi:10.1145/3820002.3828595
“The reproducibility crisis has been particularly acute in machine learning and artificial intelligence, where computational methods are central to scientific claims [3, 8].”
Vangala, B. P., Gehani, A., & Malik, T. (2026) · ACM Conference on Reproducibility and Replicability (ACM REP) · doi:10.1145/3820002.3828581
“Studies show that a significant portion of published AI/ML research cannot be reproduced due to missing dependencies, incomplete environment specifications, and undocumented requirements [8, 9, 23, 25].”
Metaphor of overreaching: effects of AI agglomeration on innovation
Lian, Y., Jian, J., & Xia, C. (2026) · The Journal of Technology Transfer · doi:10.1007/s10961-026-10390-3
AI-assisted peer review: efficiency for reviewers, burden for authors—a narrative review
Babker, A. M. A. (2026) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2026.1968385
Mitigating Domain Shift and Dataset Bias in Deep Learning Models for Cancer Diagnostics
Ameyaw, S. A., & Ameyaw, J. A. (2026) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-026-02353-9
Epifani, L. (2026) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn69854.2026.11729137
Xiao, W., Xu, S., Li, J., & Mou, J. (2026) · Electronic Markets · doi:10.1007/s12525-026-00939-3
“…perform tasks typically requiring human cognition (e.g., reasoning, learning, decision-making) or processing data at scales beyond human capability (Haibe-Kains et al., 2020), has taken the world by storm, simultaneously offering unprecedented empowerment opportunities for industrial advancement…”
Khullar, S., Sobhan, A., & Ali, S. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-8562-4.ch001
Rodríguez‐Hernández, A. G. (2025) · SAP Artificial Intelligence in Dentistry · doi:10.62486/aid202542
Ko, J. E., Hwang, J. W., Kim, S. M., & Kwon, J. Y. (2026) · Journal of Medical Internet Research · doi:10.2196/94543
“Importantly, without a statement of whether and how the intervention or its underlying prompts, parameters, and code can be obtained, subsequent researchers cannot access the exact intervention evaluated, let alone validate it [111].”
Huang, Y. (2026) · The University of North Carolina at Chapel Hill University Libraries · doi:10.17615/9da1-2v23
Metadata
- Authors
- Haibe-Kains, B., Adam, G. A., Hosny, A., Khodakarami, F., Massive Analysis Quality Control (MAQC) Society Board of Directors, Shraddha, T., Kusko, R., Sansone, S.-A., Tong, W., Wolfinger, R. D., Mason, C. E., Jones, W., Dopazo, J., Furlanello, C., Waldron, L., Wang, B., McIntosh, C., Goldenberg, A., Kundaje, A., Greene, C. S., Broderick, T., Hoffman, M. M., Leek, J. T., Korthauer, K., Huber, W., Brazma, A., Pineau, J., Tibshirani, R., Hastie, T., Ioannidis, J. P. A., Quackenbush, J., & Aerts, H. J. W. L.
- Affiliations
- University of Toronto (CA)
- Brigham and Women's Hospital (US)
- University Health Network (CA)
- National Center for Toxicological Research (US)
- University of Oxford (GB)
- SAS Institute (United States) (US)
- Q2 Solutions (United States) (US)
- City University of New York (US)
- Canadian Institute for Advanced Research (CA)
- Alex's Lemonade Stand Foundation (US)
- University of British Columbia (CA)
- European Bioinformatics Institute (GB)
- Stanford Medicine (US)
- ORCID
- Haibe-Kains, Benjamin
- Adam, George Alexandru
- Hosny, Ahmed
- Khodakarami, Farnoosh
- Kusko, Rebecca
- Sansone, Susanna-Assunta
- Tong, Weida
- Mason, Christopher E.
- Jones, Wendell
- Furlanello, Cesare
- Waldron, Levi
- Wang, Bo
- McIntosh, Chris
- Goldenberg, Anna
- Kundaje, Anshul
- Greene, Casey S.
- Broderick, Tamara
- Hoffman, Michael M.
- Leek, Jeffrey T.
- Korthauer, Keegan
- Huber, Wolfgang
- Brazma, Alvis
- Pineau, Joelle
- Tibshirani, Robert
- Hastie, Trevor
- Ioannidis, John P. A.
- Quackenbush, John
- Aerts, Hugo J. W. L.
- Venue
- Nature
- Published
- 2020-10-14
- Volume
- 586
- Issue
- 7829
- Pages
- E14-E16
- ISSN
- 0028-0836 (ISSN-L)
- 1476-4687
- PMC
- PMC8144864
- OpenAlex
- W3008058947
- Cited by
- 540
- References
- 18
- Citations by year
- 2026: 83 · 2025: 91 · 2024: 97 · 2023: 110 · 2022: 79 · 2021: 70 · 2020: 10
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Nature (publishedVersion, other-oa)
- PubMed (publishedVersion)
- Research Publications (Maastricht University) (publishedVersion, other-oa)
- PubMed Central (acceptedVersion)
- Digital Access to Scholarship at Harvard (DASH) (Harvard University) (acceptedVersion)
- arXiv (Cornell University) (submittedVersion)
- RISalud Institutional Health Repository of Andalucía (submittedVersion, other-oa)
- RePEc: Research Papers in Economics (submittedVersion)
- Research Publications (Maastricht University) (submittedVersion, other-oa)
- Funders
- Biotechnology and Biological Sciences Research Council (BB/L005069/2, BB/E025080/1, BB/L005069/1, BB/J020265/1, BB/L005050/1)
- National Cancer Institute (U24CA180996, R01CA237170, CA237170)
- National Institute of General Medical Sciences (DP2GM123485)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Decision Sciences › Information Systems and Management › Scientific Computing and Data Management
- Keywords
- reproducibility, explainability, deep learning, backtesting, generalizability, machine learning in healthcare, intellectual and developmental disabilities
- MeSH terms
- Algorithms, Artificial Intelligence, Reproducibility of Results
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
