Reporting guidelines for clinical trials evaluating artificial intelligence interventions are needed
Open accessThe claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
These frameworks primarily focus on evidence generation at the point of evaluation and provide limited guidance on how performance degradation should be detected and managed during routine clinical operation, particularly when outcome labels are delayed, incomplete, or influenced by the model's own use.
“However, these frameworks primarily focus on evidence generation at the point of evaluation and provide limited guidance on how performance degradation should be detected and managed during routine clinical operation, particularly when outcome labels are delayed, incomplete, or influenced by the model’s own use [25–27].”
From Validation is not enough: Longitudinal evidence of post-deployment fragility in clinical AI systems (2026) · cites this paper as [25]
Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
Experimental — accuracy not yet measuredSPIRIT-AI is an international initiative supported by SPIRIT and the EQUATOR Network to extend or elaborate SPIRIT 2013 where necessary to develop consensus-based AI-specific protocol guidance.
“SPIRIT-AI (as part of the SPIRIT-AI and CONSORT-AI initiative) is an international initiative supported by SPIRIT and the EQUATOR (Enhancing the Quality and Transparency of Health Research) Network to extend or elaborate on the existing SPIRIT 2013 statement where necessary, to develop consensus-based AI-specific protocol guidance27,28.”
From Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extension (2020) · cites this paper as [27]
The section headed “The CONSORT-AI and SPIRIT-AI Steering Group” says the working group is preparing international, consensus-based AI-specific extensions to the SPIRIT and CONSORT statements, using the EQUATOR Network’s guideline-development framework. It does not establish that SPIRIT or the EQUATOR Network formally supports the initiative, or explicitly identify the statement being extended as SPIRIT 2013. No page number is available in our copy. (LLM, conf=0.84)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials
Schulz, K. F., Altman, D. G., Moher, D., & for the CONSORT Group (2010) · PLoS Medicine · doi:10.1371/journal.pmed.1000251 · Cited by 2,296 · see citing works
SPIRIT 2013: new guidance for content of clinical trial protocols
Chan, A.-W., Tetzlaff, J. M., Altman, D. G., Dickersin, K., & Moher, D. (2013) · The Lancet · doi:10.1016/s0140-6736(12)62160-6 · Cited by 298 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- Chalmers I, Glasziou P, Heneghan C, Shepperd S, Loudon K, Bossuyt PM. Avoidable waste in the production and reporting of research evidence. Lancet. 2009;374(9683):86-89.
- Collins GS, Moons KGM. Reporting of artificial intelligence prediction models. The Lancet. 2019;393(10181):1577-1579. doi:10.1016/s0140-6736(19)30037-6 · DOI · Paper page
Cited by
Sorted by citations
Liu, X., Faes, L., Kale, A. U., Wagner, S. K., Fu, D. J., Bruynseels, A., Mahendiran, T., Moraes, G., Shamdas, M., Kern, C., et al. (2019) · The Lancet Digital Health · doi:10.1016/s2589-7500(19)30123-2 · Cited by 1,883 · see citing works
Liu, X., Cruz Rivera, S., Moher, D., Calvert, M. J., Denniston, A. K., The SPIRIT-AI and CONSORT-AI Working Group, SPIRIT-AI and CONSORT-AI Steering Group, Chan, A.-W., Darzi, A., Holmes, C., et al. (2020) · Nature Medicine · doi:10.1038/s41591-020-1034-x · Cited by 1,421 · see citing works
Self-citationCruz Rivera, S., Liu, X., Chan, A.-W., Denniston, A. K., Calvert, M. J., The SPIRIT-AI and CONSORT-AI Working Group, SPIRIT-AI and CONSORT-AI Steering Group, Darzi, A., Holmes, C., Yau, C., et al. (2020) · Nature Medicine · doi:10.1038/s41591-020-1037-7 · Cited by 748 · see citing works
Self-citationLiu, X., Rivera, S. C., Moher, D., Calvert, M. J., & Denniston, A. K. (2020) · BMJ · doi:10.1136/bmj.m3164 · Cited by 702 · see citing works
Ethical Machine Learning in Healthcare
Chen, I. Y., Pierson, E., Rose, S., Joshi, S., Ferryman, K., & Ghassemi, M. (2021) · Annual Review of Biomedical Data Science · doi:10.1146/annurev-biodatasci-092820-114757 · Cited by 645 · see citing works
Liu, X., Cruz Rivera, S., Moher, D., Calvert, M. J., Denniston, A. K., Ashrafian, H., Beam, A. L., Chan, A.-W., Collins, G. S., Deeks, A. D. J., et al. (2020) · The Lancet Digital Health · doi:10.1016/s2589-7500(20)30218-1 · Cited by 555 · see citing works
Cruz Rivera, S., Liu, X., Chan, A.-W., Denniston, A. K., Calvert, M. J., Ashrafian, H., Beam, A. L., Collins, G. S., Darzi, A., Deeks, J. J., et al. (2020) · The Lancet Digital Health · doi:10.1016/s2589-7500(20)30219-3 · Cited by 486 · see citing works
Rivera, S. C., Liu, X., Chan, A.-W., Denniston, A. K., & Calvert, M. J. (2020) · BMJ · doi:10.1136/bmj.m3210 · Cited by 463 · see citing works
Causality matters in medical imaging
Castro, D. C., Walker, I., & Glocker, B. (2020) · Nature Communications · doi:10.1038/s41467-020-17478-w · Cited by 364 · see citing works
Cutillo, C. M., Sharma, K. R., Foschini, L., Kundu, S., Mackintosh, M., Mandl, K. D., MI in Healthcare Workshop Working Group, Beck, T., Collier, E., Colvis, C., et al. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0254-2 · Cited by 337 · see citing works
Ethical limitations of algorithmic fairness solutions in health care machine learning
McCradden, M. D., Joshi, S., Mazwi, M., & Anderson, J. A. (2020) · The Lancet Digital Health · doi:10.1016/s2589-7500(20)30065-0 · Cited by 291 · see citing works
Presenting machine learning model information to clinical end users with model facts labels
Sendak, M. P., Gao, M., Brajer, N., & Balu, S. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0253-3 · Cited by 279 · see citing works
Sounderajah, V., Ashrafian, H., Aggarwal, R., De Fauw, J., Denniston, A. K., Greaves, F., Karthikesalingam, A., King, D., Liu, X., Markar, S. R., et al. (2020) · Nature Medicine · doi:10.1038/s41591-020-0941-1 · Cited by 270 · see citing works
Chin, M. H., Afsar-Manesh, N., Bierman, A. S., Chang, C., Colón-Rodríguez, C. J., Dullabh, P., Duran, D. G., Fair, M., Hernandez-Boussard, T., Hightower, M., et al. (2023) · JAMA Network Open · doi:10.1001/jamanetworkopen.2023.45050 · Cited by 275 · see citing works
Freeman, K., Dinnes, J., Chuchu, N., Takwoingi, Y., Bayliss, S. E., Matin, R. N., Jain, A., Walter, F. M., Williams, H. C., & Deeks, J. J. (2020) · BMJ · doi:10.1136/bmj.m127 · Cited by 260 · see citing works
Introduction to artificial intelligence in ultrasound imaging in obstetrics and gynecology
Drukker, L., Noble, J. A., & Papageorghiou, A. T. (2020) · Ultrasound in Obstetrics & Gynecology · doi:10.1002/uog.22122 · Cited by 240 · see citing works
A Clinician's Guide to Artificial Intelligence: How to Critically Appraise Machine Learning Studies
Faes, L., Liu, X., Wagner, S. K., Fu, D. J., Balaskas, K., Sim, D. A., Bachmann, L. M., Keane, P. A., & Denniston, A. K. (2020) · Translational Vision Science & Technology · doi:10.1167/tvst.9.2.7 · Cited by 162 · see citing works
PRISMA AI reporting guidelines for systematic reviews and meta-analyses on AI in healthcare
Cacciamani, G. E., Chu, T. N., Sanford, D. I., Abreu, A., Duddalwar, V., Oberai, A., Kuo, C.-C. J., Liu, X., Denniston, A. K., Vasey, B., et al. (2023) · Nature Medicine · doi:10.1038/s41591-022-02139-w · Cited by 143 · see citing works
Zhou, Q., Chen, Z.-H., Cao, Y.-H., & Peng, S. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00524-2 · Cited by 126 · see citing works
O'Reilly-Shah, V. N., Gentry, K. R., Walters, A. M., Zivot, J., Anderson, C. T., & Tighe, P. J. (2020) · British Journal of Anaesthesia · doi:10.1016/j.bja.2020.07.040 · Cited by 62 · see citing works
Use of artificial intelligence in obstetrics: Not quite ready for prime time
Sarno, L., Neola, D., Carbone, L., Saccone, G., Carlea, A., Miceli, M., Iorio, G. G., Mappa, I., Rizzo, G., Girolamo, R. D., et al. (2023) · American Journal of Obstetrics & Gynecology MFM · doi:10.1016/j.ajogmf.2022.100792 · Cited by 47 · see citing works
- Authors
- The CONSORT-AI and SPIRIT-AI Steering Group
- Corresponding author
- The CONSORT-AI and SPIRIT-AI Steering Group
- Venue
- Nature Medicine
- Published
- 2019-09-24
- Volume
- 25
- Issue
- 10
- Pages
- 1467-1468
- ISSN
- 1078-8956 (ISSN-L)
- 1546-170X
- OpenAlex
- W3202966610
- Cited by
- 143
- References
- 2
- Citations by year
- 2026: 14 · 2025: 16 · 2024: 12 · 2023: 18 · 2022: 23 · 2021: 28 · 2020: 30 · 2019: 2
- FWCI
- 13.88
- Type
- journal article
- Language
- en
- Update notices
- correction (2019-11-12)
- Open access
- bronze
- doi.org
- Locations
- Nature Medicine (publishedVersion)
- PubMed (publishedVersion)
- University of Birmingham Research Portal (University of Birmingham) (publishedVersion, other-oa)
- UCL Discovery (University College London) (submittedVersion)
- Research Explorer (The University of Manchester) (submittedVersion)
- Funders
- Wellcome Trust
- National Institute for Health and Care Research (CS-2014-14-023, NF-SI-0510-10186, NF-SI-0617-10038)
- Health Data Research UK
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Keywords
- artificial intelligence interventions, clinical trial reporting, reporting guidelines, CONSORT, alcoholic beverages, transparency, clinical trial design
- MeSH terms
- Artificial Intelligence, Clinical Trials as Topic, Humans, United States, United States Food and Drug Administration, Guidelines as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
