The DOI matches its Crossref record; 52 of the 52 references it cites were found in a registry or an index.
Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extension
Open accessAbstract
The SPIRIT 2013 statement aims to improve the completeness of clinical trial protocol reporting by providing evidence-based recommendations for the minimum set of items to be addressed. This guidance has been instrumental in promoting transparent evaluation of new interventions. More recently, there has been a growing recognition that interventions involving artificial intelligence (AI) need to undergo rigorous, prospective evaluation to demonstrate their impact on health outcomes. The SPIRIT-AI (Standard Protocol Items: Recommendations for Interventional Trials-Artificial Intelligence) extension is a new reporting guideline for clinical trial protocols evaluating interventions with an AI component. It was developed in parallel with its companion statement for trial reports: CONSORT-AI (Consolidated Standards of Reporting Trials-Artificial Intelligence). Both guidelines were developed through a staged consensus process involving literature review and expert consultation to generate 26 candidate items, which were consulted upon by an international multi-stakeholder group in a two-stage Delphi survey (103 stakeholders), agreed upon in a consensus meeting (31 stakeholders) and refined through a checklist pilot (34 participants). The SPIRIT-AI extension includes 15 new items that were considered sufficiently important for clinical trial protocols of AI interventions. These new items should be routinely reported in addition to the core SPIRIT 2013 items. SPIRIT-AI recommends that investigators provide clear descriptions of the AI intervention, including instructions and skills required for use, the setting in which the AI intervention will be integrated, considerations for the handling of input and output data, the human-AI interaction and analysis of error cases. SPIRIT-AI will help promote transparency and completeness for clinical trial protocols for AI interventions. Its use will assist editors and peer reviewers, as well as the general readership, to understand, interpret, and critically appraise the design and risk of bias for a planned clinical trial.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures


Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“the SPIRIT-AI (Standard Protocol Items: Recommendations for Interventional Trials-Artificial Intelligence) extension guidelines clearly state the importance of pre-existing evidence for AI intervention, with evidence that the AI-assisted tool produces better performance compared with the standard of care [58].”
From Randomized Controlled Trials of Artificial Intelligence in Clinical Practice: Systematic Review (2022) · cites this paper as [58]
Experimental — accuracy not yet measuredIn the citing paper
“Additional guidelines include SPIRIT-AI (Standard Protocol Items: Recommendations for Interventional Trials-Artificial Intelligence), which provides an extended set of 15 items for AI-focused protocol documents (199), and CONSORT-AI (Consolidated Standards of Reporting of Trials-Artificial Intelligence), which provides an extended set of 14 items for reporting on AI-focused clinical trials (200).”
From Artificial Intelligence in Oncology: Current Landscape, Challenges, and Future Directions (2024) · cites this paper as [199]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Dermatologist-level classification of skin cancer with deep neural networks
Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017) · Nature · doi:10.1038/nature21056 · Cited by 14,362
Self-citationSPIRIT 2013 Statement: Defining Standard Protocol Items for Clinical Trials
Chan, A.-W., Tetzlaff, J. M., Altman, D. G., Laupacis, A., Gøtzsche, P. C., Krleža-Jerić, K., Hróbjartsson, A., Mann, H., Dickersin, K., Berlin, J. A., et al. (2013) · Annals of Internal Medicine · doi:10.7326/0003-4819-158-3-201302050-00583 · Cited by 7,363
Self-citationSPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials
Chan, A.-W., Tetzlaff, J. M., Gotzsche, P. C., Altman, D. G., Mann, H., Berlin, J. A., Dickersin, K., Hrobjartsson, A., Schulz, K. F., Parulekar, W. R., et al. (2013) · BMJ · doi:10.1136/bmj.e7586 · Cited by 6,251
Self-citationInternational evaluation of an AI system for breast cancer screening
McKinney, S. M., Sieniek, M., Godbole, V., Godwin, J., Antropova, N., Ashrafian, H., Back, T., Chesus, M., Corrado, G. S., Darzi, A., et al. (2020) · Nature · doi:10.1038/s41586-019-1799-6 · Cited by 3,287
Self-citationKey challenges for delivering clinical impact with artificial intelligence
Kelly, C. J., Karthikesalingam, A., Suleyman, M., Corrado, G., & King, D. (2019) · BMC Medicine · doi:10.1186/s12916-019-1426-2 · Cited by 3,136 · cited 1×
“37,38 AI systems are dependent on their operational environment, and the protocol should provide details of the hardware and software requirements to allow technical integration of the AI intervention at each study site.”
Self-citationAn overview of clinical decision support systems: Benefits, risks, and strategies for success
Sutton, R. T., Pincock, D., Baumgart, D. C., Sadowski, D. C., Fedorak, R. N., & Kroeker, K. I. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0221-y · Cited by 3,019
Clinically applicable deep learning for diagnosis and referral in retinal disease
De Fauw, J., Ledsam, J. R., Romera-Paredes, B., Nikolov, S., Tomasev, N., Blackwell, S., Askham, H., Glorot, X., O’Donoghue, B., Visentin, D., et al. (2018) · Nature Medicine · doi:10.1038/s41591-018-0107-6 · Cited by 2,639
Self-citationThe practical implementation of artificial intelligence technologies in medicine
He, J., Baxter, S. L., Xu, J., Xu, J., Zhou, X., & Zhang, K. (2019) · Nature Medicine · doi:10.1038/s41591-018-0307-0 · Cited by 2,246
Liu, X., Faes, L., Kale, A. U., Wagner, S. K., Fu, D. J., Bruynseels, A., Mahendiran, T., Moraes, G., Shamdas, M., Kern, C., et al. (2019) · The Lancet Digital Health · doi:10.1016/s2589-7500(19)30123-2 · Cited by 1,883
Self-citationZech, J. R., Badgeley, M. A., Liu, M., Costa, A. B., Titano, J. J., & Oermann, E. K. (2018) · PLOS Medicine · doi:10.1371/journal.pmed.1002683 · Cited by 1,719
Cited by
Rajpurkar, P., Chen, E., Banerjee, O., & Topol, E. J. (2022) · Nature Medicine · doi:10.1038/s41591-021-01614-0 · Cited by 3,212
Artificial Intelligence and Machine Learning in Clinical Medicine, 2023
Haug, C. J., & Drazen, J. M. (2023) · New England Journal of Medicine · doi:10.1056/nejmra2302038 · Cited by 1,215
Self-citationVasey, B., Nagendran, M., Campbell, B., Clifton, D. A., Collins, G. S., Denaxas, S., Denniston, A. K., Faes, L., Geerts, B., Ibrahim, M., et al. (2022) · Nature Medicine · doi:10.1038/s41591-022-01772-9 · Cited by 1,039
Self-citationThe next generation of evidence-based medicine
Subbiah, V. (2023) · Nature Medicine · doi:10.1038/s41591-022-02160-z · Cited by 544
Ethical and Bias Considerations in Artificial Intelligence/Machine Learning
Hanna, M. G., Pantanowitz, L., Jackson, B., Palmer, O., Visweswaran, S., Pantanowitz, J., Deebajah, M., & Rashidi, H. H. (2025) · Modern Pathology · doi:10.1016/j.modpat.2024.100686 · Cited by 535
Metadata
- Authors
- Cruz Rivera, S., Liu, X., Chan, A.-W., Denniston, A. K., Calvert, M. J., Ashrafian, H., Beam, A. L., Collins, G. S., Darzi, A., Deeks, J. J., ElZarrad, M. K., Espinoza, C., Esteva, A., Faes, L., Ferrante di Ruffano, L., Fletcher, J., Golub, R., Harvey, H., Haug, C., Holmes, C., Jonas, A., Keane, P. A., Kelly, C. J., Lee, A. Y., Lee, C. S., Manna, E., Matcham, J., McCradden, M., Moher, D., Monteiro, J., Mulrow, C., Oakden-Rayner, L., Paltoo, D., Panico, M. B., Price, G., Rowley, S., Savage, R., Sarkar, R., Vollmer, S. J., & Yau, C.
- Affiliations
- University Hospitals Birmingham NHS Foundation Trust (GB)
- University of Toronto (CA)
- Moorfields Eye Hospital NHS Foundation Trust (GB)
- Health Data Research UK (GB)
- University of Birmingham (GB)
- Corresponding author
- Denniston, Alastair K
- ORCID
- Liu, Xiaoxuan
- Chan, An-Wen
- Denniston, Alastair K
- Calvert, Melanie J
- Ashrafian, Hutan
- Beam, Andrew L
- Darzi, Ara
- Deeks, Jonathan J
- ElZarrad, M Khair
- Esteva, Andre
- Faes, Livia
- Fletcher, John
- Harvey, Hugh
- Haug, Charlotte
- Keane, Pearse A
- Kelly, Christopher J
- Lee, Aaron Y
- Lee, Cecilia S
- McCradden, Melissa
- Moher, David
- Mulrow, Cynthia
- Rowley, Samuel
- Savage, Richard
- Sarkar, Rupa
- Vollmer, Sebastian J
- Yau, Christopher
- Venue
- The Lancet Digital Health
- Publisher
- Elsevier BV
- Published
- 2020-10
- Volume
- 2
- Issue
- 10
- Pages
- e549-e560
- ISSN
- 2589-7500 (ISSN-L)
- PMC
- PMC8212701
- OpenAlex
- W3084438218
- Cited by
- 486
- References
- 50
- Citations by year
- 2026: 117 · 2025: 88 · 2024: 85 · 2023: 82 · 2022: 63 · 2021: 47 · 2020: 4
- FWCI
- 10.12
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by-nc-nd
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
- www.elsevier.com/legal/tdmrep-license (tdm)
- creativecommons.org/licenses/by-nc-nd/4.0/ (vor)
- Locations
- The Lancet Digital Health (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- University of Birmingham Research Portal (University of Birmingham) (publishedVersion, cc-by-nc-nd)
- PubMed Central (acceptedVersion, cc-by-nc-nd)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- UCL Discovery (University College London) (submittedVersion)
- University of Birmingham Research Portal (University of Birmingham) (submittedVersion, cc-by-nc-nd)
- Spiral (Imperial College London) (submittedVersion, other-oa)
- Funders
- National Institutes of Health (7K01HL141771-02)
- Health Foundation
- Alan Turing Institute
- U.S. Food and Drug Administration
- University of Birmingham
- University Hospitals Birmingham NHS Foundation Trust
- U.S. National Library of Medicine
- European Regional Development Fund
- UK Research and Innovation
- Office of the Director
- Surgical Reconstruction and Microbiology Research Centre
- UCB Pharma
- Moorfields Eye Charity
- National Institute for Health and Care Research
- Macmillan Cancer Support
- Birmingham Biomedical Research Centre
- Wellcome Trust
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Keywords
- reporting guidelines, artificial intelligence interventions, artificial intelligence in healthcare, human-AI interaction, risk of bias, CONSORT-AI, Delphi consensus
- MeSH terms
- Artificial Intelligence, Clinical Trials as Topic, Delphi Technique, Humans, Publishing, Research Design, Guidelines as Topic, Disclosure, Consensus, Checklist, Research Report
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extension. https://app.cochat.ai/papers/10.1016/s2589-7500(20)30219-3
- The paper
- Cruz Rivera, S., Liu, X., Chan, A.-W., Denniston, A. K., Calvert, M. J., Ashrafian, H., Beam, A. L., Collins, G. S., Darzi, A., Deeks, J. J., ElZarrad, M. K., Espinoza, C., Esteva, A., Faes, L., Ferrante di Ruffano, L., Fletcher, J., Golub, R., Harvey, H., Haug, C., . . . Yau, C. (2020). Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extension. The Lancet Digital Health. https://doi.org/10.1016/s2589-7500(20)30219-3
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.