CoChat

Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extension

Open access
Cruz Rivera, Samantha, Liu, Xiaoxuan, Chan, An-Wen, Denniston, Alastair K., Calvert, Melanie J., The SPIRIT-AI and CONSORT-AI Working Group, SPIRIT-AI and CONSORT-AI Steering Group, Darzi, Ara, Holmes, Christopher, Yau, Christopher, Moher, David, Ashrafian, Hutan, Deeks, Jonathan J., Ferrante di Ruffano, Lavinia, Faes, Livia, Keane, Pearse A., Vollmer, Sebastian J., SPIRIT-AI and CONSORT-AI Consensus Group, Lee, Aaron Y., Jonas, Adrian, Esteva, Andre, Beam, Andrew L., Panico, Maria Beatrice, Lee, Cecilia S., Haug, Charlotte, Kelly, Christophe J., Yau, Christopher, Mulrow, Cynthia, Espinoza, Cyrus, Fletcher, John, Moher, David, Paltoo, Dina, Manna, Elaine, Price, Gary, Collins, Gary S., Harvey, Hugh, Matcham, James, Monteiro, Joao, ElZarrad, M. Khair, Ferrante di Ruffano, Lavinia, Oakden-Rayner, Luke, McCradden, Melissa, Keane, Pearse A., Savage, Richard, Golub, Robert, Sarkar, Rupa, Rowley, Samuel · 2020 · Nature Medicine

The DOI matches its Crossref record; 53 of the 53 references it cites were found in a registry or an index; 19 of the 34 sources its claims cite were checked for support.

Abstract

The SPIRIT 2013 statement aims to improve the completeness of clinical trial protocol reporting by providing evidence-based recommendations for the minimum set of items to be addressed. This guidance has been instrumental in promoting transparent evaluation of new interventions. More recently, there has been a growing recognition that interventions involving artificial intelligence (AI) need to undergo rigorous, prospective evaluation to demonstrate their impact on health outcomes. The SPIRIT-AI (Standard Protocol Items: Recommendations for Interventional Trials–Artificial Intelligence) extension is a new reporting guideline for clinical trial protocols evaluating interventions with an AI component. It was developed in parallel with its companion statement for trial reports: CONSORT-AI (Consolidated Standards of Reporting Trials–Artificial Intelligence). Both guidelines were developed through a staged consensus process involving literature review and expert consultation to generate 26 candidate items, which were consulted upon by an international multi-stakeholder group in a two-stage Delphi survey (103 stakeholders), agreed upon in a consensus meeting (31 stakeholders) and refined through a checklist pilot (34 participants). The SPIRIT-AI extension includes 15 new items that were considered sufficiently important for clinical trial protocols of AI interventions. These new items should be routinely reported in addition to the core SPIRIT 2013 items. SPIRIT-AI recommends that investigators provide clear descriptions of the AI intervention, including instructions and skills required for use, the setting in which the AI intervention will be integrated, considerations for the handling of input and output data, the human–AI interaction and analysis of error cases. SPIRIT-AI will help promote transparency and completeness for clinical trial protocols for AI interventions. Its use will assist editors and peer reviewers, as well as the general readership, to understand, interpret and critically appraise the design and risk of bias for a planned clinical trial.

Overview

CoChat Summary

TL;DR

The SPIRIT-AI extension provides consensus-based reporting guidance for clinical trial protocols evaluating interventions with an artificial intelligence component. It adds 15 AI-specific items—12 extensions and 3 elaborations—to the SPIRIT 2013 checklist, covering intervention description, input/output data, setting, human–AI interaction, and performance-error analysis.

Key findings

  • SPIRIT-AI recommends 15 new items for AI trial protocols: 12 extensions and 3 elaborations, all of which passed the pre-specified 80% consensus threshold.
  • The extension was developed alongside CONSORT-AI through literature review, a two-stage Delphi survey with 103 first-round respondents and 91 second-round respondents (88% of round one), a 31-stakeholder consensus meeting, and a 34-participant checklist pilot.
  • SPIRIT-AI 11a (iii), on assessing and handling poor-quality or unavailable input data, initially received 73% support, then passed after rewording with 97% support.
  • The guidance emphasizes clear descriptions of the AI intervention, required user instructions and skills, integration setting, input and output data handling, human–AI interaction, and plans to identify and analyze performance errors.

Methods

  • Guideline development followed the EQUATOR Network methodological framework and was conducted in parallel with CONSORT-AI, overseen by a 15-member international Steering Group.
  • Candidate items were generated from a 13 May 2019 ClinicalTrials.gov search using “artificial intelligence,” “machine learning,” and “deep learning,” which identified 316 registered trials, 62 completed trials, and 7 with published results; after consultation with 19 additional experts, 29 candidate items were generated, 26 relevant to both SPIRIT-AI and CONSORT-AI and 3 relevant only to CONSORT-AI.
  • Consensus used two online Delphi surveys, a two-day consensus meeting with anonymous electronic voting and an 80% inclusion threshold, and a pilot of the penultimate checklists with 34 participants.

Limitations

  • The paper states that at the time of SPIRIT-AI development there were only seven published AI trials and no published AI trial protocols, so decisions were not always supported by existing real-world examples.
  • The paper states that the literature search used “artificial intelligence,” “machine learning,” and “deep learning” but not terms such as “clinical decision support systems” and “expert systems,” so those older AI-related technologies were not actively discussed.
  • The paper states that the initial candidate item list was generated by a relatively small group of experts, although additional items from the wider Delphi group were considered and no new items were suggested during the consensus meeting or post-meeting evaluation.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1038/s41591-020-1037-7

Cites

53 of 53 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Fig. 1 CONSORT 2010 flow diagram — adapted for AI clinical trials. SPIRIT-AI 10 (i): State the inclusion and exclusion criteria at the level of participants. SPIRIT-AI 10 (ii): State the inclusion and exclusion criteria at the level of the input data. SPIRIT 13 (core CONSORT item): Time schedule of enrollment, interventions (including any run-ins and washouts), assessments, and visits for participants. A schematic diagram is highly recommended.
Fig. 1 CONSORT 2010 flow diagram — adapted for AI clinical trials. SPIRIT-AI 10 (i): State the inclusion and exclusion criteria at the level of participants. SPIRIT-AI 10 (ii): State the inclusion and exclusion criteria at the level of the input data. SPIRIT 13 (core CONSORT item): Time schedule of enrollment, interventions (including any run-ins and washouts), assessments, and visits for participants. A schematic diagram is highly recommended.CC BY · PubMed Central

Verification

Claims the paper makes

14 Valid · 3 Not valid · 5 Not checked · 41 Own results and interpretations

8 more claims

Create a free account to see all 63 claims and to verify the 2 references still open.

See all claims — free account
  • Probably not a substantial claim. 46
  • It may state two separate claims as one. 15

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

7 more claims about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Citations

Cites

Show all 53 references

Cited by

See all 933

Metadata

Authors
Cruz Rivera, S., Liu, X., Chan, A.-W., Denniston, A. K., Calvert, M. J., The SPIRIT-AI and CONSORT-AI Working Group, SPIRIT-AI and CONSORT-AI Steering Group, Darzi, A., Holmes, C., Yau, C., Moher, D., Ashrafian, H., Deeks, J. J., Ferrante di Ruffano, L., Faes, L., Keane, P. A., Vollmer, S. J., SPIRIT-AI and CONSORT-AI Consensus Group, Lee, A. Y., Jonas, A., Esteva, A., Beam, A. L., Panico, M. B., Lee, C. S., Haug, C., Kelly, C. J., Yau, C., Mulrow, C., Espinoza, C., Fletcher, J., Moher, D., Paltoo, D., Manna, E., Price, G., Collins, G. S., Harvey, H., Matcham, J., Monteiro, J., ElZarrad, M. K., Ferrante di Ruffano, L., Oakden-Rayner, L., McCradden, M., Keane, P. A., Savage, R., Golub, R., Sarkar, R., & Rowley, S.
Affiliations
  • University Hospitals Birmingham NHS Foundation Trust (GB)
  • University of Toronto (CA)
  • Imperial College London (GB)
  • The Alan Turing Institute (GB)
  • University of Oxford (GB)
  • Luzerner Kantonsspital (CH)
  • National Institute for Health and Care Excellence (GB)
  • Salesforce (United States) (US)
  • Harvard University (US)
  • University of Washington (US)
  • Google (United Kingdom) (GB)
  • Ottawa Hospital Research Institute (CA)
  • National Institutes of Health (US)
  • BMJ Group (United Kingdom) (GB)
  • AstraZeneca (United Kingdom) (GB)
  • United States Food and Drug Administration (US)
  • American Medical Association (US)
  • Medical Research Council (GB)
Corresponding author
Denniston, Alastair K.
Venue
Nature Medicine
Published
2020-09
Volume
26
Issue
9
Pages
1351-1363
ISSN
  • 1078-8956 (ISSN-L)
  • 1546-170X
OpenAlex
W3136960503
Cited by
933
References
49
Citations by year
2026: 393 · 2025: 129 · 2024: 87 · 2023: 94 · 2022: 111 · 2021: 105 · 2020: 14
FWCI
12.16
Type
journal article
Language
en
Open access
Licence
Locations
  • Nature Medicine (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • University of Birmingham Research Portal (University of Birmingham) (publishedVersion, cc-by)
  • Oxford University Research Archive (ORA) (University of Oxford) (submittedVersion, cc-by)
  • University of Birmingham Research Portal (University of Birmingham) (submittedVersion, cc-by)
  • Spiral (Imperial College London) (submittedVersion, cc-by)
Funders
  • Wellcome Trust
  • Alan Turing Institute
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
Keywords
reporting guidelines, artificial intelligence interventions, human-AI interaction, risk of bias, trial design, CONSORT-AI
MeSH terms
Artificial Intelligence, Clinical Trials as Topic, Humans, Research Design

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-15, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-15 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (qwen3.8-omni-flash and gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extension. https://app.cochat.ai/papers/10.1038/s41591-020-1037-7
The paper
Cruz Rivera, S., Liu, X., Chan, A.-W., Denniston, A. K., Calvert, M. J., The SPIRIT-AI and CONSORT-AI Working Group, SPIRIT-AI and CONSORT-AI Steering Group, Darzi, A., Holmes, C., Yau, C., Moher, D., Ashrafian, H., Deeks, J. J., Ferrante di Ruffano, L., Faes, L., Keane, P. A., Vollmer, S. J., SPIRIT-AI and CONSORT-AI Consensus Group, Lee, A. Y., . . . Rowley, S. (2020). Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extension. Nature Medicine. https://doi.org/10.1038/s41591-020-1037-7

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.