Guidelines, Consensus Statements, and Standards for the Use of Artificial Intelligence in Medicine: Systematic Review
Open accessThe DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index.
Abstract
Background: The application of artificial intelligence (AI) in the delivery of health care is a promising area, and guidelines, consensus statements, and standards on AI regarding various topics have been developed. Objective: We performed this study to assess the quality of guidelines, consensus statements, and standards in the field of AI for medicine and to provide a foundation for recommendations about the future development of AI guidelines. Methods: We searched 7 electronic databases from database establishment to April 6, 2022, and screened articles involving AI guidelines, consensus statements, and standards for eligibility. The AGREE II (Appraisal of Guidelines for Research & Evaluation II) and RIGHT (Reporting Items for Practice Guidelines in Healthcare) tools were used to assess the methodological and reporting quality of the included articles. Results: This systematic review included 19 guideline articles, 14 consensus statement articles, and 3 standard articles published between 2019 and 2022. Their content involved disease screening, diagnosis, and treatment; AI intervention trial reporting; AI imaging development and collaboration; AI data application; and AI ethics governance and applications. Our quality assessment revealed that the average overall AGREE II score was 4.0 (range 2.2-5.5; 7-point Likert scale) and the mean overall reporting rate of the RIGHT tool was 49.4% (range 25.7%-77.1%). Conclusions: The results indicated important differences in the quality of different AI guidelines, consensus statements, and standards. We made recommendations for improving their methodological and reporting quality. Trial Registration: PROSPERO International Prospective Register of Systematic Reviews (CRD42022321360); https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=321360
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Skivington, K., Matthews, L., Simpson, S. A., Craig, P., Baird, J., Blazeby, J. M., Boyd, K. A., Craig, N., French, D. P., McIntosh, E., et al. (2021) · BMJ · doi:10.1136/bmj.n2061 · Cited by 5,808
International evaluation of an AI system for breast cancer screening
McKinney, S. M., Sieniek, M., Godbole, V., Godwin, J., Antropova, N., Ashrafian, H., Back, T., Chesus, M., Corrado, G. S., Darzi, A., et al. (2020) · Nature · doi:10.1038/s41586-019-1799-6 · Cited by 3,287 · cited 1×
“AI has been widely used in medical imaging [1], disease screening [2], the prediction and evaluation of treatments [3], the development of disease models from patient trajectories [4], and other areas.”
MethodRajpurkar, P., Chen, E., Banerjee, O., & Topol, E. J. (2022) · Nature Medicine · doi:10.1038/s41591-021-01614-0 · Cited by 3,212 · cited 1×
“AI is expected to make medicine-related practice, research, and applications more accurate [5].”
BackgroundVideo-based AI for beat-to-beat assessment of cardiac function
Ouyang, D., He, B., Ghorbani, A., Yuan, N., Ebinger, J., Langlotz, C. P., Heidenreich, P. A., Harrington, R. A., Liang, D. H., Ashley, E. A., et al. (2020) · Nature · doi:10.1038/s41586-020-2145-8 · Cited by 1,153
Liu, X., Rivera, S. C., Moher, D., Calvert, M. J., & Denniston, A. K. (2020) · BMJ · doi:10.1136/bmj.m3164 · Cited by 702
Towards a guideline for evaluation metrics in medical image segmentation
Müller, D., Soto-Rey, I., & Kramer, F. (2022) · BMC Research Notes · doi:10.1186/s13104-022-06096-y · Cited by 668 · cited 2×
“Except for 20 articles [23,30,32-35, 37, 38, Although depth was lacking, 16 guideline articles considered health benefits, side effects, and hazards during the development process, and detailed the potential benefits and hazards of following the advice.”
BackgroundDisparities in dermatology AI performance on a diverse, curated clinical image set
Daneshjou, R., Vodrahalli, K., Novoa, R. A., Jenkins, M., Liang, W., Rotemberg, V., Ko, J., Swetter, S. M., Bailey, E. E., Gevaert, O., et al. (2022) · Science Advances · doi:10.1126/sciadv.abq6147 · Cited by 544
Rivera, S. C., Liu, X., Chan, A.-W., Denniston, A. K., & Calvert, M. J. (2020) · BMJ · doi:10.1136/bmj.m3210 · Cited by 463 · cited 3×
“Twenty-five articles [19-21,23-26,28,29,31-34,36-41,44-46, 48,50,51] explicitly stated their editorial independence.”
BackgroundDaneshjou, R., Smith, M. P., Sun, M. D., Rotemberg, V., & Zou, J. (2021) · JAMA Dermatology · doi:10.1001/jamadermatol.2021.3129 · Cited by 403
With an eye to AI and autonomous diagnosis
Keane, P. A., & Topol, E. J. (2018) · npj Digital Medicine · doi:10.1038/s41746-018-0048-y · Cited by 304 · cited 1×
“With the likely further clinical translation of AI systems, it will become increasingly important for AI guidelines to be both followed and regularly updated [13].”
Background
Cited by
Current status of ChatGPT use in medical education: potentials, challenges, and strategies
Xu, T., Weng, H., Liu, F., Yang, L., Luo, Y., Ding, Z., & Wang, Q. (2024) · Journal of Medical Internet Research · doi:10.2196/57896 · Cited by 84
Bailo, P., Nittari, G., Pesel, G., Basello, E., Spasari, T., & Ricci, G. (2026) · Sci · doi:10.3390/sci8020036 · Cited by 22
“These findings echo external validation failures of widely implemented proprietary models, reminding us that safety is a continuous property produced by monitoring, feedback loops, and the capacity to pause, update, or retire systems [59,146,150].”
Alami, H., Pozelli Sabio, R., Pérez, E. J., Gagnon, M.-P., Langlois, L., Denis, J.-L., Malas, K., Rivard, L., Savoldelli, M., Ag Ahmed, M. A., et al. (2026) · Journal of Medical Internet Research · doi:10.2196/87448 · Cited by 9
“In response, stakeholders have called for robust and adaptive AI governance frameworks for HS to safeguard the system and enable responsible, sustainable, and effective integration of AI-based technologies [1,12-15].”
Xu, N. Y., Litake, O., Tully, J. L., Meineke, M. N., Sinha, A., Meyer, M., & Gabriel, R. A. (2025) · Journal of Clinical Monitoring and Computing · doi:10.1007/s10877-025-01401-z · Cited by 2
“However, widespread adoption of these modalities is limited by several barriers, including more consistent consensus guidelines for the use of artificial intelligence and limitations of infrastructure capabilities [22, 23].”
Metadata
- Authors
- Wang, Y., Li, N., Chen, L., Wu, M., Meng, S., Dai, Z., Zhang, Y., & Clarke, M.
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2023-11-22
- Volume
- 25
- Pages
- e46089
- ISSN
- 1438-8871 (ISSN-L)
- OpenAlex
- W4388900228
- Cited by
- 45
- References
- 14
- FWCI
- 0.74
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Guidelines, Consensus Statements, and Standards for the Use of Artificial Intelligence in Medicine: Systematic Review. https://app.cochat.ai/papers/10.2196/46089
- The paper
- Wang, Y., Li, N., Chen, L., Wu, M., Meng, S., Dai, Z., Zhang, Y., & Clarke, M. (2023). Guidelines, Consensus Statements, and Standards for the Use of Artificial Intelligence in Medicine: Systematic Review. Journal of Medical Internet Research. https://doi.org/10.2196/46089
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.