CoChat

Health Care Professionals’ Experience of Using AI: Systematic Review With Narrative Synthesis

Open access
Ayorinde, Abimbola, Mensah, Daniel Opoku, Walsh, Julia, Ghosh, Iman, Ibrahim, Siti Aishah, Hogg, Jeffry, Peek, Niels, Griffiths, Frances · 2024 · Journal of Medical Internet Research

Published version of a preprint

The DOI matches its Crossref record; 69 of the 69 references it cites were found in a registry or an index.

Abstract

Background: There has been a substantial increase in the development of artificial intelligence (AI) tools for clinical decision support. Historically, these were mostly knowledge-based systems, but recent advances include non–knowledge-based systems using some form of machine learning. The ability of health care professionals to trust technology and understand how it benefits patients or improves care delivery is known to be important for their adoption of that technology. For non–knowledge-based AI tools for clinical decision support, these issues are poorly understood. Objective: The aim of this study is to qualitatively synthesize evidence on the experiences of health care professionals in routinely using non–knowledge-based AI tools to support their clinical decision-making. Methods: In June 2023, we searched 4 electronic databases, MEDLINE, Embase, CINAHL, and Web of Science, with no language or date limit. We also contacted relevant experts and searched reference lists of the included studies. We included studies of any design that reported the experiences of health care professionals using non–knowledge-based systems for clinical decision support in their work settings. We completed double independent quality assessment for all included studies using the Mixed Methods Appraisal Tool. We used a theoretically informed thematic approach to synthesize the findings. Results: After screening 7552 titles and 182 full-text articles, we included 25 studies conducted in 9 different countries. Most of the included studies were qualitative (n=13), and the remaining were quantitative (n=9) and mixed methods (n=3). Overall, we identified 7 themes: health care professionals’ understanding of AI applications, level of trust and confidence in AI tools, judging the value added by AI, data availability and limitations of AI, time and competing priorities, concern about governance, and collaboration to facilitate the implementation and use of AI. The most frequently occurring are the first 3 themes. For example, many studies reported that health care professionals were concerned about not understanding the AI outputs or the rationale behind them. There were issues with confidence in the accuracy of the AI applications and their recommendations. Some health care professionals believed that AI provided added value and improved decision-making, and some reported that it only served as a confirmation of their clinical judgment, while others did not find it useful at all. Conclusions: Our review identified several important issues documented in various studies on health care professionals’ use of AI tools in real-world health care settings. Opinions of health care professionals regarding the added value of AI tools for supporting clinical decision-making varied widely, and many professionals had concerns about their understanding of and trust in this technology. The findings of this review emphasize the need for concerted efforts to optimize the integration of AI tools in real-world health care settings. Trial Registration: PROSPERO CRD42022336359; https://tinyurl.com/2yunvkmb

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.2196/55766

Cites

69 of 69 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 Schematic illustration of the data coding process.
Figure 1 Schematic illustration of the data coding process.CC BY · PubMed Central
Figure 2 PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flowchart of study selection.
Figure 2 PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flowchart of study selection.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 1

  1. McAdam, WA, 1990, Ann R Coll Surg Engl, 72, 140

Cited by

See all 75

Metadata

Authors
Ayorinde, A., Mensah, D. O., Walsh, J., Ghosh, I., Ibrahim, S. A., Hogg, J., Peek, N., & Griffiths, F.
Affiliations
  • University of Warwick (GB)
  • University of Cambridge (GB)
Corresponding author
Ayorinde, Abimbola
Venue
Journal of Medical Internet Research
Published
2024-10-30
Volume
26
Pages
e55766
ISSN
1438-8871 (ISSN-L)
OpenAlex
W4400984022
Cited by
55
References
69
Citations by year
2026: 33 · 2025: 22
FWCI
1.24
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • Journal of Medical Internet Research (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • University of Birmingham Research Portal (University of Birmingham) (publishedVersion, cc-by)
  • Warwick Research Archive Portal (University of Warwick) (acceptedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • National Institute for Health and Care Research (NIHR200174, NIHR301467, NIHR200165)
  • Department of Health and Social Care
  • University of Warwick
  • Warwick Medical School
  • National Institute for Health Research Applied Research Collaboration West (NIHR 200165)
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
  • Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
Keywords
artificial intelligence in clinical decision support, health professionals' experiences, trust in AI, clinical decision-making, artificial intelligence adoption, AI governance, explainable AI, human-AI collaboration, AI implementation, time constraints in clinical practice
MeSH terms
Clinical Decision-Making, Artificial Intelligence, Health Personnel, Humans, Decision Support Systems, Clinical

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.