CoChat
Sign Up
Cited by 39Artificial Intelligence in Healthcare and EducationHealth Policy Implementation ScienceArtificial Intelligence Applications

Toward Successful Implementation of Artificial Intelligence in Health Care Practice: Protocol for a Research Program

Open access
Svedberg, Petra, Reed, Julie, Nilsen, Per, Barlow, James, Macrae, Carl, Nygren, Jens·2022·JMIR Research Protocols
DOI 10.2196/34920PMC PMC8943554

Published version of a preprint

Open this paper in CoChat — free accountOpen PDF

Abstract

Background: The uptake of artificial intelligence (AI) in health care is at an early stage. Recent studies have shown a lack of AI-specific implementation theories, models, or frameworks that could provide guidance for how to translate the potential of AI into daily health care practices. This protocol provides an outline for the first 5 years of a research program seeking to address this knowledge-practice gap through collaboration and co-design between researchers, health care professionals, patients, and industry stakeholders. Objective: The first part of the program focuses on two specific objectives. The first objective is to develop a theoretically informed framework for AI implementation in health care that can be applied to facilitate such implementation in routine health care practice. The second objective is to carry out empirical AI implementation studies, guided by the framework for AI implementation, and to generate learning for enhanced knowledge and operational insights to guide further refinement of the framework. The second part of the program addresses a third objective, which is to apply the developed framework in clinical practice in order to develop regional capacity to provide the practical resources, competencies, and organizational structure required for AI implementation; however, this objective is beyond the scope of this protocol. Methods: This research program will use a logic model to structure the development of a methodological framework for planning and evaluating implementation of AI systems in health care and to support capacity building for its use in practice. The logic model is divided into time-separated stages, with a focus on theory-driven and coproduced framework development. The activities are based on both knowledge development, using existing theory and literature reviews, and method development by means of co-design and empirical investigations. The activities will involve researchers, health care professionals, and other stakeholders to create a multi-perspective understanding. Results: The project started on July 1, 2021, with the Stage 1 activities, including model overview, literature reviews, stakeholder mapping, and impact cases; we will then proceed with Stage 2 activities. Stage 1 and 2 activities will continue until June 30, 2026. Conclusions: There is a need to advance theory and empirical evidence on the implementation requirements of AI systems in health care, as well as an opportunity to bring together insights from research on the development, introduction, and evaluation of AI systems and existing knowledge from implementation research literature. Therefore, with this research program, we intend to build an understanding, using both theoretical and empirical approaches, of how the implementation of AI systems should be approached in order to increase the likelihood of successful and widespread application in clinical practice. International Registered Report Identifier (IRRID): PRR1-10.2196/34920

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.2196/34920

Cites

9 of 9 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 35

  1. [1]Barlow J Bayer S Curry R Implementing complex innovations in fluid multi-stakeholder environments: Experiences of ‘telecare’ Technovation 2006 3 26 3 396 406 10.1016/j.technovation.2005.06.010 · DOI
  2. [2]Webster A Gardner J Aligning technology and institutional readiness: The adoption of innovation Technol Anal Strateg Manag 2019 04 02 31 10 1229 1241 10.1080/09537325.2019.1601694 · DOI
  3. [3]Mehta N Pandit A Shukla S Transforming healthcare with big data analytics and artificial intelligence: A systematic mapping study J Biomed Inform 2019 12 100 103311 10.1016/j.jbi.2019.103311 31629922 S1532-0464(19)30230-8 31629922 · DOI
  4. [6]Shaw J Rudzicz F Jamieson T Goldfarb A Artificial intelligence and the implementation challenge J Med Internet Res 2019 07 10 21 7 e13659 10.2196/13659 31293245 v21i7e13659 31293245 PMC6652121 · DOI · Paper page
  5. [7]Guo Y Hao Z Zhao S Gong J Yang F Artificial intelligence in health care: Bibliometric analysis J Med Internet Res 2020 07 29 22 7 e18228 10.2196/18228 32723713 v22i7e18228 32723713 PMC7424481 · DOI
  6. [9]Murphy K Di Ruggiero E Upshur R Willison DJ Malhotra N Cai JC Malhotra N Lui V Gibson J Artificial intelligence for good health: A scoping review of the ethics literature BMC Med Ethics 2021 02 15 22 1 14 10.1186/s12910-021-00577-8 33588803 10.1186/s12910-021-00577-8 33588803 PMC7885243 · DOI · Paper page
  7. [10]Jiang F Jiang Y Zhi H Dong Y Li H Ma S Wang Y Dong Q Shen H Wang Y Artificial intelligence in healthcare: Past, present and future Stroke Vasc Neurol 2017 12 2 4 230 243 10.1136/svn-2017-000101 29507784 svn-2017-000101 29507784 PMC5829945 · DOI · Paper page
  8. [12]Alhashmi S Alshurideh M Kurdi B Salloum S A systematic review of the factors affecting the artificial intelligence implementation in the health care sector Proceedings of the 1st International Conference on Artificial Intelligence and Computer Vision 2020 1st International Conference on Artificial Intelligence and Computer Vision April 8-9, 2020 Cairo, Egypt Cham, Switzerland Springer 37 49 10.1007/978-3-030-44289-7_4 · DOI
  9. [13]Lin SY Mahoney MR Sinsky CA Ten ways artificial intelligence will transform primary care J Gen Intern Med 2019 08 34 8 1626 1630 10.1007/s11606-019-05035-1 31090027 10.1007/s11606-019-05035-1 31090027 PMC6667610 · DOI · Paper page
  10. [14]Recht MP Dewey M Dreyer K Langlotz C Niessen W Prainsack B Smith JJ Integrating artificial intelligence into the clinical practice of radiology: Challenges and recommendations Eur Radiol 2020 06 30 6 3576 3584 10.1007/s00330-020-06672-5 32064565 10.1007/s00330-020-06672-5 32064565 · DOI · Paper page
  11. [15]Asan O Bayrak AE Choudhury A Artificial intelligence and human trust in healthcare: Focus on clinicians J Med Internet Res 2020 06 19 22 6 e15154 10.2196/15154 32558657 v22i6e15154 32558657 PMC7334754 · DOI · Paper page
  12. [16]Gooding P Kariotis T Ethics and law in research on algorithmic and data-driven technology in mental health care: Scoping review JMIR Ment Health 2021 06 10 8 6 e24668 10.2196/24668 34110297 v8i6e24668 34110297 PMC8262551 · DOI · Paper page
  13. [17]Beil M Proft I van Heerden D Sviri S van Heerden PV Ethical considerations about artificial intelligence for prognostication in intensive care Intensive Care Med Exp 2019 12 10 7 1 70 10.1186/s40635-019-0286-6 31823128 10.1186/s40635-019-0286-6 31823128 PMC6904702 · DOI · Paper page
  14. [18]Choudhury A Asan O Role of artificial intelligence in patient safety outcomes: Systematic literature review JMIR Med Inform 2020 07 24 8 7 e18599 10.2196/18599 32706688 v8i7e18599 32706688 PMC7414411 · DOI · Paper page
  15. [19]Fernandes M Vieira SM Leite F Palos C Finkelstein S Sousa JM Clinical decision support systems for triage in the emergency department using intelligent systems: A review Artif Intell Med 2020 01 102 101762 10.1016/j.artmed.2019.101762 31980099 S0933-3657(19)30126-5 31980099 · DOI
  16. [20]Yin J Ngiam KY Teo HH Role of artificial intelligence applications in real-life clinical practice: Systematic review J Med Internet Res 2021 04 22 23 4 e25759 10.2196/25759 33885365 v23i4e25759 33885365 PMC8103304 · DOI · Paper page
  17. [21]Wolff J Pauling J Keck A Baumbach J The economic impact of artificial intelligence in health care: Systematic review J Med Internet Res 2020 02 20 22 2 e16866 10.2196/16866 32130134 v22i2e16866 32130134 PMC7059082 · DOI
  18. [23]Macrae C Learning from the failure of autonomous and intelligent systems: Accidents, safety, and sociotechnical sources of risk Risk Anal 2021 11 23 1 27 10.1111/risa.13850 34814229 34814229 · DOI · Paper page
  19. [24]Eldh AC Almost J DeCorby-Watson K Gifford W Harvey G Hasson H Kenny D Moodie S Wallin L Yost J Clinical interventions, implementation interventions, and the potential greyness in between: A discussion paper BMC Health Serv Res 2017 01 07 17 1 16 10.1186/s12913-016-1958-5 28061856 10.1186/s12913-016-1958-5 28061856 PMC5219812 · DOI
  20. [26]Miake-Lye I Mak S Lam CA Lambert-Kerzner AC Delevan D Olmos-Ochoa T Shekelle P Scaling beyond early adopters: A content analysis of literature and key informant perspectives J Gen Intern Med 2021 02 36 2 383 395 10.1007/s11606-020-06142-0 33111242 10.1007/s11606-020-06142-0 33111242 PMC7878615 · DOI
  21. [27]Kirchner JE Smith JL Powell BJ Waltz TJ Proctor EK Getting a clinical innovation into practice: An introduction to implementation strategies Psychiatry Res 2020 01 283 112467 10.1016/j.psychres.2019.06.042 31488332 S0165-1781(19)30741-3 31488332 PMC7239693 · DOI
  22. [28]von Thiele Schwarz U Nielsen K Edwards K Hasson H Ipsen C Savage C Simonsen Abildgaard J Richter A Lornudd C Mazzocato P Reed JE How to design, implement and evaluate organizational interventions for maximum impact: The Sigtuna Principles Eur J Work Organ Psychol 2020 08 26 30 3 415 427 10.1080/1359432x.2020.1803960 34518756 PMC8432268 · DOI
  23. [29]Reed JE Howe C Doyle C Bell D Simple rules for evidence translation in complex systems: A qualitative study BMC Med 2018 06 20 16 1 92 10.1186/s12916-018-1076-9 29921274 10.1186/s12916-018-1076-9 29921274 PMC6009041 · DOI
  24. [31]Lysaght T Lim HY Xafis V Ngiam KY AI-assisted decision-making in healthcare: The application of an ethics framework for big data in health and research Asian Bioeth Rev 2019 09 11 3 299 314 10.1007/s41649-019-00096-0 33717318 96 33717318 PMC7747260 · DOI · Paper page
  25. [33]Ashfaq A Lönn S Nilsson H Eriksson JA Kwatra J Yasin ZM Slutzman JE Wallenfeldt T Obermeyer Z Anderson PD Lingman M Data Resource Profile: Regional healthcare information platform in Halland, Sweden Int J Epidemiol 2020 06 01 49 3 738 739f 10.1093/ije/dyz262 31930310 5701527 31930310 · DOI
  26. [34]McLaughlin JA Jordan GB Logic models: A tool for telling your programs performance story Eval Program Plann 1999 3 22 1 65 72 10.1016/s0149-7189(98)00042-1 · DOI
  27. [35]McMeekin N Wu O Germeni E Briggs A How methodological frameworks are being developed: Evidence from a scoping review BMC Med Res Methodol 2020 06 30 20 1 173 10.1186/s12874-020-01061-4 32605535 10.1186/s12874-020-01061-4 32605535 PMC7325096 · DOI · Paper page
  28. [38]Reed J Howe C Doyle C Bell D Successful Healthcare Improvements From Translating Evidence in complex systems (SHIFT-Evidence): Simple rules to guide practice and research Int J Qual Health Care 2019 04 01 31 3 238 244 10.1093/intqhc/mzy160 30085160 5066475 30085160 PMC6464098 · DOI
  29. [39]Arksey H O'Malley L Scoping studies: Towards a methodological framework Int J Soc Res Methodol 2005 02 8 1 19 32 10.1080/1364557032000119616 · DOI · Paper page
  30. [40]Peters MDJ Godfrey CM Khalil H McInerney P Parker D Soares CB Guidance for conducting systematic scoping reviews Int J Evid Based Healthc 2015 09 13 3 141 146 10.1097/XEB.0000000000000050 26134548 26134548 · DOI · Paper page
  31. [41]Hsieh H Shannon SE Three approaches to qualitative content analysis Qual Health Res 2005 11 15 9 1277 1288 10.1177/1049732305276687 16204405 15/9/1277 16204405 · DOI · Paper page
  32. [42]Graneheim UH Lundman B Qualitative content analysis in nursing research: Concepts, procedures and measures to achieve trustworthiness Nurse Educ Today 2004 02 24 2 105 112 10.1016/j.nedt.2003.10.001 14769454 S0260691703001515 14769454 · DOI · Paper page
  33. [43]Wärnestål P Nygren JM Building an experience framework for a digital peer support service for children surviving from cancer Proceedings of the 12th International Conference on Interaction Design and Children 2013 12th International Conference on Interaction Design and Children June 24-27, 2013 New York, NY 269 272 10.1145/2485760.2485794 · DOI
  34. [44]Kolko J Exposing the Magic of Design: A Practitioner’s Guide to the Methods and Theory of Synthesis 2010 Oxford, UK Oxford University Press
  35. [45]Charmaz K Constructing Grounded Theory. 2nd edition 2014 London, UK SAGE Publications

Cited by

Sorted by citations

Authors
Svedberg, P., Reed, J., Nilsen, P., Barlow, J., Macrae, C., & Nygren, J.
Affiliations
  • Halmstad University (SE)
  • Imperial College London (GB)
  • University of Nottingham (GB)
Corresponding author
Nygren, Jens
ORCID
Venue
JMIR Research Protocols
Publisher
JMIR Publications Inc.
Published
2022-03-09
Volume
11
Issue
3
Pages
e34920
ISSN
1929-0748 (ISSN-L)
DOI
10.2196/34920
PMC
PMC8943554
OpenAlex
W4221100166
Cited by
39
References
9
Citations by year
2026: 7 · 2025: 9 · 2024: 10 · 2023: 10 · 2022: 3
FWCI
2.41
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • JMIR Research Protocols (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Repository@Nottingham (University of Nottingham) (publishedVersion, cc-by)
  • Repository@Nottingham (University of Nottingham) (submittedVersion, cc-by)
  • DiVA (Linkoping University) (submittedVersion, other-oa)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
  • Spiral (Imperial College London) (submittedVersion, cc-by)
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
  • Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
Keywords
artificial intelligence in healthcare, AI implementation, implementation theory, implementation framework, logic model, co-design, stakeholder collaboration, knowledge-practice gap, capacity building, clinical practice, implementation evaluation, healthcare workers, patients, industry stakeholders, regional capacity building, stakeholder mapping

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review