Artificial Intelligence Applications in Health Care Practice: Scoping Review
Open accessThe DOI matches its Crossref record; 101 of the 101 references it cites were found in a registry or an index.
Abstract
Background: Artificial intelligence (AI) is often heralded as a potential disruptor that will transform the practice of medicine. The amount of data collected and available in health care, coupled with advances in computational power, has contributed to advances in AI and an exponential growth of publications. However, the development of AI applications does not guarantee their adoption into routine practice. There is a risk that despite the resources invested, benefits for patients, staff, and society will not be realized if AI implementation is not better understood. Objective: The aim of this study was to explore how the implementation of AI in health care practice has been described and researched in the literature by answering 3 questions: What are the characteristics of research on implementation of AI in practice? What types and applications of AI systems are described? What characteristics of the implementation process for AI systems are discernible? Methods: A scoping review was conducted of MEDLINE (PubMed), Scopus, Web of Science, CINAHL, and PsycINFO databases to identify empirical studies of AI implementation in health care since 2011, in addition to snowball sampling of selected reference lists. Using Rayyan software, we screened titles and abstracts and selected full-text articles. Data from the included articles were charted and summarized. Results: Of the 9218 records retrieved, 45 (0.49%) articles were included. The articles cover diverse clinical settings and disciplines; most (32/45, 71%) were published recently, were from high-income countries (33/45, 73%), and were intended for care providers (25/45, 56%). AI systems are predominantly intended for clinical care, particularly clinical care pertaining to patient-provider encounters. More than half (24/45, 53%) possess no action autonomy but rather support human decision-making. The focus of most research was on establishing the effectiveness of interventions (16/45, 35%) or related to technical and computational aspects of AI systems (11/45, 24%). Focus on the specifics of implementation processes does not yet seem to be a priority in research, and the use of frameworks to guide implementation is rare. Conclusions: Our current empirical knowledge derives from implementations of AI systems with low action autonomy and approaches common to implementations of other types of information systems. To develop a specific and empirically based implementation framework, further research is needed on the more disruptive types of AI systems being implemented in routine care and on aspects unique to AI implementation in health care, such as building trust, addressing transparency issues, developing explainable and interpretable solutions, and addressing ethical concerns around privacy and data protection.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Despite the ongoing development and testing of AI applications in healthcare, the implementation of medical AI systems in clinical care remains in its early stages [2, 3].”
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“For basic or straightforward questions, AI systems can provide accurate and relevant answers in real-time [6].”
From Applicability of ChatGPT in Assisting to Solve Higher Order Problems in Pathology (2023) · cites this paper as [6]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Using thematic analysis in psychology
Braun, V., & Clarke, V. (2006) · Qualitative Research in Psychology · doi:10.1191/1478088706qp063oa · Cited by 194,414 · cited 1×
“A qualitative thematic analysis [44] was used to analyze data associated with research question 3 to summarize the motives for implementation and elements in the implementation process.”
PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation
Tricco, A. C., Lillie, E., Zarin, W., O'Brien, K. K., Colquhoun, H., Levac, D., Moher, D., Peters, M. D. J., Horsley, T., Weeks, L., et al. (2018) · Annals of Internal Medicine · doi:10.7326/m18-0850 · Cited by 43,565 · cited 1×
“We chose a scoping review methodology in line with the Arksey and O’Malley framework [39] and reported according to the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) checklist (Figure 1) [40].”
Scoping studies: towards a methodological framework
Arksey, H., & O'Malley, L. (2005) · International Journal of Social Research Methodology · doi:10.1080/1364557032000119616 · Cited by 36,742 · cited 3×
“Study design We chose a scoping review methodology in line with the Arksey and O’Malley framework (43) and”
InfluentialA Dynamic Theory of Organizational Knowledge Creation
Nonaka, I. (1994) · Organization Science · doi:10.1287/orsc.5.1.14 · Cited by 17,809
Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009) · Implementation Science · doi:10.1186/1748-5908-4-50 · Cited by 14,496
Evaluating the public health impact of health promotion interventions: the RE-AIM framework.
Glasgow, R. E., Vogt, T. M., & Boles, S. M. (1999) · American Journal of Public Health · doi:10.2105/ajph.89.9.1322 · Cited by 7,217 · cited 1×
“…the use of the following implementation frameworks was mentioned: the Reach, Effectiveness, Adoption, Implementation, and Maintenance framework [90]; the Nonadoption, Abandonment, Scale-up, Spread, and Sustainability framework [91]; and the Socialization, Externalization, Combination, and…”
Artificial intelligence in healthcare: past, present and future
Jiang, F., Jiang, Y., Zhi, H., Dong, Y., Li, H., Ma, S., Wang, Y., Dong, Q., Shen, H., & Wang, Y. (2017) · Stroke and Vascular Neurology · doi:10.1136/svn-2017-000101 · Cited by 4,837 · cited 1×
“Others see a role complementary to humans; for example, decision support or decision augmentation where humans (in the roles of clinicians or programmers) provide oversight and collaborate [4-7].”
Rajpurkar, P., Chen, E., Banerjee, O., & Topol, E. J. (2022) · Nature Medicine · doi:10.1038/s41591-021-01614-0 · Cited by 3,212 · cited 2×
“The latter approach has been demonstrated to yield superior performance when compared to experts alone (8).”
Greenhalgh, T., Wherton, J., Papoutsi, C., Lynch, J., Hughes, G., A'Court, C., Hinder, S., Fahy, N., Procter, R., & Shaw, S. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.8775 · Cited by 2,887 · cited 1×
“…was mentioned: the Reach, Effectiveness, Adoption, Implementation, and Maintenance framework [90]; the Nonadoption, Abandonment, Scale-up, Spread, and Sustainability framework [91]; and the Socialization, Externalization, Combination, and Internalization model of knowledge dimensions [92].”
The practical implementation of artificial intelligence technologies in medicine
He, J., Baxter, S. L., Xu, J., Xu, J., Zhou, X., & Zhang, K. (2019) · Nature Medicine · doi:10.1038/s41591-018-0307-0 · Cited by 2,246 · cited 1×
“…(eg, precision medicine in chemotherapy and for combination drug therapy), and regulatory or administrative applications (eg, coding of records or economic evaluations), as well as for population health management (eg, public health surveillance or predictive epidemiological modeling) [15-21].”
The role of artificial intelligence in healthcare: a structured literature review
Secinaro, S., Calandra, D., Secinaro, A., Muthurangu, V., & Biancone, P. (2021) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-021-01488-9 · Cited by 1,085 · cited 2×
“The amount of data collected and available in health care, coupled with advances in computational power, has contributed to advances in AI applications [9] and an exponential growth of publications on AI in health care, with >10,000 records on PubMed in 2021 alone.”
A governance model for the application of AI in health care
Reddy, S., Allan, S., Coghlan, S., & Cooper, P. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz192 · Cited by 979
Clinical information extraction applications: A literature review
Wang, Y., Wang, L., Rastegar-Mojarad, M., Moon, S., Shen, F., Afzal, N., Liu, S., Zeng, Y., Mehrabi, S., Sohn, S., et al. (2018) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2017.11.011 · Cited by 910 · cited 1×
“The predominance of knowledge-based or symbolic models, whose greater transparency and longer existence may ease acceptance among care providers, is in line with previous reviews [103].”
Artificial intelligence and the future of global health
Schwalbe, N., & Wahl, B. (2020) · The Lancet · doi:10.1016/s0140-6736(20)30226-9 · Cited by 876 · cited 1×
“…(eg, precision medicine in chemotherapy and for combination drug therapy), and regulatory or administrative applications (eg, coding of records or economic evaluations), as well as for population health management (eg, public health surveillance or predictive epidemiological modeling) [15-21].”
Wahl, B., Cossy-Gantner, A., Germann, S., & Schwalbe, N. R. (2018) · BMJ Global Health · doi:10.1136/bmjgh-2018-000798 · Cited by 796 · cited 1×
“The following data were extracted:”
Artificial Intelligence and Surgical Decision-making.
Loftus, T. J., Tighe, P. J., Filiberto, A. C., Efron, P. A., Brakenridge, S. C., Mohr, A. M., Rashidi, P., Upchurch, G. R., & Bihorac, A. (2020) · JAMA Surgery · doi:10.1001/jamasurg.2019.4917 · Cited by 650
Peiffer-Smadja, N., Rawson, T. M., Ahmad, R., Buchard, A., Georgiou, P., Lescure, F.-X., Birgand, G., & Holmes, A. H. (2020) · Clinical Microbiology and Infection · doi:10.1016/j.cmi.2019.09.009 · Cited by 561
Artificial intelligence in healthcare: An essential guide for health leaders
Chen, M., & Decary, M. (2020) · Healthcare Management Forum · doi:10.1177/0840470419873123 · Cited by 538 · cited 1×
“Other benefits include improved patient outcomes, error reduction, health system optimization, cost reductions, and increased value [6].”
de Hond, A. A. H., Leeuwenberg, A. M., Hooft, L., Kant, I. M. J., Nijman, S. W. J., van Os, H. J. A., Aardoom, J. J., Debray, T. P. A., Schuit, E., van Smeden, M., et al. (2022) · npj Digital Medicine · doi:10.1038/s41746-021-00549-7 · Cited by 499
Role of artificial intelligence in patient safety outcomes: systematic literature review
Choudhury, A., & Asan, O. (2020) · JMIR Medical Informatics · doi:10.2196/18599 · Cited by 440
Artificial intelligence for good health: a scoping review of the ethics literature.
Murphy, K., Di Ruggiero, E., Upshur, R., Willison, D. J., Malhotra, N., Cai, J. C., Malhotra, N., Lui, V., & Gibson, J. (2021) · BMC Medical Ethics · doi:10.1186/s12910-021-00577-8 · Cited by 435
Role of Artificial Intelligence Applications in Real-Life Clinical Practice: Systematic Review
Yin, J., Ngiam, K. Y., & Teo, H. H. (2021) · Journal of Medical Internet Research · doi:10.2196/25759 · Cited by 419
Artificial intelligence for diabetic retinopathy screening: a review
Grzybowski, A., Brona, P., Lim, G., Ruamviboonsuk, P., Tan, G. S. W., Abramoff, M., & Ting, D. S. W. (2020) · Eye · doi:10.1038/s41433-019-0566-0 · Cited by 376
SHIFTing artificial intelligence to be responsible in healthcare: a systematic review
Siala, H., & Wang, Y. (2022) · Social Science & Medicine · doi:10.1016/j.socscimed.2022.114782 · Cited by 358
Strohm, L., Hehakaya, C., Ranschaert, E. R., Boon, W. P. C., & Moors, E. H. M. (2020) · European Radiology · doi:10.1007/s00330-020-06946-y · Cited by 329 · cited 3×
“…of AI effectiveness with clinicians and describing overall benefits of the technology [46,49,59], appointing champions to promote AI among peers [46,53,75], and encouraging informal communication between clinicians and IT developers to cultivate relationships and build trust in the AI [56].”
Fernandes, M., Vieira, S. M., Leite, F., Palos, C., Finkelstein, S., & Sousa, J. M. C. (2020) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2019.101762 · Cited by 312 · cited 2×
“Previous reviews have tended to focus only on some aspects of the process of implementation of AI in health care; for example, regulation and legal issues [24,25], trust and ethics [24-29], clinical and patient outcomes [30-32], and economic impact [33].”
Sendak, M. P., Ratliff, W., Sarro, D., Alderton, E., Futoma, J., Gao, M., Nichols, M., Revoir, M., Yashar, F., Miller, C., et al. (2020) · JMIR Medical Informatics · doi:10.2196/15182 · Cited by 287 · cited 10×
“However, the study by Sendak et al [46] encouraged the separation of developers and clinicians and made conscious efforts to shift focus away from the technical aspects of AI.”
InfluentialChen, Z., Lin, L., Wu, C., Li, C., Xu, R., & Sun, Y. (2021) · Cancer Communications · doi:10.1002/cac2.12215 · Cited by 285
Artificial Intelligence in Cardiology: Present and Future
Lopez-Jimenez, F., Attia, Z., Arruda-Olson, A. M., Carter, R., Chareonthaitawee, P., Jouni, H., Kapa, S., Lerman, A., Luong, C., Medina-Inojosa, J. R., et al. (2020) · Mayo Clinic Proceedings · doi:10.1016/j.mayocp.2020.01.038 · Cited by 280 · cited 1×
“2021 Nov 1 [cited 2022 Jan 18];41(11):1100–15.”
Čartolovni, A., Tomičić, A., & Lazić Mosler, E. (2022) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2022.104738 · Cited by 267
Systematic Review of Economic Impact Studies of Artificial Intelligence in Health Care
Wolff, J., Pauling, J., Keck, A., & Baumbach, J. (2020) · Journal of Medical Internet Research · doi:10.2196/16866 · Cited by 261 · cited 1×
“Earlier reviews have tended to focus only on some aspects of the implementation process of AI in healthcare, for example, regulation and legal issues (24,25), trust and ethics (24–29), clinical and patient outcomes (30–32) and economic impact (33).”
Will Artificial Intelligence Replace Radiologists?
Langlotz, C. P. (2019) · Radiology: Artificial Intelligence · doi:10.1148/ryai.2019190058 · Cited by 244 · cited 1×
“Others see a role complementary to humans; for example, decision support or decision augmentation where humans (in the roles of clinicians or programmers) provide oversight and collaborate [4-7].”
Mirbabaie, M., Stieglitz, S., & Frick, N. R. J. (2021) · Health and Technology · doi:10.1007/s12553-021-00555-5 · Cited by 241 · cited 1×
“Others have focused their studies on specific AI applications for healthcare, such as predictive medicine, diagnostics and clinical decision making (30,34–36).”
BackgroundLorenzi, N. M., Riley, R. T., Blyth, A. J. C., Southon, G., & Dixon, B. J. (1997) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.1997.0040079 · Cited by 237 · cited 1×
“…and sociocultural factors), organization (eg, organizational structure, resources, and processes), group (eg, professional values and cultures), individual (eg, attitudes, motivation, user satisfaction, and trust), and technology (eg, usability, design, accuracy, and explainability) [22,23].”
Gama, F., Tyskbo, D., Nygren, J., Barlow, J., Reed, J., & Svedberg, P. (2022) · Journal of Medical Internet Research · doi:10.2196/32215 · Cited by 216 · cited 5×
“Implementation was defined as “An intentional effort designed to change or adapt or uptake interventions into routines,” based on a review of frameworks for the translation of AI into health care practice [37].”
InfluentialSelf-citationThe three ghosts of medical AI: Can the black-box present deliver?
Quinn, T. P., Jacobs, S., Senadeera, M., Le, V., & Coghlan, S. (2022) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2021.102158 · Cited by 215
Kukafka, R., Johnson, S. B., Linfante, A., & Allegrante, J. P. (2003) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2003.09.002 · Cited by 206
Responsible AI for Digital Health: a Synthesis and a Research Agenda
Trocin, C., Mikalef, P., Papamitsiou, Z., & Conboy, K. (2023) · Information Systems Frontiers · doi:10.1007/s10796-021-10146-4 · Cited by 205 · cited 1×
“Earlier reviews have tended to focus only on some aspects of the implementation process of AI in healthcare, for example, regulation and legal issues (24,25), trust and ethics (24–29), clinical and patient outcomes (30–32) and economic impact (33).”
BackgroundDamoah, I. S., Ayakwah, A., & Tingbani, I. (2021) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2021.129598 · Cited by 190 · cited 2×
“Empirical studies improve the ability to answer the research questions compared with conceptual commentaries or viewpoints Nonempirical designs, including editorials, commentaries, opinion articles, and reports As the aim was to explore implementation in practice, studies that stop short of that, for example, proof-of…”
Review on Diagnosis of COVID-19 from Chest CT Images Using Artificial Intelligence
Ozsahin, I., Sekeroglu, B., Musa, M. S., Mustapha, M. T., & Uzun Ozsahin, D. (2020) · Computational and Mathematical Methods in Medicine · doi:10.1155/2020/9756518 · Cited by 179
Artificial intelligence in cancer therapy
Ho, D. (2020) · Science · doi:10.1126/science.aaz3023 · Cited by 164
Artificial intelligence and neural networks in urology: current clinical applications
Checcucci, E., on behalf of the Uro-technology and SoMe Working Group of the Young Academic Urologists Working Party of the European Association of Urology, Autorino, R., Cacciamani, G. E., Amparore, D., De Cillis, S., Piana, A., Piazzolla, P., Vezzetti, E., Fiori, C., et al. (2020) · Minerva Urologica e Nefrologica · doi:10.23736/s0393-2249.19.03613-0 · Cited by 146
Abbasgholizadeh Rahimi, S., Légaré, F., Sharma, G., Archambault, P., Zomahoun, H. T. V., Chandavong, S., Rheault, N., T Wong, S., Langlois, L., Couturier, Y., et al. (2021) · Journal of Medical Internet Research · doi:10.2196/29839 · Cited by 146 · cited 1×
“2011 Dec [cited 2022 May 18];80(12):863–71.”
Romero-Brufau, S., Wyatt, K. D., Boyum, P., Mickelson, M., Moore, M., & Cognetta-Rieke, C. (2020) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2019.104072 · Cited by 143 · cited 2×
“Nondisruptive workflow design was emphasized, where efforts were made to design AI systems around existing roles and functions of the intended user to avoid radical modification of current practice to fit the AI system [46,49,51].”
Artificial intelligence and primary care research: a scoping review
Kueper, J. K., Terry, A. L., Zwarenstein, M., & Lizotte, D. J. (2020) · The Annals of Family Medicine · doi:10.1370/afm.2518 · Cited by 143 · cited 2×
“…field and is similar to other studies, which highlights that many of the publications on AI in health care focus on the methods and technical aspects of applying the AI model to clinical scenarios but provide very little information on the actual process of its implementation in practice [51,99].”
Wen, A., Fu, S., Moon, S., El Wazir, M., Rosenbaum, A., Kaggal, V. C., Liu, S., Sohn, S., Liu, H., & Fan, J. (2019) · npj Digital Medicine · doi:10.1038/s41746-019-0208-8 · Cited by 142
Alhashmi, S. F. S., Alshurideh, M. T., Kurdi, B. H. A., & Salloum, S. A. (2020) · Advances in intelligent systems and computing · doi:10.1007/978-3-030-44289-7_4 · Cited by 142 · cited 1×
“A scoping review would thus enable a mapping of the “extent, range and nature of research activity” in this emerging area of research [39].”
Artificial Intelligence in Clinical Decision Support: a Focused Literature Survey
Montani, S., & Striani, M. (2019) · Yearbook of Medical Informatics · doi:10.1055/s-0039-1677911 · Cited by 129 · cited 1×
“However, the majority of recently published AI models use data-driven or hybrid technologies, and knowledge-based models comprised only a minority of the applications [104].”
Ethical considerations about artificial intelligence for prognostication in intensive care
Beil, M., Proft, I., van Heerden, D., Sviri, S., & van Heerden, P. V. (2019) · Intensive Care Medicine Experimental · doi:10.1186/s40635-019-0286-6 · Cited by 124
Zidaru, T., Morrow, E. M., & Stockley, R. (2021) · Health Expectations · doi:10.1111/hex.13299 · Cited by 119 · cited 1×
“A scoping review would thus enable a mapping of the “extent, range and nature of research activity” in this emerging area of research [39].”
Higgins, D., & Madai, V. I. (2020) · Advanced Intelligent Systems · doi:10.1002/aisy.202000052 · Cited by 118 · cited 1×
“Research on AI implementation in health care is predominantly conceptual in nature, dominated by commentaries, perspectives, opinion articles, and conceptual frameworks that raise important questions and issues but without much-needed empirical evidence [93-96].”
Five Constants of Information Technology Adoption in Healthcare
Bernstein, M. L., McCreless, T., & Côté, M. J. (2007) · Hospital Topics · doi:10.3200/htps.85.1.17-26 · Cited by 107 · cited 1×
“For example, involvement of end users in the design and implementation of IT services and applications form the basis of user-centred design, which is seen as an important driver of uptake of digital technologies (111).”
Approaching autonomy in medical artificial intelligence
Bitterman, D. S., Aerts, H. J. W. L., & Mak, R. H. (2020) · The Lancet Digital Health · doi:10.1016/s2589-7500(20)30187-4 · Cited by 107 · cited 8×
“2020 Mar 1 [cited 2022 Jan 22];27(3):491.”
InfluentialArnold, M. H. (2021) · Journal of Bioethical Inquiry · doi:10.1007/s11673-020-10080-1 · Cited by 105
Lai, L., Wittbold, K. A., Dadabhoy, F. Z., Sato, R., Landman, A. B., Schwamm, L. H., He, S., Patel, R., Wei, N., Zuccotti, G., et al. (2020) · Healthcare · doi:10.1016/j.hjdsi.2020.100493 · Cited by 100
Gooding, P., & Kariotis, T. (2021) · JMIR Mental Health · doi:10.2196/24668 · Cited by 100
Wong, J., Huang, V., Wells, D., Giambattista, J., Giambattista, J., Kolbeck, C., Otto, K. F., Saibishkumar, E. P., & Alexander, A. (2021) · Radiation Oncology · doi:10.1186/s13014-021-01831-4 · Cited by 98
O’Neill, T. J., Xi, Y., Stehel, E., Browning, T., Ng, Y. S., Baker, C., & Peshock, R. M. (2021) · Radiology: Artificial Intelligence · doi:10.1148/ryai.2020200024 · Cited by 95
Shuaib, A., Arian, H., & Shuaib, A. (2020) · International Journal of General Medicine · doi:10.2147/ijgm.s268093 · Cited by 94 · cited 2×
“2020 May 1 [cited 2022 Jan 18];95(5):1015–39.”
Litvin, C. B., Ornstein, S. M., Wessell, A. M., Nemeth, L. S., & Nietert, P. J. (2012) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2012.03.002 · Cited by 87 · cited 3×
“More controlling approaches included periodic monitoring and audits [56,84] or removing alternative ways of performing the task altogether to necessitate AI use [84].”
Romero-Brufau, S., Wyatt, K. D., Boyum, P., Mickelson, M., Moore, M., & Cognetta-Rieke, C. (2020) · Applied Clinical Informatics · doi:10.1055/s-0040-1715827 · Cited by 81 · cited 2×
“…theme described AI systems used to improve quality of services [46,71,75,87,88], reduce diagnostic errors [66], reduce hospital length of stay [73], or reduce unplanned readmissions [50,54], whereas studies in the latter theme described AI systems used to achieve better patient survival [59,70].”
Sukums, F. K., Mensah, N. K., Mpembeni, R. N. M., Massawe, S. N., Duysburgh, E., Williams, A., Kaltschmidt, J., Loukanova, S., Haefeli, W. E., & Blank, A. (2015) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2015.05.002 · Cited by 79 · cited 4×
“Most studies (n=27) were authored by a multi-disciplinary team (50,51,54–59,62,63,65,66,68,71,73,74,76,78,79,82– 84,86,90–93) with clinical and IT/informatics backgrounds being the most common combination (n=9) (51,54,59,65,74,78,83,91,93).”
InfluentialIntegrated Anesthesiologist and Room Scheduling for Surgeries: Methodology and Application
Rath, S., Rajaram, K., & Mahajan, A. (2017) · Operations Research · doi:10.1287/opre.2017.1634 · Cited by 77
Artificial Intelligence in Drug Treatment
Romm, E. L., & Tsigelny, I. F. (2019) · The Annual Review of Pharmacology and Toxicology · doi:10.1146/annurev-pharmtox-010919-023746 · Cited by 71
A Decision Support System for Operating Room scheduling
Dios, M., Molina-Pariente, J. M., Fernández-Viagas, V., Andrade-Pineda, J. L., & Framiñán, J. M. (2015) · Computers & Industrial Engineering · doi:10.1016/j.cie.2015.08.001 · Cited by 69
Artificial Intelligence in Global Health —A Framework and Strategy for Adoption and Sustainability
Hadley, T. D., Pettit, R. W., Malik, T., Khoei, A. A., & Salihu, H. M. (2020) · International Journal of Maternal and Child Health and AIDS (IJMA) · doi:10.21106/ijma.296 · Cited by 66 · cited 1×
“Research on AI implementation in health care is predominantly conceptual in nature, dominated by commentaries, perspectives, opinion articles, and conceptual frameworks that raise important questions and issues but without much-needed empirical evidence [93-96].”
Addressing User Resistance Would Have Prevented a Healthcare AI Project Failure
University of Bamberg (Germany), Reis, L., Maier, C., University of Bamberg (Germany), Mattke, J., University of Bamberg (Germany), Creutzenberg, M., University Hospital Regensburg (Germany), Weitzel, T., & University of Bamberg (Germany) (2020) · MIS Quarterly Executive · doi:10.17705/2msqe.00038 · Cited by 65 · cited 6×
“Improve provider satisfaction focused on workload reduction for health care professionals [55,69].”
InfluentialCécile, P., Aude, M., Jean-Louis, D., & Catherine, R. (2020) · Studies in Health Technology and Informatics · doi:10.3233/shti200312 · Cited by 62
Davis, M. A., Rao, B., Cedeno, P. A., Saha, A., & Zohrabian, V. M. (2022) · Current Problems in Diagnostic Radiology · doi:10.1067/j.cpradiol.2020.10.007 · Cited by 62 · cited 1×
“…theme described AI systems used to improve quality of services [46,71,75,87,88], reduce diagnostic errors [66], reduce hospital length of stay [73], or reduce unplanned readmissions [50,54], whereas studies in the latter theme described AI systems used to achieve better patient survival…”
Rahman, M. M., Khatun, F., Uzzaman, A., Sami, S. I., Bhuiyan, M. A.-A., & Kiong, T. S. (2021) · International Journal of Health Services · doi:10.1177/00207314211017469 · Cited by 62 · cited 1×
“The following data were extracted:”
Thurzo, A., Kurilová, V., & Varga, I. (2021) · Healthcare · doi:10.3390/healthcare9121695 · Cited by 56 · cited 2×
“Empirical studies improve the ability to answer the research questions compared with conceptual commentaries or viewpoints Nonempirical designs, including editorials, commentaries, opinion articles, and reports As the aim was to explore implementation in practice, studies that stop short of that, for example, proof-of…”
Implementation of an Artificial Intelligence Algorithm for sepsis detection
Gonçalves, L. S., Amaro, M. L. D. M., Romero, A. D. L. M., Schamne, F. K., Fressatto, J. L., & Bezerra, C. W. (2020) · Revista Brasileira de Enfermagem · doi:10.1590/0034-7167-2018-0421 · Cited by 53 · cited 3×
“…to building trust and promoting use by sharing evidence of AI effectiveness with clinicians and describing overall benefits of the technology [46,49,59], appointing champions to promote AI among peers [46,53,75], and encouraging informal communication between clinicians and IT developers to…”
Insights into the growing popularity of artificial intelligence in ophthalmology
Rajalakshmi, R., Dutt, S., Sivaraman, A., & Savoy, F. M. (2020) · Indian Journal of Ophthalmology · doi:10.4103/ijo.ijo_1754_19 · Cited by 52
Wijnhoven, F. (2022) · Information Systems Frontiers · doi:10.1007/s10796-021-10206-9 · Cited by 46
Pediatric decision support using adapted Arden Syntax
Anand, V., Carroll, A. E., Biondich, P., Dugan, T. M., & Downs, S. M. (2015) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2015.09.006 · Cited by 45 · cited 1×
“Practical consideration, given the investigators’language proficiency English language”
Kashyap, S., Morse, K. E., Patel, B., & Shah, N. H. (2021) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocab154 · Cited by 43
Implementing an Individual-Centric Discharge Process across Singapore Public Hospitals
Ng, R., & Tan, K. B. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18168700 · Cited by 41 · cited 2×
“%) implementation efforts involved cocreation with multidisciplinary stakeholders, starting from an ideation phase that included problem identification, requirement collection, and design or redesign of clinical workflows to facilitate AI-system integration [45,46,49,52,55,59,68,78].”
Lacey, G., Zhou, J., Li, X., Craven, C., & Gush, C. (2019) · American Journal of Infection Control · doi:10.1016/j.ajic.2019.06.015 · Cited by 40 · cited 1×
“Gamification was used to promote a feeling of reward and competition [61,65].”
Chonde, D. B., Pourvaziri, A., Williams, J., McGowan, J., Moskos, M., Alvarez, C., Narayan, A. K., Daye, D., Flores, E. J., & Succi, M. D. (2021) · Journal of the American College of Radiology · doi:10.1016/j.jacr.2021.01.013 · Cited by 40 · cited 5×
“%) implementation efforts involved cocreation with multidisciplinary stakeholders, starting from an ideation phase that included problem identification, requirement collection, and design or redesign of clinical workflows to facilitate AI-system integration [45,46,49,52,55,59,68,78].”
InfluentialSvedberg, P., Reed, J., Nilsen, P., Barlow, J., Macrae, C., & Nygren, J. (2022) · JMIR Research Protocols · doi:10.2196/34920 · Cited by 39 · cited 1×
“…AI algorithms and models (the actual code), AI applications (the innovation package ), and AI systems (the application in its context) and used standardized Medical Subject Headings terms and subject headings describing AI and its subcategories provided by the databases used for our searches [41].”
Self-citationBarriers and facilitators to the use of computer-based intensive insulin therapy
Campion, T. R., Waitman, L. R., Lorenzi, N. M., May, A. K., & Gadd, C. S. (2011) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2011.10.003 · Cited by 38 · cited 5×
“Nearly three-quarters of all AI systems were intended for clinical care (n=33) (50,53,55,57–63,65,67–77,82–84,88–93) and the majority of these concerned providing support to inform the patient-provider encounter (n = 18) (50,53,55,59,60,65,67,69,71,72,78,82,83,88,89,91,93), followed by diagnosis and prediction-based d…”
InfluentialLeveraging conversational technology to answer common COVID-19 questions
McKillop, M. M., South, B. R., Preininger, A. M., Mason, M., & Jackson, G. P. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaa316 · Cited by 38 · cited 3×
“It was surprising to find a scarcity of implementations of AI applications to handle infectious diseases (except for the study by McKillop et al [48]), given the overwhelming attention given to, and funding provided for, the management of the COVID-19 pandemic in 2020-2022.”
Sun, T. Q. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph182312682 · Cited by 37 · cited 5×
“%) proposed new frameworks, principles, or recommendations based on their presented findings and implementation experiences [49,55,56,80].”
InfluentialSchuh, C., de Bruin, J. S., & Seeling, W. (2018) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2015.11.002 · Cited by 36 · cited 1×
“%) implementation efforts involved cocreation with multidisciplinary stakeholders, starting from an ideation phase that included problem identification, requirement collection, and design or redesign of clinical workflows to facilitate AI-system integration [45,46,49,52,55,59,68,78].”
Barriers to Implementing an Artificial Intelligence Model for Unplanned Readmissions
Baxter, S. L., Bass, J. S., & Sitapati, A. M. (2020) · ACI Open · doi:10.1055/s-0040-1716748 · Cited by 30
Principles for Real-World Implementation of Bedside Predictive Analytics Monitoring
Moorman, L. P. (2021) · Applied Clinical Informatics · doi:10.1055/s-0041-1735183 · Cited by 26 · cited 7×
“%) implementation efforts involved cocreation with multidisciplinary stakeholders, starting from an ideation phase that included problem identification, requirement collection, and design or redesign of clinical workflows to facilitate AI-system integration [45,46,49,52,55,59,68,78].”
InfluentialMorales, H. M. P., Guedes, M. H., Silva, J. S., & Massuda, A. (2021) · Frontiers in Digital Health · doi:10.3389/fdgth.2021.648585 · Cited by 24
Saverino, A., Baiardi, P., Galatà, G., Pedemonte, G., Vassallo, C., & Pistarini, C. (2021) · Frontiers in Neurology · doi:10.3389/fneur.2021.643251 · Cited by 22 · cited 1×
“…necessitated by the need for access to the most up-to-date information [48], the sudden surge in demand for health care services [52], prioritization of limited resources [72], and reorganization of service delivery in response to local guidelines for prevention of infection transmission [62].”
Cruz Noa, P., Lea, C., García, M. J., Bernardino, P., & Ignacio, A. (2019) · Studies in Health Technology and Informatics · doi:10.3233/shti190285 · Cited by 19
Tan, J. R., Cheong, E. H. T., Chan, L. P., & Tham, W. P. (2020) · Current Problems in Diagnostic Radiology · doi:10.1067/j.cpradiol.2020.07.006 · Cited by 17 · cited 4×
“Among studies with authors from only one domain, the most common background was clinical (n = 8) (67,69,70,72,75,77,80,88).”
InfluentialOptimization of logistics services in hospitals
Rais, A., Alvelos, F., Figueiredo, J. P., & Nobre, A. (2017) · International Transactions in Operational Research · doi:10.1111/itor.12370 · Cited by 16
Snowdon, J. L., Robinson, B. J., Staats, C., Wolsey, K., Sands-Lincoln, M., Strasheim, T., Brotman, D., Keating, K., Schnitter, E., Jackson, G. P., et al. (2020) · Applied Clinical Informatics · doi:10.1055/s-0040-1715894 · Cited by 15 · cited 1×
“Another theme, Improve efficiency , focused on health care–cost reduction, increased service production, and optimization of public services [45,72,74,76,77].”
Development and implementation of venous thromboembolism stewardship across a hospital network
Chong, J., Curtain, C. M., Gad, F., Passam, F. H., Soo, G., Levy, R., Dunkley, S. M., Doo, N. W., Cunich, M. M., Burke, R. A., et al. (2021) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2021.104575 · Cited by 15
Kalil, A. J., Dias, V. M. D. C. H., Rocha, C. D. C., Morales, H. M. P., Fressatto, J. L., & de Faria, R. A. (2018) · Research on Biomedical Engineering · doi:10.1590/2446-4740.180021 · Cited by 14 · cited 1×
“Studies in the former theme described AI systems used to improve quality of services [46,71,75,87,88], reduce diagnostic errors [66], reduce hospital length of stay [73], or reduce unplanned readmissions [50,54], whereas studies in the latter theme described AI systems used to achieve better…”
PCaGuard: A Software Platform to Support Optimal Management of Prostate Cancer
Tamposis, I., Tsougos, I., Karatzas, A. D., Vassiou, K. G., Vlychou, M., & Tzortzis, V. (2022) · Applied Clinical Informatics · doi:10.1055/s-0041-1741481 · Cited by 14
Bermúdez, I. G., Manso, M. G., Sánchez, E. S., de Hita, A. R., Rubio, M. R., & Fernández, C. S. (2021) · Revista Clínica Española (English Edition) · doi:10.1016/j.rceng.2021.01.007 · Cited by 11
Schlicher, J., Metsker, M. T., Shah, H., & Demirkan, H. (2021) · PubMed Central · Cited by 11
Mohamed, M., Ferguson, D. J., Venugopal, A., Alam, M. K., Makki, L., & Vaid, N. R. (2021) · Seminars in Orthodontics · doi:10.1053/j.sodo.2021.05.011 · Cited by 10 · cited 1×
“Studies in the former theme described AI systems used to improve quality of services [46,71,75,87,88], reduce diagnostic errors [66], reduce hospital length of stay [73], or reduce unplanned readmissions [50,54], whereas studies in the latter theme described AI systems used to achieve better…”
Herman, B. Z., Sirichokchatchawan, W., Nantasenamat, C., & Pongpanich, S. (2021) · Journal of Health Research · doi:10.1108/jhr-11-2020-0535 · Cited by 9 · cited 2×
“…commentaries or viewpoints Nonempirical designs, including editorials, commentaries, opinion articles, and reports As the aim was to explore implementation in practice, studies that stop short of that, for example, proof-of-concept, validity, or feasibility studies, should be excluded…”
Clinical productivity system – a decision support model
Bennett, C. C. (2011) · International Journal of Productivity and Performance Management · doi:10.1108/17410401111112014 · Cited by 7
Development of a Clinical Decision Support System for the Patients of a Laboratory Service
Semenov, I., & Kopanitsa, G. (2016) · Studies in health technology and informatics · doi:10.3233/978-1-61499-678-1-90 · Cited by 3 · cited 4×
“While more than half of the AI applications had no action autonomy (n=24) (50,52– 55,57,58,61,67,70,71,74,77–79,83,85–89,91–93), a few reported applications had low- (n=2) (59,76), medium- (n=4) (62,73,75,90) or high action autonomy (n=6) (56,59,64,65,72,80) (Table 3 and Figure 2).”
Influential
Research works printed but not linked: 3
- Schlicher, J, 2021, Perspect Health Inf Manag, 18, 1g
- Semenov, I, 2016, Stud Health Technol Inform, 228, 90
- SariaSFive keys for using artificial intelligence in care delivery, and five common mistakes made in evaluating AI tools. Not All AI Is Created Equal: Strategies for Safe and Effective AdoptionNEJM Catalyst20223232022-05-30https://catalyst.nejm.org/doi/full/10.1056/CAT.22.0075 · doi:10.1056/cat.22.0075
Cited by
Mersha, M., Lam, K., Wood, J., AlShami, A. K., & Kalita, J. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128111 · Cited by 237
“Regulatory transparency and audits [161], medical device regulations [162], informed consent [163], liability and malpractice concerns [164], and intellectual property rights [21] are among the most critical legal considerations in the application of XAI in healthcare.”
BackgroundKar, A. K., Varsha, P. S., & Rajan, S. (2023) · Global Journal of Flexible Systems Management · doi:10.1007/s40171-023-00356-x · Cited by 192
“…and progression in the healthcare system to provide treatment for affordable services, and to learn about public health policy and develop a smart healthcare system to take clinical decisions for doctors (Alami et al., 2020; Baker et al., 2022; Sharma et al., 2022; Sovrano & Vitali, 2023).”
BackgroundApplicability of ChatGPT in Assisting to Solve Higher Order Problems in Pathology
Sinha, R. K., Deb Roy, A., Kumar, N., & Mondal, H. (2023) · Cureus · doi:10.7759/cureus.35237 · Cited by 187
“For basic or straightforward questions, AI systems can provide accurate and relevant answers in real-time [6].”
BackgroundNair, M., Svedberg, P., Larsson, I., & Nygren, J. M. (2024) · PLOS ONE · doi:10.1371/journal.pone.0305949 · Cited by 179
Self-citationEslam, M., Fan, J.-G., Yu, M.-L., Wong, V. W.-S., Cua, I. H., Liu, C.-J., Tanwandee, T., Gani, R., Seto, W.-K., Alam, S., et al. (2025) · Hepatology International · doi:10.1007/s12072-024-10774-3 · Cited by 170
“This ranges from predictive analytics through data mining of various large datasets, to management systems to robotic surgery and personalized medicine [412].”
Metadata
- Authors
- Sharma, M., Savage, C., Nair, M., Larsson, I., Svedberg, P., & Nygren, J. M.
- Affiliations
- Karolinska Institutet (SE)
- Halmstad University (SE)
- Corresponding author
- Nygren, Jens M
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2022-10-05
- Volume
- 24
- Issue
- 10
- Pages
- e40238
- ISSN
- 1438-8871 (ISSN-L)
- PMC
- PMC9582911
- OpenAlex
- W4293689835
- Cited by
- 198
- References
- 101
- Citations by year
- 2026: 46 · 2025: 62 · 2024: 58 · 2023: 30 · 2022: 2
- FWCI
- 2.61
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Journal of Medical Internet Research (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Keywords
- artificial intelligence in healthcare, AI implementation, implementation research, clinical decision support, implementation framework, trust in AI, explainable AI, transparency, implementation facilitators, implementation outcomes, human-AI collaboration
- MeSH terms
- Artificial Intelligence, Delivery of Health Care, Humans, Income
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
