Artificial Intelligence and the Implementation Challenge
Open accessPublished version of a preprint
The DOI matches its Crossref record; 59 of the 59 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Applications of artificial intelligence (AI) in health care have garnered much attention in recent years, but the implementation issues posed by AI have not been substantially addressed. OBJECTIVE: In this paper, we have focused on machine learning (ML) as a form of AI and have provided a framework for thinking about use cases of ML in health care. We have structured our discussion of challenges in the implementation of ML in comparison with other technologies using the framework of Nonadoption, Abandonment, and Challenges to the Scale-Up, Spread, and Sustainability of Health and Care Technologies (NASSS). METHODS: After providing an overview of AI technology, we describe use cases of ML as falling into the categories of decision support and automation. We suggest these use cases apply to clinical, operational, and epidemiological tasks and that the primary function of ML in health care in the near term will be decision support. We then outline unique implementation issues posed by ML initiatives in the categories addressed by the NASSS framework, specifically including meaningful decision support, explainability, privacy, consent, algorithmic bias, security, scalability, the role of corporations, and the changing nature of health care work. RESULTS: Ultimately, we suggest that the future of ML in health care remains positive but uncertain, as support from patients, the public, and a wide range of health care stakeholders is necessary to enable its meaningful implementation. CONCLUSIONS: If the implementation science community is to facilitate the adoption of ML in ways that stand to generate widespread benefits, the issues raised in this paper will require substantial attention in the coming years.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, health care has seen slow success in the implementation of AI use cases.”
From Barriers to and facilitators of artificial intelligence adoption in health care: scoping review (2024) · cites this paper as [20]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Clinicians expressed the need to understand both the scientific and clinical bases of the recommendations provided by the AI to confidently validate and apply the decision [19,20,36-38].”
From Barriers to and facilitators of artificial intelligence adoption in health care: scoping review (2024) · cites this paper as [20]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“However, successful utility requires successful adoption, and a number of studies have reported on the challenges encountered with implementing AI in health care [19-21].”
From Barriers to and facilitators of artificial intelligence adoption in health care: scoping review (2024) · cites this paper as [20]
In the citing paper
“Shaw et al. (2019) contend that decision support in health care can be useful only if its outputs are able to integrate with the human decision-making processes at the heart of health service delivery.”
From SHIFTing artificial intelligence to be responsible in healthcare: a systematic review (2022) · cites this paper as Shaw et al., 2019
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009) · Implementation Science · doi:10.1186/1748-5908-4-50 · Cited by 14,496 · cited 2×
“The value proposition of digital technology might be different for different stakeholder groups, and implementation frameworks direct attention to the implications of newly introduced technologies for patients, health care providers, managers, health policymakers, and others [8,25,26].”
BackgroundMethodDermatologist-level classification of skin cancer with deep neural networks
Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017) · Nature · doi:10.1038/nature21056 · Cited by 14,362 · cited 1×
“Often, ML involves supervised methods that categorize data points, for example, as images of skin cancer or otherwise given datasets in which all data points (or at least a substantial subset) are associated with a label, ordinal, or category that is meant to be predicted or inferred [15].”
BackgroundNorth, D. C. (1991) · Journal of Economic Perspectives · doi:10.1257/jep.5.1.97 · Cited by 5,232
Artificial intelligence in healthcare: past, present and future
Jiang, F., Jiang, Y., Zhi, H., Dong, Y., Li, H., Ma, S., Wang, Y., Dong, Q., Shen, H., & Wang, Y. (2017) · Stroke and Vascular Neurology · doi:10.1136/svn-2017-000101 · Cited by 4,837 · cited 1×
“We agree with a growing chorus of health care providers and researchers who suggest that ML will primarily serve to augment as opposed to replace the work of humans in the provision of health care in the near term [31], despite applications of automation in health care.”
BackgroundThe ethics of algorithms: Mapping the debate
Mittelstadt, B. D., Allo, P., Taddeo, M., Wachter, S., & Floridi, L. (2016) · Big Data & Society · doi:10.1177/2053951716679679 · Cited by 4,567 · cited 1×
“In cases where training data are partial or incomplete or only reflect a subset of a given population, the resulting model will only be relevant to the population of people represented in the dataset [47].”
BackgroundMaking sense of implementation theories, models and frameworks
Nilsen, P. (2015) · Implementation Science · doi:10.1186/s13012-015-0242-0 · Cited by 4,447 · cited 2×
“We have described use cases (and attendant value propositions) of ML in health care as more likely relating to decision support and less likely to automation, which begins to illustrate the implementation object of focus in ML initiatives [6,7].”
BackgroundMethodEvidence-Based Medicine: A New Approach to Teaching the Practice of Medicine
Guyatt, G. (1992) · JAMA · doi:10.1001/jama.1992.03490170092032 · Cited by 4,202
Machine Learning Applications in Cancer Prognosis and Prediction
Kourou, K., Exarchos, T. P., Exarchos, K. P., Karamouzis, M. V., & Fotiadis, D. I. (2015) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2014.11.005 · Cited by 3,338 · cited 1×
“Given these labels and the statistical models they help to train, ML can be very effective at determining the category in which any newly available individual data point belongs, thereby being useful in the effort to, for example, identify malignant cancers based on particular images [16].”
BackgroundBig other: Surveillance Capitalism and the Prospects of an Information Civilization
Zuboff, S. (2015) · Journal of Information Technology · doi:10.1057/jit.2015.5 · Cited by 3,205 · cited 1×
“As the innovations enabled by ML have taken on a more powerful role in driving global economies, corporations have strategically sought to acquire larger amounts of more diverse data to boost their capacity to develop ML algorithms [61].”
BackgroundInstitutions and Organizations: Ideas, Interests, and Identities
Scott, W. R. (2007) · Cited by 2,943
Research works printed but not linked: 8
- Ajay, A, 2019, The Economics of Artificial Intelligence: An Agenda
- 10.1016/j.socscimed.2012.11.021published · doi:10.1016/j.socscimed.2012.11.021published
- Agrawal, A, 2018, Prediction Machines: The Simple Economics of Artificial Intelligence
- RajpurkarPPolamreddiVBalakrishnanAResearchGate2017Cornell University LibraryMalaria likelihood prediction by effectively surveying households using deep reinforcement learning https://www.researchgate.net/publication/321324506_Malaria_Likelihood_Prediction_By_Effectively_Surveying_Households_Using_Deep_Reinforcement_Learning
- CaseyDBeyanOMurphyKFelzmannHRobot-Assisted Care for Elderly With Dementia: Is There a Potential for Genuine End-User Empowerment?Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction Extended Abstracts2015HRI'15 Extended AbstractsMarch 2-5, 2015Portland, OR2478
- RossiRAAhmedNKThe Network Data Repository with Interactive Graph Analytics and VisualizationProceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence2015AAAI'15January 25-30, 2015Austin Texas42923
- CulnaneCRubinsteinBTeagueVarXiv2017Health data in an open world https://arxiv.org/ftp/arxiv/papers/1712/1712.05627.pdf
- TsymbalAResearchGate2004Computer Science Department, Trinity College DublinThe problem of concept drift: definitions and related work https://www.researchgate.net/publication/228723141_The_Problem_of_Concept_Drift_Definitions_and_Related_Work
Cited by
Algorithmic bias: review, synthesis, and future research directions
Kordzadeh, N., & Ghasemaghaei, M. (2022) · European Journal of Information Systems · doi:10.1080/0960085x.2021.1927212 · Cited by 770
“…explains the sources, consequences, and mitigation strategies in a range of domains from employment and people analytics (Gal et al., 2017) and healthcare (Shaw et al., 2019) to education (Williams et al., 2018), credit markets (Fuster et al., 2018), and criminal justice (Mulligan et al., 2019).”
Barriers to and facilitators of artificial intelligence adoption in health care: scoping review
Hassan, M., Kushniruk, A., & Borycki, E. (2024) · JMIR Human Factors · doi:10.2196/48633 · Cited by 435
“Clinicians expressed the need to understand both the scientific and clinical bases of the recommendations provided by the AI to confidently validate and apply the decision [19,20,36-38].”
BackgroundInfluentialPetersson, L., Larsson, I., Nygren, J. M., Nilsen, P., Neher, M., Reed, J. E., Tyskbo, D., & Svedberg, P. (2022) · BMC Health Services Research · doi:10.1186/s12913-022-08215-8 · Cited by 434
“AI is thus not one single type of technology but rather many different types within various application areas, e.g., diagnosis and treatment, patient engagement and adherence, and administrative activities [5, 6].”
BackgroundRole of Artificial Intelligence Applications in Real-Life Clinical Practice: Systematic Review
Yin, J., Ngiam, K. Y., & Teo, H. H. (2021) · Journal of Medical Internet Research · doi:10.2196/25759 · Cited by 419
“Examples of key challenges that need to be addressed include data sharing and privacy issues, lack of algorithm transparency, the changing nature of health care work, financial concerns, and the demanding regulatory environment [1,3,9-13].”
BackgroundThe potential of artificial intelligence to improve patient safety: a scoping review
Bates, D. W., Levine, D., Syrowatka, A., Kuznetsova, M., Craig, K. J. T., Rui, A., Jackson, G. P., & Rhee, K. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00423-6 · Cited by 323
SHIFTing artificial intelligence to be responsible in healthcare: a systematic review
Siala, H., & Wang, Y. (2022) · Social Science & Medicine · doi:10.1016/j.socscimed.2022.114782 · Cited by 358
Zhang, B., Shi, H., & Wang, H. (2023) · Journal of Multidisciplinary Healthcare · doi:10.2147/jmdh.s410301 · Cited by 347
“Algorithm integration with EHR systems can help healthcare facilities by providing an alternative for improved resource allocation based on the information of the individual subgroups with a greater risk of cancer growth or cancer-related consequences.(100) The advent of ChatGPT by OpenAI has been recognized for its e…”
BackgroundUsing artificial intelligence to improve public health: a narrative review
Olawade, D. B., Wada, O. J., David-Olawade, A. C., Kunonga, E., Abaire, O., & Ling, J. (2023) · Frontiers in Public Health · doi:10.3389/fpubh.2023.1196397 · Cited by 274
“AI has become a potent tool for public health surveillance (67, 68) due to the rise in the amount of health-related data being produced, including that from electronic health records (EHRs), social media, and sensor data (69, 70).”
BackgroundSendak, M. P., Ratliff, W., Sarro, D., Alderton, E., Futoma, J., Gao, M., Nichols, M., Revoir, M., Yashar, F., Miller, C., et al. (2020) · JMIR Medical Informatics · doi:10.2196/15182 · Cited by 287
“Although Sepsis Watch is an instance of machine learning clinical decision support (CDS), deep learning systems do pose implementation challenges beyond traditional CDS, as detailed elsewhere [14-16].”
BackgroundGama, F., Tyskbo, D., Nygren, J., Barlow, J., Reed, J., & Svedberg, P. (2022) · Journal of Medical Internet Research · doi:10.2196/32215 · Cited by 216
“We chose to use the NASSS framework for deductive analysis of the included papers as it represents one of the most advanced frameworks dedicated to understanding implementation of health care technologies and was informed by extensive empirical research and literature review [2].”
BackgroundMethodInfluentialApplication of Artificial Intelligence for the Diagnosis and Treatment of Liver Diseases
Ahn, J. C., Connell, A., Simonetto, D. A., Hughes, C., & Shah, V. H. (2021) · Hepatology · doi:10.1002/hep.31603 · Cited by 196
Artificial Intelligence and Health Technology Assessment: Anticipating a New Level of Complexity
Alami, H., Lehoux, P., Auclair, Y., de Guise, M., Gagnon, M.-P., Shaw, J., Roy, D., Fleet, R., Ag Ahmed, M. A., & Fortin, J.-P. (2020) · Journal of Medical Internet Research · doi:10.2196/17707 · Cited by 164
“To date, few scholars have examined these issues in a systemic and holistic manner [7].”
BackgroundSelf-citationValue creation and value capture for AI business model innovation: a three-phase process framework
Åström, J., Reim, W., & Parida, V. (2022) · Review of Managerial Science · doi:10.1007/s11846-022-00521-z · Cited by 195
Exploring stakeholder attitudes towards AI in clinical practice
Scott, I. A., Carter, S. M., & Coiera, E. (2021) · BMJ Health & Care Informatics · doi:10.1136/bmjhci-2021-100450 · Cited by 193
“2 This limited uptake may be due, at least partly, to misperceptions of what the term AI actually means and negative attitudes towards AI held by key players in the health-care ecosystem.”
Vo, V., Chen, G., Aquino, Y. S. J., Carter, S. M., Nga, Q., & Woode, M. E. (2023) · Social Science & Medicine · doi:10.1016/j.socscimed.2023.116357 · Cited by 193
Applications of Artificial Intelligence in Climate-Resilient Smart-Crop Breeding
Khan, M. H. U., Wang, S., Wang, J., Ahmar, S., Saeed, S., Khan, S. U., Xu, X., Chen, H., Bhat, J. A., & Feng, X. (2022) · International Journal of Molecular Sciences · doi:10.3390/ijms231911156 · Cited by 160
“The recent research on AI tools, including ML, deep learning, and predictive analysis intended toward increasing the planning, learning, reasoning, thinking, and action-taking abilities [33].”
Artificial Intelligence in U.S. Health Care Delivery
Sahni, N. R., & Carrus, B. (2023) · New England Journal of Medicine · doi:10.1056/nejmra2204673 · Cited by 175
A Path for Translation of Machine Learning Products into Healthcare Delivery
(2020) · EMJ Innovations · doi:10.33590/emjinnov/19-00172 · Cited by 172
“92,93 To date, no machine learning product ingesting EHR data has successfully diffused and scaled across healthcare.”
BackgroundAbbasgholizadeh Rahimi, S., Légaré, F., Sharma, G., Archambault, P., Zomahoun, H. T. V., Chandavong, S., Rheault, N., T Wong, S., Langlois, L., Couturier, Y., et al. (2021) · Journal of Medical Internet Research · doi:10.2196/29839 · Cited by 146
“However, the literature also raises doubts about using and implementing AI in health care [24,25].”
Tong, H. L., Quiroz, J. C., Kocaballi, A. B., Fat, S. C. M., Dao, K. P., Gehringer, H., Chow, C. K., & Laranjo, L. (2021) · Preventive Medicine · doi:10.1016/j.ypmed.2021.106532 · Cited by 162
Establishing Data Provenance for Responsible Artificial Intelligence Systems
Werder, K., Ramesh, B., & Zhang, R. (2022) · ACM Transactions on Management Information Systems · doi:10.1145/3503488 · Cited by 188
“The lack of this understanding undermines the implementation of AI in healthcare [81].”
Backgroundd’Elia, A., Gabbay, M., Rodgers, S., Kierans, C., Jones, E., Durrani, I., Thomas, A., & Frith, L. (2022) · Family Medicine and Community Health · doi:10.1136/fmch-2022-001670 · Cited by 166
“4”
Are the Futures Computable? Knightian Uncertainty and Artificial Intelligence
Townsend, D. M., Hunt, R. A., Rady, J., Manocha, P., & Jin, J. H. (2025) · Academy of Management Review · doi:10.5465/amr.2022.0237 · Cited by 122
An Overview of Artificial Intelligence in Oncology
Farina, E., Nabhen, J. J., Dacoregio, M. I., Batalini, F., & Moraes, F. Y. (2022) · Future Science OA · doi:10.2144/fsoa-2021-0074 · Cited by 128
Adly, A. S., Adly, A. S., & Adly, M. S. (2020) · Journal of Medical Internet Research · doi:10.2196/19104 · Cited by 138
“However, evidence synthesis approaches that fit with multiple data sources can enable more robust estimations of the transmission dynamics that can be gathered from noisy data [58,59].”
BackgroundThe role of AI for developing digital twins in healthcare: The case of cancer care
Kaul, R., Ossai, C. I., Forkan, A. R. M., Jayaraman, P. P., Zelcer, J., Vaughan, S., & Wickramasinghe, N. S. (2022) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.1480 · Cited by 178
“Healthcare to date, on the other hand, has been slow to embrace the concept of digital twins (Fuller et al., 2020; Shaw et al., 2019; Wickramasinghe et al., 2021).”
BackgroundLee, T. C., Shah, N. U., Haack, A., & Baxter, S. L. (2020) · Informatics · doi:10.3390/informatics7030025 · Cited by 133
“These findings support prior studies that have expressed the importance of local validation and customization, not just for predictive models but for EHR and health information technology (IT) systems more broadly [20,71,72].”
BackgroundResultNaser, M. A., Majeed, A. A., Alsabah, M. Q., Al-Shaikhli, T. R., & Kaky, K. M. (2024) · Algorithms · doi:10.3390/a17020078 · Cited by 164
“While machine learning methods have demonstrated exceptional predictive power, understanding their decision-making processes can be a very challenging issue [188,189].”
Lu, S.-C., Swisher, C. L., Chung, C. D., Jaffray, D. A., & Sidey-Gibbons, C. J. (2023) · Frontiers in Oncology · doi:10.3389/fonc.2023.1129380 · Cited by 150
The Potential Impact of Artificial Intelligence on Healthcare Spending
Sahni, N. R., Stein, G., Zemmel, R. W., & Cutler, D. (2023) · National Bureau of Economic Research · doi:10.3386/w30857 · Cited by 126
“There are also methodological concerns such as validation and communication of uncertainty, as well as reporting difficulties such as explanations of assump tions (Bates et al. 2020; Shaw et al. 2019; Singh et al. 2020; He et al. 2019).”
Maassen, O., Fritsch, S., Palm, J., Deffge, S., Kunze, J., Marx, G., Riedel, M., Schuppert, A., & Bickenbach, J. (2021) · Journal of Medical Internet Research · doi:10.2196/26646 · Cited by 125
“Shaw et al [74] investigated the challenges of implementing AI in health care compared with other technologies, suggesting that AI implementation in medicine poses novel difficulties.”
BackgroundWang, Y., Zhang, N., & Zhao, X. (2022) · Social Science Computer Review · doi:10.1177/0894439320980132 · Cited by 107
“The advantages, benefits, risks, challenges, and regulations of AI have been extensively discussed (Armstrong et al., 2016; Bernd et al., 2019; Wirtz & Muller, 2019; Mikhaylov et al., 2018; Naud´e & Dimitri, 2009; Shaw et al., 2019; Vincent, 2014; Yudkowsky, 2008).”
BackgroundAntes, A. L., Burrous, S., Sisk, B. A., Schuelke, M. J., Keune, J. D., & DuBois, J. M. (2021) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-021-01586-8 · Cited by 99
“The adoption of these technologies must consider the perspectives of patients, as their effective implementation requires them to engage with AI technologies and share their health data [21–24].”
BackgroundArtificial Intelligence in the Fight Against COVID-19: Scoping Review
Abd-Alrazaq, A., Alajlani, M., Alhuwail, D., Schneider, J., Al-Kuwari, S., Shah, Z., Hamdi, M., & Househ, M. (2020) · Journal of Medical Internet Research · doi:10.2196/20756 · Cited by 114
“Among such tools is artificial intelligence (AI)—a branch of computer science concerned with intelligently analyzing and handling complex information [8,9]—amplifying public health efforts against COVID-19.”
BackgroundKhanijahani, A., Iezadi, S., Dudley, S., Goettler, M., Kroetsch, P., & Wise, J. (2022) · Health Policy and Technology · doi:10.1016/j.hlpt.2022.100602 · Cited by 126
Trust me, i'm a chatbot: how artificial intelligence in health care fails the Turing Test
Powell, J. (2019) · Journal of Medical Internet Research · doi:10.2196/16222 · Cited by 115
“Factors which will influence the implementation of AI tools include those related to people, such as professional and public attitudes, trust, existing work practices, training needs, and the risks of deskilling and disempowerment; those related to the health system, such as leadership and management, the positioning…”
BackgroundInfluentialImplementation frameworks for end-to-end clinical AI: derivation of the SALIENT framework
van der Vegt, A. H., Scott, I. A., Dermawan, K., Schnetler, R. J., Kalke, V. R., & Lane, P. J. (2023) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocad088 · Cited by 109
Artificial intelligence for the echocardiographic assessment of valvular heart disease
Nedadur, R., Wang, B., & Tsang, W. (2022) · Heart · doi:10.1136/heartjnl-2021-319725 · Cited by 107
“44 45”
Alami, H., Lehoux, P., Papoutsi, C., Shaw, S. E., Fleet, R. P., & Fortin, J. (2024) · BMC Health Services Research · doi:10.1186/s12913-024-11112-x · Cited by 119
“…adds to the growing literature that considers technology in a complex sociotechnical transformation perspective that requires not only technological but also human, clinical, professional, organisational, socio-political, economic, regulatory, legal, and cultural changes [27, 40, 41, 56, 59–61].”
BackgroundThe Sociotechnical Ethics of Digital Health: A Critique and Extension of Approaches From Bioethics
Shaw, J., & Donia, J. (2021) · Frontiers in Digital Health · doi:10.3389/fdgth.2021.725088 · Cited by 118
“The ways in which health-related organizations navigate the transition from analog to digital work environments is likely to have substantial implications for the nature of health care work and the nature of patient care (12, 53).”
BackgroundSelf-citationRane, N. L., Paramesha, M., Choudhary, S., & Rane, J. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4835661 · Cited by 122
Implementation of artificial intelligence in medicine: status analysis and development suggestions
Xiang, Y., Zhao, L., Liu, Z., Wu, X., Chen, J., Long, E., Lin, D., Zhu, Y., Chen, C., Lin, Z., et al. (2020) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2019.101780 · Cited by 100
Provenance documentation to enable explainable and trustworthy AI: A literature review
Kale, A., Nguyen, T., Harris, F. C., Li, C., Zhang, J., & Ma, X. (2022) · Data Intelligence · doi:10.1162/dint_a_00119 · Cited by 117
“Even if the predictions generated by AI/ML models deliver high accuracy, it is crucial to know the very roots before concluding any decision, especially in critical domains such as human activities [67, 68].”
Jadczyk, T., Wojakowski, W., Tendera, M., Henry, T. D., Egnaczyk, G. F., & Shreenivas, S. (2021) · Journal of Medical Internet Research · doi:10.2196/22959 · Cited by 109
“Key barriers that delay the adoption of new solutions and practice patterns exist at many levels in the chain of health care delivery [71,72].”
BackgroundKyung, N., & Kwon, H. E. (2022) · Production and Operations Management · doi:10.1111/poms.13785 · Cited by 114
Trustworthy and ethical AI-enabled cardiovascular care: a rapid review
Mooghali, M., Stroud, A. M., Yoo, D. W., Barry, B. A., Grimshaw, A. A., Ross, J. S., Zhu, X., & Miller, J. E. (2024) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-024-02653-6 · Cited by 94
“Another important trust barrier is the uncertainty about the reliability and quality of the data used in the algorithms, which could be incomplete, unrepresentative, or outdated [73].”
BackgroundArtificial intelligence in vaccine research and development: an umbrella review
Arab, R. A. E., Alkhunaizi, M., Alhashem, Y. N., Khatib, A. A., Bubsheet, M., & Hassanein, S. E. (2025) · Frontiers in Immunology · doi:10.3389/fimmu.2025.1567116 · Cited by 94
Artificial Intelligence in Healthcare: Perception and Reality
Akinrinmade, A. O., Adebile, T. M., Ezuma-Ebong, C., Bolaji, K., Ajufo, A., Adigun, A. O., Mohammad, M., Dike, J. C., & Okobi, O. E. (2023) · Cureus · doi:10.7759/cureus.45594 · Cited by 58
“AI is not able to implement an entire job by itself and is, rather, a task-specific tool that will serve to boost but not replace physicians [33].”
BackgroundOccupational Safety and Health Equity Impacts of Artificial Intelligence: A Scoping Review
Fisher, E., Flynn, M. A., Pratap, P., & Vietas, J. (2023) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph20136221 · Cited by 86
What Makes Artificial Intelligence Exceptional in Health Technology Assessment?
Bélisle‐Pipon, J., Couture, V., Roy, M., Ganache, I., Goetghebeur, M. M., & Cohen, I. G. (2021) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2021.736697 · Cited by 93
Sisk, B. A., Antes, A. L., Burrous, S., & DuBois, J. M. (2020) · Children · doi:10.3390/children7090145 · Cited by 89
Yung, N. D. T., Wong, W. K., Juwono, F. H., & Sim, Z. A. (2022) · International Conference on Green Energy, Computing and Sustainable Technology (GECOST) · doi:10.1109/gecost55694.2022.10010490 · Cited by 79
“AI has contributed in the data production of various applications such as advanced computing, statistical modelling, and computer coding [3].”
MethodWickramasinghe, N. S., Jayaraman, P. P., Forkan, A. R. M., Ulapane, N. N. B., Kaul, R., Vaughan, S., & Zelcer, J. (2021) · IEEE Internet Computing · doi:10.1109/mic.2021.3065381 · Cited by 88
Lämmermann, L., Hofmann, P., & Urbach, N. (2023) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2023.102728 · Cited by 93
“However, not all stakeholders in healthcare prioritize these goals and their accompanying challenges equally (Shaw et al., 2019; Sun & Medaglia, 2019).”
BackgroundBienefeld, N., Keller, E., & Grote, G. (2024) · Journal of Medical Internet Research · doi:10.2196/50852 · Cited by 88
Pedro, A. R., Dias, M. B., Laranjo, L., Cunha, A. S., & Cordeiro, J. V. (2023) · PLOS ONE · doi:10.1371/journal.pone.0290613 · Cited by 74
IoT Health Devices: Exploring Security Risks in the Connected Landscape
Affia, A.-A. O., Finch, H., Jung, W., Samori, I. A., Potter, L. N., & Palmer, X. (2023) · IoT · doi:10.3390/iot4020009 · Cited by 88
“Further, such scrutiny is reasonable to be heightened as 4IR technologies, especially that of artificial intelligence, are employed [175].”
BackgroundBienefeld, N., Keller, E., & Grote, G. (2024) · Journal of Medical Internet Research · doi:10.2196/50130 · Cited by 83
Nilsén, P., Svedberg, P., Nygren, J., Frideros, M., Johansson, J., & Schueller, S. M. (2022) · Implementation Research and Practice · doi:10.1177/26334895221112033 · Cited by 79
Ilan, Y. (2020) · Frontiers in Digital Health · doi:10.3389/fdgth.2020.569178 · Cited by 73
“These require constant adaptation of therapeutic regimens (154).”
BackgroundAn Infrastructure Framework for Remote Patient Monitoring Interventions and Research
Claggett, J. L., Petter, S., Joshi, A. M., Ponzio, T. A., & Kirkendall, E. S. (2024) · Journal of Medical Internet Research · doi:10.2196/51234 · Cited by 75
“Particularly in machine learning-based analysis techniques, biases can arise from various sources, such as data sampling, measurement errors, or algorithmic design, leading to potentially biased predictions or recommendations [71,72].”
BackgroundBukowski, M., Farkas, R., Beyan, O., Moll, L., Hahn, H., Kiessling, F., & Schmitz-Rode, T. (2020) · European Radiology · doi:10.1007/s00330-020-06874-x · Cited by 67
Challenges of AI Adoption in the UAE Healthcare
Badi, F. K. A., Hosani, K. A. A., Jabeen, F., Stachowicz-Stanusch, A., Shehzad, N., & Amann, W. (2021) · Vision The Journal of Business Perspective · doi:10.1177/0972262920988398 · Cited by 71
“Stakeholder’s support is imperative for successfully implementing ML in healthcare (Shaw et al., 2019).”
BackgroundUnveiling barriers and drivers of AI adoption for digital marketing in Malaysian SMEs
Enshassi, M., Nathan, R. J., Soekmawati, S., & Ismail, H. (2025) · Journal of Open Innovation Technology Market and Complexity · doi:10.1016/j.joitmc.2025.100519 · Cited by 75
Mamo, Y., Crompton, H., Burke, D., & Nickel, C. (2024) · TechTrends · doi:10.1007/s11528-024-00954-1 · Cited by 65
“Scholars often note that those basic jobs that AI can do should be embraced allowing people time to take on more important roles that AI cannot do (Shaw et al., 2019).”
Joshi, S., Sharma, M., Das, R. P., Rosak-Szyrocka, J., Żywiołek, J., Muduli, K., & Prasad, M. (2022) · Sustainability · doi:10.3390/su141811698 · Cited by 75
Starke, G., Gille, F., Termine, A., Aquino, Y. S. J., Chavarriaga, R., Ferrario, A., Hastings, J., Jongsma, K. R., Kellmeyer, P., Kulynych, B., et al. (2025) · Journal of Medical Internet Research · doi:10.2196/56306 · Cited by 67
“Problems arise at the level of development, with data bottlenecks impeding the training of machine learning models or a lack of user-centered and value-sensitive design procedures affecting AI acceptability [10,11], and stretch to the level of practical implementation of AI systems [12-14].”
Preti, L. M., Ardito, V., Compagni, A., Petracca, F., & Cappellaro, G. (2024) · Journal of Medical Internet Research · doi:10.2196/55897 · Cited by 62
“…that the outer setting domain was largely overlooked, although prior studies have highlighted the importance of external factors, such as data privacy and security laws, ethical issues, regulatory frameworks, and medical liability, in implementing ML applications in clinical practice [5,76-78].”
Neher, M., Petersson, L., Nygren, J. M., Svedberg, P., Larsson, I., & Nilsen, P. (2023) · Implementation Science Communications · doi:10.1186/s43058-023-00458-8 · Cited by 45
“Thus, how we use the term AI needs to be clarified to identify nuances and differences of the AI technologies and AI systems, and study the specific challenges involved in their safe and effective implementation [31].”
BackgroundMethodDebunking the myth of industry 4.0 in health care: insights from a systematic literature review
Cavallone, M., & Palumbo, R. (2020) · The TQM Journal · doi:10.1108/tqm-10-2019-0245 · Cited by 73
Upshaw, T. L., Craig-Neil, A., Macklin, J. A., Gray, C. S., Chan, T. C. Y., Gibson, J., & Pinto, A. D. (2023) · The Journal of the American Board of Family Medicine · doi:10.3122/jabfm.2022.220171r1 · Cited by 65
Adoption of Machine Learning Systems for Medical Diagnostics in Clinics: Qualitative Interview Study
Pumplun, L., Fecho, M., Wahl, N., Peters, F., & Buxmann, P. (2021) · Journal of Medical Internet Research · doi:10.2196/29301 · Cited by 57
“Guided by the call of Shaw et al [10] for more research on the adoption of ML systems in clinics and the lack of prior integral research, our study thus aims to answer the following first research question: which specific factors influence the adoption process of ML systems in medical diagnostics?”
InfluentialJahandideh, S., Ozavci, G., Sahle, B. W., Kouzani, A. Z., Magrabi, F., & Bucknall, T. K. (2023) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2023.105084 · Cited by 60
“Researchers must be cautious about evaluating clinical decision support systems based on ML analytics before widespread algorithm implementation to guarantee safety and accuracy [18,28,41,47].”
BackgroundPredictive Analytics for Glaucoma Using Data From the All of Us Research Program
Baxter, S. L., Saseendrakumar, B. R., Paul, P., Kim, J., Bonomi, L., Kuo, T.-T., Loperena, R., Ratsimbazafy, F., Boerwinkle, E., Cicek, M. S., et al. (2021) · American Journal of Ophthalmology · doi:10.1016/j.ajo.2021.01.008 · Cited by 59
Current trends in chromatographic prediction using artificial intelligence and machine learning
Singh, Y. R., Shah, D. B., Kulkarni, M., Patel, S. R., Maheshwari, D., Shah, J. S., & Shah, S. (2023) · Analytical Methods · doi:10.1039/d3ay00362k · Cited by 65
Machine Learning Operations in Health Care: A Scoping Review
Rajagopal, A., Ayanian, S., Ryu, A. J., Qian, R., Legler, S. R., Peeler, E. A., Issa, M., Coons, T. J., & Kawamoto, K. (2024) · Mayo Clinic Proceedings: Digital Health · doi:10.1016/j.mcpdig.2024.06.009 · Cited by 52
“Although the importance of stakeholder collaboration has been underscored in the literature, 8,68,107,108 real-world examples of successful interdisciplinary collaboration remain sparse.”
Public Trust in Artificial Intelligence Applications in Mental Health Care: Topic Modeling Analysis
Shan, Y., Ji, M., Xie, W., Lam, K.-Y., & Chow, C.-Y. (2022) · JMIR Human Factors · doi:10.2196/38799 · Cited by 45
“…capabilities [29], people-related factors (eg, public attitudes, trust), health system–related factors (eg, issues of regulation and service provision), and tool-related factors (eg, issues of reliability) [10-12] all potentially reduce public trust in DMHC delivered through AI apps or systems.”
BackgroundResultInfluentialNurlia, N., Daud, I., & Rosadi, M. E. (2023) · Escalate Economics and Business Journal · doi:10.61536/escalate.v1i01.6 · Cited by 52
“This finding aligns with existing research (Haanwinckel & Soares, 2021; Shaw et al., 2019) that underscores the importance of considering organizational factors in the context of AI integration.”
Olawade, D. B., Marinze, S., Qureshi, N., Weerasinghe, K., & Teke, J. (2025) · Current Research in Translational Medicine · doi:10.1016/j.retram.2025.103493 · Cited by 76
Trust in medical artificial intelligence: a discretionary account
Nickel, P. J. (2022) · Ethics and Information Technology · doi:10.1007/s10676-022-09630-5 · Cited by 55
“Therefore, for a health care provider to be entirely replaced, every single task performed by that provider would need to be automated by [a machine learning] tool or handed off to a different human’ (Shaw et al., 2019).”
MethodLi, L. T., Haley, L. C., Boyd, A. K., & Bernstam, E. V. (2023) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2023.104531 · Cited by 56
Cresswell, K. M., Rigby, M., Magrabi, F., Scott, P., Brender, J., Craven, C. K., Wong, Z. S.-Y., Kukhareva, P., Ammenwerth, E., Georgiou, A., et al. (2023) · Health Policy · doi:10.1016/j.healthpol.2023.104889 · Cited by 56
Do People Trust in Robot-Assisted Surgery? Evidence from Europe
Torrent‐Sellens, J., Jiménez‐Zarco, A. I., & Saigí-Rubió, F. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph182312519 · Cited by 54
Artificial Intelligence: A Survey on Evolution and Future Trends
Mijwil, M. M., & Abttan, R. A. (2021) · Asian Journal of Applied Sciences · doi:10.24203/ajas.v9i2.6589 · Cited by 48
“First of all, the definition of artificial intelligence must be mentioned [1] [2].”
Cornelissen, L., Egher, C., van Beek, V., Williamson, L., & Hommes, D. W. (2022) · JMIR Formative Research · doi:10.2196/33368 · Cited by 53
“Degree to which an individual believes that using AI-powered care pathways will help him or her to attain gains in productivity, efficiency, and communication [14,15] Nonmedical performance expectancya”
BackgroundResultInfluentialNair, M., Nygren, J., Nilsén, P., Gama, F., Neher, M., Larsson, I., & Svedberg, P. (2025) · Frontiers in Digital Health · doi:10.3389/fdgth.2025.1550459 · Cited by 49
Joshi, M., Mecklai, K., Rozenblum, R., & Samal, L. (2022) · JAMIA Open · doi:10.1093/jamiaopen/ooac022 · Cited by 46
““Meaningful decision support” and “explainability” have been described as 2 key implementation science barriers specific to ML models, which carried additional buy in challenges in our study.(35) In order for decision support to be “meaningful,” users need to first trust individual predictions enough to act on them.”
BackgroundAn investigation into the Impact of Artificial Intelligence on the Future of Project Management
Alshaikhi, A., & Khayyat, M. M. (2021) · International Conference of Women in Data Science at Taif University (WiDSTaif) · doi:10.1109/widstaif52235.2021.9430234 · Cited by 46
“…care workers, many stakeholders in this field perceive the automated robotics differently, and they have feared that such AI automation will eventually replace the work of human staffs in the healthcare sector [15] influenced Intelligence plays a major role on the verge of the technological world.”
BackgroundRole of deep learning in early detection of COVID-19: Scoping review
Alzubaidi, M. S., Zubaydi, H. D., Bin-Salem, A. A., Abd-Alrazaq, A. A., Ahmed, A., & Househ, M. S. (2021) · Computer Methods and Programs in Biomedicine Update · doi:10.1016/j.cmpbup.2021.100025 · Cited by 48
“Furthermore, unprocessed data can be analysed via Deep Learning (DL) to provide significant information in a relatively short period with little effort [8], augmenting public health capability against COVID-19.”
The use of artificial intelligence in the treatment of rare diseases: A scoping review
He, D., Wang, R., Xu, Z., Wang, J., Song, P., Wang, H., & Su, J. (2023) · Intractable & Rare Diseases Research · doi:10.5582/irdr.2023.01111 · Cited by 45
Designing AI-Based Work Processes: How the Timing of AI Advice Affects Diagnostic Decision Making
Yin, J., Ngiam, K. Y., Tan, S. S.-L., & Teo, H. (2025) · Management Science · doi:10.1287/mnsc.2022.01454 · Cited by 62
A Framework to Guide Implementation of AI in Health Care: Protocol for a Cocreation Research Project
Nilsén, P., Svedberg, P., Neher, M., Nair, M., Larsson, I., Petersson, L., & Nygren, J. (2023) · JMIR Research Protocols · doi:10.2196/50216 · Cited by 40
Rakers, M. M., van Buchem, M. M., Kucenko, S., de Hond, A., Kant, I. M. J., van Smeden, M., Moons, K. G. M., Leeuwenberg, A., Chavannes, N. H., Villalobos-Quesada, M. J., et al. (2024) · JAMA Network Open · doi:10.1001/jamanetworkopen.2024.32990 · Cited by 42
Irgang, L., Sestino, A., Barth, H., & Holmén, M. (2025) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2024.123967 · Cited by 43
Medical Laboratories in Healthcare Delivery: A Systematic Review of Their Roles and Impact
Adekoya, A., Okezue, M. A., & Menon, K. (2025) · Laboratories · doi:10.3390/laboratories2010008 · Cited by 52
“Streamlined workflow: AI optimizes laboratory workflows, reducing turnaround times for test results, which is crucial in emergencies where timely decisions can save [14,50].”
Svedberg, P., Reed, J., Nilsen, P., Barlow, J., Macrae, C., & Nygren, J. (2022) · JMIR Research Protocols · doi:10.2196/34920 · Cited by 39
“…implementation science has advanced our knowledge about barriers to implementing such innovations and provided guidance about what strategies can be used to overcome these barriers, this knowledge has not yet been applied for understanding or supporting the implementation of AI in health care [6].”
Application of Artificial Intelligence in Oncology Nursing: A Scoping Review
Zhou, T., Luo, Y., Li, J., Zhang, H., Meng, Z., Xiong, W., & Zhang, J. (2023) · Cancer Nursing · doi:10.1097/ncc.0000000000001254 · Cited by 42
Artificial Intelligence for Dermatopathology: Current Trends and the Road Ahead
Chen, S. B., & Novoa, R. A. (2022) · Seminars in Diagnostic Pathology · doi:10.1053/j.semdp.2022.01.003 · Cited by 37
Beyond the Randomized Clinical Trial: Innovative Data Science to Close the Pediatric Evidence Gap
Goulooze, S. C., Zwep, L. B., Vogt, J. E., Krekels, E. H. J., Hankemeier, T., van den Anker, J. N., & Knibbe, C. A. J. (2019) · Clinical Pharmacology & Therapeutics · doi:10.1002/cpt.1744 · Cited by 40
“For the implementation of machine learning as a medical decision support tool in pediatrics, there are additional challenges to overcome (Figure 3, right column).(36) Because the predictions or classifications of machine-learning tools can incorporate information of multiple variables, they are not as readily integrat…”
BackgroundAkhlaghi, H., Freeman, S., Vari, C., McKenna, B., Braitberg, G., Karro, J., & Tahayori, B. (2024) · Emergency Medicine Australasia · doi:10.1111/1742-6723.14325 · Cited by 35
Metadata
- Authors
- Shaw, J., Rudzicz, F., Jamieson, T., & Goldfarb, A.
- Affiliations
- University of Toronto (CA)
- St. Michael's Hospital (CA)
- Corresponding author
- Shaw, James
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2019-07-10
- Volume
- 21
- Issue
- 7
- Pages
- e13659
- ISSN
- 1438-8871 (ISSN-L)
- PMC
- PMC6652121
- OpenAlex
- W2947151330
- Cited by
- 389
- References
- 58
- Citations by year
- 2026: 62 · 2025: 70 · 2024: 68 · 2023: 77 · 2022: 49 · 2021: 36 · 2020: 22 · 2019: 5
- FWCI
- 8.91
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Journal of Medical Internet Research (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Keywords
- artificial intelligence in healthcare, machine learning implementation, NASSS framework, clinical decision support, explainability, algorithmic bias, privacy and consent, healthcare workforce, scalability, technology adoption, implementation science
- MeSH terms
- Machine Learning, Artificial Intelligence, Humans, Telemedicine
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.