Artificial Intelligence and the Implementation Challenge
Open accessPublished version of a preprint
The DOI matches its Crossref record; 59 of the 59 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Applications of artificial intelligence (AI) in health care have garnered much attention in recent years, but the implementation issues posed by AI have not been substantially addressed. OBJECTIVE: In this paper, we have focused on machine learning (ML) as a form of AI and have provided a framework for thinking about use cases of ML in health care. We have structured our discussion of challenges in the implementation of ML in comparison with other technologies using the framework of Nonadoption, Abandonment, and Challenges to the Scale-Up, Spread, and Sustainability of Health and Care Technologies (NASSS). METHODS: After providing an overview of AI technology, we describe use cases of ML as falling into the categories of decision support and automation. We suggest these use cases apply to clinical, operational, and epidemiological tasks and that the primary function of ML in health care in the near term will be decision support. We then outline unique implementation issues posed by ML initiatives in the categories addressed by the NASSS framework, specifically including meaningful decision support, explainability, privacy, consent, algorithmic bias, security, scalability, the role of corporations, and the changing nature of health care work. RESULTS: Ultimately, we suggest that the future of ML in health care remains positive but uncertain, as support from patients, the public, and a wide range of health care stakeholders is necessary to enable its meaningful implementation. CONCLUSIONS: If the implementation science community is to facilitate the adoption of ML in ways that stand to generate widespread benefits, the issues raised in this paper will require substantial attention in the coming years.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, health care has seen slow success in the implementation of AI use cases.”
From Barriers to and facilitators of artificial intelligence adoption in health care: scoping review (2024) · cites this paper as [20]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Clinicians expressed the need to understand both the scientific and clinical bases of the recommendations provided by the AI to confidently validate and apply the decision [19,20,36-38].”
From Barriers to and facilitators of artificial intelligence adoption in health care: scoping review (2024) · cites this paper as [20]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“However, successful utility requires successful adoption, and a number of studies have reported on the challenges encountered with implementing AI in health care [19-21].”
From Barriers to and facilitators of artificial intelligence adoption in health care: scoping review (2024) · cites this paper as [20]
In the citing paper
“Shaw et al. (2019) contend that decision support in health care can be useful only if its outputs are able to integrate with the human decision-making processes at the heart of health service delivery.”
From SHIFTing artificial intelligence to be responsible in healthcare: a systematic review (2022) · cites this paper as Shaw et al., 2019
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009) · Implementation Science · doi:10.1186/1748-5908-4-50 · Cited by 14,496 · cited 2×
“The value proposition of digital technology might be different for different stakeholder groups, and implementation frameworks direct attention to the implications of newly introduced technologies for patients, health care providers, managers, health policymakers, and others [8,25,26].”
BackgroundMethodDermatologist-level classification of skin cancer with deep neural networks
Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017) · Nature · doi:10.1038/nature21056 · Cited by 14,362 · cited 1×
“Often, ML involves supervised methods that categorize data points, for example, as images of skin cancer or otherwise given datasets in which all data points (or at least a substantial subset) are associated with a label, ordinal, or category that is meant to be predicted or inferred [15].”
BackgroundNorth, D. C. (1991) · Journal of Economic Perspectives · doi:10.1257/jep.5.1.97 · Cited by 5,232
Artificial intelligence in healthcare: past, present and future
Jiang, F., Jiang, Y., Zhi, H., Dong, Y., Li, H., Ma, S., Wang, Y., Dong, Q., Shen, H., & Wang, Y. (2017) · Stroke and Vascular Neurology · doi:10.1136/svn-2017-000101 · Cited by 4,837 · cited 1×
“We agree with a growing chorus of health care providers and researchers who suggest that ML will primarily serve to augment as opposed to replace the work of humans in the provision of health care in the near term [31], despite applications of automation in health care.”
BackgroundThe ethics of algorithms: Mapping the debate
Mittelstadt, B. D., Allo, P., Taddeo, M., Wachter, S., & Floridi, L. (2016) · Big Data & Society · doi:10.1177/2053951716679679 · Cited by 4,567 · cited 1×
“In cases where training data are partial or incomplete or only reflect a subset of a given population, the resulting model will only be relevant to the population of people represented in the dataset [47].”
BackgroundMaking sense of implementation theories, models and frameworks
Nilsen, P. (2015) · Implementation Science · doi:10.1186/s13012-015-0242-0 · Cited by 4,447 · cited 2×
“We have described use cases (and attendant value propositions) of ML in health care as more likely relating to decision support and less likely to automation, which begins to illustrate the implementation object of focus in ML initiatives [6,7].”
BackgroundMethodEvidence-Based Medicine: A New Approach to Teaching the Practice of Medicine
Guyatt, G. (1992) · JAMA · doi:10.1001/jama.1992.03490170092032 · Cited by 4,202
Machine Learning Applications in Cancer Prognosis and Prediction
Kourou, K., Exarchos, T. P., Exarchos, K. P., Karamouzis, M. V., & Fotiadis, D. I. (2015) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2014.11.005 · Cited by 3,338 · cited 1×
“Given these labels and the statistical models they help to train, ML can be very effective at determining the category in which any newly available individual data point belongs, thereby being useful in the effort to, for example, identify malignant cancers based on particular images [16].”
BackgroundBig other: Surveillance Capitalism and the Prospects of an Information Civilization
Zuboff, S. (2015) · Journal of Information Technology · doi:10.1057/jit.2015.5 · Cited by 3,205 · cited 1×
“As the innovations enabled by ML have taken on a more powerful role in driving global economies, corporations have strategically sought to acquire larger amounts of more diverse data to boost their capacity to develop ML algorithms [61].”
BackgroundInstitutions and Organizations: Ideas, Interests, and Identities
Scott, W. R. (2007) · Cited by 2,943
Research works printed but not linked: 8
- Ajay, A, 2019, The Economics of Artificial Intelligence: An Agenda
- 10.1016/j.socscimed.2012.11.021published · doi:10.1016/j.socscimed.2012.11.021published
- Agrawal, A, 2018, Prediction Machines: The Simple Economics of Artificial Intelligence
- RajpurkarPPolamreddiVBalakrishnanAResearchGate2017Cornell University LibraryMalaria likelihood prediction by effectively surveying households using deep reinforcement learning https://www.researchgate.net/publication/321324506_Malaria_Likelihood_Prediction_By_Effectively_Surveying_Households_Using_Deep_Reinforcement_Learning
- CaseyDBeyanOMurphyKFelzmannHRobot-Assisted Care for Elderly With Dementia: Is There a Potential for Genuine End-User Empowerment?Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction Extended Abstracts2015HRI'15 Extended AbstractsMarch 2-5, 2015Portland, OR2478
- RossiRAAhmedNKThe Network Data Repository with Interactive Graph Analytics and VisualizationProceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence2015AAAI'15January 25-30, 2015Austin Texas42923
- CulnaneCRubinsteinBTeagueVarXiv2017Health data in an open world https://arxiv.org/ftp/arxiv/papers/1712/1712.05627.pdf
- TsymbalAResearchGate2004Computer Science Department, Trinity College DublinThe problem of concept drift: definitions and related work https://www.researchgate.net/publication/228723141_The_Problem_of_Concept_Drift_Definitions_and_Related_Work
Cited by
Items 401–443 of 443
Sneha, S., Awofisayo, R. M., Nino, V., Rischie, I., & Kumah, L. I. (2026) · JMIR AI · doi:10.2196/preprints.98274
Carpenter, C. R., Juhn, Y. J., Montori, V. M., Beebe, T. J., Melton, G. B., Hawley, N. C., Spiten, M. J., Weis, J. A., Cerhan, J. R., Hidaka, B. H., et al. (2026) · Mayo Clinic Proceedings · doi:10.1016/j.mayocp.2026.01.028
Shati, M. A., Fatema, K., & Mitu, M. A. (2024) · World Journal of Advanced Research and Reviews · doi:10.30574/wjarr.2024.23.2.2556
The Human Vanguard: Reclaiming Agency in the Age of Algorithmic Deification
Mu, J. (2026) · Future of business and finance · doi:10.1007/978-3-032-25134-3_8
Valverde, D. A., Quaresma, A. B., Villamil, M. P. G., de Moraes, T. P., & Kotze, P. G. (2026) · Arquivos de Gastroenterologia · doi:10.1590/s0004-2803.24612025-066
“The use of technology to capture and analyze data saves time from researchers and can optimize resources as a consequence to fast generating results (14-17) .”
Artificial intelligence and public services: Enhancing effectiveness and citizen satisfaction
Zhang, M., & Alshammari, S. S. (2026) · Journal of Infrastructure Policy and Development · doi:10.24294/jipd11933
“Shaw highlighted the need for public trust, transparency, and scalability in healthcare by providing valuable insights for overcoming the challenges faced by AI in public administration, particularly in improving operational efficiency (Shaw et al., 2019).”
Hill, C., Keast, J. M., Dahil, A., & Dambha‐Miller, H. (2026) · Journal of Multimorbidity and Comorbidity · doi:10.1177/26335565261444423
Artificial intelligence-supported health counseling scale (AI-HCS): a reliability and validity study
GÖDE, A. (2026) · PeerJ Computer Science · doi:10.7717/peerj-cs.3937
AI and healthcare systems: technological doundations and societal implications
Dubini, O. (2025) · Portuguese National Funding Agency for Science, Research and Technology (RCAAP Project by FCT)
Sun, Y. (2026) · Progress in Human Geography · doi:10.1177/03091325261459102
Democratising AI in qualitative psychology: the role of public involvement
Hitch, D., & Edwards, J. M. (2026) · Qualitative Research in Psychology · doi:10.1080/14780887.2026.2687462
Evaluating AI in Higher Education: Metrics, Evidence Quality, and Meta-Analytic Synthesis
Consuegra, D., Montero, K. D. C. R., Vásquez, M. M., de Sucre, A. J., & Velandia, B. B. (2026) · JoSME : · doi:10.69821/josme.v4i1.25
Dzramado, V. L., Lasim, O. U., & Owusu-Aboagye, D. (2026) · Frontiers in Digital Health · doi:10.3389/fdgth.2026.1834800
“…the primary search strategy, driven by subscription access constraints, may reduce retrieval sensitivity for grey literature and non-African-indexed studies; this is partially mitigated by the extensive African-specific database coverage and grey literature searches described in Table 1 (37, 39).”
Abreu, M., Magalhães, T., Seringa, J., Faria, J., Panitz, L. M., & Vilas-Boas, P. (2026) · doi:10.2196/preprints.102781
Elmosaad, Y. M. (2026) · Healthcare · doi:10.3390/healthcare14142114
“…used in the domains of social health, including social interaction, economic, environmental, and structural determinants, by providing accessible and affordable communication platforms that reduce disparities, improve social service delivery, and foster interaction in various forms [18,29,66].”
Zaghloul, H. B., Arabi, B., Al-Ani, M., Abdullah, A., El-Masri, R., AboMuslim, O., Ahdab, F., Rizwan, M. R. M., Tag, Z. M., Zaghlool, S. B., et al. (2026) · medRxiv · doi:10.64898/2026.07.17.26358328
Katipoğlu, B., & Karaman, F. (2026) · Journal of the American Medical Directors Association · doi:10.1016/j.jamda.2026.106381
Athukorala, S. (2026) · medRxiv · doi:10.64898/2026.07.22.26358733
任栋慧, 曲玉芳, & Sun, H. (2026) · Frontiers in Public Health · doi:10.3389/fpubh.2026.1891645
“Combined with escalating cybersecurity risks associated with large-scale data aggregation, these challenges demand robust governance frameworks to protect patient privacy, data security, and professional autonomy (44).”
Dai, D., Xie, F., Cui, J., Wang, S., Cai, J., Yao, F., & Wang, G. (2026) · Journal of Medical Internet Research · doi:10.2196/93779 · Cited by 1
“Furthermore, AI tools that do not sufficiently incorporate the specific requirements and environmental contexts of end users frequently encounter barriers to adoption and sustained use [15].”
da Ponte, G., & Almeida, S. P. (2026) · IGI Global eBooks · doi:10.4018/979-8-3373-8973-8.ch017
Su, J., Liao, C.-C., Wang, I., Wu, C., & Ho, Y. (2026) · Digital Health · doi:10.1177/20552076261481014
“39”
Vasquez-Cevallos, L., Grunauer-Robalino, R., Muñoz-Hernández, S., Herranz, Á., Salazar, P., & Rodriguez, P. E. D. S. (2026) · Applied Sciences · doi:10.3390/app16168282
Liability Framework to Regulate Artificial Intelligence in Healthcare
Kureel, A. K., & Yadav, B. S. (2026) · International Journal of Law Management & Humanities · doi:10.63108/ijlmh.12676
Lei, Q.-K., Wu, T.-X., Zhang, C.-L., & Zheng, Y. (2026) · Liver Research · doi:10.1016/j.livres.2026.08.001
“207,208 Because HCC management in patients with cirrhosis or chronic viral hepatitis depends on liver reserve, AFP trends, prior imaging, transplant eligibility, and patient goals, AI outputs should be interpreted within this clinical context.”
Alqubaisi, F. A. J. K., & Azizan, A. Z. B. (2026) · International Journal of Social Science Research · doi:10.5296/ijssr.v14i3.24037
Pałubska, J., Muller, O., Rafałowska, Z., Rutz, S., Wojtala, A., Kret, J., Wyrostkiewicz, A., Sadkowski, A., Bigda, P., Gdula, J., et al. (2026) · International Journal of Innovative Technologies in Social Science · doi:10.31435/ijitss.3(51).2026.6007
“Shaw et al. (2019) framed implementation as a major challenge precisely because healthcare AI is usually introduced into already complex systems characterized by legacy technologies, professional norms, regulatory constraints, and interdependent workflows.”
InfluentialDondapati, A., & Dehury, R. K. (2026) · Journal of Health Organization and Management · doi:10.1108/jhom-08-2024-0323
Popa, D. I., Gîndac, C., Buleu, F. N., Elena, R. R., & Levai, C. M. (2026) · European Journal of Investigation in Health Psychology and Education · doi:10.3390/ejihpe16100153
“explainability, governance readiness, and medico-legal uncertainty as recurrent determinants of implementation (Castagno & Khalifa, 2020; Kelly et al., 2019; Lambert et al., 2023; Scheetz et al., 2021; Scott et al., 2021; Shaw et al., 2019).”
Integrating chemistry disciplines with artificial intelligence
Thabit, Q. Q., & Radhi, N. K. (2026) · Advanced Journal of Scientific Research · doi:10.67364/sr4njc07
ВИКОРИСТАННЯ ШТУЧНОГО ІНТЕЛЕКТУ ДЛЯ ПЕРСОНАЛІЗАЦІЇ ТРЕНУВАЛЬНОГО ПРОЦЕСУ ЗА ДОПОМОГОЮ СМАРТПРИСТРОЇВ
Каськів, В., & Luchkevych, M. (2025) · UNIVERSUM
Role of Artificial Intelligence in Healthcare and Pharmaceutical Sector
Tirumalareddy, N. R. (2025) · UPI Journal of Chemical and Life Sciences
Sayed, R. (2023) · Law Business and Sustainability Herald · Cited by 3
A review on harnessing AI for precision dentistry: A new era in oral healthcare
Anushka Saxsena, P. P., Panigrahi, P., & Mishra, P. (2024) · Library Progress (International) · Cited by 1
IDENTIFYING BARRIERS TO THE IMPLEMENTATION OF AI IN SAUDI NON-FINANCIAL FIRMS
Almutairi, D. S. (2026) · Acta Innovations · doi:10.62441/actainnovations.v63i1s.1540
Commitment of criminal offenses by misuse of artificial intelligence
Antonović, R. D. (2026) · International Scientific Conference Law Days – Prof. Slavko Carić, PhD · doi:10.5937/pdsc26035a
Aralihond, A., Ashok, S., & Kumar, A. (2026) · Elsevier eBooks · doi:10.1016/b978-0-443-45156-0.00001-7
Gap analysis of AI ethical guidelines in the medical sector: a comprehensive review
Ashtaiwi, A. (2026) · Discover Artificial Intelligence · doi:10.1007/s44163-026-02071-9
“As AI systems become more integrated into clinical workflows and healthcare decision-making [9], addressing these ethical challenges is necessary to support equitable, safe, and responsible healthcare innovation [10].”
Hald, K., Fagerström, M., Thomsen, J. L., Johnsen, S. P., & Nilsen, P. (2026) · BMJ Open · doi:10.1136/bmjopen-2026-124097
Potentials of GenAI for the Efficiency of Medium-Sized Enterprises
Ruoff, N., Sengül, M., Banzhaf, F., Härting, R.-C., & Scheppach, J. (2026) · Procedia Computer Science · doi:10.1016/j.procs.2026.09.420
Machine Learning and Trading: A Cloud-Based Approach for Financial Forecasting
Pal, N., Verma, T., Singh, A. K., Saifi, R., Srivastava, P. C., & Singh, R. (2026) · International Conference on Advances in Computer Science, Electrical, Electronics, and Communication Technologies (CE2CT) · doi:10.1109/ce2ct68422.2026.11756125
Tobar, S. K. (2026) · Journal of Nursing Management · doi:10.1155/jonm/4870265
Gurha, S., Shrivastava, S., Suravajhala, P., & Singh, P. (2026) · Health Science Reports · doi:10.1002/hsr2.71518
Metadata
- Authors
- Shaw, J., Rudzicz, F., Jamieson, T., & Goldfarb, A.
- Affiliations
- University of Toronto (CA)
- St. Michael's Hospital (CA)
- Corresponding author
- Shaw, James
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2019-07-10
- Volume
- 21
- Issue
- 7
- Pages
- e13659
- ISSN
- 1438-8871 (ISSN-L)
- PMC
- PMC6652121
- OpenAlex
- W2947151330
- Cited by
- 389
- References
- 58
- Citations by year
- 2026: 62 · 2025: 70 · 2024: 68 · 2023: 77 · 2022: 49 · 2021: 36 · 2020: 22 · 2019: 5
- FWCI
- 8.91
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Journal of Medical Internet Research (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Keywords
- artificial intelligence in healthcare, machine learning implementation, NASSS framework, clinical decision support, explainability, algorithmic bias, privacy and consent, healthcare workforce, scalability, technology adoption, implementation science
- MeSH terms
- Machine Learning, Artificial Intelligence, Humans, Telemedicine
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.