Barriers to Implementing an Artificial Intelligence Model for Unplanned Readmissions
Open accessAbstract
Background Electronic health record (EHR) vendors now offer “off-the-shelf” artificial intelligence (AI) models to client organizations. Our health system faced difficulties in promoting end-user utilization of a new AI model for predicting readmissions embedded in the EHR. Objectives The aim is to conduct a case study centered on identifying barriers to uptake/utilization. Methods A qualitative study was conducted using interviews with stakeholders. The interviews were used to identify relevant stakeholders, understand current workflows, identify implementation barriers, and formulate future strategies. Results We discovered substantial variation in existing workflows around readmissions. Some stakeholders did not perform any formal readmissions risk assessment. Others accustomed to using existing risk scores such as LACE+ had concerns about transitioning to a new model. Some stakeholders had existing workflows in place that could accommodate the new model, but they were not previously aware that the new model was in production. Concerns expressed by end-users included: whether the model's predictors were relevant to their work, need for adoption of additional workflow processes, need for training and change management, and potential for unintended consequences (e.g., increased health care resource utilization due to potentially over-referring discharged patients to home health services). Conclusion AI models for risk stratification, even if “off-the-shelf” by design, are unlikely to be “plug-and-play” in health care settings. Seeking out key stakeholders and defining clear use cases early in the implementation process can better facilitate utilization of these models.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Baxter et al reported concerns from adopters around the lack of explainability regarding the prediction of the AI algorithm embedded in the EHR to predict unplanned readmission; specifically, the lack of explainability regarding what features of the algorithm were driving the output was an impediment to trust among adopters.
“Baxter et al [35] reported concerns from adopters around the lack of explainability regarding the prediction of the AI algorithm embedded in the EHR to predict unplanned readmission; specifically, the lack of explainability regarding what features of the algorithm were driving the output was an impediment to trust among adopters.”
From Barriers to and facilitators of artificial intelligence adoption in health care: scoping review (2024) · cites this paper as [35]
Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
Experimental — accuracy not yet measuredDissatisfaction was reported if recommendations were too similar, inappropriate, or not useful.
“This also included dissatisfaction if recommendations were too similar, inappropriate, or not useful [35,37,38,40,56].”
From Barriers to and facilitators of artificial intelligence adoption in health care: scoping review (2024) · cites this paper as [35]
In Results > Concerns Regarding Model Implementation (page unavailable), stakeholders questioned whether the risk score was useful and whether its predictors were relevant and actionable. This supports concerns about usefulness and appropriateness, but the paper discusses risk scores rather than recommendations and does not report dissatisfaction because recommendations were too similar. (LLM, conf=0.82)
Experimental — accuracy not yet measuredPhysicians expressed that they would be less likely to use an AI given their familiarity with the patient's condition and the value of intuition in clinical decision-making.
“From a clinician standpoint, physicians expressed that they would be less likely to use an AI, given their familiarity with the patient’s condition and the value of intuition in clinical decision-making [32,35].”
From Barriers to and facilitators of artificial intelligence adoption in health care: scoping review (2024) · cites this paper as [35]
In Results > Current Workflows and Readiness for Adoption (page unavailable), physicians said they would be less likely to use the readmissions model because they knew their patients’ risk factors. Results > Concerns Regarding Model Implementation adds that some considered their knowledge of patients’ histories more accurate than risk scores. The paper does not separately identify the value of intuition in clinical decision-making as a reason. (LLM, conf=0.83)
Experimental — accuracy not yet measuredSpecific concerns related to adoption include models being trained on data not representative of the patient population or not containing diverse data as related to social determinants of health.
“Specific concerns related to adoption include models being trained on data not representative of the patient population or not containing diverse data as related to social determinants of health [19,35,36].”
From Barriers to and facilitators of artificial intelligence adoption in health care: scoping review (2024) · cites this paper as [35]
The Results section (Concerns Regarding Model Implementation; page not provided) reports stakeholders’ concern that social determinants of health were not included among the model’s predictors. This supports the concern about missing social-determinants information, but the paper does not say that the model was trained on data unrepresentative of the patient population or that its training data lacked diversity. (LLM, conf=0.82)
Experimental — accuracy not yet measuredImproving algorithmic transparency will be a key consideration to change attitudes and build the trust of adopters.
“To facilitate adoption, improving algorithmic transparency will be a key consideration to change attitudes and build the trust of adopters [35,40].”
From Barriers to and facilitators of artificial intelligence adoption in health care: scoping review (2024) · cites this paper as [35]
Clinical stakeholders expressed concerns around the relevance of the AI model's output, especially because the algorithm did not consider social determinants of health to predict risk outcomes for readmissions.
“Baxter et al [35] and Chua et al [49] reported clinical stakeholders’ concerns around the relevance of the AI model’s output, especially because the algorithm did not consider social determinants of health to predict risk outcomes for readmissions.”
From Barriers to and facilitators of artificial intelligence adoption in health care: scoping review (2024) · cites this paper as [35]
Shortages of personnel with the required skills and the quality of IT infrastructure available for AI implementation were reported as barriers.
“Shortages of personnel with the required skills were reported as barriers, along with the quality of IT infrastructure available for AI implementation [32,35,36,40,45,50].”
From Barriers to and facilitators of artificial intelligence adoption in health care: scoping review (2024) · cites this paper as [35]
The Results (Concerns Regarding Model Implementation; page not provided) discuss staff-training time, and the Conclusion (page not provided) lists workflow and adoption barriers. Searches for skilled-personnel shortages and IT-infrastructure quality found no passage reporting either as a barrier. (LLM, conf=0.72)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Valente, T. W. (2003) · Genetics in Medicine · doi:10.1097/01.gim.0000061743.67794.c4 · Cited by 20,555 · see citing works
Rehospitalizations among Patients in the Medicare Fee-for-Service Program
Jencks, S. F., Williams, M. V., & Coleman, E. A. (2009) · New England Journal of Medicine · doi:10.1056/nejmsa0803563 · Cited by 5,164 · see citing works
Greenhalgh, T., Wherton, J., Papoutsi, C., Lynch, J., Hughes, G., A'Court, C., Hinder, S., Fahy, N., Procter, R., & Shaw, S. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.8775 · Cited by 2,887 · cited 4× · see citing works
“Using the NASSS framework, 16 our fi rst objective was to identify key stakeholders relevant to the following domains: condition (readmissions), adopters, and organization.”
BackgroundMethodInfluentialThe practical implementation of artificial intelligence technologies in medicine
He, J., Baxter, S. L., Xu, J., Xu, J., Zhou, X., & Zhang, K. (2019) · Nature Medicine · doi:10.1038/s41591-018-0307-0 · Cited by 2,246 · see citing works
Self-citationSittig, D. F., & Singh, H. (2010) · Quality and Safety in Health Care · doi:10.1136/qshc.2010.042085 · Cited by 897 · see citing works
Readmissions, Observation, and the Hospital Readmissions Reduction Program
Zuckerman, R. B., Sheingold, S. H., Orav, E. J., Ruhter, J., & Epstein, A. M. (2016) · New England Journal of Medicine · doi:10.1056/nejmsa1513024 · Cited by 826 · see citing works
Opening the black box: low-dimensional dynamics in high-dimensional recurrent neural networks
Sussillo, D., & Barak, O. (2013) · Neural Computation · doi:10.1162/neco_a_00409 · Cited by 575 · see citing works
Artificial intelligence and the implementation challenge
Shaw, J., Rudzicz, F., Jamieson, T., & Goldfarb, A. (2019) · Journal of Medical Internet Research · doi:10.2196/13659 · Cited by 389 · see citing works
A Path Forward on Medicare Readmissions
Joynt, K. E., & Jha, A. K. (2013) · New England Journal of Medicine · doi:10.1056/nejmp1300122 · Cited by 302 · see citing works
Implementing Electronic Health Care Predictive Analytics: Considerations And Challenges
Amarasingham, R., Patzer, R. E., Huesch, M. D., Nguyen, N. Q., & Xie, B. (2014) · Health Affairs · doi:10.1377/hlthaff.2014.0352 · Cited by 180 · see citing works
Integrating Predictive Analytics Into High-Value Care
Parikh, R. B., Kakad, M., & Bates, D. W. (2016) · JAMA · doi:10.1001/jama.2015.19417 · Cited by 169 · see citing works
Economic Evaluation of Quality Improvement Interventions Designed to Prevent Hospital Readmission
Nuckols, T. K., Keeler, E. B., Morton, S. C., Anderson, L., Doyle, B. J., Pevnick, J. M., Booth, M. S., Shanman, R., Arifkhanova, A., & Shekelle, P. (2017) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2017.1136 · Cited by 107 · see citing works
Ibrahim, A. M., Koester, C., Al‐Akchar, M., Tandan, N., Regmi, M. R., Bhattarai, M., Al-bast, B., Kulkarni, A., & ROBINSON, R. L. (2019) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2019-111271 · Cited by 47 · see citing works
Benda, N. C., Das, L. T., Abramson, E. L., Blackburn, K., Thoman, A., Kaushal, R., Zhang, Y., & Ancker, J. S. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaa021 · Cited by 45 · see citing works
Path From Predictive Analytics to Improved Patient Outcomes
Harris, A. H. S. (2016) · Annals of Surgery · doi:10.1097/sla.0000000000002023 · Cited by 37 · see citing works
LACE+ Index as Predictor of 30-Day Readmission in Brain Tumor Population
Caplan, I. F., Sullivan, P. Z., Kung, D. K., O’Rourke, D. M., Choudhri, O., Glauser, G., Osiemo, B., Goodrich, S., McClintock, S. D., & Malhotra, N. R. (2019) · World Neurosurgery · doi:10.1016/j.wneu.2019.03.169 · Cited by 24 · see citing works
Caplan, I. F., Sullivan, P. Z., Glauser, G., Choudhri, O., Kung, D. K., O’Rourke, D. M., Osiemo, B., Goodrich, S., McClintock, S. D., & Malhotra, N. R. (2019) · Clinical Neurology and Neurosurgery · doi:10.1016/j.clineuro.2019.04.026 · Cited by 19 · see citing works
Utility of the LACE Scoring System in Predicting Readmission Following Tracheotomy and Laryngectomy
Ettyreddy, A. R., Kao, W. T. K., Roland, L. T., Rich, J. T., & Chi, J. J. (2019) · Ear Nose & Throat Journal · doi:10.1177/0145561319827908 · Cited by 4 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- [7]Institute of Medicine (U.S.) Rewarding Provider Performance: Aligning Incentives in Medicare. Washington, DC: National Academies Press; 2007
Cited by
Sorted by citations
A Systematic Review of the Barriers to the Implementation of Artificial Intelligence in Healthcare
Ahmed, M. I., Spooner, B., Isherwood, J., Lane, M., Orrock, E., & Dennison, A. (2023) · Cureus · doi:10.7759/cureus.46454 · Cited by 364 · see citing works
Barriers to and facilitators of artificial intelligence adoption in health care: scoping review
Hassan, M., Kushniruk, A., & Borycki, E. (2024) · JMIR Human Factors · doi:10.2196/48633 · Cited by 324 · see citing works
Artificial Intelligence Applications in Health Care Practice: Scoping Review
Sharma, M., Savage, C., Nair, M., Larsson, I., Svedberg, P., & Nygren, J. M. (2022) · Journal of Medical Internet Research · doi:10.2196/40238 · Cited by 198 · see citing works
Nair, M., Svedberg, P., Larsson, I., & Nygren, J. M. (2024) · PLOS ONE · doi:10.1371/journal.pone.0305949 · Cited by 179 · see citing works
Rahimi, A. K., Pienaar, O., Ghadimi, M., Canfell, O. J., Pole, J. D., Shrapnel, S., van der Vegt, A. H., & Sullivan, C. M. (2024) · Journal of Medical Internet Research · doi:10.2196/49655 · Cited by 131 · see citing works
Zając, H. D., Li, D., Dai, X., Carlsen, J. F., Kensing, F., & Andersen, T. O. (2023) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3582430 · Cited by 76 · see citing works
Patient Perspectives on AI for Mental Health Care: Cross-Sectional Survey Study
Benda, N. C., Desai, P., Reza, Z., Zheng, A., Kumar, S., Harkins, S., Hermann, A. D., Zhang, Y., Joly, R., Kim, J., et al. (2024) · JMIR Mental Health · doi:10.2196/58462 · Cited by 75 · see citing works
Jesso, S. T., Kelliher, A., Sanghavi, H. K., Martin, T., & Parker, S. H. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.830345 · Cited by 59 · see citing works
Preti, L. M., Ardito, V., Compagni, A., Petracca, F., & Cappellaro, G. (2024) · Journal of Medical Internet Research · doi:10.2196/55897 · Cited by 62 · see citing works
Impact of Artificial Intelligence–Based Technology on Nurse Management: A Systematic Review
Gonzalez-Garcia, A., Pérez-González, S., Benavides, C., Pinto-Carral, A., Quiroga-Sánchez, E., & Marqués-Sánchez, P. (2024) · Journal of Nursing Management · doi:10.1155/2024/3537964 · Cited by 53 · see citing works
Nair, M., Nygren, J., Nilsén, P., Gama, F., Neher, M., Larsson, I., & Svedberg, P. (2025) · Frontiers in Digital Health · doi:10.3389/fdgth.2025.1550459 · Cited by 38 · see citing works
The Impact of Artificial Intelligence on Allergy Diagnosis and Treatment
Khan, M., Banerjee, S., Muskawad, S., Maity, R., Chowdhury, S. R., Ejaz, R., Kuuzie, E., & Satnarine, T. (2024) · Current Allergy and Asthma Reports · doi:10.1007/s11882-024-01152-y · Cited by 19 · see citing works
Cao, C., Zhao, W., Guo, J., Wang, Z., Lü, A., & Liang, C. C. (2025) · The Innovation Medicine · doi:10.59717/j.xinn-med.2025.100154 · Cited by 14 · see citing works
"It depends": Configuring AI to Improve Clinical Usefulness Across Contexts
Zając, H. D., Ribeiro, J., Ingala, S., Gentile, S., Wanjohi, R., Gitau, S. N., Carlsen, J. F., Nielsen, M. B., & Andersen, T. O. (2024) · Designing Interactive Systems Conference · doi:10.1145/3643834.3660707 · Cited by 14 · see citing works
Zając, H. D., Andersen, T. O., Kwasa, E., Wanjohi, R., Onyinkwa, M. K., Mwaniki, E. K., Gitau, S. N., Yaseen, S. S., Carlsen, J. F., Fraccaro, M., et al. (2025) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3715115 · Cited by 12 · see citing works
Thieme, A., Rajamohan, A. M., Cooper, B., Groombridge, H., Simister, R. J., Wong, B., Woznitza, N., Pinnock, M. A., Wetscherek, M. T., Morrison, C. P., et al. (2025) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3716500 · Cited by 12 · see citing works
Just how transformative will AI/ML be for immuno-oncology?
Bottomly, D., & McWeeney, S. K. (2024) · Journal for ImmunoTherapy of Cancer · doi:10.1136/jitc-2023-007841 · Cited by 9 · see citing works
Can we predict the future of respiratory failure prediction?
Pearce, A. K., Nemati, S., Goligher, E. C., Hough, C. L., Holder, A. L., Wardi, G., Yang, P., Boussina, A. E., Lyons, P. G., Sahetya, S. K., et al. (2025) · Critical Care · doi:10.1186/s13054-025-05484-7 · Cited by 7 · see citing works
Butler, J. M., Doubleday, A., Sattar, U., Nies, M. A., Jeppesen, A., Wright, M. C., Reese, T., Kawamoto, K., Fiol, G. D., & Madaras‐Kelly, K. (2025) · Applied Clinical Informatics · doi:10.1055/a-2505-7743 · Cited by 5 · see citing works
Benda, N. C., Desai, P. M., Reza, Z., Zhang, A., Kumar, S., Harkins, S., Hermann, A. D., Zhang, Y., Joly, R., Kim, J., et al. (2024) · Research Square · doi:10.21203/rs.3.rs-3793389/v1 · Cited by 5 · see citing works
Big data and artificial intelligence in Orthopaedics: Trends and future directions
Loh, J. Y., Tay, X. K. K., Ng, Y. H., Howe, T. S., Koh, J. S. B., & Yang, Y. O. (2026) · Journal of orthopaedic surgery · doi:10.1177/10225536261424038 · Cited by 4 · see citing works
Artificial Intelligence Applications in Health Care Practice: A Scoping Review (Preprint)Preprint
Sharma, M., Savage, C., Nair, M., Larsson, I., Svedberg, P., & Nygren, J. (2022) · doi:10.2196/preprints.40238 · Cited by 3 · see citing works
Naidoo, C. M., Obi, C. L., & Mkolo, N. M. (2025) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3693-9301-7.ch006 · Cited by 2 · see citing works
Performance Characteristics of a Machine-Learning Tool to Predict 7-Day Hospital Readmissions
Morrison, J. M., Casey, B., Sochet, A. A., Dudas, R. A., Rehman, M., Goldenberg, N. A., Ahumada, L. M., & Dees, P. (2022) · Hospital Pediatrics · doi:10.1542/hpeds.2022-006527 · Cited by 1 · see citing works
Cheng, M., & Wang, H. (2025) · Frontiers in Medicine · doi:10.3389/fmed.2025.1708409
Implementation Science for AI Projects
Liebe, J.-D., & Hübner, U. H. (2026) · Health informatics · doi:10.1007/978-3-032-11938-4_5
Sneha, S., Awofisayo, R. M., Nino, V., Rischie, I., & Kumah, L. I. (2026) · JMIR AI · doi:10.2196/preprints.98274
From Algorithm to Augmentation: Redefining Human–AI Dynamics in Modern Healthcare
Gresbek, M., & Schirmer, H. (2026) · IntechOpen eBooks · doi:10.5772/intechopen.1017039
Balazadeh, K., Nygren, J., Gama, F., da Silva, M. A. R., Ploug, T., & Svedberg, P. (2026) · Research Square · doi:10.21203/rs.3.rs-11085511/v1
- Authors
- Baxter, S. L., Bass, J. S., & Sitapati, A. M.
- Affiliations
- University of California San Diego (US)
- Corresponding author
- Baxter, Sally L.
- Venue
- ACI Open
- Publisher
- Georg Thieme Verlag KG
- Published
- 2020-07
- Volume
- 04
- Issue
- 02
- Pages
- e108-e113
- ISSN
- 2566-9346 (ISSN-L)
- PMC
- PMC7710326
- OpenAlex
- W3106808021
- Cited by
- 30
- References
- 18
- Citations by year
- 2026: 6 · 2025: 8 · 2024: 10 · 2023: 2 · 2022: 4
- FWCI
- 0.44
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- Locations
- ACI Open (publishedVersion, cc-by)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Funders
- University of California, San Diego
- U.S. National Library of Medicine (T15LM011271)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Keywords
- unplanned hospital readmission, AI implementation, electronic health records, readmission prediction, risk stratification, stakeholder barriers, workflow integration, change management, healthcare resource utilization, use cases
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
