Factors Influencing Outcomes of Clinical Information Systems Implementation: A Systematic Review
Abstract
Healthcare agencies spend significant resources to acquire or develop clinical information systems. However, implementation of clinical information systems often report significant failures. A systematic review of the research literature identified processes and outcomes of clinical information system implementation and factors that influenced success or failure. Of 124 original papers, 18 met the primary inclusion criteria-clinical systems implementation, healthcare facility, and outcome measures. Data extraction elements included study characteristics, outcomes, and implementation risk factors classified according to the Expanded Systems Life Cycle. The quality of each study was also assessed. Forty-nine outcomes of clinical information system implementation were identified. No single implementation strategy proved completely effective. The findings of this synthesis direct the attention of managers and decision makers to the importance of clinical context to successful implementation of clinical information systems. The highest number of factors influencing success or failure was reported during implementation and system "go-live." End-user support or lack thereof was the important factor in both successful and failed implementations, respectively. Following the Expanded Systems Life Cycle management model instead of a traditional project management approach may contribute to greater success over time, by paying particular attention to the underrecognized maintenance phase of implementation.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
One review emphasised the need for ongoing effort after the initial 'go-live' phase, referring to the 'under-recognised maintenance phase of implementation'.
“One review emphasised the need for ongoing effort after the initial ‘go-live’ phase, referring to the ‘under-recognised maintenance phase of implementation’ [29].”
From Factors that influence the implementation of e-health: a systematic review of systematic reviews (an update) (2016) · cites this paper as [29]
Evaluation was seen as important to ensure system benefits, to increase health professional acceptance through demonstration of benefits, and to secure ongoing funding.
“Evaluation was seen as important to ensure system benefits [29, 48], to increase health professional acceptance through demonstration of benefits [31, 34, 41, 43] and to secure ongoing funding [53],”
From Factors that influence the implementation of e-health: a systematic review of systematic reviews (an update) (2016) · cites this paper as [29]
Cites
Evaluating computerised health information systems: hard lessons still to be learnt
Littlejohns, P., Wyatt, J. C., & Garvican, L. (2003) · BMJ · doi:10.1136/bmj.326.7394.860 · Cited by 407 · see citing works
Implementation research: state of the art and future directions
Linton, J. D. (2002) · Technovation · doi:10.1016/s0166-4972(01)00075-x · Cited by 126 · see citing works
Nahm, R., & Poston, I. (2000) · PubMed · Cited by 96 · see citing works
LARRABEE, J. H., Boldreghini, S., Elder-Sorrells, K., Turner, Z., Wender, R. G., Hart, J. M., & Lenzi, P. S. (2001) · PubMed · Cited by 87 · see citing works
Hospital information systems: perspectives on problems and prospects, 1979 and 2002
Ball, M. (2003) · International Journal of Medical Informatics · doi:10.1016/s1386-5056(02)00098-9 · Cited by 82 · see citing works
Attitudes of Australian nurses toward the implementation of a clinical information system.
Marasovic, C., Kenney, C., Elliott, D., & Sindhusake, D. P. (1997) · PubMed · Cited by 81 · see citing works
Impact of clinical information systems on nursing practice. Nurses' perspectives.
Axford, R. L., & Carter, B. E. (1996) · PubMed · Cited by 47 · see citing works
Nursing Self-efficacy of an Integrated Clinical and Administrative Information System
Dillon, T. W., Lending, D., Crews, T. R., & Blankenship, R. J. (2003) · CIN Computers Informatics Nursing · doi:10.1097/00024665-200307000-00010 · Cited by 41 · see citing works
Managing Change During an Information Systems Transition
Barr, B. J. (2002) · AORN Journal · doi:10.1016/s0001-2092(06)61610-4 · Cited by 28 · see citing works
Introducing Information Technology in the Clinical Setting: Lessons Learned in a Trauma Center
Paré, G., & Elam, J. J. (1998) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462300012290 · Cited by 23 · see citing works
The "big bang" implementation: not for the faint of heart.
Anderson, L. K., & Stafford, C. J. (2002) · PubMed · Cited by 22 · see citing works
HEALTHCARE INFORMATION SYSTEMS
Husting, P. M., & Cintron, L. (2003) · Journal for Nurses in Staff Development · doi:10.1097/00124645-200309000-00008 · Cited by 14 · see citing works
LaDuke, S. D. (2001) · JONA The Journal of Nursing Administration · doi:10.1097/00005110-200106000-00003 · Cited by 14 · see citing works
Analysis, processes, and techniques. Case study.
Thompson, C. B., Snyder‐Halpern, R., & Staggers, N. (1999) · PubMed · Cited by 8 · see citing works
Implementation of NANDA nursing diagnoses online. North American Nursing Diagnosis Association.
Brooks, B. A., & Massanari, K. (1998) · PubMed · Cited by 7 · see citing works
Managing the human side of change to automation.
Doyle, K., & Kowba, M. (1997) · PubMed · Cited by 6 · see citing works
Halley, E. C., & Kambic, P. M. (1996) · PubMed · Cited by 5 · see citing works
Are Staffs Prepared for the New Information-Based Hospital Enterprise?
Simpson, R. L. (1997) · Nursing Administration Quarterly · doi:10.1097/00006216-199702120-00013 · Cited by 5 · see citing works
Healing Health Care, Healing Nursing in the 21st Century
Simpson, R. L. (2002) · Nursing Administration Quarterly · doi:10.1097/00006216-200210000-00013 · Cited by 5 · see citing works
The need for quick computer response with minimal input: Development of an ED computer system
Travers, D. A. (1997) · Journal of Emergency Nursing · doi:10.1016/s0099-1767(97)90025-8 · Cited by 3 · see citing works
A training plan for bedside computers.
Bl, W., Sk, H., & Bl, V. (1997) · PubMed · Cited by 3 · see citing works
Education strategy for the implementation of computerized nursing documentation.
Murphy, F., Karmali, K., McFarlane, G., & Agustin, K. I. (1995) · PubMed · Cited by 3 · see citing works
A clinical information system transition: a nursing perspective.
Stebbins, J., Gentes, J. M., Kagen-Fishkind, J. E., LaMountain, S., Sheehan, S. M., & Colburn, C. (1997) · PubMed · Cited by 3 · see citing works
Functional health pattern assessment on-line. Lessons learned.
Johnson, T. (2000) · PubMed · Cited by 2 · see citing works
End-user support improvement at Memorial Sloan-Kettering Cancer Center.
Wt, C., & Hamilton, M. (1997) · PubMed · Cited by 1 · see citing works
Flagged first, then by citations
3 of the 28 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
The Impact of eHealth on the Quality and Safety of Health Care: A Systematic Overview
Black, A. D., Car, J., Pagliari, C., Anandan, C., Cresswell, K., Bokun, T., McKinstry, B., Procter, R., Majeed, A., & Sheikh, A. (2011) · PLoS Medicine · doi:10.1371/journal.pmed.1000387 · Cited by 1,442 · see citing works
Ross, J., Stevenson, F., Lau, R., & Murray, E. (2016) · Implementation Science · doi:10.1186/s13012-016-0510-7 · Cited by 1,011 · see citing works
Cresswell, K., & Sheikh, A. (2013) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2012.10.007 · Cited by 690 · see citing works
Mair, F. S., May, C., O’Donnell, C., Finch, T., Sullivan, F., & Murray, E. (2012) · Bulletin of the World Health Organization · doi:10.2471/blt.11.099424 · Cited by 553 · see citing works
Gustafsson, L. L., Wettermark, B., Godman, B., Karlsson, E., Bergman, U., Hasselström, J., Hensjö, L., Hjemdahl, P., Jägre, I., Julander, M., et al. (2011) · Basic & Clinical Pharmacology & Toxicology · doi:10.1111/j.1742-7843.2011.00682.x · Cited by 227 · see citing works
Fritz, F., Tilahun, B., & Dugas, M. (2015) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocu038 · Cited by 131 · see citing works
An importance-performance analysis of hospital information system attributes: A nurses' perspective
Cohen, J. F., Coleman, E. C., & Kangethe, J. M. (2015) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2015.10.010 · Cited by 91 · see citing works
Factors Associated With Canadian Nurses' Informatics Competency
Kleib, M., & Nagle, L. (2018) · CIN Computers Informatics Nursing · doi:10.1097/cin.0000000000000434 · Cited by 73 · see citing works
Medication administration technologies and patient safety: a mixed-method systematic review
Wulff, K., Cummings, G. G., Marck, P., & Yurtseven, Ö. (2011) · Journal of Advanced Nursing · doi:10.1111/j.1365-2648.2011.05676.x · Cited by 60 · see citing works
Self-citationFreire, M. L., Noronha, B. P., Cota, G., & Silva, S. N. (2025) · BMC Health Services Research · doi:10.1186/s12913-025-13117-6 · Cited by 58 · see citing works
deRiel, E., Puttkammer, N. H., Hyppolite, N., Diallo, J. O., Wagner, S., Honoré, J. G., Balan, J. G., Celestin, N., Valles, J. S., Duval, N., et al. (2017) · Health Policy and Planning · doi:10.1093/heapol/czx171 · Cited by 42 · see citing works
The Impact of eHealth on the Quality and Safety of Healthcare
Majeed, A., Black, A., Car, J., Anandan, C., Cresswell, K. M., McKinstry, B., Pagliari, C., Procter, R., & Sheikh, A. (2009) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-642-11284-3_21 · Cited by 34 · see citing works
Wye, L., Lasseter, G., Simmonds, B., Duncan, L., Percival, J., & Purdy, S. (2016) · Journal of research in nursing · doi:10.1177/1744987116628922 · Cited by 31 · see citing works
Moghaddam, N. M., Kazemi, A., & Alimoradnori, M. (2019) · Informatics in Medicine Unlocked · doi:10.1016/j.imu.2019.100251 · Cited by 31 · see citing works
Cheung, A., van Velden, F. H. P., Lagerburg, V., & Minderman, N. (2015) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2014.12.002 · Cited by 31 · see citing works
Schweitzer, M., & Hoerbst, A. (2015) · Yearbook of Medical Informatics · doi:10.15265/iy-2015-018 · Cited by 29 · see citing works
Impact of digital health on the safety and quality of health care
Shaw, T. J., Hines, M., & Kielly-Carroll, C. (2018) · Cited by 27 · see citing works
Eiermann, B., Bastholm, P., Korkmaz, S., Landberg, C., Lilja, B., Shemeikka, T., Vég, A., Wettermark, B., & Lars, L. (2010) · InTech eBooks · doi:10.5772/39458 · Cited by 23 · see citing works
Ahmad, N., Du, S., Ahmed, F., Amin, N. U., & Yi, X. (2023) · Information Technology and Management · doi:10.1007/s10799-023-00407-w · Cited by 22 · see citing works
Patient-Centered Personal Health Record and Portal Implementation Toolkit for Ambulatory Clinics
Nahm, E., Diblasi, C., Gonzales, E., Silver, K. D., Zhu, S., Sagherian, K., & Kongs, K. (2016) · CIN Computers Informatics Nursing · doi:10.1097/cin.0000000000000318 · Cited by 21 · see citing works
Tun, S. Y. Y., & Madanian, S. (2023) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocad011 · Cited by 18 · see citing works
Evaluating Maturity Models in Healthcare Information Systems: A Comprehensive Review
Gomes, J., & Rom�ão, M. (2025) · Healthcare · doi:10.3390/healthcare13151847 · Cited by 17 · see citing works
Katehakis, D. G., Halkiotis, S., & Kouroubali, A. (2011) · Journal of Healthcare Engineering · doi:10.1260/2040-2295.2.3.389 · Cited by 13 · see citing works
McLane, S., & Turley, J. P. (2010) · JONA The Journal of Nursing Administration · doi:10.1097/nna.0b013e3181fc19d6 · Cited by 12 · see citing works
Distributed IT championing: A process theory
Negoita, B., Rahrovani, Y., Lapointe, L., & Pinsonneault, A. (2021) · Journal of Information Technology · doi:10.1177/02683962211019406 · Cited by 8 · see citing works
Impact of a nursing information system in clinical practice: a longitudinal study project.
D’Agostino, F., Zega, M., Rocco, G., Luzzi, L., Vellone, E., & Alvaro, R. (2013) · PubMed · doi:10.7416/ai.2013.1935 · Cited by 8 · see citing works
Successful IS/IT Projects in Healthcare: Pretesting a Questionnaire
Gomes, J., Rom�ão, M., & Carvalho, H. (2016) · Procedia Computer Science · doi:10.1016/j.procs.2016.09.172 · Cited by 8 · see citing works
Staffing resource allocation, budgets and management
Hovenga, E. J. S., & Lowe, C. (2020) · Elsevier eBooks · doi:10.1016/b978-0-12-816977-3.00007-1 · Cited by 7 · see citing works
Los adjusted clinicals groups: un sistema de clasificación de pacientes por ajuste de riesgos
Sicras-Mainar, A., & Navarro‐Artieda, R. (2014) · Revista Peruana de Medicina Experimental y Salud Pública · doi:10.17843/rpmesp.2013.302.210 · Cited by 6 · see citing works
Cohen, J. F., & Kangethe, J. M. (2015) · Journal of Organizational and End User Computing · doi:10.4018/joeuc.2015070103 · Cited by 6 · see citing works
Impact of four training conditions on physician use of a web-based clinical decision support system
Kealey, E. M., Leckman‐Westin, E., & Finnerty, M. T. (2013) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2013.03.003 · Cited by 6 · see citing works
Bahnassy, A. (2018) · Journal of Nursing & Patient Care · doi:10.4172/2573-4571-c3-019 · Cited by 6 · see citing works
Status and trends of e-Health tools in Kuwait: A narrative review
Alaslawi, H., Berrou, I., Alhuwail, D., & Aslanpour, Z. (2019) · UWE Research Repository (UWE Bristol) · Cited by 5 · see citing works
Niazkhani, Z., Pirnejad, H., Bont, A. A. D., & Aarts, J. (2010) · The Open Medical Informatics Journal · doi:10.2174/1874431101004010206 · Cited by 5 · see citing works
Olson, B., & FAUCHALD, S. K. (2011) · CIN Computers Informatics Nursing · doi:10.1097/ncn.0b013e318224b450 · Cited by 5 · see citing works
Fanta, G. B., & Erasmus, L. D. (2014) · INCOSE International Symposium · doi:10.1002/j.2334-5837.2014.00006.x · Cited by 4 · see citing works
The case of Inca´s National Tumor Bank Management System in Brazil
Gonçalves, A. A., Pitassi, C., & Moreno, V. D. A., Jr. (2014) · Journal of Information Systems and Technology Management · doi:10.4301/s1807-17752014000300003 · Cited by 4 · see citing works
Technology Implementation Case Studies: Lincus Software as a Service
Blanchard, A., Prior, F., Gilbert, L. A., & Dawson, T. (2017) · Springer eBooks · doi:10.1007/978-3-319-60137-3_7 · Cited by 3 · see citing works
Wu, M.-W., Lee, T., Tsai, T.-C., Huang, C.-Y., Wu, F. F.-S., & Mills, M. E. (2015) · Open Journal of Nursing · doi:10.4236/ojn.2015.57071 · Cited by 3 · see citing works
Ahmad, N., Amin, N. U., Ali, S., & Yi, X. (2025) · Behaviour and Information Technology · doi:10.1080/0144929x.2025.2484390 · Cited by 2 · see citing works
Digital health ecosystems: Use of informatics, connectivity and system interoperability
Hovenga, E. J. S., & Lowe, C. (2020) · Elsevier eBooks · doi:10.1016/b978-0-12-816977-3.00009-5 · Cited by 2 · see citing works
Burton, S. D. (2022) · Social and cultural studies of robots and AI · doi:10.1007/978-3-030-88615-8_8 · Cited by 2 · see citing works
A classification framework for clinical information system implementation in hospitals
Meulendijks, A., Batenburg, R. S., & van de Wetering, R. (2012) · International Journal of Organisational Design and Engineering · doi:10.1504/ijode.2012.051443 · Cited by 2 · see citing works
Wen, I. V. K., & Long, C. S. (2023) · International Journal of Academic Research in Business and Social Sciences · doi:10.6007/ijarbss/v13-i9/17926 · Cited by 2 · see citing works
Galimany‐Masclans, J., Garrido-Aguilar, E., Estrada-Masllorens, J. M., & García, M. R. G. I. (2013) · Educación Médica · doi:10.33588/fem.163.689 · Cited by 1 · see citing works
Ross, J. (2016) · UCL Discovery (University College London) · Cited by 1 · see citing works
Zobair, K. M. (2019) · Griffith Research Online · doi:10.25904/1912/1330 · Cited by 1 · see citing works
Understanding the human factor: the key to information systems implementation in service operations
Kekäle, T., Spens, K. M., & Phusavat, K. (2011) · International Journal of Services Economics and Management · doi:10.1504/ijsem.2011.042787
DRUG-DRUG INTERACTIONS - FROM KNOWLEDGE BASE TO CLINICAL IMPACT
Andersson, M. L. (2014) · Figshare
Evaluating Your Information System Implementation
Wilson, M. L. (2009) · Journal of PeriAnesthesia Nursing · doi:10.1016/j.jopan.2009.08.004
Understanding Perspectives of Risk Awareness
Park, B. R. (2014) · UVic’s Research and Learning Repository (University of Victoria)
THE CASE OF INCA´S NATIONAL TUMOR BANK MANAGEMENT SYSTEM IN BRAZIL
Gonçalves, A. A., Pitassi, C., & Moreno, V. D. A., Jr. (2022) · Figshare · doi:10.6084/m9.figshare.20011711
Cohen, J. F., & Kangethe, J. M. (2019) · IGI Global eBooks · doi:10.4018/978-1-5225-8356-1.ch066
Information and Communication Systems and Technologies in Health
Gomes, J., & Rom�ão, M. (2023) · Advances in civil and industrial engineering book series · doi:10.4018/978-1-6684-7874-5.ch012
Digital Mental Health Interventions for Children and Youth
Machado, A. S., Ferreira, B., Pereira, C. Y., Carrola, C., de Melo, B. B. V., & Campos, C. (2023) · Advances in medical technologies and clinical practice book series · doi:10.4018/978-1-6684-9251-2.ch002
Historia clínica electrónica de atención primaria de salud: Valoración de las enfermeras
Masclans, J. G. (2014) · LA Referencia (Red Federada de Repositorios Institucionales de Publicaciones Científicas)
- Authors
- GRUBER, D., CUMMINGS, G. G., LEBLANC, L., & SMITH, D. L.
- Affiliations
- Alberta Health Services (CA)
- Venue
- CIN: Computers, Informatics, Nursing
- Published
- 2009-05
- Volume
- 27
- Issue
- 3
- Pages
- 151-163
- ISSN
- 1538-2931 (ISSN-L)
- OpenAlex
- W2045492644
- Cited by
- 56
- References
- 28
- Citations by year
- 2025: 3 · 2023: 5 · 2022: 1 · 2021: 1 · 2020: 2 · 2019: 4 · 2018: 3 · 2017: 2 · 2016: 5 · 2015: 6 · 2014: 6 · 2013: 4 · 2012: 3
- FWCI
- 9.85
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- CIN Computers Informatics Nursing (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Social Sciences › Sociology and Political Science › Information Systems Theories and Implementation
- Social Sciences › Business, Management and Accounting › Management Information Systems › ERP Systems Implementation and Impact
- Keywords
- clinical information systems, health information technology implementation, implementation success and failure, go-live, end-user support, implementation outcomes, care facility management
- MeSH terms
- Database Management Systems, Education, Continuing, Nursing Informatics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
