Effects of clinical decision-support systems on practitioner performance and patient outcomes: a synthesis of high-quality systematic review findings
Open accessAbstract
OBJECTIVE: To synthesize the literature on clinical decision-support systems' (CDSS) impact on healthcare practitioner performance and patient outcomes. DESIGN: Literature search on Medline, Embase, Inspec, Cinahl, Cochrane/Dare and analysis of high-quality systematic reviews (SRs) on CDSS in hospital settings. Two-stage inclusion procedure: (1) selection of publications on predefined inclusion criteria; (2) independent methodological assessment of preincluded SRs by the 11-item measurement tool, AMSTAR. Inclusion of SRs with AMSTAR score 9 or above. SRs were thereafter rated on level of evidence. Each stage was performed by two independent reviewers. RESULTS: 17 out of 35 preincluded SRs were of high methodological quality and further analyzed. Evidence that CDSS significantly impacted practitioner performance was found in 52 out of 91 unique studies of the 16 SRs examining this effect (57%). Only 25 out of 82 unique studies of the 16 SRs reported evidence that CDSS positively impacted patient outcomes (30%). CONCLUSIONS: Few studies have found any benefits on patient outcomes, though many of these have been too small in sample size or too short in time to reveal clinically important effects. There is significant evidence that CDSS can positively impact healthcare providers' performance with drug ordering and preventive care reminder systems as most clear examples. These outcomes may be explained by the fact that these types of CDSS require a minimum of patient data that are largely available before the advice is (to be) generated: at the time clinicians make the decisions.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
A recent review of reviews of decision-support systems found that, although 52 (57%) of 91 unique studies demonstrated improved practitioner performance, only 25 (30%) of the 82 in which patient outcomes were assessed showed benefits to patient outcomes.
“A recent review of reviews of decisionsupport systems found that, although 52 (57%) of 91 unique studies demonstrated improved practitioner performance, only 25 (30%) of the 82 in which patient outcomes were assessed showed benefits to patient outcomes.”
From Impacts of e-health on the outcomes of care in low- and middle-income countries: where do we go from here? (2012) · cites this paper as [5]
Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
Experimental — accuracy not yet measuredSome systematic evidence of the benefits of e-health in general, and of specific areas of e-health such as decision-support systems for clinicians or patient-targeted text messaging, already exists.
“Some systematic evidence of the benefits of e-health in general, [2][3][4] and of specific areas of e-health, such as decision-support systems for clinicians 5,6 or patient-targeted text messaging, 7-10 already exists.”
From Impacts of e-health on the outcomes of care in low- and middle-income countries: where do we go from here? (2012) · cites this paper as [5]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Evidence based medicine: what it is and what it isn't
Sackett, D. L., Rosenberg, W. M. C., Gray, J. A. M., Haynes, R. B., & Richardson, W. S. (1996) · BMJ · doi:10.1136/bmj.312.7023.71 · Cited by 15,769 · see citing works
From best evidence to best practice: effective implementation of change in patients' care
Grol, R., & Grimshaw, J. (2003) · The Lancet · doi:10.1016/s0140-6736(03)14546-1 · Cited by 4,531 · see citing works
External Validation of a Measurement Tool to Assess Systematic Reviews (AMSTAR)
Shea, B. J., Bouter, L. M., Peterson, J., Boers, M., Andersson, N., Ortiz, Z., Ramsay, T., Bai, A., Shukla, V. K., & Grimshaw, J. M. (2007) · PLoS ONE · doi:10.1371/journal.pone.0001350 · Cited by 569 · see citing works
Mollon, B., Chong, J. J., Holbrook, A. M., Sung, M., Thabane, L., & Foster, G. (2009) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-9-11 · Cited by 124 · see citing works
Clinical decision support systems in the pediatric intensive care unit*
Mack, E. H., Wheeler, D. S., & Embi, P. J. (2009) · Pediatric Critical Care Medicine · doi:10.1097/pcc.0b013e3181936b23 · Cited by 76 · see citing works
Sintchenko, V., Magrabi, F., & Tipper, S. (2007) · Medical Informatics and the Internet in Medicine · doi:10.1080/14639230701447701 · Cited by 47 · see citing works
Flagged first, then by citations
Research works printed but not linked: 34
- Montori VM, Wilczynski NL, Morgan D, et al. Optimal search strategies for retrieving systematic reviews from Medline: analytical survey. BMJ 2005;330:68.
- Shea BJ, Grimshaw JM, Wells GA, et al. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol 2007;7:10.
- Shea BJ, Hamel C, Wells GA, et al. AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. J Clin Epidemiol 2009;62:1013e20.
- Nie `s J, Colombet I, Degoulet P, et al. Determinants of success for computerized clinical decision support systems integrated in CPOE systems: a systematic review. AMIA Annu Symp Proc 2006:594e8.
- Shiffman RN, Liaw Y, Brandt CA, et al. Computer-based guideline implementation systems: a systematic review of functionality and effectiveness. J Am Med Inform Assoc 1999;6:104e14.
- Kaplan B. Evaluating informatics applicationsdclinical decision support systems literature review. Int J Med Inform 2001;64:15e37.
- Montani S, Bellazzi R, Quaglini S, et al. Meta-analysis of the effect of the use of computer-based systems on the metabolic control of patients with diabetes mellitus. Diabetes Technol Ther 2001;3:347e56.
- Jackson CL, Bolen S, Brancati FL, et al. A systematic review of interactive computer-assisted technology in diabetes care. Interactive information technology in diabetes care. J Gen Intern Med 2006;21:105e10.
- Chaudhry B, Wang J, Wu S, et al. Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Ann Intern Med 2006;144:742e52.
- Conroy S, Sweis D, Planner C, et al. Interventions to reduce dosing errors in children: a systematic review of the literature. Drug Saf 2007;30:1111e25.
- Eslami S, de Keizer NF, Abu-Hanna A. The impact of computerized physician medication order entry in hospitalized patientsda systematic review. Int J Med Inform 2008;77:365e76.
- Georgiou A, Williamson M, Westbrook JI, et al. The impact of computerised physician order entry systems on pathology services: a systematic review. Int J Med Inform 2007;76:514e29.
- Rommers MK, Teepe-Twiss IM, Guchelaar HJ. Preventing adverse drug events in hospital practice: an overview. Pharmacoepidemiol Drug Saf 2007;16:1129e35.
- Shebl NA, Franklin BD, Barber N. Clinical decision support systems and antibiotic use. Pharm World Sci 2007;29:342e9.
- Ammenwerth E, Schnell-Inderst P, Machan C, et al. The effect of electronic prescribing on medication errors and adverse drug events: a systematic review. J Am Med Inform Assoc 2008;15:585e600.
- Van Rosse F, Maat B, Rademaker CM, et al. The effect of computerized physician order entry on medication prescription errors and clinical outcome in pediatric and intensive care: a systematic review. Pediatrics 2009;123:1184e90.
- Schedlbauer A, Prasad V, Mulvaney C, et al. What evidence supports the use of computerized alerts and prompts to improve clinicians' prescribing behavior? J Am Med Inform Assoc 2009;16:531e8.
- Johnston ME, Langton KB, Haynes RB, et al. Effects of computer-based clinical decision support systems on clinician performance and patient outcome. A critical appraisal of research. Ann Intern Med 1994;120:135e42.
- Hunt DL, Haynes RB, Hanna E, et al. Effects of computer-based clinical decision support systems on physician performance and patient outcomes: a systematic review. JAMA 1998;280:1339e46.
- Walton R, Dovey S, Harvey E, et al. Information in practice. Computer support for determining drug dose: systematic review and meta-analysis. BMJ 1999;318:984e90.
- Kaushal R, Shojania KG, Bates DW. Effects of computerized physician order entry and clinical decision support systems on medication safety: a systematic review. Arch Intern Med 2003;163:1409e16.
- Garg AX, Adhikari NK, McDonald H, et al. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA 2005;293:1223e38.
- Shamliyan TA, Duval S, Du J, et al. Just what the doctor ordered. Review of the evidence of impact of compurized physician order entry systems on medication errors. Health Serv Res 2008;43:32e53.
- Montgomery AA, Fahey T. A systematic review of the use of computers in the management of hypertension. J Epidemiol Community Health 1998;52:520e5.
- Balas EA, Krishna S, Kretschmer RA, et al. Computerized knowledge management in diabetes care. Med Care 2004;42:610e21.
- Liu LY, Wyatt JC, Deeks JJ, et al. Systematic reviews of clinical decision tools for acute abdominal pain. Health Technol Assess 2006;10:1e167. iiieiv.
- Sanders DL, Aronsky D. Biomedical informatics applications for asthma care: a systematic review. J Am Med Inform Assoc 2006;13:418e27.
- Yourman L, Concato J, Agostini JV. Use of computer decision support interventions to improve medication prescribing in older adults: a systematic review. Am J Geriatr Pharmacother 2008;6:119e29.
- Randell R, Mitchell N, Dowding D, et al. Effects of computerized decision support systems on nursing performance and patient outcomes: a systematic review. J Health Serv Res Policy 2007;12:242e9.
- Chatellier G, Colombet I, Degoulet P. An overview of the effect of computer- assisted anticoagulant therapy on the quality of anticoagulation. Int J Med Inform 1998;49:311e20.
- Wolfstadt JI, Gurwitz JH, Field TS, et al. The effect of computerized physician order entry with clinical decision support on the rates of adverse drug events: a systematic review. J Gen Intern Med 2008;23:451e8.
- Khajouei R, Jaspers MW. The impact of CPOE medication systems' design aspects on USABILITY, Workflow and medication orders. Methods Inf Med 2010;49:3e19.
- Kawamoto K, Houlihan CA, Balas EA, et al. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ 2005;330:765.
- Patel VL, Shortliffe EH, Stefanelli M, et al. The coming of age of artificial medicine. Artif Intell Med 2009;46:5e17.
Cited by
Sorted by citations
Liberati, E. G., Ruggiero, F., Galuppo, L., Gorli, M., González-Lorenzo, M., Maraldi, M., Ruggieri, P., Polo Friz, H., Scaratti, G., Kwag, K. H., et al. (2017) · Implementation Science · doi:10.1186/s13012-017-0644-2 · Cited by 301 · see citing works
Yang, Q., Steinfeld, A., & Zimmerman, J. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300468 · Cited by 247 · see citing works
Harnessing the Power of Clinical Decision Support Systems: Challenges and Opportunities
Chen, Z., Liang, N., Zhang, H., Li, H., Yang, Y., Zong, X., Chen, Y., Wang, Y., & Shi, N. (2023) · Open Heart · doi:10.1136/openhrt-2023-002432 · Cited by 244 · see citing works
Evaluating the impact of prediction models: lessons learned, challenges, and recommendations
Kappen, T. H., van Klei, W. A., van Wolfswinkel, L., Kalkman, C. J., Vergouwe, Y., & Moons, K. G. M. (2018) · Diagnostic and Prognostic Research · doi:10.1186/s41512-018-0033-6 · Cited by 215 · see citing works
Piette, J., Lun, K., Moura, L., Fraser, H., Mechael, P., Powell, J., & Khoja, S. (2012) · Bulletin of the World Health Organization · doi:10.2471/blt.11.099069 · Cited by 188 · see citing works
Digital health interventions in palliative care: a systematic meta-review
Finucane, A. M., O’Donnell, H., Lugton, J., Gibson-Watt, T., Swenson, C., & Pagliari, C. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00430-7 · Cited by 163 · see citing works
Vasey, B., Ursprung, S., Beddoe, B., Taylor, E. H., Marlow, N., Bilbro, N., Watkinson, P., & McCulloch, P. (2021) · JAMA Network Open · doi:10.1001/jamanetworkopen.2021.1276 · Cited by 154 · see citing works
General Practitioners’ Attitudes Toward Artificial Intelligence–Enabled Systems: Interview Study
Buck, C., Doctor, E., Hennrich, J., Jöhnk, J., & Eymann, T. (2022) · Journal of Medical Internet Research · doi:10.2196/28916 · Cited by 147 · see citing works
Van de Velde, S., Heselmans, A., Delvaux, N., Brandt, L., Marco-Ruiz, L., Spitaels, D., Cloetens, H., Kortteisto, T., Roshanov, P., Kunnamo, I., et al. (2018) · Implementation Science · doi:10.1186/s13012-018-0790-1 · Cited by 156 · see citing works
Jones, K. R., Lekhak, N., & Kaewluang, N. (2014) · Worldviews on Evidence-Based Nursing · doi:10.1111/wvn.12030 · Cited by 114 · see citing works
Yang, Q., Zimmerman, J., Steinfeld, A., Carey, L., & Antaki, J. F. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858373 · Cited by 105 · see citing works
Laka, M., Milazzo, A., & Merlin, T. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18041901 · Cited by 78 · see citing works
Nurek, M., Kostopoulou, O., Delaney, B. C., & Esmail, A. (2015) · European Journal of General Practice · doi:10.3109/13814788.2015.1043123 · Cited by 60 · see citing works
Scheepers-Hoeks, A.-M. J., Grouls, R. J., Neef, C., Ackerman, E. W., & Korsten, E. H. (2013) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2013.05.002 · Cited by 53 · see citing works
Bayor, A. A., Li, J., Yang, I. A., & Varnfield, M. (2025) · Journal of Medical Internet Research · doi:10.2196/63733 · Cited by 46 · see citing works
“Studies show that CDSS improve clinician performance [3-5], assist with medical condition management [2,6], and enhance screening procedures [7, 8].”
Participatory design of probability-based decision support tools for in-hospital nurses
Jeffery, A. D., Novak, L. L., Kennedy, B., Dietrich, M. S., & Mion, L. C. (2017) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocx060 · Cited by 46 · see citing works
Tao, L., Zhang, C., Zeng, L., Zhu, S., Li, N., Li, W., Zhang, H., Zhao, Y., Zhan, S., & Ji, H. (2020) · JMIR Medical Informatics · doi:10.2196/16912 · Cited by 42 · see citing works
Applications of Electronic Health Information in Public Health: Uses, Opportunities and Barriers
Tomines, A., Readhead, H., Readhead, A., & Teutsch, S. (2013) · eGEMs (Generating Evidence & Methods to improve patient outcomes) · doi:10.13063/2327-9214.1019 · Cited by 40 · see citing works
Spithoff, S., Mathieson, S., Sullivan, F., Guan, Q., Sud, A., Hum, S., & O'Brien, M. A. (2020) · The Journal of the American Board of Family Medicine · doi:10.3122/jabfm.2020.04.190199 · Cited by 25 · see citing works
Chen, J., Cutrona, S. L., Dharod, A., Moses, A., Bridges, A., Ostasiewski, B., Foley, K. L., Houston, T. K., iDAPT (Implementation & Informatics Developing Adaptable Processes and Technologies for Cancer Control) Implementation Science Center for Cancer Control, Chen, J., et al. (2025) · JAMIA Open · doi:10.1093/jamiaopen/ooaf050 · Cited by 3 · see citing works
- Authors
- Jaspers, M. W. M., Smeulers, M., Vermeulen, H., & Peute, L. W.
- Affiliations
- Amsterdam UMC Location University of Amsterdam (NL)
- Corresponding author
- Jaspers, M. W. M.
- Venue
- Journal of the American Medical Informatics Association
- Publisher
- Oxford University Press (OUP)
- Published
- 2011-05-01
- Volume
- 18
- Issue
- 3
- Pages
- 327-334
- ISSN
- 1067-5027 (ISSN-L)
- 1527-974X
- PMC
- PMC3078663
- OpenAlex
- W2107318542
- Cited by
- 525
- References
- 6
- Citations by year
- 2026: 13 · 2025: 27 · 2024: 30 · 2023: 42 · 2022: 36 · 2021: 43 · 2020: 37 · 2019: 40 · 2018: 34 · 2017: 38 · 2016: 38 · 2015: 47 · 2014: 31 · 2013: 40 · 2012: 19
- FWCI
- 4.91
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Journal of the American Medical Informatics Association (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- UvA-DARE (University of Amsterdam) (publishedVersion, other-oa)
- UvA-DARE (University of Amsterdam) (submittedVersion, other-oa)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- Pure Amsterdam UMC (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
- Topics
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Family Practice › Clinical Reasoning and Diagnostic Skills
- Keywords
- clinical decision support systems, patient outcomes, drug ordering, hospital settings, quality of evidence, AMSTAR, methodological quality of systematic reviews
- MeSH terms
- Clinical Competence, Humans, Therapy, Computer-Assisted, Reminder Systems, Outcome Assessment, Health Care, Decision Support Systems, Clinical, Medical Order Entry Systems, Electronic Prescribing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
