The DOI matches its Crossref record; 1 of the 1 references it cites were found in a registry or an index.
Effects of Computer-Based Clinical Decision Support Systems on Physician Performance and Patient Outcomes: A Systematic Review
Abstract
CONTEXT: Many computer software developers and vendors claim that their systems can directly improve clinical decisions. As for other health care interventions, such claims should be based on careful trials that assess their effects on clinical performance and, preferably, patient outcomes. OBJECTIVE: To systematically review controlled clinical trials assessing the effects of computer-based clinical decision support systems (CDSSs) on physician performance and patient outcomes. DATA SOURCES: We updated earlier reviews covering 1974 to 1992 by searching the MEDLINE, EMBASE, INSPEC, SCISEARCH, and the Cochrane Library bibliographic databases from 1992 to March 1998. Reference lists and conference proceedings were reviewed and evaluators of CDSSs were contacted. STUDY SELECTION: Studies were included if they involved the use of a CDSS in a clinical setting by a health care practitioner and assessed the effects of the system prospectively with a concurrent control. DATA EXTRACTION: The validity of each relevant study (scored from 0-10) was evaluated in duplicate. Data on setting, subjects, computer systems, and outcomes were abstracted and a power analysis was done on studies with negative findings. DATA SYNTHESIS: A total of 68 controlled trials met our criteria, 40 of which were published since 1992. Quality scores ranged from 2 to 10, with more recent trials rating higher (mean, 7.7) than earlier studies (mean, 6.4) (P<.001). Effects on physician performance were assessed in 65 studies and 43 found a benefit (66%). These included 9 of 15 studies on drug dosing systems, 1 of 5 studies on diagnostic aids, 14 of 19 preventive care systems, and 19 of 26 studies evaluating CDSSs for other medical care. Six of 14 studies assessing patient outcomes found a benefit. Of the remaining 8 studies, only 3 had a power of greater than 80% to detect a clinically important effect. CONCLUSIONS: Published studies of CDSSs are increasing rapidly, and their quality is improving. The CDSSs can enhance clinical performance for drug dosing, preventive care, and other aspects of medical care, but not convincingly for diagnosis. The effects of CDSSs on patient outcomes have been insufficiently studied.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Tierney, W. M., Hui, S. L., & McDonald, C. J. (1986) · Medical Care · doi:10.1097/00005650-198608000-00001 · Cited by 337
Cited by
From best evidence to best practice: effective implementation of change in patients' care
Grol, R., & Grimshaw, J. (2003) · The Lancet · doi:10.1016/s0140-6736(03)14546-1 · Cited by 4,531
Garg, A. X., Adhikari, N. K. J., McDonald, H., Rosas-Arellano, M. P., Devereaux, P. J., Beyene, J., Sam, J., & Haynes, R. B. (2005) · JAMA · doi:10.1001/jama.293.10.1223 · Cited by 2,936
Self-citationEvaluating non-randomised intervention studies
Deeks, J., Dinnes, J., D'Amico, R., Sowden, A., Sakarovitch, C., Song, F., Petticrew, M., & Altman, D. (2003) · Health Technology Assessment · doi:10.3310/hta7270 · Cited by 2,907
Kawamoto, K., Houlihan, C. A., Balas, E. A., & Lobach, D. F. (2005) · BMJ · doi:10.1136/bmj.38398.500764.8f · Cited by 2,703
(2001) · PubMed · Cited by 1,709
Diagnostic error in internal medicine
Graber, M. L., Franklin, N., & Gordon, R. (2005) · Archives of Internal Medicine · doi:10.1001/archinte.165.13.1493 · Cited by 1,549
Improving Safety with Information Technology
Bates, D. W., & Gawande, A. A. (2003) · New England Journal of Medicine · doi:10.1056/nejmsa020847 · Cited by 1,544
Bates, D. W., Kuperman, G. J., Wang, S., Gandhi, T., Kittler, A., Volk, L., Spurr, C., Khorasani, R., Tanasijevic, M., & Middleton, B. (2003) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1370 · Cited by 1,346
Kaushal, R., Shojania, K. G., & Bates, D. W. (2003) · Archives of Internal Medicine · doi:10.1001/archinte.163.12.1409 · Cited by 30
The Impact of Computerized Physician Order Entry on Medication Error Prevention
Bates, D. W., Teich, J. M., Lee, J., Seger, D., Kuperman, G. J., Ma'Luf, N., Boyle, D., & Leape, L. (1999) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.1999.00660313 · Cited by 1,260
Effect of Clinical Decision-Support Systems: A Systematic Review
Bright, T. J., Wong, A., Dhurjati, R., Bristow, E., Bastian, L., Coeytaux, R. R., Samsa, G., Hasselblad, V., Williams, J. W., Musty, M. D., et al. (2012) · Annals of Internal Medicine · doi:10.7326/0003-4819-157-1-201207030-00450 · Cited by 1,200
Overriding of Drug Safety Alerts in Computerized Physician Order Entry
van der Sijs, H., Aarts, J., Vulto, A., & Berg, M. (2006) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1809 · Cited by 1,117
Poissant, L. (2005) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1700 · Cited by 901
A cost-benefit analysis of electronic medical records in primary care
Wang, S. J., Middleton, B., Prosser, L. A., Bardon, C. G., Spurr, C. D., Carchidi, P. J., Kittler, A. F., Goldszer, R. C., Fairchild, D. G., Sussman, A. J., et al. (2003) · The American Journal of Medicine · doi:10.1016/s0002-9343(03)00057-3 · Cited by 740
Clinical decision support systems for the practice of evidence-based medicine
Sim, I., Gorman, P., Greenes, R. A., Haynes, R. B., Kaplan, B., Lehmann, H., & Tang, P. C. (2001) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2001.0080527 · Cited by 732
Self-citationJaspers, M. W. M., Smeulers, M., Vermeulen, H., & Peute, L. W. (2011) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000094 · Cited by 525
A Computerized Reminder System to Increase the Use of Preventive Care for Hospitalized Patients
Dexter, P. R., Perkins, S., Overhage, J. M., Maharry, K., Kohler, R. B., & McDonald, C. J. (2001) · New England Journal of Medicine · doi:10.1056/nejmsa010181 · Cited by 519
Costs and benefits of health information technology.
Shekelle, P. G., Morton, S. C., & Keeler, E. B. (2006) · doi:10.23970/ahrqepcerta132 · Cited by 501
Electronic Health Record Use and the Quality of Ambulatory Care in the United States
Linder, J. A., Ma, J., Bates, D. W., Middleton, B., & Stafford, R. S. (2007) · Archives of Internal Medicine · doi:10.1001/archinte.167.13.1400 · Cited by 438
Evidence-Based Implementation of Evidence-Based Medicine
Grol, R., & Grimshaw, J. (1999) · The Joint Commission Journal on Quality Improvement · doi:10.1016/s1070-3241(16)30464-3 · Cited by 421
The Epidemiology of Prescribing Errors: The Potential Impact of Computerized Prescriber Order Entry
Bobb, A., Gleason, K., Husch, M., Feinglass, J., Yarnold, P. R., & Noskin, G. A. (2004) · Archives of Internal Medicine · doi:10.1001/archinte.164.7.785 · Cited by 359
Evaluating informatics applications – clinical decision support systems literature review
Kaplan, B. (2001) · International Journal of Medical Informatics · doi:10.1016/s1386-5056(01)00183-6 · Cited by 350
Quality of Diabetes Care in U.S. Academic Medical Centers
Grant, R. W., Buse, J. B., Meigs, J. B., & for the University HealthSystem Consortium (UHC) Diabetes Benchmarking Project Team (2005) · Diabetes Care · doi:10.2337/diacare.28.2.337 · Cited by 311
Investigating evaluation frameworks for health information systems
Yusof, M. M., Papazafeiropoulou, A., Paul, R. J., & Stergioulas, L. K. (2008) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2007.08.004 · Cited by 304
Bates, D. W., Kuperman, G. J., Rittenberg, E., Teich, J. M., Fiskio, J., Ma’luf, N., Onderdonk, A., Wybenga, D., Winkelman, J., Brennan, T. A., et al. (1999) · The American Journal of Medicine · doi:10.1016/s0002-9343(98)00410-0 · Cited by 292
Diagnostic Errors in Medicine: A Case of Neglect
Graber, M. (2005) · The Joint Commission Journal on Quality and Patient Safety · doi:10.1016/s1553-7250(05)31015-4 · Cited by 259
Harnessing the Power of Clinical Decision Support Systems: Challenges and Opportunities
Chen, Z., Liang, N., Zhang, H., Li, H., Yang, Y., Zong, X., Chen, Y., Wang, Y., & Shi, N. (2023) · Open Heart · doi:10.1136/openhrt-2023-002432 · Cited by 244
Evaluating information technology in health care: barriers and challenges
Heathfield, H., Pitty, D., & Hanka, R. (1998) · BMJ · doi:10.1136/bmj.316.7149.1959 · Cited by 226
Human-centered XAI: Developing design patterns for explanations of clinical decision support systems
Schoonderwoerd, T. A. J., Jorritsma, W., Neerincx, M. A., & van den Bosch, K. (2021) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2021.102684 · Cited by 210
The Relationship between Electronic Health Record Use and Quality of Care over Time
Zhou, L., Soran, C. S., Jenter, C. A., Volk, L. A., Orav, E. J., Bates, D. W., & Simon, S. R. (2009) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m3128 · Cited by 194
Friedman, C. P., Gatti, G. G., Franz, T. M., Murphy, G. C., Wolf, F. M., Heckerling, P. S., Fine, P. L., Miller, T. M., & Elstein, A. S. (2005) · Journal of General Internal Medicine · doi:10.1111/j.1525-1497.2005.30145.x · Cited by 187
A framework for analyzing the cognitive complexity of computer-assisted clinical ordering
Horsky, J., Kaufman, D. R., Oppenheim, M. I., & Patel, V. L. (2003) · Journal of Biomedical Informatics · doi:10.1016/s1532-0464(03)00062-5 · Cited by 137
Thomsen, T. F., Davidsen, M., Ibsen, H., Jorgensen, T., Jensen, G., & Borch-Johnsen, K. (2001) · European Journal of Cardiovascular Prevention & Rehabilitation · doi:10.1177/174182670100800508 · Cited by 127
Opportunities for and challenges of computerisation
Powsner, S. M., Wyatt, J. C., & Wright, P. (1998) · The Lancet · doi:10.1016/s0140-6736(98)08309-3 · Cited by 119
Human Factors Barriers to the Effective Use of Ten HIV Clinical Reminders
Patterson, E. S., Nguyen, A. D., Halloran, J. P., & Asch, S. M. (2004) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1364 · Cited by 115
Mahadevaiah, G., RV, P., Bermejo, I., Jaffray, D., Dekker, A., & Wee, L. (2020) · Medical Physics · doi:10.1002/mp.13562 · Cited by 114
Wells, S., Furness, S., Rafter, N., Horn, E., Whittaker, R., Stewart, A., Moodabe, K., Roseman, P., Selak, V., Bramley, D., et al. (2008) · European Journal of Cardiovascular Prevention & Rehabilitation · doi:10.1097/hjr.0b013e3282f13af4 · Cited by 109
Ash, J. S., Sittig, D. F., Guappone, K. P., Dykstra, R. H., Richardson, J., Wright, A., Carpenter, J., McMullen, C., Shapiro, M., Bunce, A., et al. (2012) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-12-6 · Cited by 103
Dreiseitl, S., & Binder, M. (2005) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2004.07.007 · Cited by 102
Hicks, L. S., Sequist, T. D., Ayanian, J. Z., Shaykevich, S., Fairchild, D. G., Orav, E. J., & Bates, D. W. (2008) · Journal of General Internal Medicine · doi:10.1007/s11606-007-0403-1 · Cited by 93
(2006) · PubMed · Cited by 89
Clinical decision support systems in the pediatric intensive care unit*
Mack, E. H., Wheeler, D. S., & Embi, P. J. (2009) · Pediatric Critical Care Medicine · doi:10.1097/pcc.0b013e3181936b23 · Cited by 76
Aronsky, D., & Haug, P. J. (2000) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2000.0070055 · Cited by 74
Rosenbloom, S. T., Geissbuhler, A. J., Dupont, W. D., Giuse, D. A., Talbert, D. A., Tierney, W. M., Plummer, W. D., Stead, W. W., & Miller, R. A. (2005) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1627 · Cited by 71
the eCRT Research Team, McDermott, L., Yardley, L., Little, P., Ashworth, M., & Gulliford, M. (2010) · BMC Family Practice · doi:10.1186/1471-2296-11-90 · Cited by 62
The case for randomized controlled trials to assess the impact of clinical information systems
Liu, J. L. Y., & Wyatt, J. C. (2011) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2010.010306 · Cited by 66
Flottorp, S., & Oxman, A. D. (2003) · BMC Health Services Research · doi:10.1186/1472-6963-3-3 · Cited by 62
Mold, J. W., Aspy, C. A., & Nagykaldi, Z. (2008) · The Journal of the American Board of Family Medicine · doi:10.3122/jabfm.2008.04.080006 · Cited by 61
Computerized Physician Order Entry and Online Decision Support
Handler, J. A., Feied, C. F., Coonan, K., Vozenilek, J., Gillam, M., Peacock, P. R., Sinert, R., & Smith, M. S. (2004) · Academic Emergency Medicine · doi:10.1197/j.aem.2004.08.007 · Cited by 58
(2006) · PubMed Central · Cited by 53
Zakane, S. A., Gustafsson, L. L., Tomson, G., Loukanova, S., Sié, A., Nasiell, J., & Bastholm-Rahmner, P. (2014) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2014.01.013 · Cited by 48
Sintchenko, V., Magrabi, F., & Tipper, S. (2007) · Medical Informatics and the Internet in Medicine · doi:10.1080/14639230701447701 · Cited by 47
Ash, J. S., McCormack, J. L., Sittig, D. F., Wright, A., McMullen, C., & Bates, D. W. (2012) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000705 · Cited by 41
From Physician to Consumer: The Effectiveness of Strategies to Manage Health Care Utilization
Flynn, K. E., Smith, M. A., & Davis, M. K. (2002) · Medical Care Research and Review · doi:10.1177/107755802237811 · Cited by 36
The Effect of Computerized Provider Order Entry on Medical Student Clerkship Experiences
Knight, A. M. (2005) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1839 · Cited by 34
Artificial Intelligence to Improve Antibiotic Prescribing: A Systematic Review
Amin, D., Garzόn-Orjuela, N., Garcia Pereira, A., Parveen, S., Vornhagen, H., & Vellinga, A. (2023) · Antibiotics · doi:10.3390/antibiotics12081293 · Cited by 34
Maurice, P., Dhombres, F., Blondiaux, E., Friszer, S., Guilbaud, L., Lelong, N., Khoshnood, B., Charlet, J., Perrot, N., Jauniaux, E., et al. (2017) · Journal of Gynecology Obstetrics and Human Reproduction · doi:10.1016/j.jogoh.2017.03.004 · Cited by 33
Byng, R., & Jones, R. (2004) · Journal of Evaluation in Clinical Practice · doi:10.1111/j.1365-2753.2003.00439.x · Cited by 31
(2006) · PubMed · Cited by 20
Steffny, L., Dahlem, N., Reichl, L., Gisa, K., Greff, T., & Werth, D. (2023) · Frontiers in Artificial Intelligence and Applications · doi:10.3233/faia230090 · Cited by 8
Aldredge, N. H., Rodriguez, D., González, J., & Burt, D. R. (2020) · Annals of Global Health · doi:10.5334/aogh.3041 · Cited by 6
Folks, B., LeBlanc, W. G., Staton, E. W., & Pace, W. D. (2011) · Implementation Science · doi:10.1186/1748-5908-6-65 · Cited by 5
Qualitative Study of Inpatient Hospital Medicine Interruptive Alerts and Clinical Decision Support
Khan, A., & Schnipper, J. (2026) · Applied Clinical Informatics · doi:10.1055/a-2955-9555
Metadata
- Authors
- Hunt, D. L., Haynes, R. B., Hanna, S. E., & Smith, K.
- Venue
- JAMA
- Publisher
- American Medical Association (AMA)
- Published
- 1998-10-21
- Volume
- 280
- Issue
- 15
- Pages
- 1339
- ISSN
- 0098-7484
- OpenAlex
- W2798513298
- Cited by
- 1,669
- References
- 1
- FWCI
- 149.77
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.