Ten commandments for effective clinical decision support: making the practice of evidence-based medicine a reality

Bates, David W.; Kuperman, Gilad J.; Wang, Samuel; Gandhi, Tejal; Kittler, Anne; Volk, Lynn; Spurr, Cynthia; Khorasani, Ramin; Tanasijevic, Milenko; Middleton, Blackford

doi:10.1197/jamia.m1370

Cited by 1,346Electronic Health Records SystemsHealth Sciences Research and EducationArtificial Intelligence in Healthcare and Education

Ten commandments for effective clinical decision support: making the practice of evidence-based medicine a reality

Open access

Bates, David W., Kuperman, Gilad J., Wang, Samuel, Gandhi, Tejal, Kittler, Anne, Volk, Lynn, Spurr, Cynthia, Khorasani, Ramin +2 more·2003·Journal of the American Medical Informatics Association

DOI 10.1197/jamia.m1370PMC PMC264429

Open this paper in CoChat — free account Open PDF

Abstract

While evidence-based medicine has increasingly broad-based support in health care, it remains difficult to get physicians to actually practice it. Across most domains in medicine, practice has lagged behind knowledge by at least several years. The authors believe that the key tools for closing this gap will be information systems that provide decision support to users at the time they make decisions, which should result in improved quality of care. Furthermore, providers make many errors, and clinical decision support can be useful for finding and preventing such errors. Over the last eight years the authors have implemented and studied the impact of decision support across a broad array of domains and have found a number of common elements important to success. The goal of this report is to discuss these lessons learned in the interest of informing the efforts of others working to make the practice of evidence-based medicine a reality.

Integrity signals

Registry

Crossref · 10.1197/jamia.m1370

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

The diagnosis could be inferred from routine EHR data, which could aid with triggering of decision support or clinical trial recruitment.
“This is a proof-of-concept that demonstrates that the diagnosis could be inferred from routine EHR data, which could aid with triggering of decision support68,72 or clinical trial recruitment.”
From Scalable and accurate deep learning with electronic health records (2018) · cites this paper as [72]
Many of the themes identified are long accepted requirements for CDS as outlined by Bates et al. in 2003, and are not unique to XAI or hospital settings as described in a review of AI adoption in healthcare.
“Many of these themes are long accepted requirements for CDS as outlined by Bates et al. in 2003 [39], and are not unique to XAI or hospital settings as described in a review of AI adoption in healthcare [40].”
From Explainable AI in hospital clinical decision support systems: A scoping review of healthcare professionals’ perspectives (2026) · cites this paper as [39]
AI systems are socio-technical: their impact depends on how clinicians interpret, trust, ignore, override, or overuse their outputs.
“Third, AI systems are socio-technical: their impact depends on how clinicians interpret, trust, ignore, override, or overuse their outputs [41].”
From The AI Hospital Formulary: A Practical Governance Framework for Prescribing, Monitoring, and Deprescribing Artificial Intelligence in Hospitals (2026) · cites this paper as [41]

Effect of computerised evidence based guidelines on management of asthma and angina in adults in primary care: cluster randomised controlled trial
Eccles, M. (2002) · BMJ · doi:10.1136/bmj.325.7370.941 · Cited by 354 · see citing works
Controlled Trial of Direct Physician Order Entry: Effects on Physicians' Time Utilization in Ambulatory Primary Care Internal Medicine Practices
Overhage, J. M., Perkins, S., Tierney, W. M., & McDonald, C. J. (2001) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2001.0080361 · Cited by 232 · see citing works

Flagged first, then by citations

Sorted by citations

An overview of clinical decision support systems: Benefits, risks, and strategies for success
Sutton, R. T., Pincock, D., Baumgart, D. C., Sadowski, D. C., Fedorak, R. N., & Kroeker, K. I. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0221-y · Cited by 3,019 · see citing works
Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success
Kawamoto, K., Houlihan, C. A., Balas, E. A., & Lobach, D. F. (2005) · BMJ · doi:10.1136/bmj.38398.500764.8f · Cited by 2,703 · see citing works
Scalable and accurate deep learning with electronic health records
Rajkomar, A., Oren, E., Chen, K., Dai, A. M., Hajaj, N., Hardt, M., Liu, P. J., Liu, X., Marcus, J., Sun, M., et al. (2018) · npj Digital Medicine · doi:10.1038/s41746-018-0029-1 · Cited by 2,648 · see citing works
Overriding of Drug Safety Alerts in Computerized Physician Order Entry
van der Sijs, H., Aarts, J., Vulto, A., & Berg, M. (2006) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1809 · Cited by 1,117 · see citing works
The Impact of Electronic Health Records on Time Efficiency of Physicians and Nurses: A Systematic Review
Poissant, L. (2005) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1700 · Cited by 901 · see citing works
Medication-related clinical decision support in computerized provider order entry systems: a review
Kuperman, G. J., Bobb, A., Payne, T. H., Avery, A. J., Gandhi, T. K., Burns, G., Classen, D. C., & Bates, D. W. (2007) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2170 · Cited by 763 · see citing works
Self-citation
Checklists to Reduce Diagnostic Errors
Ely, J. W., Graber, M. L., & Croskerry, P. (2011) · Academic Medicine · doi:10.1097/acm.0b013e31820824cd · Cited by 366 · see citing works
Improving Acceptance of Computerized Prescribing Alerts in Ambulatory Care
Shah, N. R., Seger, A. C., Seger, D. L., Fiskio, J. M., Kuperman, G. J., Blumenfeld, B., Recklet, E. G., Bates, D. W., & Gandhi, T. K. (2006) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1868 · Cited by 353 · see citing works
Self-citation
The impact of electronic health record systems on clinical documentation times: a systematic review
Baumann, L. A., Baker, J., & Elshaug, A. G. (2018) · Health Policy · doi:10.1016/j.healthpol.2018.05.014 · Cited by 320 · see citing works
Implementation and adoption of nationwide electronic health records in secondary care in England: final qualitative results from prospective national evaluation in early adopter hospitals
Sheikh, A., Cornford, T., Barber, N., Avery, A., Takian, A., Lichtner, V., Petrakaki, D., Crowe, S., Marsden, K., Robertson, A., et al. (2011) · BMJ · doi:10.1136/bmj.d6054 · Cited by 314 · see citing works
Computerized clinical decision support for prescribing: provision does not guarantee uptake
Moxey, A., Robertson, J., Newby, D., Hains, I., Williamson, M., & Pearson, S.-A. (2010) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m3170 · Cited by 305 · see citing works
What evidence supports the use of computerized alerts and prompts to improve clinicians' prescribing behavior?
Schedlbauer, A., Prasad, V., Mulvaney, C., Phansalkar, S., Stanton, W., Bates, D. W., & Avery, A. J. (2009) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2910 · Cited by 262 · see citing works
Self-citation
Harnessing the Power of Clinical Decision Support Systems: Challenges and Opportunities
Chen, Z., Liang, N., Zhang, H., Li, H., Yang, Y., Zong, X., Chen, Y., Wang, Y., & Shi, N. (2023) · Open Heart · doi:10.1136/openhrt-2023-002432 · Cited by 244 · see citing works
Do patients trust computers?
Promberger, M., & Baron, J. (2006) · Journal of Behavioral Decision Making · doi:10.1002/bdm.542 · Cited by 242 · see citing works
A lesson in implementation: A pre-post study of providers’ experience with artificial intelligence-based clinical decision support
Romero-Brufau, S., Wyatt, K. D., Boyum, P., Mickelson, M., Moore, M., & Cognetta-Rieke, C. (2020) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2019.104072 · Cited by 143 · see citing works
Exploring the persistence of paper with the electronic health record
Saleem, J. J., Russ, A. L., Justice, C. F., Hagg, H., Ebright, P. R., Woodbridge, P. A., & Doebbeling, B. N. (2009) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2009.04.001 · Cited by 145 · see citing works
The anatomy of decision support during inpatient care provider order entry (CPOE): Empirical observations from a decade of CPOE experience at Vanderbilt
Miller, R. A., Waitman, L. R., Chen, S., & Rosenbloom, S. T. (2005) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2005.08.009 · Cited by 144 · see citing works
The Development and Evaluation of an Integrated Electronic Prescribing and Drug Management System for Primary Care
Tamblyn, R., Huang, A., Kawasumi, Y., Bartlett, G., Grad, R., Jacques, A., Dawes, M., Abrahamowicz, M., Perreault, R., Taylor, L., et al. (2006) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1887 · Cited by 137 · see citing works
Using EMR-enabled computerized decision support systems to reduce prescribing of potentially inappropriate medications: a narrative review
Scott, I. A., Pillans, P. I., Barras, M., & Morris, C. (2018) · Therapeutic Advances in Drug Safety · doi:10.1177/2042098618784809 · Cited by 114 · see citing works
Public health communications and alert fatigue
Baseman, J. G., Revere, D., Painter, I., Toyoji, M., Thiede, H., & Duchin, J. (2013) · BMC Health Services Research · doi:10.1186/1472-6963-13-295 · Cited by 105 · see citing works
Do physicians value decision support? A look at the effect of decision support systems on physician opinion
Dreiseitl, S., & Binder, M. (2005) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2004.07.007 · Cited by 102 · see citing works
Clinical Decision Support and Electronic Prescribing Systems: A Time for Responsible Thought and Action
Miller, R. A., Gardner, R. M., Johnson, K. B., & Hripcsak, G. (2005) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1830 · Cited by 99 · see citing works
Governance for clinical decision support: case studies and recommended practices from leading institutions
Wright, A., Sittig, D. F., Ash, J. S., Bates, D. W., Feblowitz, J., Fraser, G., Maviglia, S. M., McMullen, C., Nichol, W. P., Pang, J. E., et al. (2011) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2009.002030 · Cited by 92 · see citing works
Self-citation
Applications of Artificial Intelligence in the Radiology Roundtrip: Process Streamlining, Workflow Optimization, and Beyond
Pierre, K., Haneberg, A. G., Kwak, S., Peters, K. R., Hochhegger, B., Sananmuang, T., Tunlayadechanont, P., Tighe, P. J., Mancuso, A., & Forghani, R. (2023) · Seminars in Roentgenology · doi:10.1053/j.ro.2023.02.003 · Cited by 79 · see citing works
Clinical decision support systems in the pediatric intensive care unit*
Mack, E. H., Wheeler, D. S., & Embi, P. J. (2009) · Pediatric Critical Care Medicine · doi:10.1097/pcc.0b013e3181936b23 · Cited by 76 · see citing works
Effect of CPOE User Interface Design on User-Initiated Access to Educational and Patient Information during Clinical Care
Rosenbloom, S. T., Geissbuhler, A. J., Dupont, W. D., Giuse, D. A., Talbert, D. A., Tierney, W. M., Plummer, W. D., Stead, W. W., & Miller, R. A. (2005) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1627 · Cited by 71 · see citing works
Decision Support System for Triage Management: A Hybrid Approach Using Rule-Based Reasoning and Fuzzy Logic
Dehghani Soufi, M., Samad-Soltani, T., Shams Vahdati, S., & Rezaei-Hachesu, P. (2018) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2018.03.008 · Cited by 70 · see citing works
Health Decision Making: Lynchpin of Evidence-Based Practice
Spring, B. (2008) · Medical Decision Making · doi:10.1177/0272989x08326146 · Cited by 61 · see citing works
A Decision-Support Tool for Renal Mass Classification
Kunapuli, G., Varghese, B. A., Ganapathy, P., Desai, B., Cen, S., Aron, M., Gill, I., & Duddalwar, V. (2018) · Journal of Digital Imaging · doi:10.1007/s10278-018-0100-0 · Cited by 58 · see citing works
Development of a Prediction Model of Early Acute Kidney Injury in Critically Ill Children Using Electronic Health Record Data
Sanchez-Pinto, L. N., & Khemani, R. G. (2016) · Pediatric Critical Care Medicine · doi:10.1097/pcc.0000000000000750 · Cited by 52 · see citing works
Physicians’ responses to clinical decision support on an intensive care unit—Comparison of four different alerting methods
Scheepers-Hoeks, A.-M. J., Grouls, R. J., Neef, C., Ackerman, E. W., & Korsten, E. H. (2013) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2013.05.002 · Cited by 53 · see citing works
‘Too much, too late’: mixed methods multi-channel video recording study of computerized decision support systems and GP prescribing
Hayward, J., Thomson, F., Milne, H., Buckingham, S., Sheikh, A., Fernando, B., Cresswell, K., Williams, R., & Pinnock, H. (2013) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2012-001484 · Cited by 48 · see citing works
Perceived Increase in Mortality After Process and Policy Changes Implemented With Computerized Physician Order Entry
Rosenbloom, S. T., Harrell, F. E., Lehmann, C. U., Schneider, J. H., Spooner, S. A., & Johnson, K. B. (2006) · Pediatrics · doi:10.1542/peds.2005-3163 · Cited by 38 · see citing works
Addressing Alert Fatigue by Replacing a Burdensome Interruptive Alert with Passive Clinical Decision Support
Fallon, A., Haralambides, K., Mazzillo, J., & Gleber, C. (2024) · Applied Clinical Informatics · doi:10.1055/a-2226-8144 · Cited by 22 · see citing works
Use of Simulation to Study Nurses’ Acceptance and Nonacceptance of Clinical Decision Support Suggestions
SOUSA, V. E. C., LOPEZ, K. D., FEBRETTI, A., STIFTER, J., YAO, Y., JOHNSON, A., WILKIE, D. J., & KEENAN, G. M. (2015) · CIN: Computers, Informatics, Nursing · doi:10.1097/cin.0000000000000185 · Cited by 18 · see citing works
Get To The Point! Problem-Based Curated Data Views To Augment Care For Critically Ill Patients
Zhang, M., Ehrmann, D., Mazwi, M., Eytan, D., Ghassemi, M., & Chevalier, F. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501887 · Cited by 16 · see citing works
Explainable AI in hospital clinical decision support systems: A scoping review of healthcare professionals’ perspectives
Van Dort, B. A., Engelsma, T., Sneekes, P., Peute, L., & Medlock, S. (2026) · PLOS Digital Health · doi:10.1371/journal.pdig.0001417 · Cited by 9 · see citing works
A model curriculum for an emergency medicine residency rotation in clinical informatics
Baker, C., Maniam, N., Schnapp, B., Genes, N., Nielson, J., Mohan, V., Hersh, W., & Slovis, B. (2022) · Journal of Education and Teaching in Emergency Medicine · doi:10.5070/m57459214 · Cited by 6 · see citing works
The AI Hospital Formulary: A Practical Governance Framework for Prescribing, Monitoring, and Deprescribing Artificial Intelligence in Hospitals
Epelde, F. (2026) · Hospitals · doi:10.3390/hospitals3030015
Qualitative Study of Inpatient Hospital Medicine Interruptive Alerts and Clinical Decision Support
Khan, A., & Schnipper, J. (2026) · Applied Clinical Informatics · doi:10.1055/a-2955-9555

Authors: Bates, D. W., Kuperman, G. J., Wang, S., Gandhi, T., Kittler, A., Volk, L., Spurr, C., Khorasani, R., Tanasijevic, M., & Middleton, B.
Affiliations: Brigham and Women's Hospital (US)
Corresponding author: Bates, David W.
ORCID: Bates, David W.
Kuperman, Gilad J.
Wang, Samuel
Volk, Lynn
Khorasani, Ramin
Venue: Journal of the American Medical Informatics Association
Publisher: Oxford University Press (OUP)
Published: 2003-11
Volume: 10
Issue: 6
Pages: 523-530
ISSN: 1067-5027 (ISSN-L)
1527-974X
DOI: 10.1197/jamia.m1370
PMC: PMC264429
OpenAlex: W2111175405
Cited by: 1,346
References: 2
Citations by year: 2026: 71 · 2025: 50 · 2024: 49 · 2023: 52 · 2022: 53 · 2021: 66 · 2020: 70 · 2019: 72 · 2018: 70 · 2017: 58 · 2016: 69 · 2015: 84 · 2014: 57 · 2013: 81 · 2012: 70
FWCI: 61.90
Type: journal article
Language: en
Open access: bronze
doi.org
Locations: Journal of the American Medical Informatics Association (publishedVersion)
PubMed Central (publishedVersion)
PubMed (publishedVersion)
Funders: Agency for Healthcare Research and Quality (R01 HS07107)
Topics: Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
Health Sciences › Health Professions › General Health Professions › Health Sciences Research and Education
Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
Keywords: clinical decision support, evidence-based medicine, evidence-based practice implementation, clinical information systems, medical errors, error prevention, quality of care, lessons learned, knowledge-practice gap
MeSH terms: Decision Making, Computer-Assisted, Decision Support Techniques, Humans, User-Computer Interface, Evidence-Based Medicine, Decision Support Systems, Clinical

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review