Grand challenges in clinical decision support
Open accessAbstract
There is a pressing need for high-quality, effective means of designing, developing, presenting, implementing, evaluating, and maintaining all types of clinical decision support capabilities for clinicians, patients and consumers. Using an iterative, consensus-building process we identified a rank-ordered list of the top 10 grand challenges in clinical decision support. This list was created to educate and inspire researchers, developers, funders, and policy-makers. The list of challenges in order of importance that they be solved if patients and organizations are to begin realizing the fullest benefits possible of these systems consists of: improve the human-computer interface; disseminate best practices in CDS design, development, and implementation; summarize patient-level information; prioritize and filter recommendations to the user; create an architecture for sharing executable CDS modules and services; combine recommendations for patients with co-morbidities; prioritize CDS content development and implementation; create internet-accessible clinical decision support repositories; use freetext information to drive clinical decision support; mine large clinical databases to create new CDS. Identification of solutions to these challenges is critical if clinical decision support is to achieve its potential and improve the quality, safety and efficiency of healthcare.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
A key challenge with clinical decision support is knowledge management as guidelines and best practices change over time, as well as governance.
“A key challenge with clinical decision support is knowledge management as guidelines and best practices change over time,40–42 as well as governance.43–45”
From Clinician collaboration to improve clinical decision support: the Clickbusters initiative (2022) · cites this paper as [41]
Section 2.3.3 identifies formal knowledge-management processes as essential and says CDS content must accommodate ongoing upgrades and new knowledge. It also discusses standards for sponsorship and trust, but our copy does not explicitly identify governance as a separate challenge. (LLM, conf=0.82)
Experimental — accuracy not yet measuredClinical decision-support systems have faced issues with adoption and effectiveness despite their promise.
“Despite their promise, clinical decision-support systems have faced issues with adoption and effectiveness [57].”
From To Recommend or Not to Recommend: Designing and Evaluating AI-Enabled Decision Support for Time-Critical Medical Events (2025) · cites this paper as [57]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Boyd, C. M., Darer, J., Boult, C., Fried, L. P., Boult, L., & Wu, A. W. (2005) · JAMA · doi:10.1001/jama.294.6.716 · Cited by 2,581 · see citing works
Miller, R. A., Waitman, L. R., Chen, S., & Rosenbloom, S. T. (2005) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2005.08.009 · Cited by 144 · see citing works
Flagged first, then by citations
Research works printed but not linked: 22
- Osheroff JA, Teich JM, Middleton B, Steen EB, Wright A, Detmer DE. A roadmap for national action on clinical decision support. J Am Med Inform Assoc 2007;14(2):141-5.
- West M, Ginsburg GS, Huang AT, Nevins JR. Embracing the complexity of genomic data for personalized medicine. Genome Res 2006;16(5):559-66.
- Chaudhry B, Wang J, Wu S, Maglione M, Mojica W, Roth E, et al. Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Ann Intern Med 2006;144(10):742-52. Table
- Sittig DF. Grand challenges in medical informatics? J Am Med Inform Assoc 1994;1(5):412-3.
- Berner ES, Moss J. Informatics challenges for the impending patient information explosion. J Am Med Inform Assoc 2005;12(6):614-7.
- Weingart SN, Toth M, Sands DZ, Aronson MD, Davis RB, Phillips RS. Physicians' decisions to override computerized drug alerts in primary care. Arch Intern Med 2003;163(21):2625-31.
- Zeng Q, Cimino JJ. A knowledge-based, concept-oriented view generation system for clinical data. J Biomed Inform 2001;34(2):112-28.
- Petersen LA, Orav EJ, Teich JM, O'Neil AC, Brennan TA. Using a computerized sign-out to improve continuity of inpatient care and prevent adverse events. Jt Comm J Qual Improv 1998;24(2):77-87.
- Durso SC. Using clinical guidelines designed for older adults with diabetes mellitus and complex health status. JAMA 2006;295(16): 1935-40.
- Hripcsak G, Friedman C, Alderson PO, DuMouchel W, Johnson SB, Clayton PD. Unlocking clinical data from narrative reports: a study of natural language processing. Ann Intern Med 1995;122(9):681-8.
- Hicks J. The potential of claims data to support the measurement of health care quality. Ph.D. dissertation. RAND Graduate School, 2003. Available as of December 20, 2004, at: http://www.rand.org/ cgibin/Abstracts/ordi/getabbydoc.pl?doc=RGSD-171.
- Safran C, Bloomrosen M, Hammond WE, Labkoff S, Markel-Fox S, Tang PC, et al. Toward a national framework for the secondary use of health data: an American Medical Informatics Association White Paper. J Am Med Inform Assoc 2007;14(1):1-9.
- Warner Jr HR. Iliad: moving medical decision-making into new frontiers. Methods Inf Med 1989;28(4):370-2.
- Ash JS, Sittig DF, Campbell E, Guappone K, Dykstra R. Some unintended consequences of clinical decision support systems. AMIA Annu Symp Proc 2007; in press.
- Dexter PR, Perkins SM, Maharry KS, Jones K, McDonald CJ. Inpatient computer-based standing orders vs physician reminders to increase influenza and pneumococcal vaccination rates: a randomized trial. JAMA 2004;292(19):2366-71.
- Leonard KJ, Sittig DF. Improving information technology adoption and implementation through the identification of appropriate bene- fits: creating IMPROVE-IT. J Med Internet Res 2007;9(2):e9.
- Thompson DI, Osheroff J, Classen D, Sittig DF. A review of methods to estimate the benefits of electronic medical records in hospitals and the need for a national benefits database. J Healthc Inf Manag 2007;21(1):62-8.
- Bates DW, Kuperman GJ, Wang S, Gandhi T, Kittler A, Volk L, et al. Ten commandments for effective clinical decision support: making the practice of evidence-based medicine a reality. J Am Med Inform Assoc 2003;10(6):523-30.
- Kawamoto K, Lobach DF. Proposal for fulfilling strategic objectives of the U.S. Roadmap for national action on clinical decision support through a service-oriented architecture leveraging HL7 services. J Am Med Inform Assoc 2007;14(2):146-55.
- Hripcsak G, Wigertz OB, Kahn MG, Clayton PD, Pryor TA. ASTM E31.15 on health knowledge representation: the Arden Syntax. Stud Health Technol Inform 1993;6:105-12.
- Wright A. SANDS: a service-oriented architecture for clinical decision support in a national health information network. Ph.D. dissertation, Oregon Health & Science University, Portland, OR; 2007.
- Simon SR, Kaushal R, Cleary PD, Jenter CA, Volk LA, Orav EJ, et al. Physicians and electronic health records: a statewide survey. Arch Intern Med 2007;167(5):507-12.
Cited by
Sorted by citations
Wearables and the Medical Revolution
Dunn, J., Runge, R., & Snyder, M. (2018) · Personalized Medicine · doi:10.2217/pme-2018-0044 · Cited by 654 · see citing works
Health information technology: fallacies and sober realities
Karsh, B.-T., Weinger, M. B., Abbott, P. A., & Wears, R. L. (2010) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2010.005637 · Cited by 365 · see citing works
Horng, S., Sontag, D. A., Halpern, Y., Jernite, Y., Shapiro, N. I., & Nathanson, L. A. (2017) · PLOS ONE · doi:10.1371/journal.pone.0174708 · Cited by 310 · see citing works
Yang, Q., Steinfeld, A., & Zimmerman, J. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300468 · Cited by 247 · see citing works
Harnessing the Power of Clinical Decision Support Systems: Challenges and Opportunities
Chen, Z., Liang, N., Zhang, H., Li, H., Yang, Y., Zong, X., Chen, Y., Wang, Y., & Shi, N. (2023) · Open Heart · doi:10.1136/openhrt-2023-002432 · Cited by 244 · see citing works
Junior physician's use of Web 2.0 for information seeking and medical education: a qualitative study
Hughes, B., Joshi, I., Lemonde, H., & Wareham, J. (2009) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2009.04.008 · Cited by 217 · see citing works
Wang, D., Wang, L., Zhang, Z., Wang, D., Zhu, H., Gao, Y., Fan, X., & Tian, F. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445432 · Cited by 209 · see citing works
Barton, C., Chettipally, U., Zhou, Y., Jiang, Z., Lynn-Palevsky, A., Le, S., Calvert, J., & Das, R. (2019) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2019.04.027 · Cited by 175 · see citing works
Jacobs, M., He, J., F. Pradier, M., Lam, B., Ahn, A. C., McCoy, T. H., Perlis, R. H., Doshi-Velez, F., & Gajos, K. Z. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445385 · Cited by 177 · see citing works
Henry, K. E., Adams, R., Parent, C., Soleimani, H., Sridharan, A., Johnson, L., Hager, D. N., Cosgrove, S. E., Markowski, A., Klein, E. Y., et al. (2022) · Nature Medicine · doi:10.1038/s41591-022-01895-z · Cited by 127 · see citing works
Mahadevaiah, G., RV, P., Bermejo, I., Jaffray, D., Dekker, A., & Wee, L. (2020) · Medical Physics · doi:10.1002/mp.13562 · Cited by 114 · see citing works
Yang, Q., Zimmerman, J., Steinfeld, A., Carey, L., & Antaki, J. F. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858373 · Cited by 105 · see citing works
The state of the art in clinical knowledge management: an inventory of tools and techniques
Sittig, D. F., Wright, A., Simonaitis, L., Carpenter, J. D., Allen, G. O., Doebbeling, B. N., Sirajuddin, A. M., Ash, J. S., & Middleton, B. (2010) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2009.09.003 · Cited by 101 · see citing works
“In a recent article, we identified “disseminate best practices in Clinical Decision Support (CDS) design, development, and implementation” as one of the “grand challenges” in CDS [7].”
Self-citationOlakotan, O. O., & Mohd Yusof, M. (2021) · Health Informatics Journal · doi:10.1177/14604582211007536 · Cited by 101 · see citing works
Wright, A., Sittig, D. F., Ash, J. S., Bates, D. W., Feblowitz, J., Fraser, G., Maviglia, S. M., McMullen, C., Nichol, W. P., Pang, J. E., et al. (2011) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2009.002030 · Cited by 92 · see citing works
Self-citationMachine learning and artificial intelligence: applications in healthcare epidemiology
Hamilton, A. J., Strauss, A. T., Martinez, D. A., Hinson, J. S., Levin, S., Lin, G., & Klein, E. Y. (2021) · Antimicrobial Stewardship & Healthcare Epidemiology · doi:10.1017/ash.2021.192 · Cited by 58 · see citing works
Wright, A., Sittig, D. F., Ash, J. S., Erickson, J. L., Hickman, T. T., Paterno, M., Gebhardt, E., McMullen, C., Tsurikova, R., Dixon, B. E., et al. (2015) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2015.08.008 · Cited by 55 · see citing works
Self-citationSanchez-Pinto, L. N., & Khemani, R. G. (2016) · Pediatric Critical Care Medicine · doi:10.1097/pcc.0000000000000750 · Cited by 52 · see citing works
Hayward, J., Thomson, F., Milne, H., Buckingham, S., Sheikh, A., Fernando, B., Cresswell, K., Williams, R., & Pinnock, H. (2013) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2012-001484 · Cited by 48 · see citing works
Clinician collaboration to improve clinical decision support: the Clickbusters initiative
McCoy, A. B., Russo, E. M., Johnson, K. B., Addison, B., Patel, N., Wanderer, J. P., Mize, D. E., Jackson, J. G., Reese, T. J., Littlejohn, S., et al. (2022) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocac027 · Cited by 41 · see citing works
Self-citationClinical decision support for therapeutic decision-making in cancer: a systematic review
Beauchemin, M., Murray, M. T., Sung, L., Hershman, D. L., Weng, C., & Schnall, R. (2019) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2019.07.019 · Cited by 38 · see citing works
McCulloh, R. J., Fouquet, S. D., Herigon, J., Biondi, E. A., Kennedy, B., Kerns, E., DePorre, A., Markham, J. L., Chan, Y. R., Nelson, K., et al. (2018) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocy069 · Cited by 36 · see citing works
Get To The Point! Problem-Based Curated Data Views To Augment Care For Critically Ill Patients
Zhang, M., Ehrmann, D., Mazwi, M., Eytan, D., Ghassemi, M., & Chevalier, F. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501887 · Cited by 16 · see citing works
Mastrianni, A., Kim, M. S., Sullivan, T. M., Sippel, G. J., Burd, R. S., Gajos, K. Z., & Sarcevic, A. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757512 · Cited by 6 · see citing works
“Despite their promise, clinical decision-support systems have faced issues with adoption and effectiveness [57].”
- Authors
- Sittig, D. F., Wright, A., Osheroff, J. A., Middleton, B., Teich, J. M., Ash, J. S., Campbell, E., & Bates, D. W.
- Affiliations
- Oregon Health & Science University (US)
- Brigham and Women's Hospital (US)
- University of Pennsylvania Health System (US)
- Corresponding author
- Sittig, Dean F.
- Venue
- Journal of Biomedical Informatics
- Publisher
- Elsevier BV
- Published
- 2008-04
- Volume
- 41
- Issue
- 2
- Pages
- 387-392
- ISSN
- 1532-0464 (ISSN-L)
- PMC
- PMC2660274
- OpenAlex
- W2097855313
- Cited by
- 582
- References
- 2
- Citations by year
- 2026: 15 · 2025: 16 · 2024: 11 · 2023: 23 · 2022: 41 · 2021: 32 · 2020: 37 · 2019: 38 · 2018: 41 · 2017: 39 · 2016: 40 · 2015: 39 · 2014: 41 · 2013: 50 · 2012: 38
- FWCI
- 2.71
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- Locations
- Journal of Biomedical Informatics (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Funders
- National Institutes of Health
- U.S. Public Health Service (ASMMI0031)
- U.S. National Library of Medicine (R01LM006942, LM06942, R56LM006942)
- Topics
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Technology and Human Factors in Education and Health
- Keywords
- clinical decision support, societal challenges, human-machine interaction, comorbidities, healthcare quality and safety
- MeSH terms
- Database Management Systems, Decision Support Techniques, User-Computer Interface, Medical Records Systems, Computerized, Decision Support Systems, Clinical, Internationality
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
