Using Commercial Knowledge Bases for Clinical Decision Support: Opportunities, Hurdles, and Recommendations
Open accessThe DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.
Abstract
The quality and safety of health care leaves much to be desired.1,2 Automated clinical decision support (CDS) tools embedded in clinical information systems (CISs) such as computer provider order entry (CPOE) and electronic health records (EHR) applications have the potential to improve care and should be part of any comprehensive approach to improve quality.3,4,5,6 Medication prescribing is a component of health care with well documented quality and safety problems that can be improved by CDS.7,8,9 Medication-related CDS requires that pharmaceutical knowledge be represented in a computable, explicit and unambiguous form. Creating an automated representation of medical knowledge often is the most time consuming step in the development of a CDS system and is known as the “knowledge acquisition bottleneck.”10 For a time, it was hoped that the move toward explicit guidelines in medicine would decrease the knowledge acquisition effort,11 but that has not happened.12 Experiments on data sharing from over a decade ago have not progressed.13 As a result, just a few organizations, primarily academic medical centers, are creating rules and benefiting from CDS,14 but most health care organizations do not have the expertise or resources to create such knowledge bases themselves. One potential solution to the problem of access to automated medication-related knowledge is the set of commercial vendors that supply medication-related knowledge bases for pharmacy and prescribing applications. These vendors' products contain such knowledge as drug-drug and drug-disease interactions, minimum and maximum dosing suggestions, drug-allergy cross-sensitivity groupings, and groupings of medications by therapeutic class. Developers of CISs (either vendor-based or “homegrown”), with appropriate licensing, can incorporate commercial knowledge bases into their products. The knowledge base vendor receives a licensing fee for each CIS implementation and can amortize the costs of knowledge creation over several clients. Any given health care organization pays just a fraction of the total knowledge development costs and is spared the effort of developing the knowledge itself. The CIS developers can create CDS features based on the commercial knowledge base, for example, a drug-drug interaction checking feature within a CPOE application. To the end-user of the CIS, it may not be evident that the CIS application and the knowledge contained therein are two distinct components. The CIS developer may provide end-user configuration tools that allow certain decision support features to be turned on and off. However, several end-user organizations have reported that, when used as the basis for CDS within clinical systems, commercial knowledge bases generate excessive number of alerts, many of which are clinically unhelpful.15 Specific examples include overly sensitive drug-allergy interaction checking,16 drug-drug interaction checking,17 and dose limit checks.18 Such frequent alerts are bothersome to practicing clinicians who deem them to be “nuisance” or “nonsense” alerts. Such nuisance alerts are problematic in various ways. First, they are not helpful in the particular circumstance. Second, they degrade clinician confidence in the entire category of alerts (e.g., a clinically irrelevant maximum dose alert can decrease clinicians' confidence in all maximum dose alerts). Third, excessive nuisance alerts can decrease confidence in the alerting system and CDS as a whole and can cause clinicians to ignore clinically relevant alerts. Fourth, inappropriate CDS may cause clinicians to be dissatisfied with the CIS within which the CDS resides.19,20,21 Such dissatisfaction may impede the nation's progress towards the policy goal22 of rapid and broad dissemination of health information technology. Another problem with the use of commercial knowledge bases is that the tools within the CIS that allow the CDS features to be turned on and off may be inadequate to achieve the alerting functionality desired by the end-user. For example, the tools may permit all class-based duplicate medication alerts to be turned off, but the same tools may not have the capability to turn off the alerting feature for some medications (e.g., antibiotics) while leaving it on for others (e.g., potassium preparations). The result of all these problems is that, in at least some instances, decision support features in CISs that are based on commercial medication knowledge bases have not been well received by the clinician community.15,19 Perhaps it should not be surprising that the use of commercial knowledge bases within CISs results in a large number of alerts. Knowledge base vendors and CIS developers may have incentives (legal as well as financial) to point out when any possible breach of reasonable behavior has occurred. A naïve perspective may be that “any information is good” and some parties may not appreciate the costs of false positive alerts and the human factors involved
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The quality of health care delivered to adults in the United States
McGlynn, E. A., Asch, S. M., Adams, J., Keesey, J., Hicks, J., DeCristofaro, A., & Kerr, E. A. (2003) · New England Journal of Medicine · doi:10.1056/nejmsa022615 · Cited by 5,079
Garg, A. X., Adhikari, N. K. J., McDonald, H., Rosas-Arellano, M. P., Devereaux, P. J., Beyene, J., Sam, J., & Haynes, R. B. (2005) · JAMA · doi:10.1001/jama.293.10.1223 · Cited by 2,936
Role of Computerized Physician Order Entry Systems in Facilitating Medication Errors
Koppel, R. (2005) · JAMA · doi:10.1001/jama.293.10.1197 · Cited by 2,126
Ash, J. S. (2003) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1471 · Cited by 1,789
Systems Analysis of Adverse Drug Events
Leape, L. L. (1995) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.1995.03530010049034 · Cited by 1,404
Bates, D. W., Kuperman, G. J., Wang, S., Gandhi, T., Kittler, A., Volk, L., Spurr, C., Khorasani, R., Tanasijevic, M., & Middleton, B. (2003) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1370 · Cited by 1,346
Self-citationComputer Technology and Clinical Work: Still Waiting for Godot
Wears, R. L., & Berg, M. (2005) · JAMA · doi:10.1001/jama.293.10.1261 · Cited by 448 · cited 1×
“Such frequent alerts are bothersome to practicing clinicians who deem them to be “nuisance” or “nonsense” alerts. Such nuisance alerts are problematic in various ways. First, they are not helpful in the particular circumstance. Second, they degrade clinician confidence in the entire category of alerts (e.g., a clinica…”
BackgroundComputer Physician Order Entry: Benefits, Costs, and Issues
Kuperman, G. J., & Gibson, R. F. (2003) · Annals of Internal Medicine · doi:10.7326/0003-4819-139-1-200307010-00010 · Cited by 438 · cited 1×
“As a result, just a few organizations, primarily academic medical centers, are creating rules and benefiting from CDS,”
BackgroundSelf-citationImproving Acceptance of Computerized Prescribing Alerts in Ambulatory Care
Shah, N. R., Seger, A. C., Seger, D. L., Fiskio, J. M., Kuperman, G. J., Blumenfeld, B., Recklet, E. G., Bates, D. W., & Gandhi, T. K. (2006) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1868 · Cited by 353 · cited 1×
“Examples of such filters and other local customizations include excluding certain drug-allergy cross-sensitivity rules,(16) increasing maximum drug dose levels,(18) and filtering out certain drug-drug interactions.(23) In addition, these organizations have found that with just a moderate amount of additional knowledge…”
BackgroundSelf-citationHsieh, T. C., Kuperman, G. J., Jaggi, T., Hojnowski-Diaz, P., Fiskio, J., Williams, D. H., Bates, D. W., & Gandhi, T. K. (2004) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1556 · Cited by 232
Self-citation
Cited by
Medication-related clinical decision support in computerized provider order entry systems: a review
Kuperman, G. J., Bobb, A., Payne, T. H., Avery, A. J., Gandhi, T. K., Burns, G., Classen, D. C., & Bates, D. W. (2007) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2170 · Cited by 763
“Application vendors’ tools should allow provider organizations to make local customizations to the knowledge base, and these customizations should survive vendor product upgrades.(13) Application vendors should encourage the research required to improve the quality and breadth of currently available drug information d…”
InfluentialSelf-citationTurning Off Frequently Overridden Drug Alerts: Limited Opportunities for Doing It Safely
van der Sijs, H., Aarts, J., van Gelder, T., Berg, M., & Vulto, A. (2008) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2311 · Cited by 175
“national standard for drug interaction information that could be locally customized, and included: (1) generic names of interacting drugs, (2) a brief human-readable but computable standard set of descriptions for the clinical nature of the interactions, (3) an indication of the strength of the evidence base for the i…”
BackgroundLin, C.-P., Payne, T. H., Nichol, W. P., Hoey, P. J., Anderson, C. L., & Gennari, J. H. (2008) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2453 · Cited by 110
“who recommend that drug knowledge base creators need to provide the necessary tools to understand, customize and share rule information and that organizations need to create policy and procedure infrastructure to support the use of these tools.(12)”
BackgroundThe state of the art in clinical knowledge management: an inventory of tools and techniques
Sittig, D. F., Wright, A., Simonaitis, L., Carpenter, J. D., Allen, G. O., Doebbeling, B. N., Sirajuddin, A. M., Ash, J. S., & Middleton, B. (2010) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2009.09.003 · Cited by 101
Simpao, A. F., Ahumada, L. M., Desai, B. R., Bonafide, C. P., Gálvez, J. A., Rehman, M. A., Jawad, A. F., Palma, K. L., & Shelov, E. D. (2015) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2013-002538 · Cited by 94
Genco, E. K., Forster, J. E., Flaten, H., Goss, F., Heard, K. J., Hoppe, J., & Monte, A. A. (2016) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2015.09.020 · Cited by 66
Field, T. S., Rochon, P., Lee, M., Gavendo, L., Subramanian, S., Hoover, S., Baril, J., & Gurwitz, J. (2008) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2589 · Cited by 60
Metadata
- Authors
- Kuperman, G. J., Reichley, R. M., & Bailey, T. C.
- Venue
- Journal of the American Medical Informatics Association
- Publisher
- Oxford University Press (OUP)
- Published
- 2006-07-01
- Volume
- 13
- Issue
- 4
- Pages
- 369-371
- ISSN
- 1067-5027 (ISSN-L)
- 1527-974X
- OpenAlex
- W2188589943
- Cited by
- 78
- References
- 12
- FWCI
- 15.04
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.