Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 91 of the 91 references it cites were found in a registry or an index.

Evaluating informatics applications – clinical decision support systems literature review

Kaplan, Bonnie · 2001 · International Journal of Medical Informatics

Abstract

This paper reviews clinical decision support systems (CDSS) literature, with a focus on evaluation. The literature indicates a general consensus that clinical decision support systems are thought to have the potential to improve care. Evidence is more equivocal for guidelines and for systems to aid physicians with diagnosis. There also is general consensus that a variety of systems are little used despite demonstrated or potential benefits. In the evaluation literature, the main emphasis is on how clinical performance changes. Most studies use an experimental or randomized controlled clinical trials design (RCT) to assess system performance or to focus on changes in clinical performance that could affect patient care. Few studies involve field tests of a CDSS and almost none use a naturalistic design in routine clinical settings with real patients. In addition, there is little theoretical discussion, although papers are permeated by a rationalist perspective that excludes contextual issues related to how and why systems are used. The studies mostly concern physicians rather than other clinicians. Further, CDSS evaluation studies appear to be insulated from evaluations of other informatics applications. Consequently, there is a lack of information useful for understanding why CDSSs may or may not be effective, resulting in making less informed decisions about these technologies and, by extension, other medical informatics applications.

Overview

Integrity signals

Registry

Crossref · 10.1016/s1386-5056(01)00183-6

Cites

91 of 91 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 91 references

Research works printed but not linked: 4

  1. R.D. Zielstorff, M.J. Teich, M.D. Paterno, M. Segal, P-CAPE: A high-level tool for entering and processing clinical practice guidelines, Proc. AMIA Symp. 1998; 478–82.
  2. R.P. van der Loo, Overview of published assessment and evaluation studies. In: E.M.S. van Gennip, J.L. Talmon (Eds.), Assessment and evaluation of information technologies, Studies in Health Technology and Informatics, vol. 17. Amsterdam, IOS Press, 1995, pp. 261–282.
  3. L.A. Moore, Reaction of medical personnel to a medical expert system for stroke. In: J.G. Anderson, C.E. Aydin, S.J. Jay (Eds.), Evaluating health care information systems: approaches and applications. Sage, Thousand Oaks, CA, 1994, pp. 226–244.
  4. R.R. Weaver, Evaluating the Problem Knowledge Coupler: a case study. In: J.G. Anderson, C.E. Aydin, S.J. Jay (Eds.), Evaluating health care information systems: approaches and applications. Sage, Thousand Oaks, CA, 1994, pp. 203–225.

Cited by

See all 350

Metadata

Authors
Kaplan, B.
Affiliations
Yale University (US)
Corresponding author
Kaplan, Bonnie
Venue
International Journal of Medical Informatics
Publisher
Elsevier BV
Published
2001-11
Volume
64
Issue
1
Pages
15-37
ISSN
1386-5056 (ISSN-L)
OpenAlex
W2136375912
Cited by
350
References
91
Citations by year
2026: 2 · 2025: 4 · 2024: 10 · 2023: 3 · 2022: 12 · 2021: 10 · 2020: 9 · 2019: 7 · 2018: 14 · 2017: 13 · 2016: 19 · 2015: 12 · 2014: 13 · 2013: 27 · 2012: 24
FWCI
32.45
Type
journal article
Language
en
Open access
closed
Licence
www.elsevier.com/tdm/userlicense/1.0/ (tdm)
Locations
  • International Journal of Medical Informatics (publishedVersion)
  • PubMed (publishedVersion)
Funders
National Institute of Standards and Technology
Topics
  • Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Health Sciences › Medicine › Family Practice › Clinical Reasoning and Diagnostic Skills
Keywords
clinical decision support systems, randomized controlled trials, field testing, clinical performance, guideline adherence, diagnostic support, technology adoption
MeSH terms
Humans, Medical Informatics, Quality of Health Care, Randomized Controlled Trials as Topic, Practice Guidelines as Topic, Decision Support Systems, Clinical

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Evaluating informatics applications – clinical decision support systems literature review. https://app.cochat.ai/papers/10.1016/s1386-5056(01)00183-6
The paper
Kaplan, B. (2001). Evaluating informatics applications – clinical decision support systems literature review. International Journal of Medical Informatics. https://doi.org/10.1016/s1386-5056(01)00183-6

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.