Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 94 of the 94 references it cites were found in a registry or an index.

Evaluating informatics applications – some alternative approaches: theory, social interactionism, and call for methodological pluralism.

Kaplan, Bonnie · 2001 · International Journal of Medical Informatics

Abstract

A review of evaluation literature concerning CDSSs indicates that randomized controlled clinical trials (RCTs) are the 'gold standard' for evaluation. While this approach is excellent for studying system or clinical performance, it is not well suited to answering questions concerning whether systems will be used or how they will be used. Because lack of use of CDSS has been of concern for some years, other evaluation research designs are needed to address those issues. This paper critiques RCT and experimental evaluation approaches and presents alternative approaches to evaluation that address questions outside the scope of the usual RCT and experimental designs. A wide range of literature is summarized to illustrate the value of evaluations that take into account social, organizational, professional, and other contextual considerations. Many of these studies go beyond the usual measures of systems performance or physicians' behavior by focusing on 'fit' of the system with other aspects of professional and organizational life. Because there is little explicit theory that informs many evaluations, the paper then reviews CDSS evaluations informed by social science theories. Lastly, it proposes a theoretical social science base of social interactionism. An example of such an approach is given. It involves a CDSS in psychiatry and is based on Kaplan's 4Cs, which focus on communication, control, care, and context. Although the example is a CDSS, the evaluation approach also is use…

Overview

Integrity signals

Registry

Crossref · 10.1016/s1386-5056(01)00184-8

Cites

94 of 94 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

5 more claims about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Citations

Cites

Show all 93 references

Research works printed but not linked: 11

  1. J.G. Anderson, C.E. Aydin, S.J. Jay (Eds.), Evaluating health care information systems: approaches and applications. Sage, Thousand Oaks, CA, 1994.
  2. E.M.S. van Gennip, J.L. Talmon (Eds.), Assessment and evaluation of information technologies, Studies in Health Technology and Informatics, vol. 17, Amsterdam IOS Press, 1995
  3. F. Grémy, M. Bonnin, Evaluation of automatic health information systems. What and how?, In: E.M.S. van Gennip, J.L. Talmon (Eds.), Assessment and evaluation of information technologies, Studies in Health Technology and Informatics, vol 17, Amsterdam IOS Press, 1995, pp. 9–20.
  4. T. Jørgensen, Measuring effects, In: E.M.S. van Gennip, J.L. Talmon (Eds.), Assessment and evaluation of information technologies, Studies in Health Technology and Informatics, vol. 17, Amsterdam IOS Press, 1995, pp. 99–109.
  5. C. Aydin, Computerized order entry in a large medical center: evaluating interactions between departments, in: J.G., Anderson, C.E., Aydin, S.J., Jay, (Eds.), Evaluating health care information systems: approaches and applications, Sage, Thousand Oaks, CA, 1994, pp. 260–275.
  6. Covvey, 2001, When is a system only a system?, Healthcare Inform. Manage. Commun. Canada, 15, 46
  7. Kaplan, 2001, Computer rorschach test: what do you see when you look at a computer?, Physicians Comput., 18, 12
  8. L.A. Moore, Reaction of medical personnel to a medical expert system for stroke. In: J.G. Anderson, C.E. Aydin, S.J. Jay (Eds.), Evaluating health care information systems: Approaches and applications, Sage, Thousand Oaks, CA, 1994, pp. 226–244.
  9. E.M. Rogers, Diffusion of innovations, third ed., The Free Press, New York, 1983.
  10. R.E. Rice, J.G. Anderson, Social networks and health care information systems: a structural approach to evaluation. In: J.G. Anderson, C.E. Aydin, S.J. Jay (Eds.), Evaluating health care information systems: approaches and applications, Sage, Thousand Oaks, CA, 1994, pp. 135–163.
  11. R.R. Weaver, Evaluating the Problem Knowledge Coupler: a case study. In: J.G. Anderson, C.E. Aydin, S.J. Jay (Eds.), Evaluating health care information systems: approaches and applications, Sage, Thousand Oaks, CA, 1994, pp. 203–225.

Cited by

See all 381

Metadata

Authors
Kaplan, B.
Affiliations
Yale University (US)
Corresponding author
Kaplan, Bonnie
Venue
International Journal of Medical Informatics
Publisher
Elsevier BV
Published
2001-11
Volume
64
Issue
1
Pages
39-56
ISSN
1386-5056 (ISSN-L)
OpenAlex
W2090760643
Cited by
380
References
93
Citations by year
2026: 1 · 2025: 4 · 2024: 4 · 2023: 5 · 2022: 11 · 2021: 5 · 2020: 9 · 2019: 12 · 2018: 13 · 2017: 10 · 2016: 15 · 2015: 22 · 2014: 20 · 2013: 16 · 2012: 21
FWCI
39.51
Type
journal article
Language
en
Open access
closed
Licence
www.elsevier.com/tdm/userlicense/1.0/ (tdm)
Locations
  • International Journal of Medical Informatics (publishedVersion)
  • PubMed (publishedVersion)
Funders
National Institute of Standards and Technology
Topics
  • Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
  • Social Sciences › Business, Management and Accounting › Management Information Systems › Information Systems Theories and Implementation
  • Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
Keywords
clinical decision support systems, randomized controlled trials, methodological pluralism, social interactionism, social science theory, system adoption, system use, contextual factors, guideline implementation, organizational context
MeSH terms
Attitude, Humans, Quality of Health Care, Social Conditions, User-Computer Interface, Organizational Culture, Program Evaluation, Randomized Controlled Trials as Topic, Practice Guidelines as Topic, Decision Support Systems, Clinical

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Evaluating informatics applications – some alternative approaches: theory, social interactionism, and call for methodological pluralism. https://app.cochat.ai/papers/10.1016/s1386-5056(01)00184-8
The paper
Kaplan, B. (2001). Evaluating informatics applications – some alternative approaches: theory, social interactionism, and call for methodological pluralism. International Journal of Medical Informatics. https://doi.org/10.1016/s1386-5056(01)00184-8

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.