CoChat

How good are clinical MEDLINE searches? A comparative study of clinical end-user and librarian searches

McKibbon, K.Ann, Haynes, R.Brian, Walker Dilks, Cynthia J., Ramsden, Michael F., Ryan, Nancy C., Baker, Lynda, Flemming, Tom, Fitzgerald, Dorothy · 1990 · Computers and Biomedical Research

The DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.

Abstract

The objective of this study was to determine the quality of MEDLINE searches done by physicians, physician trainees, and expert searchers (clinicians and librarians). Its design was an analytic survey with independent replication in a setting of self-service online searching from medical wards, an intensive care unit, a coronary care unit, an emergency room, and an ambulatory clinic in a 300-bed teaching hospital. Participating were all M.D. clinical clerks, house, and attending staff responsible for patients in the above settings. Intervention for all participants consisted of a 2-h small group class and 1-h practice session on MEDLINE searching (GRATEFUL MED) before free access to MEDLINE. Search questions from 104 randomly selected novice searches were given to 1 of 13 clinicians with prior search experience and 1 of 3 librarians to run independent searches (triplicated searches). Measurements and main results from these unique citations of the triplicated searches were sent to expert clinicians to rate for relevance (7-point scale). Recall (number of relevant citations retrieved from an individual search divided by the total number of relevant citations from all searches on the same topic) and precision (proportion of relevant citations retrieved in each search) were calculated. Librarians were significantly better than novices for both. Librarians had equivalent recall to, and better precision than, experienced end-users. Unexpectedly, only 20% of relevant citations were retrieved by more than one search of the set of three, with the conclusion that novice searchers on MEDLINE via GRATEFUL MED after brief training have relatively low recall and precision. Recall improves with experience but precision remains suboptimal. Further research is needed to determine the "learning curve," evaluate training interventions, and explore the non-overlapping retrieval of relevant citations by different searchers.

Overview

Integrity signals

Registry

Crossref · 10.1016/0010-4809(90)90042-b

Cites

15 of 15 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 15 references

Research works printed but not linked: 1

  1. White, 1988, Putting end-users to work, Library J., 5, 41

Cited by

See all 108

Metadata

Authors
McKibbon, K. A., Haynes, R. B., Walker Dilks, C. J., Ramsden, M. F., Ryan, N. C., Baker, L., Flemming, T., & Fitzgerald, D.
Affiliations
McMaster University (CA)
Venue
Computers and Biomedical Research
Publisher
Elsevier BV
Published
1990-12
Volume
23
Issue
6
Pages
583-593
ISSN
0010-4809 (ISSN-L)
OpenAlex
W2161479801
Cited by
108
References
14
Citations by year
2025: 1 · 2024: 2 · 2022: 1 · 2020: 1 · 2018: 1 · 2017: 4 · 2015: 2 · 2014: 1 · 2013: 6 · 2012: 5
FWCI
17.38
Type
journal article
Language
en
Open access
closed
Licence
www.elsevier.com/tdm/userlicense/1.0/ (tdm)
Locations
  • Computers and Biomedical Research (publishedVersion)
  • PubMed (publishedVersion)
Funders
  • National Institutes of Health (5r01lm004696-05)
  • U.S. National Library of Medicine (R01 LM 04696-01)
Topics
  • Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
  • Health Sciences › Health Professions › General Health Professions › Health Sciences Research and Education
  • Social Sciences › Library and Information Sciences › Library Science and Information Literacy
Keywords
MEDLINE searching, search quality, precision and recall, Grateful Med, physician training, information retrieval, learning curve, citation retrieval
MeSH terms
Evaluation Studies as Topic, Subject Headings, User-Computer Interface, Reproducibility of Results, MEDLINE, Grateful Med

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: How good are clinical MEDLINE searches? A comparative study of clinical end-user and librarian searches. https://app.cochat.ai/papers/10.1016/0010-4809(90)90042-b
The paper
McKibbon, K. A., Haynes, R. B., Walker Dilks, C. J., Ramsden, M. F., Ryan, N. C., Baker, L., Flemming, T., & Fitzgerald, D. (1990). How good are clinical MEDLINE searches? A comparative study of clinical end-user and librarian searches. Computers and Biomedical Research. https://doi.org/10.1016/0010-4809(90)90042-b

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.