The DOI matches its Crossref record.
Identifying Diagnostic Studies in MEDLINE: Reducing the Number Needed to Read
Open accessAbstract
OBJECTIVES: The search filters in PubMed have become a cornerstone in information retrieval in evidence-based practice. However, the filter for diagnostic studies is not fully satisfactory, because sensitive searches have low precision. The objective of this study was to construct and validate better search strategies to identify diagnostic articles recorded on MEDLINE with special emphasis on precision. DESIGN: A comparative, retrospective analysis was conducted. Four medical journals were hand-searched for diagnostic studies published in 1989 and 1994. Four other journals were hand-searched for 1999. The three sets of studies identified were used as gold standards. A new search strategy was constructed and tested using the 1989-subset of studies and validated in both the 1994 and 1999 subsets. We identified candidate text words for search strategies using a word frequency analysis of the abstracts. According to the frequency of identified terms, searches were run for each term independently. The sensitivity, precision, and number needed to read (1/precision) of every candidate term were calculated. Terms with the highest sensitivity x precision product were used as free text terms in combination with the MeSH term "SENSITIVITY AND SPECIFICITY" using the Boolean operator OR. In the 1994 and 1999 subsets, we performed head-to-head comparisons of the currently available PubMed filter with the one we developed. MEASUREMENTS: The sensitivity, precision and the number needed to read (1/precision) were measured for different search filters. RESULTS: The most frequently occurring three truncated terms (diagnos*; predict* and accura*) in combination with the MeSH term "SENSITIVITY AND SPECIFICITY" produced a sensitivity of 98.1 percent (95% confidence interval: 89.9-99.9%) and a number needed to read of 8.3 (95% confidence interval: 6.7-11.3%). In direct comparisons of the new filter with the currently available one in PubMed using the 1994 and 1999 subsets, the new filter achieved better precision (12.0% versus 8.2% in 1994 and 5.0% versus 4.3% in 1999. The 95% confidence intervals for the differences range from 0.05% to 7.5% (p = 0.041) and -1.0% to 2.3% (p = 0.45), respectively). The new filter achieved slightly better sensitivities than the currently available one in both subsets, namely 98.1 and 96.1% (p = 0.32) versus 95.1 and 88.8% (p = 0.125). CONCLUSIONS: The quoted performance of the currently available filter for diagnostic studies in PubMed may be overstated. It appears that even single external validation may lead to over optimistic views of a filter's performance. Precision appears to be more unstable than sensitivity. In terms of sensitivity, our filter for diagnostic studies performed slightly better than the currently available one and it performed better with regards to precision in the 1994 subset. Additional research is required to determine whether these improvements are beneficial to searches in practice.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cited by
Systematic Reviews of Diagnostic Test Accuracy
Leeflang, M. M. G., Deeks, J. J., Gatsonis, C., & Bossuyt, P. M. M. (2008) · Annals of Internal Medicine · doi:10.7326/0003-4819-149-12-200812160-00008 · Cited by 1,124
A systematic review of rapid diagnostic tests for the detection of tuberculosis infection
Dinnes, J., Deeks, J., Kunst, H., Gibson, A., Cummins, E., Waugh, N., Drobniewski, F., & Lalvani, A. (2007) · Health Technology Assessment · doi:10.3310/hta11030 · Cited by 666
Haynes, R. B., & Wilczynski, N. L. (2004) · BMJ · doi:10.1136/bmj.38068.557998.ee · Cited by 547
Geersing, G.-J., Bouwmeester, W., Zuithoff, P., Spijker, R., Leeflang, M., & Moons, K. (2012) · PLoS ONE · doi:10.1371/journal.pone.0032844 · Cited by 350
Physical examination for lumbar radiculopathy due to disc herniation in patients with low-back pain
van der Windt, D. A., Simons, E., Riphagen, I. I., Ammendolia, C., Verhagen, A. P., Laslett, M., Devillé, W., Deyo, R. A., Bouter, L. M., de Vet, H. C., et al. (2010) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd007431.pub2 · Cited by 334
Metadata
- Authors
- Bachmann, L. M.
- Venue
- Journal of the American Medical Informatics Association
- Publisher
- Oxford University Press (OUP)
- Published
- 2002-07-23
- Volume
- 9
- Issue
- 6
- Pages
- 653-658
- ISSN
- 1067-5027 (ISSN-L)
- 1527-974X
- OpenAlex
- W2110097433
- Cited by
- 173
- References
- 0
- FWCI
- 14.34
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Identifying Diagnostic Studies in MEDLINE: Reducing the Number Needed to Read. https://app.cochat.ai/papers/10.1197/jamia.m1124
- The paper
- Bachmann, L. M. (2002). Identifying Diagnostic Studies in MEDLINE: Reducing the Number Needed to Read. Journal of the American Medical Informatics Association. https://doi.org/10.1197/jamia.m1124
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.