CoChat
Sign Up
Cited by 1,861Meta-analysis and systematic reviewsStatistical Methods in EpidemiologyClinical Reasoning and Diagnostic Skills

The DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.

Empirical evidence of design-related bias in studies of diagnostic tests.

Lijmer, Jeroen G.·1999·JAMA
DOI 10.1001/jama.282.11.1061
Open this paper in CoChat — free account

Abstract

CONTEXT: The literature contains a large number of potential biases in the evaluation of diagnostic tests. Strict application of appropriate methodological criteria would invalidate the clinical application of most study results. OBJECTIVE: To empirically determine the quantitative effect of study design shortcomings on estimates of diagnostic accuracy. DESIGN AND SETTING: Observational study of the methodological features of 184 original studies evaluating 218 diagnostic tests. Meta-analyses on diagnostic tests were identified through a systematic search of the literature using MEDLINE, EMBASE, and DARE databases and the Cochrane Library (1996-1997). Associations between study characteristics and estimates of diagnostic accuracy were evaluated with a regression model. MAIN OUTCOME MEASURES: Relative diagnostic odds ratio (RDOR), which compared the diagnostic odds ratios of studies of a given test that lacked a particular methodological feature with those without the corresponding shortcomings in design. RESULTS: Fifteen (6.8%) of 218 evaluations met all 8 criteria; 64 (30%) met 6 or more. Studies evaluating tests in a diseased population and a separate control group overestimated the diagnostic performance compared with studies that used a clinical population (RDOR, 3.0; 95% confidence interval [CI], 2.0-4.5). Studies in which different reference tests were used for positive and negative results of the test under study overestimated the diagnostic performance compared with studies using a single reference test for all patients (RDOR, 2.2; 95% CI, 1.5-3.3). Diagnostic performance was also overestimated when the reference test was interpreted with knowledge of the test result (RDOR, 1.3; 95% CI, 1.0-1.9), when no criteria for the test were described (RDOR, 1.7; 95% CI, 1.1-2.5), and when no description of the population under study was provided (RDOR, 1.4; 95% CI, 1.1-1.7). CONCLUSION: These data provide empirical evidence that diagnostic studies with methodological shortcomings may overestimate the accuracy of a diagnostic test, particularly those including nonrepresentative patients or applying different reference standards.

Overview

Integrity signals

Registry

Crossref · 10.1001/jama.282.11.1061

Cites

38 of 38 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Cited by

Sorted by citations

See all 1,861

Related

See all

Metadata

Authors
Lijmer, J. G.
Affiliations
Amsterdam UMC Location University of Amsterdam (NL)
Corresponding author
Lijmer, Jeroen G.
ORCID
Lijmer, Jeroen G.
Venue
JAMA
Publisher
American Medical Association (AMA)
Published
1999-09-15
Volume
282
Issue
11
Pages
1061
ISSN
  • 0002-9955 (ISSN-L)
  • 0098-7484
DOI
10.1001/jama.282.11.1061
OpenAlex
W1971348077
Cited by
1,861
References
38
Citations by year
2026: 56 · 2025: 38 · 2024: 38 · 2023: 45 · 2022: 45 · 2021: 48 · 2020: 74 · 2019: 50 · 2018: 73 · 2017: 66 · 2016: 79 · 2015: 82 · 2014: 73 · 2013: 78 · 2012: 79
FWCI
92.03
Type
journal article
Language
en
Open access
closed
Locations
  • JAMA (publishedVersion)
  • PubMed (publishedVersion)
  • Data Archiving and Networked Services (DANS) (submittedVersion)
  • Pure Amsterdam UMC (submittedVersion)
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
  • Health Sciences › Medicine › Family Practice › Clinical Reasoning and Diagnostic Skills
Keywords
diagnostic accuracy, study design bias, methodological quality, diagnostic odds ratio, reference materials
MeSH terms
Diagnostic Tests, Routine, Evaluation Studies as Topic, Quality Control, Reference Standards, Regression Analysis, Research Design, Sensitivity and Specificity, Bias, Odds Ratio

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review