The DOI matches its Crossref record; 61 of the 61 references it cites were found in a registry or an index.
Sources of Variation and Bias in Studies of Diagnostic Accuracy: A Systematic Review
Open accessAbstract
BACKGROUND: Studies of diagnostic accuracy are subject to different sources of bias and variation than studies that evaluate the effectiveness of an intervention. Little is known about the effects of these sources of bias and variation. PURPOSE: To summarize the evidence on factors that can lead to bias or variation in the results of diagnostic accuracy studies. DATA SOURCES: MEDLINE, EMBASE, and BIOSIS, and the methodologic databases of the Centre for Reviews and Dissemination and the Cochrane Collaboration. Methodologic experts in diagnostic tests were contacted. STUDY SELECTION: Studies that investigated the effects of bias and variation on measures of test performance were eligible for inclusion, which was assessed by one reviewer and checked by a second reviewer. Discrepancies were resolved through discussion. DATA EXTRACTION: Data extraction was conducted by one reviewer and checked by a second reviewer. DATA SYNTHESIS: The best-documented effects of bias and variation were found for demographic features, disease prevalence and severity, partial verification bias, clinical review bias, and observer and instrument variation. For other sources, such as distorted selection of participants, absent or inappropriate reference standard, differential verification bias, and review bias, the amount of evidence was limited. Evidence was lacking for other features, including incorporation bias, treatment paradox, arbitrary choice of threshold value, and dropouts. CONCLUSIONS: Many issues in the design and conduct of diagnostic accuracy studies can lead to bias or variation; however, the empirical evidence about the size and effect of these issues is limited.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Empirical evidence of design-related bias in studies of diagnostic tests.
Lijmer, J. G. (1999) · JAMA · doi:10.1001/jama.282.11.1061 · Cited by 1,861
Problems of Spectrum and Bias in Evaluating the Efficacy of Diagnostic Tests
Ransohoff, D. F., & Feinstein, A. R. (1978) · New England Journal of Medicine · doi:10.1056/nejm197810262991705 · Cited by 1,734
The STARD Statement for Reporting Studies of Diagnostic Accuracy: Explanation and Elaboration
Bossuyt, P. M., Reitsma, J. B., Bruns, D. E., Gatsonis, C. A., Glasziou, P. P., Irwig, L. M., Moher, D., Rennie, D., de Vet, H. C. W., & Lijmer, J. G. (2003) · Annals of Internal Medicine · doi:10.7326/0003-4819-138-1-200301070-00012-w1 · Cited by 1,124
Self-citationClinical Epidemiology: The Architecture of Clinical Research.
(1985) · Annals of Internal Medicine · doi:10.7326/0003-4819-103-4-649_2 · Cited by 949
NHLBI funding policies. Enhancing stability, predictability, and cost control.
Lenfant, C. (1994) · Circulation · doi:10.1161/01.cir.90.1.1 · Cited by 762
Assessment of diagnostic tests when disease verification is subject to selection bias
Begg, C. B., & Greenes, R. A. (1983) · Biometrics · doi:10.2307/2530820 · Cited by 708
Variability in Radiologists' Interpretations of Mammograms
Elmore, J. G., Wells, C. K., Lee, C. H., Howard, D. H., & Feinstein, A. R. (1994) · New England Journal of Medicine · doi:10.1056/nejm199412013312206 · Cited by 601
Levy, D., Labib, S. B., Anderson, K. M., Christiansen, J. C., Kannel, W. B., & Castelli, W. P. (1990) · Circulation · doi:10.1161/01.cir.81.3.815 · Cited by 531
Development and validation of methods for assessing the quality of diagnostic accuracy studies
Whiting, P., Rutjes, A., Dinnes, J., Reitsma, J., Bossuyt, P., & Kleijnen, J. (2004) · Health Technology Assessment · doi:10.3310/hta8250 · Cited by 372
Self-citationFactors affecting sensitivity and specificity of exercise electrocardiography
Hlatky, M. A., Pryor, D. B., Harrell, F. E., Califf, R. M., Mark, D. B., & Rosati, R. A. (1984) · The American Journal of Medicine · doi:10.1016/0002-9343(84)90437-6 · Cited by 371
Cited by
QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies
Whiting, P. F., Rutjes, A. W. S., Westwood, M. E., Mallett, S., Deeks, J. J., Reitsma, J. B., Leeflang, M. M. G., Sterne, J. A. C., Bossuyt, P. M. M., & the QUADAS-2 Group* (2011) · Annals of Internal Medicine · doi:10.7326/0003-4819-155-8-201110180-00009 · Cited by 14,199
Self-citationWhiting, P., Rutjes, A. W., Reitsma, J. B., Bossuyt, P. M., & Kleijnen, J. (2003) · BMC Medical Research Methodology · doi:10.1186/1471-2288-3-25 · Cited by 3,592
Self-citationReitsma, J. B., Glas, A. S., Rutjes, A. W. S., Scholten, R. J. P. M., Bossuyt, P. M., & Zwinderman, A. H. (2005) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2005.02.022 · Cited by 3,369
“Second, our understanding of how sources of bias and variability can affect estimates of diagnostic accuracy is largely based on thesemeasures [27,28].”
BackgroundSelf-citationSystematic reviews: CRD's guidance for undertaking reviews in health care
Tacconelli, E. (2010) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(10)70065-7 · Cited by 3,245
STARD 2015 guidelines for reporting diagnostic accuracy studies: explanation and elaboration
Cohen, J. F., Korevaar, D. A., Altman, D. G., Bruns, D. E., Gatsonis, C. A., Hooft, L., Irwig, L., Levine, D., Reitsma, J. B., de Vet, H. C. W., et al. (2016) · BMJ Open · doi:10.1136/bmjopen-2016-012799 · Cited by 2,570
Self-citation
Metadata
- Authors
- Whiting, P., Rutjes, A. W. S., Reitsma, J. B., Glas, A. S., Bossuyt, P. M. M., & Kleijnen, J.
- Venue
- Annals of Internal Medicine
- Publisher
- American College of Physicians
- Published
- 2004-02-03
- Volume
- 140
- Issue
- 3
- Pages
- 189-202
- ISSN
- 0003-4819 (ISSN-L)
- 1539-3704
- OpenAlex
- W2088323328
- Cited by
- 798
- References
- 61
- FWCI
- 53.36
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Sources of Variation and Bias in Studies of Diagnostic Accuracy: A Systematic Review. https://app.cochat.ai/papers/10.7326/0003-4819-140-3-200402030-00010
- The paper
- Whiting, P., Rutjes, A. W. S., Reitsma, J. B., Glas, A. S., Bossuyt, P. M. M., & Kleijnen, J. (2004). Sources of Variation and Bias in Studies of Diagnostic Accuracy: A Systematic Review. Annals of Internal Medicine. https://doi.org/10.7326/0003-4819-140-3-200402030-00010
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.