Variation of a test’s sensitivity and specificity with disease prevalence
Open accessAbstract
BACKGROUND: Anecdotal evidence suggests that the sensitivity and specificity of a diagnostic test may vary with disease prevalence. Our objective was to investigate the associations between disease prevalence and test sensitivity and specificity using studies of diagnostic accuracy. METHODS: We used data from 23 meta-analyses, each of which included 10-39 studies (416 total). The median prevalence per review ranged from 1% to 77%. We evaluated the effects of prevalence on sensitivity and specificity using a bivariate random-effects model for each meta-analysis, with prevalence as a covariate. We estimated the overall effect of prevalence by pooling the effects using the inverse variance method. RESULTS: Within a given review, a change in prevalence from the lowest to highest value resulted in a corresponding change in sensitivity or specificity from 0 to 40 percentage points. This effect was statistically significant (p < 0.05) for either sensitivity or specificity in 8 meta-analyses (35%). Overall, specificity tended to be lower with higher disease prevalence; there was no such systematic effect for sensitivity. INTERPRETATION: The sensitivity and specificity of a test often vary with disease prevalence; this effect is likely to be the result of mechanisms, such as patient spectrum, that affect prevalence, sensitivity and specificity. Because it may be difficult to identify such mechanisms, clinicians should use prevalence as a guide when selecting studies that most closely match their situation.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Such fundamental differences in the spectrum of skin conditions compared with the general population means that poorer accuracy is likely to be observed in a real world setting.
“Such fundamental differences in the spectrum of skin conditions compared with the general population means that poorer accuracy is likely to be observed in a real world setting.38 39”
From Algorithm based smartphone apps to assess risk of skin cancer in adults: systematic review of diagnostic accuracy studies (2020) · cites this paper as [38]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Cochrane Handbook for Systematic Reviews of Interventions: Cochrane Book Series
(2008) · doi:10.1002/9780470712184 · Cited by 25,048 · see citing works
Evidence-Based Medicine: How to Practice and Teach EBM
Sackett, D. L. (2000) · Cited by 5,950 · see citing works
Reitsma, J. B., Glas, A. S., Rutjes, A. W. S., Scholten, R. J. P. M., Bossuyt, P. M., & Zwinderman, A. H. (2005) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2005.02.022 · Cited by 3,369 · see citing works
Self-citationTowards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative
Bossuyt, P. M., Reitsma, J. B., Bruns, D. E., Gatsonis, C. A., Glasziou, P. P., Irwig, L. M., Lijmer, J. G., Moher, D., Rennie, D., & de Vet, H. C. W. (2003) · BMJ · doi:10.1136/bmj.326.7379.41 · Cited by 3,215
Self-citationTowards Complete and Accurate Reporting of Studies of Diagnostic Accuracy: The STARD Initiative
Bossuyt, P. M., Reitsma, J. B., Bruns, D. E., Gatsonis, C. A., Glasziou, P., Irwig, L., Lijmer, J. G., Moher, D., Rennie, D., de Vet, H. C. W., et al. (2003) · Clinical Biochemistry · doi:10.1016/s0009-9120(02)00443-5 · Cited by 3,182 · see citing works
Self-citationProblems of Spectrum and Bias in Evaluating the Efficacy of Diagnostic Tests
Ransohoff, D. F., & Feinstein, A. R. (1978) · New England Journal of Medicine · doi:10.1056/nejm197810262991705 · Cited by 1,739 · see citing works
Epidemiology: Beyond the Basics
Szklo, M., & Nieto, F. J. (1999) · Cited by 1,455 · see citing works
Accuracy of Positron Emission Tomography for Diagnosis of Pulmonary Nodules and Mass Lesions
Gould, M. K., Maclean, C. C., Kuschner, W. G., Rydzak, C. E., & Owens, D. K. (2001) · JAMA · doi:10.1001/jama.285.7.914 · Cited by 1,092 · see citing works
Bossuyt, P. M., Reitsma, J. B., Bruns, D. E., Gatsonis, C., Glasziou, P., Irwig, L., Lijmer, J. G., Moher, D., Rennie, D., & de Vet, H. C. W. (2003) · PubMed · doi:10.1373/49.1.1 · Cited by 861 · see citing works
Self-citationThe rational clinical examination
Devitt, N. F. (1992) · JAMA · doi:10.1001/jama.268.16.2164c · Cited by 616 · see citing works
Evidence of bias and variation in diagnostic accuracy studies
Rutjes, A. W., Reitsma, J. B., Nisio, M. D., Smidt, N., van Rijn, J. C., & Bossuyt, P. M. (2006) · Canadian Medical Association Journal · doi:10.1503/cmaj.050090 · Cited by 613 · see citing works
Self-citationDiagnostic test accuracy may vary with prevalence: implications for evidence-based diagnosis
Leeflang, M., Bossuyt, P. M., & Irwig, L. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2008.04.007 · Cited by 485 · see citing works
Self-citationHow to Use an Article About a Diagnostic Test
Scales, C. D., Dahm, P., Sultan, S., Campbell-Scherer, D. L., & Devereaux, P. J. (2008) · The Journal of Urology · doi:10.1016/j.juro.2008.04.026 · Cited by 457 · see citing works
Meta-analysis of exercise testing to detect coronary artery disease in women
Kwok, Y. S., Kim, C., Grady, D. G., Segal, M. R., & Redberg, R. F. (1999) · The American Journal of Cardiology · doi:10.1016/s0002-9149(98)00963-1 · Cited by 450 · see citing works
Linden, A. (2006) · Journal of Evaluation in Clinical Practice · doi:10.1111/j.1365-2753.2005.00598.x · Cited by 449 · see citing works
Kelly, S., Harris, K. M., Berry, E., Hutton, J., Roderick, P., Cullingworth, J., Gathercole, L., & Smith, M. A. (2001) · Gut · doi:10.1136/gut.49.4.534 · Cited by 365 · see citing works
Analysing and Presenting Results
Macaskill, P., Gatsonis, C., Deeks, J. J., Harbord, R., & Takwoingi, Y. (2010) · Bristol Research (University of Bristol) · Cited by 337 · see citing works
Spectrum Bias or Spectrum Effect? Subgroup Variation in Diagnostic Test Evaluation
Engel, S. A. M., & Miller, W. C. (2002) · Annals of Internal Medicine · doi:10.7326/0003-4819-137-7-200210010-00011 · Cited by 331 · see citing works
Devillé, W. L. J. M., van der Windt, D., Dzaferagić, A., Bezemer, P. D., & Bouter, L. M. (2000) · Spine · doi:10.1097/00007632-200005010-00016 · Cited by 264 · see citing works
Irwig, L. (2002) · BMJ · doi:10.1136/bmj.324.7338.669 · Cited by 246 · see citing works
The quality of diagnostic accuracy studies since the STARD statement
Smidt, N., Rutjes, A. W., van der Windt, D., Ostelo, R., Bossuyt, P. M., Reitsma, J. B., Bouter, L. M., & de Vet, H. C. W. (2006) · Neurology · doi:10.1212/01.wnl.0000238386.41398.30 · Cited by 230 · see citing works
Self-citationPharmacologic stress testing for coronary disease diagnosis: A meta-analysis
Kim, C., Kwok, Y. S., Heagerty, P. J., et al. (2001) · American Heart Journal · doi:10.1067/mhj.2001.119761 · Cited by 220 · see citing works
Does This Patient Have Abdominal Aortic Aneurysm?
Lederle, F. A. (1999) · JAMA · doi:10.1001/jama.281.1.77 · Cited by 217 · see citing works
Peripheral Arterial Disease: Meta-analysis of the Diagnostic Performance of MR Angiography
Nelemans, P. J., Leiner, T., de Vet, H. C. W., & van Engelshoven, J. M. A. (2000) · Radiology · doi:10.1148/radiology.217.1.r00oc11105 · Cited by 215 · see citing works
Visser, K., & Hunink, M. G. M. (2000) · Radiology · doi:10.1148/radiology.216.1.r00jl0367 · Cited by 182 · see citing works
A meta-analysis of the Papanicolaou smear and wet mount for the diagnosis of vaginal trichomoniasis
Wiese, W., Patel, S. R., Patel, S. C., Ohl, C. A., & Estrada, C. A. (2000) · The American Journal of Medicine · doi:10.1016/s0002-9343(99)00466-0 · Cited by 150 · see citing works
Balk, E. M., Ioannidis, J. P. A., Salem, D. N., et al. (2001) · Annals of Emergency Medicine · doi:10.1067/mem.2001.114905 · Cited by 144 · see citing works
Lau, J., Ioannidis, J. P. A., Balk, E. M., et al. (2001) · Annals of Emergency Medicine · doi:10.1067/mem.2001.114903 · Cited by 141 · see citing works
Abdominal Symptoms: Do They Predict Gallstones?: A Systematic Review
Berger, J. J. I. M. V. D. V. M. Y. (2000) · Scandinavian Journal of Gastroenterology · doi:10.1080/003655200750024560 · Cited by 140 · see citing works
Thibodeau, L. A. (1981) · Biometrics · doi:10.2307/2530161 · Cited by 118 · see citing works
Magnetic Resonance Imaging of the Wrist: Diagnostic Performance Statistics
Hobby, J., Tom, B. D. M., Bearcroft, P. W. P., & Dixon, A. K. (2001) · Clinical Radiology · doi:10.1053/crad.2000.0571 · Cited by 116 · see citing works
Misguided efforts and future challenges for research on “diagnostic tests”
Feinstein, A. R. (2002) · Journal of Epidemiology & Community Health · doi:10.1136/jech.56.5.330 · Cited by 116 · see citing works
Reporting of measures of accuracy in systematic reviews of diagnostic literature
Honest, H., & Khan, K. S. (2002) · BMC Health Services Research · doi:10.1186/1472-6963-2-4 · Cited by 113 · see citing works
Hoogendam, A. (1999) · Family Practice · doi:10.1093/fampra/16.6.621 · Cited by 101 · see citing works
CT pulmonary angiography in the detection of pulmonary emboli
Safriel, Y. I., & Zinn, H. L. (2002) · Clinical Imaging · doi:10.1016/s0899-7071(01)00366-7 · Cited by 100 · see citing works
Huicho, L., Campos, M. A. L., & Alamo, C. (2002) · The Pediatric Infectious Disease Journal · doi:10.1097/00006454-200201000-00002 · Cited by 97 · see citing works
Sonnad, S. S., Langlotz, C. P., & Schwartz, J. S. (2001) · Academic Radiology · doi:10.1016/s1076-6332(01)90095-9 · Cited by 76 · see citing works
Hoffman, R. M., Clanon, D. L., Littenberg, B., et al. (2000) · Journal of General Internal Medicine · doi:10.1046/j.1525-1497.2000.90907.x · Cited by 67 · see citing works
van Zeijts, T. E. J., & Zijlstra, G. (1999) · Agronomy monograph/Agronomy · doi:10.2134/agronmonogr38.c31 · Cited by 58 · see citing works
Gates, G. A., Stewart, I. A., Northern, J. L., Downs, M. P., Ferrer, H. P., Hannley, M., Karma, P. H., Marchant, C. D., Renvall, U., Ruben, R. J., et al. (1994) · Annals of Otology Rhinology & Laryngology · doi:10.1177/00034894941030s810 · Cited by 58 · see citing works
Screening and Syndromic Approaches to Identify Gonorrhea and Chlamydial Infection among Women
Sloan, N. L., Winikoff, B., Haberland, N. A., Coggins, C., & Elias, C. J. (2000) · Studies in Family Planning · doi:10.1111/j.1728-4465.2000.00055.x · Cited by 57 · see citing works
EFFECT OF STUDY DESIGN ON THE ASSOCIATION BETWEEN NUCHAL TRANSLUCENCY MEASUREMENT AND DOWN SYNDROME
Mol, B. W., Lijmer, J. G., van der Meulen, J., Pajkrt, E., Bilardo, C. M., & Bossuyt, P. M. (1999) · Obstetrics and Gynecology · doi:10.1016/s0029-7844(99)00496-2 · Cited by 51 · see citing works
Self-citationScreening for squamous intraepithelial lesions with fluorescence spectroscopy*1
Mitchell, M. F. (1999) · Obstetrics and Gynecology · doi:10.1016/s0029-7844(99)00408-1 · Cited by 45 · see citing works
Concordance of EndotoxemiaWith Gram-Negative Bacteremia
Hurley, J. C. (2000) · Archives of Pathology & Laboratory Medicine · doi:10.5858/2000-124-1157-coegnb · Cited by 24 · see citing works
Pharmacologic stress testing for coronary disease diagnosis: a meta-analysis
Kim, C., Kwok, Y. S., Heagerty, P. J., et al. (2002) · ACC Current Journal Review · doi:10.1016/s1062-1458(02)00615-3 · Cited by 11 · see citing works
Efficacy of morphine sulfate-augmented hepatobiliary imaging in acute cholecystitis.
VASQUEZ, T. E., RIMKUS, D. S., Hass, M. G., et al. (2000) · PubMed · Cited by 9 · see citing works
Screening for Alcohol Problems in Primary Care
Fiellin, D. A., Reid, M. C., & O'Connor, P. G. (2000) · Archives of Internal Medicine · doi:10.1001/archinte.160.13.1977 · Cited by 3 · see citing works
Hosmer (1997) · doi:10.1002/(sici)1097-0258(19970515)16:9
Flagged first, then by citations
Research works printed but not linked: 5
- Haynes RB, Sackett DL, Guyatt GH, et al. editors. New York (NY): Lippincott William and Wilkins; 2006:294-5.
- Brenner H, Gefeller O. Variation of sensitivity, specificity, likeli- hood ratios and predictive values with disease prevalence. Stat Med 1997;16:981-91.
- Higgins JPT, Green S, editors. Cochrane handbook for systematic reviews of interventions. 5.1.0 ed. Oxford (UK): The Cochrane Collaboration; 2011. Available: www.cochrane-handbook.org (accessed 2012 Mar. 27).
- Mitchell MF, Cantor SB, Brookner C, et al. Screening for squa- mous intraepithelial lesions with fluorescence spectroscopy. Obstet Gynecol 1999;94:889-96.
- Irwig L, Bossuyt P, Glasziou P, et al. Designing studies to ensure that estimates of test accuracy are transferable. BMJ 2002; 324:669-71.
Cited by
Sorted by citations
STARD 2015 guidelines for reporting diagnostic accuracy studies: explanation and elaboration
Cohen, J. F., Korevaar, D. A., Altman, D. G., Bruns, D. E., Gatsonis, C. A., Hooft, L., Irwig, L., Levine, D., Reitsma, J. B., de Vet, H. C. W., et al. (2016) · BMJ Open · doi:10.1136/bmjopen-2016-012799 · Cited by 2,570 · see citing works
Self-citationRiley, R. D., Ensor, J., Snell, K. I. E., Debray, T. P. A., Altman, D. G., Moons, K. G. M., & Collins, G. S. (2016) · BMJ · doi:10.1136/bmj.i3140 · Cited by 585 · see citing works
Freeman, K., Dinnes, J., Chuchu, N., Takwoingi, Y., Bayliss, S. E., Matin, R. N., Jain, A., Walter, F. M., Williams, H. C., & Deeks, J. J. (2020) · BMJ · doi:10.1136/bmj.m127 · Cited by 260 · see citing works
The spectrum effect in tests for risk prediction, screening, and diagnosis
Usher-Smith, J. A., Sharp, & Griffin, S. J. (2016) · BMJ · doi:10.1136/bmj.i3139 · Cited by 250 · see citing works
Thombs, B. D., Kwakkenbos, L., Levis, A. W., & Benedetti, A. (2018) · Canadian Medical Association Journal · doi:10.1503/cmaj.170691 · Cited by 231 · see citing works
Xpert <sup>®</sup> MTB/RIF assay for extrapulmonary tuberculosis and rifampicin resistance
Kohli, M., Schiller, I., Dendukuri, N., Dheda, K., Denkinger, C. M., Schumacher, S. G., & Steingart, K. R. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012768.pub2 · Cited by 245 · see citing works
Kohli, M., Schiller, I., Dendukuri, N., Yao, M., Dheda, K., Denkinger, C. M., Schumacher, S. G., & Steingart, K. R. (2021) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012768.pub3 · Cited by 215 · see citing works
Teledermatology for diagnosing skin cancer in adults.
Chuchu, N., Dinnes, J., Takwoingi, Y., Matin, R. N., Bayliss, S. E., Davenport, C., Moreau, J. F., Bassett, O., Godfrey, K., O'Sullivan, C., et al. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013193 · Cited by 175 · see citing works
Paddick, S.-M., Gray, W. K., McGuire, J., Richardson, J., Dotchin, C., & Walker, R. W. (2017) · International Psychogeriatrics · doi:10.1017/s1041610216001976 · Cited by 96 · see citing works
Diagnosing occupational asthma
Vandenplas, O., Suojalehto, H., & Cullinan, P. (2017) · Clinical & Experimental Allergy · doi:10.1111/cea.12858 · Cited by 87 · see citing works
Abdallah, A., Hewson, J., Francoz, D., Selim, H., & Buczinski, S. (2016) · Journal of Veterinary Internal Medicine · doi:10.1111/jvim.13975 · Cited by 39 · see citing works
- Authors
- Leeflang, M. M. G., Rutjes, A. W. S., Reitsma, J. B., Hooft, L., & Bossuyt, P. M. M.
- Affiliations
- University of Bern (CH)
- Venue
- Canadian Medical Association Journal
- Publisher
- CMA Joule Inc.
- Published
- 2013-06-24
- Volume
- 185
- Issue
- 11
- Pages
- E537-E544
- ISSN
- 0820-3946 (ISSN-L)
- 1488-2329
- PMC
- PMC3735771
- OpenAlex
- W2162467118
- Cited by
- 530
- References
- 49
- Citations by year
- 2026: 28 · 2025: 31 · 2024: 54 · 2023: 59 · 2022: 68 · 2021: 68 · 2020: 66 · 2019: 40 · 2018: 34 · 2017: 20 · 2016: 29 · 2015: 20 · 2014: 9 · 2013: 2
- FWCI
- 22.03
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- Canadian Medical Association Journal (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Pure Amsterdam UMC (publishedVersion)
- IRIS UNIMORE (University of Modena and Reggio Emilia) (submittedVersion, cc-by-nc-nd)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- Bern Open Repository and Information System (University of Bern) (submittedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
- Keywords
- diagnostic accuracy, disease prevalence, test sensitivity, test specificity, bivariate random-effects model
- MeSH terms
- Diagnostic Tests, Routine, Disease, False Negative Reactions, False Positive Reactions, Female, Humans, Male, Models, Theoretical, Sensitivity and Specificity, Prevalence
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
