CoChat
Sign Up
Cited by 250Statistical Methods in EpidemiologyMeta-analysis and systematic reviewsClinical Laboratory Practices and Quality Control

The spectrum effect in tests for risk prediction, screening, and diagnosis

Open access
Usher-Smith, Juliet A, Sharp,, Stephen J, Griffin, Simon J·2016·BMJ
DOI 10.1136/bmj.i3139PMC PMC4916916
Open this paper in CoChat — free accountOpen PDF

Abstract

The spectrum effect describes the variation between settings in performance of tests used to predict, screen for, and diagnose disease. In particular, the predictive use of a test may be different when it is applied in a general population rather than in the study sample in which it was first developed. This article discusses the impact of the spectrum effect on measures of test performance, and its implications for the development, evaluation, application, and implementation of such tests.

AI snapshot

Written from the paper’s full text.

TL;DR

The spectrum effect describes how test performance—sensitivity, specificity, and likelihood ratios—varies across populations because of differences in disease prevalence and the distribution of disease or risk. The paper argues that developers, evaluators, and policymakers should interpret test accuracy in the context of the population where it will be used.

Key findings

  • In simulation scenario 1, as disease prevalence decreased, positive likelihood ratio, negative likelihood ratio, and specificity increased while sensitivity decreased, with 10-fold changes in likelihood ratios and about 30% variation in sensitivity and specificity.
  • In simulation scenario 2, changing the distribution of the underlying variable altered performance: at 20% disease prevalence, sensitivity and specificity were 85% and 95% in scenario 1 versus 98% and 99% in scenario 2.
  • A review of 23 meta-analyses of diagnostic test accuracy found statistically significant associations between prevalence and sensitivity or specificity in one of three reviews, with specificity decreasing by 0.1% to 0.5% for every 1 percentage point increase in prevalence in pooled analysis.
  • An example from Moons and colleagues showed the positive likelihood ratio for an exercise test for coronary artery disease was 3.8 in individuals with systolic blood pressure 141–240 mm Hg and 17.0 in those with systolic blood pressure 100–140 mm Hg.

Methods

  • Narrative review and simulation study illustrating the spectrum effect, using a continuous variable X where true disease is defined as X ≥ 7 and an estimate Z is obtained by measurement or risk prediction.
  • Scenario 1 varied disease prevalence by changing the mean of X while keeping the threshold constant; scenario 2 varied prevalence by altering the distribution of X using a bimodal distribution.
  • The paper also draws on published literature and examples, including a review of 23 meta-analyses and a clinical example of exercise testing for coronary artery disease.

Limitations

  • The paper does not state a formal limitations section.
  • Evident from the text: the simulations are illustrative and depend on assumptions about the distribution of X, measurement or prediction error, and a threshold-based disease definition; the supplementary appendix containing simulation details is referenced but not included in the provided text.
  • Evident from the text: several examples are hypothetical or drawn from selected literature rather than from a systematic empirical study of the spectrum effect.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1136/bmj.i3139

Cites

16 of 16 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 13 cited sources checked · 0 supported · 0 flagged

  • Findingreasoning

    When X has a distribution as shown in figure 2C, a greater proportion of people with the disease will have a value of X close to the threshold (and hence be at risk of test misclassification) than the corresponding proportion in a distribution as shown in figure 2D.

    “When X has a distribution as shown in figure 2C, a greater proportion of people with the disease will have a value of X close to the threshold (and hence be at risk of test misclassification) than the corresponding proportion in a distribution as shown in figure 2D.”
    Quote in the paperSubstantial claim
  • Interpretationreasoning

    Change in prevalence is equivalent to performing a test in different health settings.

    “Change in prevalence is equivalent to performing a test in different health settings.”
    Quote in the paperSubstantial claim
  • Interpretationreasoning

    The probability of misclassification of disease status varies between individuals, with individuals with true values of the underlying characteristics close to the cut-off point for diagnosis more likely to be misclassified than others.

    “The consequence of this combination of error in the test result and the continuum of true disease status is that the probability of misclassification of disease status varies between individuals with individuals with true values of the underlying characteristics close to the cut-off point for diagnosis more likely to be misclassified than others.”
    Quote in the paperSubstantial claim
  • Interpretationreasoning

    The most commonly used measures of test performance (sensitivity, specificity, and likelihood ratios) vary with the prevalence and distribution of disease in a population or sample.

    “The implication of the spectrum effect is that the most commonly used measures of the performance of tests (sensitivity, specificity, and likelihood ratios) vary with the prevalence and distribution of disease in a population or sample.”
    Quote in the paperSubstantial claim
  • Interpretationcitation

    The same effect of lower sensitivity and higher specificity in lower prevalence settings is seen when applying the test in subgroups known to be at higher risk (the referral filter).

    “The same effect is seen when applying the test in subgroups known to be at higher risk (the referral filter3).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [3] Leeflang MMG, Bossuyt PMM, Irwig L. Diagnostic test accuracy may vary with prevalence: implications for evidence-based diagnosis. J Clin Epidemiol 2009; 62: 5- 12. 10.1016/j.jclinepi.2008.04.007 pmid:18778913. 18778913

      Not checked in the free run.

  • May combine two claims. 17
  • Classified as probably not a substantial claim. 8

  • May combine two claims.

    Findingnone given

    As prevalence decreases, the positive likelihood ratio, negative likelihood ratio, and specificity increase, while sensitivity decreases.

    “As prevalence decreases, the positive likelihood ratio (LR+), negative likelihood ratio (LR−), and specificity increase, while sensitivity decreases”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingreasoning

    With decreasing prevalence, there are 10-fold changes in likelihood ratios and variation of 30% in sensitivity and specificity.

    “with 10-fold changes in likelihood ratios and variation of 30% in sensitivity and specificity”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingreasoning

    As the proportion of the overall population with the disease increases, the risk of test misclassification among those with the disease decreases and-with this particular underlying distribution of X-the result is an increase in sensitivity and decrease in the negative likelihood ratio.

    “In other words, as the proportion of the overall population with the disease increases, the risk of test misclassification among those with the disease decreases and—with this particular underlying distribution of X—the result is an increase in sensitivity and decrease in the negative likelihood ratio (fig 2A and 2B).”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Findingnone given

    A test developed in a population with a higher prevalence of disease (or at higher risk) will typically have a lower sensitivity and higher specificity when applied in a population with lower disease prevalence (or at lower risk).

    “A test developed in a population with a higher prevalence of disease (or at higher risk) will typically have a lower sensitivity and higher specificity when applied in a population with lower disease prevalence (or at lower risk)”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingreasoning

    The performance of tests is influenced by both the prevalence of the condition or disease in the sample in which it is assessed and the characteristics of the sample.

    “Therefore, the performance of tests is influenced by both the prevalence of the condition or disease in the sample in which it is assessed and the characteristics of the sample.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingreasoning

    The wide dispersion of the values of X in the bimodal distribution means that there are fewer individuals with values close to the threshold than in scenario 1, resulting in a higher sensitivity and specificity for the same prevalence.

    “The wide dispersion of the values of X in this bimodal distribution also means that in figure 2C and 2D, there are fewer individuals with values close to the threshold than in scenario 1 (fig 1C and 1D). The result is a higher sensitivity and specificity for the same prevalence.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingreasoning

    When 20% of individuals have the disease, the sensitivity and specificity in scenario 1 are 85% and 95%, respectively, compared with 98% and 99% in scenario 2.

    “For example, when 20% of individuals have the disease, the sensitivity and specificity in scenario 1 are 85% and 95%, respectively, compared with the corresponding values of 98% and 99% in scenario 2.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Premisenone given

    All tests are subject to error, including simple measurement error, subjective error, and the influence of unmeasured covariates on test results.

    “Further, all tests are subject to error, including:”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    If a test designed for population based screening of asymptomatic individuals is used in a high risk population, the specificity will be lower and the sensitivity higher than estimated in the original development population.

    “if the test had been designed for population based screening of asymptomatic individuals, the specificity in that high risk population will be lower—and the sensitivity higher—than estimated in the original development population.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    A diagnostic procedure developed in secondary care among patients suspected of having a particular disease will typically have lower sensitivity and higher specificity when applied as a screening tool in primary care where disease prevalence is lower.

    “A diagnostic procedure developed in secondary care among patients suspected of having a particular disease will typically have lower sensitivity and higher specificity when applied as a screening tool in primary care where disease prevalence is lower.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    The variation in likelihood ratio, sensitivity, and specificity due to the spectrum effect has been well described.

    “However, while the variation in likelihood ratio, sensitivity, and specificity due to the phenomenon known as the “spectrum effect” or “spectrum bias” has been well described,1 2 3 4 5 it is frequently overlooked by the wider medical and policy-making community when interpreting data on test performance.”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [1] Diamond GA. Reverend Bayes’ silent majority. An alternative factor affecting sensitivity and specificity of exercise electrocardiography. Am J Cardiol 1986; 57: 1175- 80. 10.1016/0002-9149(86)90694-6 pmid:3754686. 3754686

      Not checked in the free run.

    • cites [2] Irwig L, Bossuyt P, Glasziou P, Gatsonis C, Lijmer J. Designing studies to ensure that estimates of test accuracy are transferable. BMJ 2002; 324: 669- 71. 10.1136/bmj.324.7338.669 pmid:11895830. 11895830 PMC1122584

      Not checked in the free run.

    • cites [3] Leeflang MMG, Bossuyt PMM, Irwig L. Diagnostic test accuracy may vary with prevalence: implications for evidence-based diagnosis. J Clin Epidemiol 2009; 62: 5- 12. 10.1016/j.jclinepi.2008.04.007 pmid:18778913. 18778913

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Ransohoff and Feinstein first described the spectrum effect in 1978, observing that the performance of a test in practice might be misrepresented by clinical studies that include too narrow a range of individuals with or without the disease of interest.

    “When Ransohoff and Feinstein first described the spectrum effect in 1978,5 they observed that the performance of a test in practice might be misrepresented by clinical studies that include too narrow a range of individuals with or without the disease of interest.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [5] Ransohoff DF, Feinstein AR. Problems of spectrum and bias in evaluating the efficacy of diagnostic tests. N Engl J Med 1978; 299: 926- 30. 10.1056/NEJM197810262991705 pmid:692598. 692598

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Logistic modelling can produce estimates of test performance among small groups, and facilitates investigation of patient characteristics that are multicategorical or continuous, enabling the investigator to model separately factors that affect sensitivity and specificity.

    “As Mulherin and Miller point out,6 logistic modelling can produce estimates of test performance17 among small groups, and facilitates investigation of patient characteristics that are multicategorical or continuous, enabling the investigator to model separately factors that affect sensitivity and specificity.4”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [6] Mulherin SA, Miller WC. Spectrum bias or spectrum effect? Subgroup variation in diagnostic test evaluation. Ann Intern Med 2002; 137: 598- 602. 10.7326/0003-4819-137-7-200210010-00011 pmid:12353947. 12353947

      Not checked in the free run.

    • cites [17] Coughlin SS, Trock B, Criqui MH, Pickle LW, Browner D, Tefft MC. The logistic modeling of sensitivity, specificity, and predictive value of a diagnostic test. J Clin Epidemiol 1992; 45: 1- 7. 10.1016/0895-4356(92)90180-U pmid:1738006. 1738006

      Not checked in the free run.

    • cites [4] Moons KG, van Es GA, Deckers JW, Habbema JD, Grobbee DE. Limitations of sensitivity, specificity, likelihood ratio, and bayes’ theorem in assessing diagnostic probabilities: a clinical example. Epidemiology 1997; 8: 12- 7. 10.1097/00001648-199701000-00002 pmid:9116087. 9116087

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    In a review of 23 meta-analyses of diagnostic test accuracy, statistically significant associations between prevalence and sensitivity or specificity were found in one of three reviews, with specificity decreasing by between 0.1% and 0.5% for every 1 percentage point increase in prevalence in pooled analysis.

    “In a review of 23 meta-analyses of diagnostic test accuracy, statistically significant associations between prevalence and sensitivity or specificity were found in one of three reviews, with specificity decreasing by between 0.1% and 0.5% for every 1 percentage point increase in prevalence in pooled analysis.10”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [10] Leeflang MMG, Rutjes AWS, Reitsma JB, Hooft L, Bossuyt PM. Variation of a test’s sensitivity and specificity with disease prevalence. CMAJ 2013; 185: E537- 44. 10.1503/cmaj.121286 pmid:23798453. 23798453 PMC3735771

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Moons and colleagues demonstrated that the performance of an exercise test for coronary artery disease varied across patients with different characteristics; the positive likelihood ratio was 3.8 in individuals with systolic blood pressure 141-240 mm Hg, and 17.0 in those with systolic blood pressure 100-140 mm Hg.

    “For example, Moons and colleagues4 demonstrated that the performance of an exercise test for coronary artery disease varied across patients with different characteristics; the positive likelihood ratio was 3.8 in individuals with systolic blood pressure 141-240 mm Hg, and 17.0 in those with systolic blood pressure 100-140 mm Hg.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [4] Moons KG, van Es GA, Deckers JW, Habbema JD, Grobbee DE. Limitations of sensitivity, specificity, likelihood ratio, and bayes’ theorem in assessing diagnostic probabilities: a clinical example. Epidemiology 1997; 8: 12- 7. 10.1097/00001648-199701000-00002 pmid:9116087. 9116087

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    Z is unlikely to be exactly equal to X for all individuals, because of measurement error if X represents a measurable trait, or prediction error if X represents an underlying risk.

    “Z is unlikely to be exactly equal to X for all individuals, because of measurement error if X represents a measurable trait, or prediction error if X represents an underlying risk.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationcitation

    Apparent differences in test performance arising from variation in prevalence and study populations are widespread and can be large enough to influence clinical practice.

    “Although the examples given above are for a situation in which the target condition is based on measurement of a continuous variable, with the presence or absence of the condition based on the value of that variable being above or below a threshold, apparent differences in test performance arising from variation in prevalence and study populations are widespread and can be large enough to influenc”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [10] Leeflang MMG, Rutjes AWS, Reitsma JB, Hooft L, Bossuyt PM. Variation of a test’s sensitivity and specificity with disease prevalence. CMAJ 2013; 185: E537- 44. 10.1503/cmaj.121286 pmid:23798453. 23798453 PMC3735771

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The STARD statement for reporting studies of diagnostic accuracy recommends that authors describe the study participants in detail, including the distribution of disease severity.

    “The STARD statement for reporting studies of diagnostic accuracy15 recommends that authors describe the study participants in detail, including the distribution of disease severity.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [15] Bossuyt PM, Reitsma JB, Bruns DE, et al. STARD Group. STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies. BMJ 2015; 351: h5527 10.1136/bmj.h5527 pmid:26511519. 26511519 PMC4623764

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    The more advanced or severe the disease or condition is among those individuals affected, the greater the separation in the measured variable(s) between people with and without the condition, and the better the test will appear to be at discriminating between the two groups.

    “The more advanced or severe the disease or condition is among those individuals affected, the greater the separation in the measured variable(s) between people those with and without the condition, and the better the test will appear to be at discriminating between the two groups.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationreasoning

    Using data from two different cohort studies (one for development and one for evaluation) would allow for some exploration of the spectrum effect and ensure that the test's performance is not overestimated by use of a sample that is unrepresentative of the wider population.

    “This approach would allow for some exploration of the spectrum effect, and ensure that the test’s performance is not overestimated by use of a sample that is unrepresentative of the wider population.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    The effect of errors on overall test performance depends on the proportion of individuals misclassified, which in turn depends on the number of individuals close to the cut-off point of the test.

    “The effect of these errors on overall test performance depends on the proportion of individuals misclassified, which in turn depends on the number of individuals close to the cut-off point of the test.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    Situations like those shown in figure 2 could arise in a study in which individuals with and without disease are selected from different populations, for example, a case-control study with cases identified from hospital and controls from the community.

    “Situations like those shown in figure 2 could arise in a study in which individuals with and without disease are selected from different populations, for example, a case-control study with cases identified from hospital and controls from the community.”
    Quote in the paperNot a substantial claim
22 of 22

  • The paper relies on this without support.

    The variation in likelihood ratio, sensitivity, and specificity due to the spectrum effect is frequently overlooked by the wider medical and policy-making community when interpreting data on test performance.

    “However, while the variation in likelihood ratio, sensitivity, and specificity due to the phenomenon known as the “spectrum effect” or “spectrum bias” has been well described,1 2 3 4 5 it is frequently overlooked by the wider medical and policy-making community when interpreting data on test performance.”

  • The paper relies on this without support.

    The positive predictive value varies with the prevalence of the condition.

    “The positive predictive value is well known to vary with the prevalence of the condition.”
2 of 2

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Cites

Flagged first, then by citations

Research works printed but not linked: 1

  1. [9]National Institute for Health and Care Excellence. Lipid modification: cardiovascular risk assessment and the modification of blood lipids for the primary and secondary prevention of cardiovascular disease (CG181). 2014. https://www.nice.org.uk/guidance/cg181/resources/guidance-lipid-modification-cardiovascular-risk-assessment-and-the-modification-of-blood-lipids-for-the-primary-and-secondary-prevention-of-cardiovascular-disease-pdf. 25340243

Cited by

Sorted by citations

Related

Authors
Usher-Smith, J. A., Sharp, & Griffin, S. J.
Affiliations
University of Cambridge (GB)
ORCID
Venue
BMJ
Publisher
BMJ
Published
2016-06-22
Volume
353
Pages
i3139
ISSN
1756-1833 (ISSN-L)
DOI
10.1136/bmj.i3139
PMC
PMC4916916
OpenAlex
W2351656844
Semantic Scholar
a1b5fbcbe1f64273ea06ff34b76fce7cc3677414
Cited by
250
References
16
Citations by year
2026: 27 · 2025: 22 · 2024: 30 · 2023: 25 · 2022: 30 · 2021: 36 · 2020: 26 · 2019: 20 · 2018: 23 · 2017: 9 · 2016: 2
FWCI
18.52
Type
journal article
Language
en
Open access
Licence
Locations
  • BMJ (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • HighWire Press Open Archive (submittedVersion)
  • Apollo (University of Cambridge) (submittedVersion, cc-by)
Funders
  • National Institute for Health and Care Research (CL-2012-14-007)
  • Medical Research Council (MC_UU_12015/1, MC_UU_12015/4)
Topics
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Health Sciences › Medicine › Physiology › Clinical Laboratory Practices and Quality Control
Keywords
test performance, risk prediction, disease screening, diagnosis, generalizability, test evaluation
MeSH terms
Humans, Mass Screening, Predictive Value of Tests, Risk Factors, Sensitivity and Specificity, Bias, Prevalence, Likelihood Functions, Risk Assessment, Diagnostic Techniques and Procedures

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-08.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-08.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review