CoChat
Sign Up
Cited by 9,619Reliability and Agreement in MeasurementClinical Laboratory Practices and Quality ControlMeta-analysis and systematic reviews

Measuring agreement in method comparison studies

Bland, J.M., Altman, D.G.·1999·Statistical Methods in Medical Research
DOI 10.1191/096228099673819272
Open this paper in CoChat — free account

Abstract

Agreement between two methods of clinical measurement can be quantified using the differences between observations made using the two methods on the same subjects. The 95% limits of agreement, estimated by mean difference +/- 1.96 standard deviation of the differences, provide an interval within which 95% of differences between measurements by the two methods are expected to lie. We describe how graphical methods can be used to investigate the assumptions of the method and we also give confidence intervals. We extend the basic approach to data where there is a relationship between difference and magnitude, both with a simple logarithmic transformation approach and a new, more general, regression approach. We discuss the importance of the repeatability of each method separately and compare an estimate of this to the limits of agreement. We extend the limits of agreement approach to data with repeated measurements, proposing new estimates for equal numbers of replicates by each method on each subject, for unequal numbers of replicates, and for replicated data collected in pairs, where the underlying value of the quantity being measured is changing. Finally, we describe a nonparametric approach to comparing methods.

Integrity signals

Registry

Crossref · 10.1191/096228099673819272

Cites

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The inter-rater agreement between the two authors for quality appraisal was 85%, with Cohen's kappa = .70, indicating good to substantial agreement.

    “The inter-rater agreement between the two authors was 85% (McHugh, 2012), with the inter-rater reliability score (Cohen's kappa = .70; Bland & Altman, 1999, based on Landis & Koch, 1977) indicating a good to substantial level of agreement.”

    From Unraveling the Complexities of Offender Strategies as Part of Online Sexual Grooming and Technology-Assisted Child Sexual Abuse: A Systematic Review (2026) · cites this paper as Bland & Altman, 1999

  • Bland and Altman introduced the Bland-Altman (B&A) plot to describe agreement between two quantitative measurements.

    “Bland and Altman introduced the Bland-Altman (B&A) plot to describe agreement between two quantitative measurements (6).”

    From Understanding bland altman analysis (2015) · cites this paper as [6]

  • Bland and Altman established a method to quantify agreement between two quantitative measurements by constructing limits of agreement.

    “They established a method to quantify agreement between two quantitative measurements by constructing limits of agreement.”

    From Understanding bland altman analysis (2015) · cites this paper as [6]

  • Bland and Altman recommended that 95% of the data points should lie within ±2 standard deviations of the mean difference.

    “B&A recommended that 95% of the data points should lie within ± 2s of the mean difference.”

    From Understanding bland altman analysis (2015) · cites this paper as [6]

Cited by

Sorted by citations

Authors
Bland, J. M., & Altman, D. G.
Affiliations
  • St George's, University of London (GB)
  • University of Oxford (GB)
ORCID
Altman, D.G.
Venue
Statistical Methods in Medical Research
Publisher
SAGE Publications
Published
1999-06-01
Volume
8
Issue
2
Pages
135-160
ISSN
  • 0962-2802 (ISSN-L)
  • 1477-0334
DOI
10.1191/096228099673819272
OpenAlex
W2159759971
Cited by
9,619
References
0
Citations by year
2026: 393 · 2025: 421 · 2024: 483 · 2023: 494 · 2022: 559 · 2021: 681 · 2020: 610 · 2019: 506 · 2018: 485 · 2017: 457 · 2016: 412 · 2015: 437 · 2014: 461 · 2013: 388 · 2012: 401
FWCI
47.68
Type
journal article
Language
en
Open access
closed
Locations
  • Statistical Methods in Medical Research (publishedVersion)
  • PubMed (publishedVersion)
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
  • Health Sciences › Medicine › Physiology › Clinical Laboratory Practices and Quality Control
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
Keywords
method comparison, measurement agreement, limits of agreement, mean difference, repeatability, regression analysis, logarithmic transformation, repeated measures, nonparametric comparison, graphical method, confidence intervals
MeSH terms
Humans, Medical Laboratory Science, Reproducibility of Results, Confidence Intervals, Linear Models, Statistics, Nonparametric, Clinical Laboratory Techniques

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review