The DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.
Measuring agreement in method comparison studies
Abstract
Agreement between two methods of clinical measurement can be quantified using the differences between observations made using the two methods on the same subjects. The 95% limits of agreement, estimated by mean difference 1.96 standard deviation of the differences, provide an interval within which 95% of differences between measurements by the two methods are expected to lie. We describe how graphical methods can be used to investigate the assumptions of the method and we also give confidence intervals. We extend the basic approach to data where there is a relationship between difference and magnitude, both with a simple logarithmic transformation approach and a new, more general, regression approach. We discuss the importance of the repeatability of each method separately and compare an estimate of this to the limits of agreement. We extend the limits of agreement approach to data with repeated measurements, proposing new estimates for equal numbers of replicates by each method on each subject, for unequal numbers of replicates, and for replicated data collected in pairs, where the underlying value of the quantity being measured is changing. Finally, we describe a nonparametric approach to comparing methods.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“For the calculations, the mean of the Moxy passive trials was used against the corresponding active trial and therefore proper adjustments to SD were made as proposed by Bland and Altman.40”
From Near-infrared spectroscopy-derived muscle oxygen saturation on a 0% to 100% scale: reliability and validity of the Moxy Monitor (2019) · cites this paper as [40]
In the citing paper
“The variance for single differences between pairs of measurements on different subject is found by summing the between subjects and within subjects variances (Bland and Altman 1999).”
From Agreement Between Methods of Measurement with Multiple Observations Per Individual (2007) · cites this paper as Bland and Altman 1999
In the citing paper
“(In our earlier paper (Bland and Altman 1999), we incorrectly stated that this number should be used to multiply the difference in the sum of squares, rather than divide it.)”
From Agreement Between Methods of Measurement with Multiple Observations Per Individual (2007) · cites this paper as Bland and Altman 1999
In the citing paper
“It is valuable, however, to obtain replicate measurements by each method on each individual so that the repeatability of the two methods can be compared (Bland and Altman 1999).”
From Agreement Between Methods of Measurement with Multiple Observations Per Individual (2007) · cites this paper as Bland and Altman 1999
In the citing paper
“Although these have been described previously (Bland and Altman 1999) no numerical example was given for the case when the true value of the quantity being measured varied and also there was an error in the mathematical description.”
From Agreement Between Methods of Measurement with Multiple Observations Per Individual (2007) · cites this paper as Bland and Altman 1999
In the citing paper
“As described in section 5 in Bland and Altman (1999), we derive the estimated standard deviation for individual differences from the variance of the subject mean differences and an extra term derived from the separate measurement error of each method.”
From Agreement Between Methods of Measurement with Multiple Observations Per Individual (2007) · cites this paper as Bland and Altman (1999)
In the citing paper
“Bland–Altman comparisons reported good agreement for LLR vs. Navlit and Navopt, with > 95% and > 91.8% of measurements within the limits of agreement across all CPAK parameters, respectively.”
From Imageless robotic total knee arthroplasty determines similar coronal plane alignment of the knee (CPAK) parameters to long leg radiographs (2024) · cites this paper as [31]
In the citing paper
“Bland–Altman comparisons reported good agreement for LLR vs. Navlit and Navopt, with > 95% and > 91.8% of measurements within the limits of agreement across all CPAK parameters, respectively.”
From Imageless robotic total knee arthroplasty determines similar coronal plane alignment of the knee (CPAK) parameters to long leg radiographs (2024) · cites this paper as [31]
In the citing paper
“Current research into the HR accuracy of wrist-worn wearable devices attempt to compare the measurement between wearable PPG and a reference method, such as an ECG, in order to determine whether or not measurements between the devices are outside of clinically important limits of agreement, so that devices can supplement, eventually replace, or be used interchangeably37,38.”
From Guidelines for wrist-worn consumer wearable assessment of heart rate in biobehavioral research (2020) · cites this paper as [37]
Citations
Cites
STATISTICAL METHODS FOR ASSESSING AGREEMENT BETWEEN TWO METHODS OF CLINICAL MEASUREMENT
Martin Bland, J., & Altman, D. (1986) · The Lancet · doi:10.1016/s0140-6736(86)90837-8 · Cited by 47,965
Self-citationMeasurement in Medicine: The Analysis of Method Comparison Studies
Altman, D. G., & Bland, J. M. (1983) · The Statistician · doi:10.2307/2987937 · Cited by 4,300
Self-citationComparing methods of measurement: why plotting difference against standard method is misleading
Bland, J. M., & Altman, D. G. (1995) · The Lancet · doi:10.1016/s0140-6736(95)91748-9 · Cited by 2,343
Self-citationClinical assessment of gestational age in the newborn infant
Dubowitz, L. M. S., Dubowitz, V., & Goldberg, C. (1970) · The Journal of Pediatrics · doi:10.1016/s0022-3476(70)80038-5 · Cited by 2,230
Searle, S. R., Casella, G., & McCulloch, C. E. (1992) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316856 · Cited by 1,926
The Journal of Laboratory and Clinical Medicine
(2003) · Journal of Laboratory and Clinical Medicine · doi:10.1016/s0022-2143(03)00127-6 · Cited by 1,803
International Journal of Methods in Psychiatric Research
Bhugra, D., Bhui, K. S., Desai, M., Singh, J., & Baldwin, D. (2006) · International Journal of Methods in Psychiatric Research · doi:10.1002/(issn)1557-0657 · Cited by 1,492
Bland, M., & Altman, D. G. (1990) · Computers in Biology and Medicine · doi:10.1016/0010-4825(90)90013-f · Cited by 689
Self-citationSPECIAL ARTICLE COMPARING METHODS OF MEASUREMENT
Ludbrook, J. (1997) · Clinical and Experimental Pharmacology and Physiology · doi:10.1111/j.1440-1681.1997.tb01807.x · Cited by 363
Confidence Intervals on Variance Components
Burdick (1992) · CRC Press eBooks · doi:10.1201/9781482277142 · Cited by 351
Use and Interpretation of Common Statistical Tests in Method Comparison Studies
Westgard, J. O. (2008) · Clinical Chemistry · doi:10.1373/clinchem.2007.094060 · Cited by 311
Beevers, G., Lip, G. Y. H., & O'Brien, E. T. (1988) · Nursing Standard · doi:10.7748/ns.2.46.22.s48 · Cited by 307
Construction of age‐related reference centiles using absolute residuals
Altman, D. G. (1993) · Statistics in Medicine · doi:10.1002/sim.4780121003 · Cited by 296
Self-citationUse and Interpretation of Common Statistical Tests in Method-Comparison Studies
Westgard, J. O., & Hunt, M. R. (1973) · Clinical Chemistry · doi:10.1093/clinchem/19.1.49 · Cited by 265
Determination of Serum Immunoreactive Erythropoietin in the Investigation of Erythrocytosis
Cotes, P. M., Doré, C. J., Yin, J. A. L., Lewis, S. M., Messinezy, M., Pearson, T. C., & Reid, C. D. L. (1986) · New England Journal of Medicine · doi:10.1056/nejm198607313150503 · Cited by 127
Evaluation of method-comparison data.
Eksborg, S. (1981) · Clinical Chemistry · doi:10.1093/clinchem/27.7.1311 · Cited by 95
HPLC with enzymatic detection as a candidate reference method for serum creatinine
Línnet, K., & Bruunshuus, I. (1991) · Clinical Chemistry · doi:10.1093/clinchem/37.10.1669 · Cited by 44
Evaluating agreement between clinical assessment methods
Marshall, G. N., Hays, R. D., & Nicholas, R. (1995) · Cited by 43
Statistics in medical research: principles versus practices.
Schoolman, H. M., Becktel, J. M., Best, W. R., & Johnson, A. F. (1968) · PubMed · Cited by 33
Feinstein, A. R. (1976) · Clinical Pharmacology & Therapeutics · doi:10.1002/cpt1976205617 · Cited by 28
An evaluation of the Vita-Stat automatic blood pressure measuring device.
Polk, B. F., ROSNER, B. A., Feudo, R., & VanDenburgh, M. (1980) · Hypertension · doi:10.1161/01.hyp.2.2.221 · Cited by 22
Leverton, M. J. (1991) · The Lancet · doi:10.1016/0140-6736(91)91206-a · Cited by 22
Clinical assessment of gestational age in the newborn infant. Comparison of two methods
Latis, G., Simionato, L., & Ferraris, G. (1981) · Early Human Development · doi:10.1016/0378-3782(81)90068-2 · Cited by 21
BOWLING, L. S., Sageman, W. S., OʼCONNOR, S. M., et al. (1993) · Critical Care Medicine · doi:10.1097/00003246-199310000-00021 · Cited by 20
An automated enzymic micromethod for the measurement of fat in human milk
Lucas, A., Hudson, G. J., Simpson, P., Cole, T. J., & Baker, B. A. (1987) · Journal of Dairy Research · doi:10.1017/s0022029900025693 · Cited by 12
Research works printed but not linked: 2
- O'Brien E, 1993, Journal of Hypertension, 11, S43
- Altman DG, 1987, Applied Statistics, 36, 224
Cited by
Sorted by citations
An overview of heart rate variability metrics and norms
Shaffer, F., & Ginsberg, J. P. (2017) · Frontiers in Public Health · doi:10.3389/fpubh.2017.00258 · Cited by 7,185
Understanding bland altman analysis
Giavarina, D. (2015) · Biochemia Medica · doi:10.11613/bm.2015.015 · Cited by 4,022
Agreement Between Methods of Measurement with Multiple Observations Per Individual
Bland, J. M., & Altman, D. G. (2007) · Journal of Biopharmaceutical Statistics · doi:10.1080/10543400701329422 · Cited by 1,864
Self-citationDevelopment, validation and utilisation of food-frequency questionnaires – a review
Cade, J., Thompson, R., Burley, V., & Warm, D. (2002) · Public Health Nutrition · doi:10.1079/phn2001318 · Cited by 1,423
Marino, M., Li, Y., Rueschman, M. N., Winkelman, J. W., Ellenbogen, J. M., Solet, J. M., Dulin, H., Berkman, L. F., & Buxton, O. M. (2013) · Sleep · doi:10.5665/sleep.3142 · Cited by 1,056
Metadata
- Authors
- Bland, J. M., & Altman, D. G.
- Affiliations
- St George's, University of London (GB)
- University of Oxford (GB)
- ORCID
- Altman, Douglas G
- Venue
- Statistical Methods in Medical Research
- Publisher
- SAGE Publications
- Published
- 1999-04
- Volume
- 8
- Issue
- 2
- Pages
- 135-160
- ISSN
- 0962-2802 (ISSN-L)
- 1477-0334
- OpenAlex
- W2159759971
- Cited by
- 9,619
- References
- 25
- Citations by year
- 2026: 393 · 2025: 421 · 2024: 483 · 2023: 494 · 2022: 559 · 2021: 681 · 2020: 610 · 2019: 506 · 2018: 485 · 2017: 457 · 2016: 412 · 2015: 437 · 2014: 461 · 2013: 388 · 2012: 401
- FWCI
- 47.68
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Statistical Methods in Medical Research (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
- Health Sciences › Health Professions › Medical Laboratory Technology › Clinical Laboratory Practices and Quality Control
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- method comparison, measurement agreement, limits of agreement, mean difference, repeatability, regression analysis, logarithmic transformation, repeated measures, nonparametric comparison, graphical method, confidence intervals
- MeSH terms
- Humans, Medical Laboratory Science, Reproducibility of Results, Confidence Intervals, Linear Models, Statistics, Nonparametric, Clinical Laboratory Techniques
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
