The DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.
Measuring agreement in method comparison studies
Abstract
Agreement between two methods of clinical measurement can be quantified using the differences between observations made using the two methods on the same subjects. The 95% limits of agreement, estimated by mean difference 1.96 standard deviation of the differences, provide an interval within which 95% of differences between measurements by the two methods are expected to lie. We describe how graphical methods can be used to investigate the assumptions of the method and we also give confidence intervals. We extend the basic approach to data where there is a relationship between difference and magnitude, both with a simple logarithmic transformation approach and a new, more general, regression approach. We discuss the importance of the repeatability of each method separately and compare an estimate of this to the limits of agreement. We extend the limits of agreement approach to data with repeated measurements, proposing new estimates for equal numbers of replicates by each method on each subject, for unequal numbers of replicates, and for replicated data collected in pairs, where the underlying value of the quantity being measured is changing. Finally, we describe a nonparametric approach to comparing methods.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“For the calculations, the mean of the Moxy passive trials was used against the corresponding active trial and therefore proper adjustments to SD were made as proposed by Bland and Altman.40”
From Near-infrared spectroscopy-derived muscle oxygen saturation on a 0% to 100% scale: reliability and validity of the Moxy Monitor (2019) · cites this paper as [40]
In the citing paper
“The variance for single differences between pairs of measurements on different subject is found by summing the between subjects and within subjects variances (Bland and Altman 1999).”
From Agreement Between Methods of Measurement with Multiple Observations Per Individual (2007) · cites this paper as Bland and Altman 1999
In the citing paper
“(In our earlier paper (Bland and Altman 1999), we incorrectly stated that this number should be used to multiply the difference in the sum of squares, rather than divide it.)”
From Agreement Between Methods of Measurement with Multiple Observations Per Individual (2007) · cites this paper as Bland and Altman 1999
In the citing paper
“It is valuable, however, to obtain replicate measurements by each method on each individual so that the repeatability of the two methods can be compared (Bland and Altman 1999).”
From Agreement Between Methods of Measurement with Multiple Observations Per Individual (2007) · cites this paper as Bland and Altman 1999
In the citing paper
“Although these have been described previously (Bland and Altman 1999) no numerical example was given for the case when the true value of the quantity being measured varied and also there was an error in the mathematical description.”
From Agreement Between Methods of Measurement with Multiple Observations Per Individual (2007) · cites this paper as Bland and Altman 1999
In the citing paper
“As described in section 5 in Bland and Altman (1999), we derive the estimated standard deviation for individual differences from the variance of the subject mean differences and an extra term derived from the separate measurement error of each method.”
From Agreement Between Methods of Measurement with Multiple Observations Per Individual (2007) · cites this paper as Bland and Altman (1999)
In the citing paper
“Bland–Altman comparisons reported good agreement for LLR vs. Navlit and Navopt, with > 95% and > 91.8% of measurements within the limits of agreement across all CPAK parameters, respectively.”
From Imageless robotic total knee arthroplasty determines similar coronal plane alignment of the knee (CPAK) parameters to long leg radiographs (2024) · cites this paper as [31]
In the citing paper
“Bland–Altman comparisons reported good agreement for LLR vs. Navlit and Navopt, with > 95% and > 91.8% of measurements within the limits of agreement across all CPAK parameters, respectively.”
From Imageless robotic total knee arthroplasty determines similar coronal plane alignment of the knee (CPAK) parameters to long leg radiographs (2024) · cites this paper as [31]
In the citing paper
“Current research into the HR accuracy of wrist-worn wearable devices attempt to compare the measurement between wearable PPG and a reference method, such as an ECG, in order to determine whether or not measurements between the devices are outside of clinically important limits of agreement, so that devices can supplement, eventually replace, or be used interchangeably37,38.”
From Guidelines for wrist-worn consumer wearable assessment of heart rate in biobehavioral research (2020) · cites this paper as [37]
Citations
Cites
STATISTICAL METHODS FOR ASSESSING AGREEMENT BETWEEN TWO METHODS OF CLINICAL MEASUREMENT
Martin Bland, J., & Altman, D. (1986) · The Lancet · doi:10.1016/s0140-6736(86)90837-8 · Cited by 47,965
Self-citationMeasurement in Medicine: The Analysis of Method Comparison Studies
Altman, D. G., & Bland, J. M. (1983) · The Statistician · doi:10.2307/2987937 · Cited by 4,300
Self-citationComparing methods of measurement: why plotting difference against standard method is misleading
Bland, J. M., & Altman, D. G. (1995) · The Lancet · doi:10.1016/s0140-6736(95)91748-9 · Cited by 2,343
Self-citationClinical assessment of gestational age in the newborn infant
Dubowitz, L. M. S., Dubowitz, V., & Goldberg, C. (1970) · The Journal of Pediatrics · doi:10.1016/s0022-3476(70)80038-5 · Cited by 2,230
Searle, S. R., Casella, G., & McCulloch, C. E. (1992) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316856 · Cited by 1,926
The Journal of Laboratory and Clinical Medicine
(2003) · Journal of Laboratory and Clinical Medicine · doi:10.1016/s0022-2143(03)00127-6 · Cited by 1,803
International Journal of Methods in Psychiatric Research
Bhugra, D., Bhui, K. S., Desai, M., Singh, J., & Baldwin, D. (2006) · International Journal of Methods in Psychiatric Research · doi:10.1002/(issn)1557-0657 · Cited by 1,492
Bland, M., & Altman, D. G. (1990) · Computers in Biology and Medicine · doi:10.1016/0010-4825(90)90013-f · Cited by 689
Self-citationSPECIAL ARTICLE COMPARING METHODS OF MEASUREMENT
Ludbrook, J. (1997) · Clinical and Experimental Pharmacology and Physiology · doi:10.1111/j.1440-1681.1997.tb01807.x · Cited by 363
Confidence Intervals on Variance Components
Burdick (1992) · CRC Press eBooks · doi:10.1201/9781482277142 · Cited by 351
Research works printed but not linked: 2
- O'Brien E, 1993, Journal of Hypertension, 11, S43
- Altman DG, 1987, Applied Statistics, 36, 224
Cited by
Sorted by citations
Items 101–113 of 113
Haxha, S., Nwibor, C., Ali, M., Sakel, M., Saunders, K., Dyo, V., & Nabakooza, S. (2024) · Sensors · doi:10.3390/s24113301 · Cited by 12
Rossato, S. L., Mosele, F., Moreira, L. B., Rodrigues, M. P., Lima, R. F., Fuchs, F. D., & Fuchs, S. C. (2021) · Nutrients · doi:10.3390/nu13113881 · Cited by 11
Karapinar, T., Tumer, K. C., & Buczinski, S. (2020) · Journal of Veterinary Internal Medicine · doi:10.1111/jvim.15794 · Cited by 11
Edelstein, A. I., Orsi, A. D., Plaskos, C., Coffey, S., & Suleiman, L. I. (2024) · Arthroplasty · doi:10.1186/s42836-024-00231-9 · Cited by 11
“Bland-Altman plots were used to assess agreement between LLR observers, LLR mean and Nav lit , and LLR mean and Nav opt , and the percentage of cases within the expected 95% limits of agreement (LOA, i.e., within ± 1.96 SD) were calculated for all CPAK parameters [31, 32].”
MethodInfluentialAutomated determination of distance visual acuity: towards teleophthalmology services
Kumar, S., Bulsara, M., & Yogesan, K. (2008) · Clinical and Experimental Optometry · doi:10.1111/j.1444-0938.2008.00267.x · Cited by 10
Sørensen, J. S., Vilstrup, M. H., Holm, J., Vogsen, M., Bülow, J. L., Ljungstrøm, L., Braad, P.-E., Gerke, O., & Hildebrandt, M. G. (2020) · Diagnostics · doi:10.3390/diagnostics10121001 · Cited by 10
Validity of self-reported weight and height among female young adults in the United Arab Emirates
Haroun, D., & Ehsanallah, A. (2024) · PLOS ONE · doi:10.1371/journal.pone.0302439 · Cited by 5
Kim, B., Youm, C., Park, H., Choi, H., & Shin, S. (2024) · Frontiers in Public Health · doi:10.3389/fpubh.2024.1376736 · Cited by 4
Östh, S., Helge, T., & Ekblom-Bak, E. (2025) · Sport Sciences for Health · doi:10.1007/s11332-025-01396-9 · Cited by 2
Kostrna, J., Oparina, E., Palacios, C., Rodriguez, A. J., Pei, J., Ajmal, A., & Ramella-Roman, J. C. (2026) · Sensors · doi:10.3390/s26102922 · Cited by 5
Ventura, T. P., Borszcz, F. K., Aguiar, R. A. D., de Lucas, R. D., & Turnes, T. (2026) · Applied Physiology, Nutrition, and Metabolism · doi:10.1139/apnm-2025-0240 · Cited by 1
Vázquez-Bautista, M. A., Robles-Sardin, A. E., Reséndiz-Sandoval, M., Navarro-Ibarra, M. J., Hernández López, J., & Caire-Juvera, G. (2025) · BIOtecnia · doi:10.18633/biotecnia.v27.2512
“To evaluate the agreement between methods, Bland-Altman analysis was performed with a one sample t -test comparing the difference between methods against a null hypothesis equals to zero to identify systematic bias (Bland and Altman, 1986; 1999).”
Porcelli, C., Anderson, F., & Kloess, J. A. (2026) · Trauma, Violence, & Abuse · doi:10.1177/15248380251411261
Metadata
- Authors
- Bland, J. M., & Altman, D. G.
- Affiliations
- St George's, University of London (GB)
- University of Oxford (GB)
- ORCID
- Altman, Douglas G
- Venue
- Statistical Methods in Medical Research
- Publisher
- SAGE Publications
- Published
- 1999-04
- Volume
- 8
- Issue
- 2
- Pages
- 135-160
- ISSN
- 0962-2802 (ISSN-L)
- 1477-0334
- OpenAlex
- W2159759971
- Cited by
- 9,619
- References
- 25
- Citations by year
- 2026: 393 · 2025: 421 · 2024: 483 · 2023: 494 · 2022: 559 · 2021: 681 · 2020: 610 · 2019: 506 · 2018: 485 · 2017: 457 · 2016: 412 · 2015: 437 · 2014: 461 · 2013: 388 · 2012: 401
- FWCI
- 47.68
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Statistical Methods in Medical Research (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
- Health Sciences › Health Professions › Medical Laboratory Technology › Clinical Laboratory Practices and Quality Control
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- method comparison, measurement agreement, limits of agreement, mean difference, repeatability, regression analysis, logarithmic transformation, repeated measures, nonparametric comparison, graphical method, confidence intervals
- MeSH terms
- Humans, Medical Laboratory Science, Reproducibility of Results, Confidence Intervals, Linear Models, Statistics, Nonparametric, Clinical Laboratory Techniques
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
