Measuring agreement in method comparison studies
Abstract
Agreement between two methods of clinical measurement can be quantified using the differences between observations made using the two methods on the same subjects. The 95% limits of agreement, estimated by mean difference 1.96 standard deviation of the differences, provide an interval within which 95% of differences between measurements by the two methods are expected to lie. We describe how graphical methods can be used to investigate the assumptions of the method and we also give confidence intervals. We extend the basic approach to data where there is a relationship between difference and magnitude, both with a simple logarithmic transformation approach and a new, more general, regression approach. We discuss the importance of the repeatability of each method separately and compare an estimate of this to the limits of agreement. We extend the limits of agreement approach to data with repeated measurements, proposing new estimates for equal numbers of replicates by each method on each subject, for unequal numbers of replicates, and for replicated data collected in pairs, where the underlying value of the quantity being measured is changing. Finally, we describe a nonparametric approach to comparing methods.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
For calculations comparing active and passive trials, the mean of the Moxy passive trials was used against the corresponding active trial and proper adjustments to SD were made as proposed by Bland and Altman.
“For the calculations, the mean of the Moxy passive trials was used against the corresponding active trial and therefore proper adjustments to SD were made as proposed by Bland and Altman.40”
From Near-infrared spectroscopy-derived muscle oxygen saturation on a 0% to 100% scale: reliability and validity of the Moxy Monitor (2019) · cites this paper as [40]
In the non-constant case, the variance for single differences between pairs of measurements on different subjects is found by summing the between-subjects and within-subjects variances.
“The variance for single differences between pairs of measurements on different subject is found by summing the between subjects and within subjects variances (Bland and Altman 1999).”
From Agreement Between Methods of Measurement with Multiple Observations Per Individual (2007) · cites this paper as Bland and Altman 1999
In Bland and Altman (1999), we incorrectly stated that this number should be used to multiply the difference in the sum of squares, rather than divide it.
“(In our earlier paper (Bland and Altman 1999), we incorrectly stated that this number should be used to multiply the difference in the sum of squares, rather than divide it.)”
From Agreement Between Methods of Measurement with Multiple Observations Per Individual (2007) · cites this paper as Bland and Altman 1999
Obtaining replicate measurements by each method on each individual allows the repeatability of the two methods to be compared.
“It is valuable, however, to obtain replicate measurements by each method on each individual so that the repeatability of the two methods can be compared (Bland and Altman 1999).”
From Agreement Between Methods of Measurement with Multiple Observations Per Individual (2007) · cites this paper as Bland and Altman 1999
Although these methods have been described previously (Bland and Altman 1999), no numerical example was given for the case when the true value varied, and there was an error in the mathematical description.
“Although these have been described previously (Bland and Altman 1999) no numerical example was given for the case when the true value of the quantity being measured varied and also there was an error in the mathematical description.”
From Agreement Between Methods of Measurement with Multiple Observations Per Individual (2007) · cites this paper as Bland and Altman 1999
In the constant case, the estimated standard deviation for individual differences is derived from the variance of subject mean differences and an extra term derived from the separate measurement error of each method.
“As described in section 5 in Bland and Altman (1999), we derive the estimated standard deviation for individual differences from the variance of the subject mean differences and an extra term derived from the separate measurement error of each method.”
From Agreement Between Methods of Measurement with Multiple Observations Per Individual (2007) · cites this paper as Bland and Altman (1999)
Bland-Altman comparisons showed that more than 91.8% of measurements were within the limits of agreement for LLR vs Navopt across all CPAK parameters.
“Bland–Altman comparisons reported good agreement for LLR vs. Navlit and Navopt, with > 95% and > 91.8% of measurements within the limits of agreement across all CPAK parameters, respectively.”
From Imageless robotic total knee arthroplasty determines similar coronal plane alignment of the knee (CPAK) parameters to long leg radiographs (2024) · cites this paper as [31]
Bland-Altman comparisons showed that more than 95% of measurements were within the limits of agreement for LLR vs Navlit across all CPAK parameters.
“Bland–Altman comparisons reported good agreement for LLR vs. Navlit and Navopt, with > 95% and > 91.8% of measurements within the limits of agreement across all CPAK parameters, respectively.”
From Imageless robotic total knee arthroplasty determines similar coronal plane alignment of the knee (CPAK) parameters to long leg radiographs (2024) · cites this paper as [31]
Current research into heart rate accuracy of wrist-worn wearable devices compares wearable photoplethysmography measurement with a reference method such as electrocardiogram to determine whether measurements between devices are outside clinically important limits of agreement.
“Current research into the HR accuracy of wrist-worn wearable devices attempt to compare the measurement between wearable PPG and a reference method, such as an ECG, in order to determine whether or not measurements between the devices are outside of clinically important limits of agreement, so that devices can supplement, eventually replace, or be used interchangeably37,38.”
From Guidelines for wrist-worn consumer wearable assessment of heart rate in biobehavioral research (2020) · cites this paper as [37]
Cites
STATISTICAL METHODS FOR ASSESSING AGREEMENT BETWEEN TWO METHODS OF CLINICAL MEASUREMENT
Martin Bland, J., & Altman, D. (1986) · The Lancet · doi:10.1016/s0140-6736(86)90837-8 · Cited by 47,965 · see citing works
Self-citationMeasurement in Medicine: The Analysis of Method Comparison Studies
Altman, D. G., & Bland, J. M. (1983) · The Statistician · doi:10.2307/2987937 · Cited by 4,300 · see citing works
Self-citationComparing methods of measurement: why plotting difference against standard method is misleading
Bland, J. M., & Altman, D. G. (1995) · The Lancet · doi:10.1016/s0140-6736(95)91748-9 · Cited by 2,343 · see citing works
Self-citationClinical assessment of gestational age in the newborn infant
Dubowitz, L. M. S., Dubowitz, V., & Goldberg, C. (1970) · The Journal of Pediatrics · doi:10.1016/s0022-3476(70)80038-5 · Cited by 2,223 · see citing works
Searle, S. R., Casella, G., & McCulloch, C. E. (1992) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316856 · Cited by 1,926 · see citing works
The Journal of Laboratory and Clinical Medicine
(2003) · Journal of Laboratory and Clinical Medicine · doi:10.1016/s0022-2143(03)00127-6 · Cited by 1,767 · see citing works
International Journal of Methods in Psychiatric Research
Bhugra, D., Bhui, K. S., Desai, M., Singh, J., & Baldwin, D. (2006) · International Journal of Methods in Psychiatric Research · doi:10.1002/(issn)1557-0657 · Cited by 1,492 · see citing works
Bland, M., & Altman, D. G. (1990) · Computers in Biology and Medicine · doi:10.1016/0010-4825(90)90013-f · Cited by 689 · see citing works
Self-citationSPECIAL ARTICLE COMPARING METHODS OF MEASUREMENT
Ludbrook, J. (1997) · Clinical and Experimental Pharmacology and Physiology · doi:10.1111/j.1440-1681.1997.tb01807.x · Cited by 363 · see citing works
Confidence Intervals on Variance Components
Burdick (1992) · CRC Press eBooks · doi:10.1201/9781482277142 · Cited by 351 · see citing works
Use and Interpretation of Common Statistical Tests in Method Comparison Studies
Westgard, J. O. (2008) · Clinical Chemistry · doi:10.1373/clinchem.2007.094060 · Cited by 311 · see citing works
Beevers, G., Lip, G. Y. H., & O'Brien, E. T. (1988) · Nursing Standard · doi:10.7748/ns.2.46.22.s48 · Cited by 307 · see citing works
Construction of age‐related reference centiles using absolute residuals
Altman, D. G. (1993) · Statistics in Medicine · doi:10.1002/sim.4780121003 · Cited by 296 · see citing works
Self-citationUse and Interpretation of Common Statistical Tests in Method-Comparison Studies
Westgard, J. O., & Hunt, M. R. (1973) · Clinical Chemistry · doi:10.1093/clinchem/19.1.49 · Cited by 265 · see citing works
Determination of Serum Immunoreactive Erythropoietin in the Investigation of Erythrocytosis
Cotes, P. M., Doré, C. J., Yin, J. A. L., Lewis, S. M., Messinezy, M., Pearson, T. C., & Reid, C. D. L. (1986) · New England Journal of Medicine · doi:10.1056/nejm198607313150503 · Cited by 127 · see citing works
Evaluation of method-comparison data.
Eksborg, S. (1981) · Clinical Chemistry · doi:10.1093/clinchem/27.7.1311 · Cited by 95 · see citing works
HPLC with enzymatic detection as a candidate reference method for serum creatinine
Línnet, K., & Bruunshuus, I. (1991) · Clinical Chemistry · doi:10.1093/clinchem/37.10.1669 · Cited by 44 · see citing works
Evaluating agreement between clinical assessment methods
Marshall, G. N., Hays, R. D., & Nicholas, R. (1995) · Cited by 43 · see citing works
Statistics in medical research: principles versus practices.
Schoolman, H. M., Becktel, J. M., Best, W. R., & Johnson, A. F. (1968) · PubMed · Cited by 33 · see citing works
Feinstein, A. R. (1976) · Clinical Pharmacology & Therapeutics · doi:10.1002/cpt1976205617 · Cited by 28 · see citing works
An evaluation of the Vita-Stat automatic blood pressure measuring device.
Polk, B. F., ROSNER, B. A., Feudo, R., & VanDenburgh, M. (1980) · Hypertension · doi:10.1161/01.hyp.2.2.221 · Cited by 22 · see citing works
Leverton, M. J. (1991) · The Lancet · doi:10.1016/0140-6736(91)91206-a · Cited by 22 · see citing works
Clinical assessment of gestational age in the newborn infant. Comparison of two methods
Latis, G., Simionato, L., & Ferraris, G. (1981) · Early Human Development · doi:10.1016/0378-3782(81)90068-2 · Cited by 21 · see citing works
BOWLING, L. S., Sageman, W. S., OʼCONNOR, S. M., et al. (1993) · Critical Care Medicine · doi:10.1097/00003246-199310000-00021 · Cited by 20 · see citing works
An automated enzymic micromethod for the measurement of fat in human milk
Lucas, A., Hudson, G. J., Simpson, P., Cole, T. J., & Baker, B. A. (1987) · Journal of Dairy Research · doi:10.1017/s0022029900025693 · Cited by 12 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
An overview of heart rate variability metrics and norms
Shaffer, F., & Ginsberg, J. P. (2017) · Frontiers in Public Health · doi:10.3389/fpubh.2017.00258 · Cited by 7,185 · see citing works
Understanding bland altman analysis
Giavarina, D. (2015) · Biochemia Medica · doi:10.11613/bm.2015.015 · Cited by 4,022 · see citing works
Agreement Between Methods of Measurement with Multiple Observations Per Individual
Bland, J. M., & Altman, D. G. (2007) · Journal of Biopharmaceutical Statistics · doi:10.1080/10543400701329422 · Cited by 1,864 · see citing works
Self-citationDevelopment, validation and utilisation of food-frequency questionnaires – a review
Cade, J., Thompson, R., Burley, V., & Warm, D. (2002) · Public Health Nutrition · doi:10.1079/phn2001318 · Cited by 1,423 · see citing works
Marino, M., Li, Y., Rueschman, M. N., Winkelman, J. W., Ellenbogen, J. M., Solet, J. M., Dulin, H., Berkman, L. F., & Buxton, O. M. (2013) · Sleep · doi:10.5665/sleep.3142 · Cited by 1,056 · see citing works
Interrater agreement and interrater reliability: Key concepts, approaches, and applications
Gisev, N., Bell, J. S., & Chen, T. F. (2013) · Research in Social and Administrative Pharmacy · doi:10.1016/j.sapharm.2012.04.004 · Cited by 924 · see citing works
Analysis of the Accuracy and Robustness of the Leap Motion Controller
Weichert, F., Bachmann, D., Rudak, B., & Fisseler, D. (2013) · Sensors · doi:10.3390/s130506380 · Cited by 913 · see citing works
The new AASM criteria for scoring hypopneas: impact on the apnea hypopnea index
Ruehland, W. R., Rochford, P. D., O’Donoghue, F. J., Pierce, R. J., Singh, P., & Thornton, A. T. (2009) · SLEEP · doi:10.1093/sleep/32.2.150 · Cited by 696 · see citing works
Warren, J. M., Ekelund, U., Besson, H., Mezzani, A., Geladas, N., & Vanhees, L. (2010) · European Journal of Cardiovascular Prevention & Rehabilitation · doi:10.1097/hjr.0b013e32832ed875 · Cited by 587 · see citing works
Wang, C., Catlin, D. H., Demers, L. M., Starcevic, B., & Swerdloff, R. S. (2004) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/jc.2003-031287 · Cited by 580 · see citing works
Wearable Sleep Technology in Clinical and Research Settings
DE ZAMBOTTI, M., CELLINI, N., GOLDSTONE, A., COLRAIN, I. M., & BAKER, F. C. (2019) · Medicine & Science in Sports & Exercise · doi:10.1249/mss.0000000000001947 · Cited by 564 · see citing works
Food-frequency questionnaires: a review of their design, validation and utilisation
Cade, J. E., Burley, V. J., Warm, D. L., Thompson, R. L., & Margetts, B. M. (2004) · Nutrition Research Reviews · doi:10.1079/nrr200370 · Cited by 518 · see citing works
Celis-Morales, C. A., Perez-Bravo, F., Ibañez, L., Salas, C., Bailey, M. E. S., & Gill, J. M. R. (2012) · PLoS ONE · doi:10.1371/journal.pone.0036345 · Cited by 486 · see citing works
McAlinden, C., Khadka, J., & Pesudovs, K. (2011) · Ophthalmic and Physiological Optics · doi:10.1111/j.1475-1313.2011.00851.x · Cited by 436 · see citing works
Li, X., Dunn, J., Salins, D., Zhou, G., Zhou, W., Schüssler-Fiorenza Rose, S. M., Perelman, D., Colbert, E., Runge, R., Rego, S., et al. (2017) · PLOS Biology · doi:10.1371/journal.pbio.2001402 · Cited by 431 · see citing works
Short-term variability in measures of glycemia and implications for the classification of diabetes
Selvin, E. (2007) · Archives of Internal Medicine · doi:10.1001/archinte.167.14.1545 · Cited by 402 · see citing works
Diarrhoea-predominant IBS patients show mast cell activation and hyperplasia in the jejunum
Guilarte, M., Santos, J., de Torres, I., Alonso, C., Vicario, M., Ramos, L., Martinez, C., Casellas, F., Saperas, E., & Malagelada, J. R. (2007) · Gut · doi:10.1136/gut.2006.100594 · Cited by 385 · see citing works
Palmqvist, S., Zetterberg, H., Blennow, K., Vestberg, S., Andreasson, U., Brooks, D. J., Owenius, R., Hägerström, D., Wollmer, P., Minthon, L., et al. (2014) · JAMA Neurology · doi:10.1001/jamaneurol.2014.1358 · Cited by 368 · see citing works
Cecconi, M., Rhodes, A., Poloniecki, J., Della Rocca, G., & Grounds, R. M. (2009) · Critical Care · doi:10.1186/cc7129 · Cited by 351 · see citing works
Schaffarczyk, M., Rogers, B., Reer, R., & Gronwald, T. (2022) · Sensors · doi:10.3390/s22176536 · Cited by 349 · see citing works
Influence of hemolysis on routine clinical chemistry testing
Lippi, G., Luca Salvagno, G., Montagnana, M., Brocco, G., & Cesare Guidi, G. (2006) · Clinical Chemistry and Laboratory Medicine (CCLM) · doi:10.1515/cclm.2006.054 · Cited by 344 · see citing works
Goligher, E. C., Laghi, F., Detsky, M. E., Farias, P., Murray, A., Brace, D., Brochard, L. J., Bolz, S.-S., Rubenfeld, G. D., Kavanagh, B. P., et al. (2015) · Intensive Care Medicine · doi:10.1007/s00134-015-3724-2 · Cited by 315 · see citing works
Test-retest reliability of evoked BOLD signals from a cognitive-emotive fMRI test battery.
Plichta, M. M., Schwarz, A. J., Grimm, O., Morgen, K., Mier, D., Haddad, L., Gerdes, A. B. M., Sauer, C., Tost, H., Esslinger, C., et al. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.01.129 · Cited by 287 · see citing works
Nelson, B. W., & Allen, N. B. (2019) · JMIR mHealth and uHealth · doi:10.2196/10828 · Cited by 310 · see citing works
Thermal Indices and Thermophysiological Modeling for Heat Stress
Havenith, G., & Fiala, D. (2015) · Comprehensive Physiology · doi:10.1002/cphy.c140051 · Cited by 262 · see citing works
Rafi, Z., & Greenland, S. (2020) · BMC Medical Research Methodology · doi:10.1186/s12874-020-01105-9 · Cited by 243 · see citing works
Guidelines for wrist-worn consumer wearable assessment of heart rate in biobehavioral research
Nelson, B. W., Low, C. A., Jacobson, N., Areán, P., Torous, J., & Allen, N. B. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0297-4 · Cited by 241 · see citing works
Sharman, J. E., Avolio, A. P., Baulmann, J., Benetos, A., Blacher, J., Blizzard, C. L., Boutouyrie, P., Chen, C.-H., Chowienczyk, P., Cockcroft, J. R., et al. (2017) · European Heart Journal · doi:10.1093/eurheartj/ehw632 · Cited by 231 · see citing works
Detection of anaemia from retinal fundus images via deep learning
Mitani, A., Huang, A., Venugopalan, S., Corrado, G. S., Peng, L., Webster, D. R., Hammel, N., Liu, Y., & Varadarajan, A. V. (2019) · Nature Biomedical Engineering · doi:10.1038/s41551-019-0487-z · Cited by 230 · see citing works
A Primer on the Use of Equivalence Testing for Evaluating Measurement Agreement
DIXON, P. M., SAINT-MAURICE, P. F., KIM, Y., HIBBING, P., BAI, Y., & WELK, G. J. (2018) · Medicine & Science in Sports & Exercise · doi:10.1249/mss.0000000000001481 · Cited by 227 · see citing works
Cuff-Less and Continuous Blood Pressure Monitoring: A Methodological Review
Sharma, M., Barbosa, K., Ho, V., Griggs, D., Ghirmai, T., Krishnan, S., Hsiai, T., Chiao, J.-C., & Cao, H. (2017) · Technologies · doi:10.3390/technologies5020021 · Cited by 226 · see citing works
Methods of Blood Pressure Measurement in the ICU*
Lehman, L.-W. H., Saeed, M., Talmor, D., Mark, R., & Malhotra, A. (2013) · Critical Care Medicine · doi:10.1097/ccm.0b013e318265ea46 · Cited by 224 · see citing works
Repeatability of SUV in Oncologic <sup>18</sup>F-FDG PET
Lodge, M. A. (2017) · Journal of Nuclear Medicine · doi:10.2967/jnumed.116.186353 · Cited by 214 · see citing works
Janzen, N., Peter, M., Sander, S., Steuerwald, U., Terhardt, M., Holtkamp, U., & Sander, J. (2007) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/jc.2006-2890 · Cited by 211 · see citing works
Simonsick, E. M., Fan, E., & Fleg, J. L. (2006) · Journal of the American Geriatrics Society · doi:10.1111/j.1532-5415.2005.00530.x · Cited by 199 · see citing works
Accurate Determination of Energy Needs in Hospitalized Patients
Boullata, J., Williams, J., Cottrell, F., Hudson, L., & Compher, C. (2007) · Journal of the American Dietetic Association · doi:10.1016/j.jada.2006.12.014 · Cited by 201 · see citing works
Piche, T., Saint-Paul, M. C., Dainese, R., Marine-Barjoan, E., Iannelli, A., Montoya, M. L., Peyron, J. F., Czerucka, D., Cherikh, F., Filippi, J., et al. (2008) · Gut · doi:10.1136/gut.2007.127068 · Cited by 198 · see citing works
Oscillometric Blood Pressure Estimation: Past, Present, and Future
Forouzanfar, M., Dajani, H. R., Groza, V. Z., Bolic, M., Rajan, S., & Batkin, I. (2015) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2015.2434215 · Cited by 196 · see citing works
Novel quantitative phenotypes of exercise training in mouse models
De Bono, J. P., Adlam, D., Paterson, D. J., & Channon, K. M. (2006) · American Journal of Physiology-Regulatory, Integrative and Comparative Physiology · doi:10.1152/ajpregu.00694.2005 · Cited by 197 · see citing works
Correlation of serum allergy (IgE) tests performed by different assay systems
Wang, J., Godbold, J. H., & Sampson, H. A. (2008) · Journal of Allergy and Clinical Immunology · doi:10.1016/j.jaci.2007.12.1150 · Cited by 188 · see citing works
Surveillance for ocular hypertension: an evidence synthesis and economic evaluation.
Burr, J., Botello-Pinzon, P., Takwoingi, Y., Hernández, R., Vazquez-Montes, M., Elders, A., Asaoka, R., Banister, K., van der Schoot, J., Fraser, C., et al. (2012) · Health Technology Assessment · doi:10.3310/hta16290 · Cited by 180 · see citing works
Tully, M. A., McBride, C., Heron, L., & Hunter, R. F. (2014) · BMC Research Notes · doi:10.1186/1756-0500-7-952 · Cited by 172 · see citing works
Feldmann, A., Schmitz, R. W., & Erlacher, D. (2019) · Journal of Biomedical Optics · doi:10.1117/1.jbo.24.11.115001 · Cited by 159 · see citing works
“For the calculations, the mean of the Moxy passive trials was used against the corresponding active trial and therefore proper adjustments to SD were made as proposed by Bland and Altman.40 2.5.3 Face validity: 0% to 100% scale To determine face validity, a comparison was made between SmO2 results from the AOM passive…”
MethodInfluentialStressing the accuracy: Wrist‐worn wearable sensor validation over different conditions
Menghini, L., Gianfranchi, E., Cellini, N., Patron, E., Tagliabue, M., & Sarlo, M. (2019) · Psychophysiology · doi:10.1111/psyp.13441 · Cited by 157 · see citing works
Assessment of the Fitbit Charge 2 for monitoring heart rate
Benedetto, S., Caldato, C., Bazzan, E., Greenwood, D. C., Pensabene, V., & Actis, P. (2018) · PLOS ONE · doi:10.1371/journal.pone.0192691 · Cited by 149 · see citing works
Ilyas, H., Mikhaeel, N. G., Dunn, J. T., Rahman, F., Møller, H., Smith, D., & Barrington, S. F. (2018) · European Journal of Nuclear Medicine and Molecular Imaging · doi:10.1007/s00259-018-3953-z · Cited by 150 · see citing works
Measuring physical activity during pregnancy
Harrison, C. L., Thompson, R. G., Teede, H. J., & Lombard, C. B. (2011) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/1479-5868-8-19 · Cited by 139 · see citing works
von Wowern, E., Östling, G., Nilsson, P. M., & Olofsson, P. (2015) · PLOS ONE · doi:10.1371/journal.pone.0135659 · Cited by 134 · see citing works
“We described agreement between the two DPA measurements with Bland-Altman plots [36,37].”
Thompson, D., Batterham, A. M., Bock, S., Robson, C., & Stokes, K. (2006) · The Journal of Nutrition · doi:10.1093/jn/136.4.1037 · Cited by 129 · see citing works
Accuracy of a PDA-based dietary assessment program
Beasley, J., Riley, W. T., & Jean-Mary, J. (2005) · Nutrition · doi:10.1016/j.nut.2004.11.006 · Cited by 124 · see citing works
A non-invasive device to continuously determine heat strain in humans
Gunga, H.-C., Sandsund, M., Reinertsen, R. E., Sattler, F., & Koch, J. (2008) · Journal of Thermal Biology · doi:10.1016/j.jtherbio.2008.03.004 · Cited by 129 · see citing works
Teufl, W., Miezal, M., Taetz, B., Fröhlich, M., & Bleser, G. (2019) · PLOS ONE · doi:10.1371/journal.pone.0213064 · Cited by 129 · see citing works
Valbuena, V. S. M., Seelye, S., Sjoding, M. W., Valley, T. S., Dickson, R. P., Gay, S. E., Claar, D., Prescott, H. C., & Iwashyna, T. J. (2022) · BMJ · doi:10.1136/bmj-2021-069775 · Cited by 127 · see citing works
Protocols for evaluating equivalency of accelerometry-based activity monitors
WELK, G. J., MCCLAIN, J., & AINSWORTH, B. E. (2012) · Medicine & Science in Sports & Exercise · doi:10.1249/mss.0b013e3182399d8f · Cited by 124 · see citing works
Gunga, H.-C., Werner, A., Stahn, A., Steinach, M., Schlabs, T., Koralewski, E., Kunz, D., Belavý, D. L., Felsenberg, D., Sattler, F., et al. (2009) · Respiratory Physiology & Neurobiology · doi:10.1016/j.resp.2009.04.005 · Cited by 122 · see citing works
Mulder, E. R., de Jong, R. A., Knol, D. L., van Schijndel, R. A., Cover, K. S., Visser, P. J., Barkhof, F., & Vrenken, H. (2014) · NeuroImage · doi:10.1016/j.neuroimage.2014.01.058 · Cited by 122 · see citing works
Day-to-day variability in spot urine albumin-creatinine ratio
Naresh, C. N., Hayen, A., Weening, A., Craig, J. C., & Chadban, S. J. (2013) · American Journal of Kidney Diseases · doi:10.1053/j.ajkd.2013.06.016 · Cited by 107 · see citing works
Drawz, P. E., Agarwal, A., Dwyer, J. P., Horwitz, E., Lash, J., Lenoir, K., McWilliams, A., Oparil, S., Rahbari-Oskoui, F., Rahman, M., et al. (2020) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2020.5028 · Cited by 105 · see citing works
Jacobs, E. M. G., Hendriks, J. C. M., van Tits, B. L. J. H., Evans, P. J., Breuer, W., Liu, D. Y., Jansen, E. H. J. M., Jauhiainen, K., Sturm, B., Porter, J. B., et al. (2005) · Analytical Biochemistry · doi:10.1016/j.ab.2005.03.008 · Cited by 101 · see citing works
Farah, N. M., Saw Yee, T., & Mohd Rasdi, H. F. (2019) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph16234750 · Cited by 99 · see citing works
Mortensen, K. H., Hjerrild, B. E., Stochholm, K., Andersen, N. H., Sørensen, K. E., Lundorf, E., Hørlyck, A., Pedersen, E. M., Christiansen, J. S., & Gravholt, C. H. (2011) · Journal of Cardiovascular Magnetic Resonance · doi:10.1186/1532-429x-13-24 · Cited by 99 · see citing works
Reproducibility and repeatability of MRI‐based body composition analysis
Borga, M., Ahlgren, A., Romu, T., Widholm, P., Dahlqvist Leinhard, O., & West, J. (2020) · Magnetic Resonance in Medicine · doi:10.1002/mrm.28360 · Cited by 94 · see citing works
Ball, T. J., Joy, E. A., Gren, L. H., & Shaw, J. M. (2016) · Preventing Chronic Disease · doi:10.5888/pcd13.150228 · Cited by 89 · see citing works
Venema, L. F., Post, W. J., Hendriks, H. G., Huet, R. C., de Wolf, J. T., & de Vries, A. J. (2010) · Anesthesia & Analgesia · doi:10.1213/ane.0b013e3181e368bc · Cited by 92 · see citing works
Measurement of blood glucose: comparison between different types of specimens
Carstensen, B., Lindström, J., Sundvall, J., Borch-Johnsen, K., & Tuomilehto, J. (2008) · Annals of Clinical Biochemistry: International Journal of Laboratory Medicine · doi:10.1258/acb.2007.006212 · Cited by 88 · see citing works
Hanggi, M. P., Teuscher, U. M., Roos, M., & Peltomaki, T. A. (2008) · The European Journal of Orthodontics · doi:10.1093/ejo/cjn055 · Cited by 85 · see citing works
Assessment of ACTH assay variability: a multicenter study
Pecori Giraldi, F., Saccani, A., Cavagnini, F., & _, _. (2011) · European Journal of Endocrinology · doi:10.1530/eje-10-0962 · Cited by 80 · see citing works
Stöckl, D., Rodríguez Cabaleiro, D., Van Uytfanghe, K., & Thienpont, L. M. (2004) · Clinical Chemistry · doi:10.1373/clinchem.2004.036095 · Cited by 79 · see citing works
ADHD in Swedish 3- to 7-year-old children
KADESJÖ, C., KADESJÖ, B., HÄGGLÖF, B., & GILLBERG, C. (2001) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1097/00004583-200109000-00010 · Cited by 72 · see citing works
Roski, F., Hammel, J., Mei, K., Baum, T., Kirschke, J. S., Laugerette, A., Kopp, F. K., Bodden, J., Pfeiffer, D., Pfeiffer, F., et al. (2019) · European Radiology · doi:10.1007/s00330-019-06263-z · Cited by 70 · see citing works
Short-term venous stasis influences routine coagulation testing
Lippi, G., Salvagno, G. L., Montagnana, M., & Guidi, G. C. (2005) · Blood Coagulation & Fibrinolysis · doi:10.1097/01.mbc.0000178828.59866.03 · Cited by 66 · see citing works
Tractenberg, R. E., Jonklaas, J., & Soldin, S. J. (2010) · International Journal of Analytical Chemistry · doi:10.1155/2010/234808 · Cited by 64 · see citing works
Gonzales, T. I., Westgate, K., Strain, T., Hollidge, S., Jeon, J., Christensen, D. L., Jensen, J., Wareham, N. J., & Brage, S. (2021) · Scientific Reports · doi:10.1038/s41598-021-94768-3 · Cited by 59 · see citing works
Yu, S., Powell, A., Khow, K., & Visvanathan, R. (2016) · Nutrients · doi:10.3390/nu8040189 · Cited by 58 · see citing works
Rogers, B., Giles, D., Draper, N., Mourot, L., & Gronwald, T. (2021) · Sensors · doi:10.3390/s21030821 · Cited by 54 · see citing works
Spineli, L. M. (2019) · BMC Medical Research Methodology · doi:10.1186/s12874-019-0731-y · Cited by 47 · see citing works
Awadh, A. M. A., Prefumo, F., Bland, J. M., & Carvalho, J. S. (2006) · Ultrasound in Obstetrics & Gynecology · doi:10.1002/uog.2813 · Cited by 50 · see citing works
Self-citationPecoraro, V., Petri, D., Costantino, G., Squizzato, A., Moja, L., Virgili, G., & Lucenteforte, E. (2021) · Internal and Emergency Medicine · doi:10.1007/s11739-020-02556-0 · Cited by 49 · see citing works
“96 times the standard deviation (SD) of these differences (random error), which are the two components of the 95% coefficient of reproducibility [14].”
A new simple technique for 3-dimensional sonographic assessment of the pelvic floor muscles
Youssef, A., Montaguti, E., Sanlorenzo, O., Cariello, L., Awad, E. E., Pacella, G., Ghi, T., Pilu, G., & Rizzo, N. (2015) · Journal of Ultrasound in Medicine · doi:10.7863/ultra.34.1.65 · Cited by 48 · see citing works
Riley, L. E., Chen, G. J., & Latham, H. E. (2017) · Blood Pressure Monitoring · doi:10.1097/mbp.0000000000000258 · Cited by 43 · see citing works
Bilo, G., Sala, O., Perego, C., Faini, A., Gao, L., Głuszewska, A., Ochoa, J. E., Pellegrini, D., Lonati, L. M., & Parati, G. (2017) · Hypertension Research · doi:10.1038/hr.2016.184 · Cited by 43 · see citing works
An evaluation of two capillary sample collection kits for laboratory measurement of HbA1c
Beck, R. W., Bocchino, L. E., Lum, J. W., Kollman, C., Barnes-Lomen, V., Sulik, M., Haller, M. J., Bode, B., Cernich, J. T., Killeen, A. A., et al. (2021) · Diabetes Technology & Therapeutics · doi:10.1089/dia.2021.0023 · Cited by 42 · see citing works
Rogers, B., Schaffarczyk, M., Clauß, M., Mourot, L., & Gronwald, T. (2022) · Sensors · doi:10.3390/s22052032 · Cited by 41 · see citing works
Using photoplethysmography in heart rate monitoring of patients with epilepsy
van Andel, J., Ungureanu, C., Aarts, R., Leijten, F., & Arends, J. (2015) · Epilepsy & Behavior · doi:10.1016/j.yebeh.2015.02.018 · Cited by 39 · see citing works
Haghayegh, S., Khoshnevis, S., Smolensky, M. H., Diller, K. R., & Castriotta, R. J. (2019) · Chronobiology International · doi:10.1080/07420528.2019.1679826 · Cited by 38 · see citing works
Lee, Q. Y., Redmond, S. J., Chan, G. S., Middleton, P. M., Steel, E., Malouf, P., Critoph, C., Flynn, G., O’Lone, E., & Lovell, N. H. (2013) · BioMedical Engineering OnLine · doi:10.1186/1475-925x-12-19 · Cited by 34 · see citing works
Gillette-Guyonnet, S. (2003) · Mechanisms of Ageing and Development · doi:10.1016/s0047-6374(02)00199-9 · Cited by 32 · see citing works
Wireless Vital Sign Monitoring in Pregnant Women: A Functionality and Acceptability Study
Boatin, A. A., Wylie, B. J., Goldfarb, I., Azevedo, R., Pittel, E., Ng, C., & Haberer, J. E. (2016) · Telemedicine and e-Health · doi:10.1089/tmj.2015.0173 · Cited by 30 · see citing works
Mielke, G. I., de Almeida Mendes, M., Ekelund, U., Rowlands, A. V., Reichert, F. F., & Crochemore‐Silva, I. (2023) · Scandinavian Journal of Medicine & Science in Sports · doi:10.1111/sms.14416 · Cited by 26 · see citing works
James, D., Umekwe, N., Edeoga, C., Nyenwe, E., & Dagogo-Jack, S. (2020) · Metabolism · doi:10.1016/j.metabol.2020.154263 · Cited by 23 · see citing works
Automated quantitative evaluation of fetal atrioventricular annular plane systolic excursion
Herling, L., Johnson, J., Ferm‐Widlund, K., Zamprakou, A., Westgren, M., & Acharya, G. (2021) · Ultrasound in Obstetrics & Gynecology · doi:10.1002/uog.23703 · Cited by 22 · see citing works
van Hooff, M., Meijer, E. J., Scheltinga, M. R. M., Savelberg, H. H. C. M., & Schep, G. (2022) · Clinical Physiology and Functional Imaging · doi:10.1111/cpf.12738 · Cited by 20 · see citing works
Ginsburg, A. S., Nkwopara, E., Macharia, W., Ochieng, R., Waiyego, M., Zhou, G., Karasik, R., Xu, S., & Ansermino, J. M. (2020) · BMJ Open · doi:10.1136/bmjopen-2019-035184 · Cited by 16 · see citing works
Proper use of noncontact infrared thermometry for temperature screening during COVID-19
Hussain, A. S., Hussain, H. S., Betcher, N., Behm, R., & Cagir, B. (2021) · Scientific Reports · doi:10.1038/s41598-021-90100-1 · Cited by 16 · see citing works
Pianta, M. J., Makrai, E., Verspoor, K. M., Cohn, T. A., & Downie, L. E. (2018) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2018.07.015 · Cited by 15 · see citing works
Dante, A., Franconi, I., Marucci, A. R., Alfes, C. M., & Lancia, L. (2020) · Journal of Pediatric Nursing · doi:10.1016/j.pedn.2019.11.014 · Cited by 13 · see citing works
Turnes, T., Penteado dos Santos, R., de Aguiar, R. A., Loch, T., Possamai, L. T., & Caputo, F. (2019) · International Journal of Sports Physiology and Performance · doi:10.1123/ijspp.2018-0675 · Cited by 15 · see citing works
Therapy monitoring in congenital adrenal hyperplasia by dried blood samples
Wieacker, I., Peter, M., Borucki, K., Empting, S., Roehl, F.-W., & Mohnike, K. (2015) · Journal of Pediatric Endocrinology and Metabolism · doi:10.1515/jpem-2014-0303 · Cited by 14 · see citing works
Haxha, S., Nwibor, C., Ali, M., Sakel, M., Saunders, K., Dyo, V., & Nabakooza, S. (2024) · Sensors · doi:10.3390/s24113301 · Cited by 12 · see citing works
Rossato, S. L., Mosele, F., Moreira, L. B., Rodrigues, M. P., Lima, R. F., Fuchs, F. D., & Fuchs, S. C. (2021) · Nutrients · doi:10.3390/nu13113881 · Cited by 11 · see citing works
- Authors
- Bland, J. M., & Altman, D. G.
- Affiliations
- St George's, University of London (GB)
- University of Oxford (GB)
- ORCID
- Altman, Douglas G
- Venue
- Statistical Methods in Medical Research
- Publisher
- SAGE Publications
- Published
- 1999-04
- Volume
- 8
- Issue
- 2
- Pages
- 135-160
- ISSN
- 0962-2802 (ISSN-L)
- 1477-0334
- OpenAlex
- W2159759971
- Cited by
- 9,619
- References
- 25
- Citations by year
- 2026: 393 · 2025: 421 · 2024: 483 · 2023: 494 · 2022: 559 · 2021: 681 · 2020: 610 · 2019: 506 · 2018: 485 · 2017: 457 · 2016: 412 · 2015: 437 · 2014: 461 · 2013: 388 · 2012: 401
- FWCI
- 47.68
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Statistical Methods in Medical Research (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
- Health Sciences › Medicine › Physiology › Clinical Laboratory Practices and Quality Control
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- method comparison, measurement agreement, limits of agreement, mean difference, repeatability, regression analysis, logarithmic transformation, repeated measures, nonparametric comparison, graphical method, confidence intervals
- MeSH terms
- Humans, Medical Laboratory Science, Reproducibility of Results, Confidence Intervals, Linear Models, Statistics, Nonparametric, Clinical Laboratory Techniques
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
