The DOI matches its Crossref record; 27 of the 27 references it cites were found in a registry or an index.
Statistical Methods Used to Test for Agreement of Medical Instruments Measuring Continuous Variables in Method Comparison Studies: A Systematic Review
Open accessAbstract
BACKGROUND: Accurate values are a must in medicine. An important parameter in determining the quality of a medical instrument is agreement with a gold standard. Various statistical methods have been used to test for agreement. Some of these methods have been shown to be inappropriate. This can result in misleading conclusions about the validity of an instrument. The Bland-Altman method is the most popular method judging by the many citations of the article proposing this method. However, the number of citations does not necessarily mean that this method has been applied in agreement research. No previous study has been conducted to look into this. This is the first systematic review to identify statistical methods used to test for agreement of medical instruments. The proportion of various statistical methods found in this review will also reflect the proportion of medical instruments that have been validated using those particular methods in current clinical practice. METHODOLOGY/FINDINGS: Five electronic databases were searched between 2007 and 2009 to look for agreement studies. A total of 3,260 titles were initially identified. Only 412 titles were potentially related, and finally 210 fitted the inclusion criteria. The Bland-Altman method is the most popular method with 178 (85%) studies having used this method, followed by the correlation coefficient (27%) and means comparison (18%). Some of the inappropriate methods highlighted by Altman and Bland since the 1980s are still in use. CONCLUSIONS: This study finds that the Bland-Altman method is the most popular method used in agreement research. There are still inappropriate applications of statistical methods in some studies. It is important for a clinician or medical researcher to be aware of this issue because misleading conclusions from inappropriate analyses will jeopardize the quality of the evidence, which in turn will influence quality of care given to patients in the future.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
STATISTICAL METHODS FOR ASSESSING AGREEMENT BETWEEN TWO METHODS OF CLINICAL MEASUREMENT
Martin Bland, J., & Altman, D. (1986) · The Lancet · doi:10.1016/s0140-6736(86)90837-8 · Cited by 47,965
Statistical methods for assessing agreement between two methods of clinical measurement
Bland, J. M., & Altman, D. G. (2010) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2009.10.001 · Cited by 5,220
Measurement in Medicine: The Analysis of Method Comparison Studies
Altman, D. G., & Bland, J. M. (1983) · The Statistician · doi:10.2307/2987937 · Cited by 4,300
Guidelines for Reporting Reliability and Agreement Studies (GRRAS) were proposed
Kottner, J., Audigé, L., Brorson, S., Donner, A., Gajewski, B. J., Hróbjartsson, A., Roberts, C., Shoukri, M., & Streiner, D. L. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.03.002 · Cited by 2,174
Validation and reproducibility of food frequency questionnaire for Korean genome epidemiologic study
Ahn, Y., Kwon, E., Shim, J. E., Park, M. K., Joo, Y., Kimm, K., Park, C., & Kim, D. H. (2007) · European Journal of Clinical Nutrition · doi:10.1038/sj.ejcn.1602657 · Cited by 784
Reliability: what is it, and how is it measured?
Bruton, A., Conway, J. H., & Holgate, S. T. (2000) · Physiotherapy · doi:10.1016/s0031-9406(05)61211-4 · Cited by 763 · cited 1×
“If the variance between subjects is high, the value of ICC will certainly appear to be high [33].”
BackgroundBland, J. M., & Altman, D. G. (1990) · Computers in Biology and Medicine · doi:10.1016/0010-4825(90)90013-f · Cited by 691
Interpretation and Uses of Medical Statistics
Daly, L. E., & Bourke, G. J. (2000) · Wiley eBooks · doi:10.1002/9780470696750 · Cited by 558
Jaffrin, M. Y., & Morel, H. (2008) · Medical Engineering & Physics · doi:10.1016/j.medengphy.2008.06.009 · Cited by 490
Statistical evaluation of agreement between two methods for measuring a quantitative variable
Lee, J., Koh, D., & Ong, C. N. (1989) · Computers in Biology and Medicine · doi:10.1016/0010-4825(89)90036-x · Cited by 473
Cited by
Nelson, B. W., & Allen, N. B. (2019) · JMIR mHealth and uHealth · doi:10.2196/10828 · Cited by 310
A Primer on the Use of Equivalence Testing for Evaluating Measurement Agreement
DIXON, P. M., SAINT-MAURICE, P. F., KIM, Y., HIBBING, P., BAI, Y., & WELK, G. J. (2018) · Medicine & Science in Sports & Exercise · doi:10.1249/mss.0000000000001481 · Cited by 227
Haghayegh, S., Kang, H.-A., Khoshnevis, S., Smolensky, M. H., & Diller, K. R. (2020) · Physiological Measurement · doi:10.1088/1361-6579/ab86d6 · Cited by 152
Validation of an IMU Gait Analysis Algorithm for Gait Monitoring in Daily Life Situations
Zhou, L., Tunca, C., Fischer, E., Brahms, C. M., Ersoy, C., Granacher, U., & Arnrich, B. (2020) · 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC) · doi:10.1109/embc44109.2020.9176827 · Cited by 63
Metadata
- Authors
- Zaki, R., Bulgiba, A., Ismail, R., & Ismail, N. A.
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2012-05-25
- Volume
- 7
- Issue
- 5
- Pages
- e37908
- ISSN
- 1932-6203 (ISSN-L)
- OpenAlex
- W2029660335
- Cited by
- 293
- References
- 27
- FWCI
- 4.25
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Statistical Methods Used to Test for Agreement of Medical Instruments Measuring Continuous Variables in Method Comparison Studies: A Systematic Review. https://app.cochat.ai/papers/10.1371/journal.pone.0037908
- The paper
- Zaki, R., Bulgiba, A., Ismail, R., & Ismail, N. A. (2012). Statistical Methods Used to Test for Agreement of Medical Instruments Measuring Continuous Variables in Method Comparison Studies: A Systematic Review. PLoS ONE. https://doi.org/10.1371/journal.pone.0037908
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.