The DOI matches its Crossref record; 47 of the 47 references it cites were found in a registry or an index.
Quantitative imaging biomarkers: A review of statistical methods for technical performance assessment
Open accessAbstract
Technological developments and greater rigor in the quantitative measurement of biological features in medical images have given rise to an increased interest in using quantitative imaging biomarkers to measure changes in these features. Critical to the performance of a quantitative imaging biomarker in preclinical or clinical settings are three primary metrology areas of interest: measurement linearity and bias, repeatability, and the ability to consistently reproduce equivalent results when conditions change, as would be expected in any clinical trial. Unfortunately, performance studies to date differ greatly in designs, analysis method, and metrics used to assess a quantitative imaging biomarker for clinical use. It is therefore difficult or not possible to integrate results from different studies or to use reported results to design studies. The Radiological Society of North America and the Quantitative Imaging Biomarker Alliance with technical, radiological, and statistical experts developed a set of technical performance analysis methods, metrics, and study designs that provide terminology, metrics, and methods consistent with widely accepted metrological standards. This document provides a consistent framework for the conduct and evaluation of quantitative imaging biomarker performance studies so that results from multiple studies can be compared, contrasted, or combined.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
STATISTICAL METHODS FOR ASSESSING AGREEMENT BETWEEN TWO METHODS OF CLINICAL MEASUREMENT
Martin Bland, J., & Altman, D. (1986) · The Lancet · doi:10.1016/s0140-6736(86)90837-8 · Cited by 47,965
Intraclass correlations: Uses in assessing rater reliability.
Shrout, P. E., & Fleiss, J. L. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.2.420 · Cited by 24,532
(2005) · Technometrics · doi:10.1198/tech.2005.s303 · Cited by 17,923
Continuous Univariate Distributions.
Omori, Y., Johnson, N. L., Kotz, S., & Balakrishnan, N. (1995) · Journal of the American Statistical Association · doi:10.2307/2291547 · Cited by 9,084
A concordance correlation coefficient to evaluate reproducibility.
Lin, L. I.-K. (1989) · Biometrics · doi:10.2307/2532051 · Cited by 8,785
Biomarkers and surrogate endpoints: Preferred definitions and conceptual framework
(2001) · Clinical Pharmacology & Therapeutics · doi:10.1067/mcp.2001.113989 · Cited by 6,762
Introduction to Linear Regression Analysis.
Assuncao, R., Sampson, P. D., Montgomery, D. C., & Peck, E. A. (1993) · Journal of the American Statistical Association · doi:10.2307/2290746 · Cited by 5,388
Draper, N. R., & Smith, H. (1998) · Wiley Series in Probability and Statistics · doi:10.1002/9781118625590 · Cited by 5,024
Fitzmaurice, G. M., Laird, N. M., & Ware, J. H. (2011) · Wiley Series in Probability and Statistics · doi:10.1002/9781119513469 · Cited by 4,639
Applied Linear Regression Models
(1997) · Technometrics · doi:10.1080/00401706.1997.10485142 · Cited by 4,615
Small sample inference for fixed effects from restricted maximum likelihood
Kenward, M. G., & Roger, J. H. (1997) · Biometrics · doi:10.2307/2533558 · Cited by 4,552 · cited 2×
“A more complete discussion may be found in Kenward and Roger.16 In general, random effects randomly choose from a large population of levels of a factor while fixed effects do not randomly select, though the distinction is not always clearly defined in practice.”
Gunst, R. F. (1999) · Technometrics · doi:10.1080/00401706.1999.10485680 · Cited by 4,002
Dhamu Kp Ramamoorthy, K. (2016) · Cited by 2,962
Tietz Textbook of Clinical Chemistry and Molecular Diagnostics
Burtis, E. B. C. A., Ashwood, E. R., & Bruns, D. E. (2011) · Medical Entomology and Zoology · Cited by 2,190
The Path to Personalized Medicine
Hamburg, M. A., & Collins, F. S. (2010) · New England Journal of Medicine · doi:10.1056/nejmp1006304 · Cited by 1,970
Searle, S. R., Casella, G., & McCulloch, C. E. (1992) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316856 · Cited by 1,926
Contributions to Probability and Statistics: Essays in Honor of Harold Hotelling.
David, F. N., Olkin, I., Ghurye, S. G., Hoeffding, W., Madow, W. G., & Mann, H. B. (1961) · Journal of the Royal Statistical Society. Series A (General) · doi:10.2307/2984135 · Cited by 1,914
Introduction to linear regression analysis
Ober, P. B. (2013) · Journal of Applied Statistics · doi:10.1080/02664763.2013.816069 · Cited by 1,633
Transformation and Weighting in Regression
Carroll, R. J., & Ruppert, D. (1988) · Springer eBooks · doi:10.1007/978-1-4899-2873-3 · Cited by 1,299
Percentage Points for a Generalized ESD Many-Outlier Procedure
Rosner, B. (1983) · Technometrics · doi:10.1080/00401706.1983.10487848 · Cited by 1,268
Transformation and Weighting in Regression.
Lachenbruch, P. A., Carroll, R. J., & Ruppert, D. (1990) · Journal of the American Statistical Association · doi:10.2307/2289809 · Cited by 1,132
Guidelines for Reporting Reliability and Agreement Studies (GRRAS) were proposed
Kottner, J., Audige, L., Brorson, S., Donner, A., Gajewski, B. J., Hróbjartsson, A., Roberts, C., Shoukri, M., & Streiner, D. L. (2011) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2011.01.016 · Cited by 997
Analysis of Messy Data, Volume I: Designed Experiments
Schmee, J. (1985) · Technometrics · doi:10.1080/00401706.1985.10488628 · Cited by 923
An introduction to linear statistical models.
Graybill, F. A. (2013) · Medical Entomology and Zoology · Cited by 740
An Introduction to Linear Statistical Models
Zyskind, G. (1964) · Technometrics · doi:10.1080/00401706.1964.10490152 · Cited by 597
Berkson, J. (1950) · Journal of the American Statistical Association · doi:10.1080/01621459.1950.10483349 · Cited by 530
Rudd, J. H. F., Myers, K. S., Bansilal, S., Machac, J., Pinto, C. A., Tong, C., Rafique, A., Hargeaves, R., Farkouh, M., Fuster, V., et al. (2008) · Journal of Nuclear Medicine · doi:10.2967/jnumed.107.050294 · Cited by 444
Confidence Intervals on Variance Components
Burdick (1992) · CRC Press eBooks · doi:10.1201/9781482277142 · Cited by 350
An Overview on Assessing Agreement with Continuous Measurements
Barnhart, H. X., Haber, M. J., & Lin, L. I. (2007) · Journal of Biopharmaceutical Statistics · doi:10.1080/10543400701376480 · Cited by 319
Kessler, L. G., Barnhart, H. X., Buckler, A. J., Choudhury, K. R., Kondratovich, M. V., Toledano, A., Guimaraes, A. R., Filice, R., Zhang, Z., Sullivan, D. C., et al. (2015) · Statistical Methods in Medical Research · doi:10.1177/0962280214537333 · Cited by 293
Self-citationSpringer Handbook of Metrology and Testing
(2011) · Springer eBooks · doi:10.1007/978-3-642-16641-9 · Cited by 234
Analysis of Messy Data Volume 1: Designed Experiments, Second Edition
Milliken, G. A., & Johnson, D. E. (2009) · Chapman & Hall/CRC eBooks · doi:10.1201/ebk1584883340 · Cited by 223
Quantitative imaging biomarkers: A review of statistical methods for computer algorithm comparisons
Obuchowski, N. A., Reeves, A. P., Huang, E. P., Wang, X.-F., Buckler, A. J., Kim, H. J. (., Barnhart, H. X., Jackson, E. F., Giger, M. L., Pennello, G., et al. (2015) · Statistical Methods in Medical Research · doi:10.1177/0962280214537390 · Cited by 172
Self-citationBarnhart, H. X., & Barboriak, D. P. (2009) · Translational Oncology · doi:10.1593/tlo.09268 · Cited by 146
Velasquez, L. M., Boellaard, R., Kollia, G. D., Hayes, W., Hoekstra, O. S., Lammertsma, A. A., & Galbraith, S. M. (2009) · Journal of Nuclear Medicine · doi:10.2967/jnumed.109.063347 · Cited by 135
Confidence Intervals on Nonnegative Linear Combinations of Variances
Graybill, F. A., & Wang, C. (1980) · Journal of the American Statistical Association · doi:10.1080/01621459.1980.10477565 · Cited by 116
Phantom-less QCT BMD system as screening tool for osteoporosis without additional radiation
Mueller, D., Kutscherenko, A., Bartel, H., Vlassenbroek, A., Ouředníček, P., & Erckenbrecht, J. F. (2010) · European Journal of Radiology · doi:10.1016/j.ejrad.2010.02.008 · Cited by 116
A Comparison of Multivariate Outlier Detection Methods for Clinical Laboratory Safety Data
Penny, K. I., & Jolliffe, I. T. (2001) · Journal of the Royal Statistical Society Series D (The Statistician) · doi:10.1111/1467-9884.00279 · Cited by 104
Comparison of ICC and CCC for assessing agreement for data without and with replications
Chen, C., & Barnhart, H. X. (2008) · Computational Statistics & Data Analysis · doi:10.1016/j.csda.2008.09.026 · Cited by 99
Learning how to Differ: Agreement and Reliability Statistics in Psychiatry
Streiner, D. L. (1995) · The Canadian Journal of Psychiatry · doi:10.1177/070674379504000202 · Cited by 95
Computer programs for the concordance correlation coefficient
Crawford, S. B., Kosinski, A. S., Lin, H., Williamson, J. M., & Barnhart, H. X. (2007) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2007.07.003 · Cited by 81
Bhavane, R. C., Badea, C. T., Ghaghada, K. B., Clark, D. P., Vela, D. C., Moturu, A., Annapragada, A. V., Johnson, G. A., Willerson, J. T., & Annapragada, A. (2013) · Circulation Cardiovascular Imaging · doi:10.1161/circimaging.112.000119 · Cited by 75
Jacene, H. A., Leboulleux, S., Baba, S., Chatzifotiadis, D., Goudarzi, B., Teytelbaum, O., Horton, K. M., Kamel, I. R., Macura, K. J., Tsai, H., et al. (2009) · Journal of Nuclear Medicine · doi:10.2967/jnumed.109.063321 · Cited by 73
Self-citationHuang, E. P., Wang, X. F., Choudhury, K. R., McShane, L. M., Gönen, M., Ye, J., Buckler, A. J., Kinahan, P. E., Reeves, A. P., Jackson, E. F., et al. (2014) · Statistical Methods in Medical Research · doi:10.1177/0962280214537394 · Cited by 52
Self-citationSelection and Analytical Evaluations of Methods-With Statistical Techniques
Línnet, K., & Boyd, J. C. (2006) · Research at the University of Copenhagen (University of Copenhagen) · Cited by 43
Dybkær, R. (1997) · PubMed · Cited by 42
Haeckel, R., Wosniok, W., & Klauke, R. (2013) · Journal of Laboratory Medicine · doi:10.1515/labmed-2013-0003 · Cited by 22
Cited by
AI in Medical Imaging Informatics: Current Challenges and Future Directions
Panayides, A. S., Amini, A., Filipovic, N. D., Sharma, A., Tsaftaris, S. A., Young, A., Foran, D., Do, N., Golemati, S., Kurc, T., et al. (2020) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2020.2991043 · Cited by 713
Metrology Standards for Quantitative Imaging Biomarkers
Sullivan, D. C., Obuchowski, N. A., Kessler, L. G., Raunig, D. L., Gatsonis, C., Huang, E. P., Kondratovich, M., McShane, L. M., Reeves, A. P., Barboriak, D. P., et al. (2015) · Radiology · doi:10.1148/radiol.2015142202 · Cited by 435
Self-citationShukla‐Dave, A., Obuchowski, N. A., Chenevert, T. L., Jambawalikar, S., Schwartz, L. H., Malyarenko, D., Huang, W., Noworolski, S. M., Young, R. J., Shiroishi, M. S., et al. (2019) · Journal of Magnetic Resonance Imaging · doi:10.1002/jmri.26518 · Cited by 389
Park, J. E., Park, S. Y., Kim, H. J., & Kim, H. S. (2019) · Korean Journal of Radiology · doi:10.3348/kjr.2018.0070 · Cited by 388
Yokoo, T., Serai, S. D., Pirasteh, A., Bashir, M. R., Hamilton, G., Hernando, D., Hu, H. H., Hetterich, H., Kühn, J.-P., Kukuk, G. M., et al. (2018) · Radiology · doi:10.1148/radiol.2017170550 · Cited by 338
Metadata
- Authors
- Raunig, D. L., McShane, L. M., Pennello, G., Gatsonis, C., Carson, P. L., Voyvodic, J. T., Wahl, R. L., Kurland, B. F., Schwarz, A. J., Gönen, M., Zahlmann, G., Kondratovich, M. V., O’Donnell, K., Petrick, N., Cole, P. E., Garra, B., Sullivan, D. C., & QIBA Technical Performance Working Group
- Venue
- Statistical Methods in Medical Research
- Publisher
- SAGE Publications
- Published
- 2014-06-11
- Volume
- 24
- Issue
- 1
- Pages
- 27-67
- ISSN
- 0962-2802 (ISSN-L)
- 1477-0334
- OpenAlex
- W2008750507
- Cited by
- 394
- References
- 47
- FWCI
- 18.76
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.