The DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
Reproducibility of radiomics for deciphering tumor phenotype with imaging
Open accessAbstract
Radiomics (radiogenomics) characterizes tumor phenotypes based on quantitative image features derived from routine radiologic imaging to improve cancer diagnosis, prognosis, prediction and response to therapy. Although radiomic features must be reproducible to qualify as biomarkers for clinical care, little is known about how routine imaging acquisition techniques/parameters affect reproducibility. To begin to fill this knowledge gap, we assessed the reproducibility of a comprehensive, commonly-used set of radiomic features using a unique, same-day repeat computed tomography data set from lung cancer patients. Each scan was reconstructed at 6 imaging settings, varying slice thicknesses (1.25 mm, 2.5 mm and 5 mm) and reconstruction algorithms (sharp, smooth). Reproducibility was assessed using the repeat scans reconstructed at identical imaging setting (6 settings in total). In separate analyses, we explored differences in radiomic features due to different imaging parameters by assessing the agreement of these radiomic features extracted from the repeat scans reconstructed at the same slice thickness but different algorithms (3 settings in total). Our data suggest that radiomic features are reproducible over a wide range of imaging settings. However, smooth and sharp reconstruction algorithms should not be used interchangeably. These findings will raise awareness of the importance of properly setting imaging acquisition parameters in radiomics/radiogenomics research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
R: A Language and Environment for Statistical Computing
R Core Team (2025) · doi:10.32614/r.manuals · Cited by 352,181
A concordance correlation coefficient to evaluate reproducibility.
Lin, L. I.-K. (1989) · Biometrics · doi:10.2307/2532051 · Cited by 8,785
Radiomics: extracting more information from medical images using advanced feature analysis
Lambin, P., Rios-Velazquez, E., Leijenaar, R., Carvalho, S., van Stiphout, R. G. P. M., Granton, P., Zegers, C. M. L., Gillies, R., Boellard, R., Dekker, A., et al. (2012) · European Journal of Cancer · doi:10.1016/j.ejca.2011.11.036 · Cited by 6,084
Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach
Aerts, H. J. W. L., Velazquez, E. R., Leijenaar, R. T. H., Parmar, C., Grossmann, P., Carvalho, S., Bussink, J., Monshouwer, R., Haibe-Kains, B., Rietveld, D., et al. (2014) · Nature Communications · doi:10.1038/ncomms5006 · Cited by 5,132
Radiomics: the process and the challenges
Kumar, V., Gu, Y., Basu, S., Berglund, A., Eschrich, S. A., Schabath, M. B., Forster, K., Aerts, H. J. W. L., Dekker, A., Fenstermacher, D., et al. (2012) · Magnetic Resonance Imaging · doi:10.1016/j.mri.2012.06.010 · Cited by 2,258
Dixon, W. J. (1953) · Biometrics · doi:10.2307/3001634 · Cited by 868
Tixier, F., Le Rest, C. C., Hatt, M., Albarghach, N., Pradier, O., Metges, J.-P., Corcos, L., & Visvikis, D. (2011) · Journal of Nuclear Medicine · doi:10.2967/jnumed.110.082404 · Cited by 696
CT-based radiomic signature predicts distant metastasis in lung adenocarcinoma
Coroller, T. P., Grossmann, P., Hou, Y., Rios Velazquez, E., Leijenaar, R. T. H., Hermann, G., Lambin, P., Haibe-Kains, B., Mak, R. H., & Aerts, H. J. W. L. (2015) · Radiotherapy and Oncology · doi:10.1016/j.radonc.2015.02.015 · Cited by 695
Robust Radiomics Feature Quantification Using Semiautomatic Volumetric Segmentation
Parmar, C., Rios Velazquez, E., Leijenaar, R., Jermoumi, M., Carvalho, S., Mak, R. H., Mitra, S., Shankar, B. U., Kikinis, R., Haibe-Kains, B., et al. (2014) · PLoS ONE · doi:10.1371/journal.pone.0102107 · Cited by 611
Leijenaar, R. T. H., Carvalho, S., Velazquez, E. R., van Elmpt, W. J. C., Parmar, C., Hoekstra, O. S., Hoekstra, C. J., Boellaard, R., Dekker, A. L. A. J., Gillies, R. J., et al. (2013) · Acta Oncologica · doi:10.3109/0284186x.2013.812798 · Cited by 427
Cited by
Radiomics: the bridge between medical imaging and personalized medicine
Lambin, P., Leijenaar, R. T. H., Deist, T. M., Peerlings, J., de Jong, E. E. C., van Timmeren, J., Sanduleanu, S., Larue, R. T. H. M., Even, A. J. G., Jochems, A., et al. (2017) · Nature Reviews Clinical Oncology · doi:10.1038/nrclinonc.2017.141 · Cited by 5,688
Artificial intelligence in cancer imaging: Clinical challenges and applications
Bi, W. L., Hosny, A., Schabath, M. B., Giger, M. L., Birkbak, N. J., Mehrtash, A., Allison, T., Arnaout, O., Abbosh, C., Dunn, I. F., et al. (2019) · CA: A Cancer Journal for Clinicians · doi:10.3322/caac.21552 · Cited by 1,846
Self-citationRadiomics in medical imaging—“how-to” guide and critical reflection
van Timmeren, J. E., Cester, D., Tanadini-Lang, S., Alkadhi, H., & Baessler, B. (2020) · Insights into Imaging · doi:10.1186/s13244-020-00887-2 · Cited by 1,316
Nioche, C., Orlhac, F., Boughdad, S., Reuzé, S., Goya-Outi, J., Robert, C., Pellot-Barakat, C., Soussan, M., Frouin, F., & Buvat, I. (2018) · Cancer Research · doi:10.1158/0008-5472.can-18-0125 · Cited by 1,167
Liu, Z., Wang, S., Dong, D., Wei, J., Fang, C., Zhou, X., Sun, K., Li, L., Li, B., Wang, M., et al. (2019) · Theranostics · doi:10.7150/thno.30309 · Cited by 967
Repeatability and Reproducibility of Radiomic Features: A Systematic Review
Traverso, A., Wee, L., Dekker, A., & Gillies, R. (2018) · International Journal of Radiation Oncology*Biology*Physics · doi:10.1016/j.ijrobp.2018.05.053 · Cited by 903
Limkin, E. J., Sun, R., Dercle, L., Zacharaki, E. I., Robert, C., Reuzé, S., Schernberg, A., Paragios, N., Deutsch, E., & Ferté, C. (2017) · Annals of Oncology · doi:10.1093/annonc/mdx034 · Cited by 757
Intrinsic dependencies of CT radiomic features on voxel size and number of gray levels
Shafiq‐ul‐Hassan, M., Zhang, G. G., Latifi, K., Ullah, G., Hunt, D. C., Balagurunathan, Y., Abdalah, M. A., Schabath, M. B., Goldgof, D. G., Mackin, D., et al. (2017) · Medical Physics · doi:10.1002/mp.12123 · Cited by 585
Radiomics strategies for risk assessment of tumour failure in head-and-neck cancer
Vallières, M., Kay-Rivest, E., Perrin, L. J., Liem, X., Furstoss, C., Aerts, H. J. W. L., Khaouam, N., Nguyen-Tan, P. F., Wang, C.-S., Sultanem, K., et al. (2017) · Scientific Reports · doi:10.1038/s41598-017-10371-5 · Cited by 542
Beyond imaging: The promise of radiomics
Avanzo, M., Stancanello, J., & El Naqa, I. (2017) · Physica Medica · doi:10.1016/j.ejmp.2017.05.071 · Cited by 483
Radiomics and radiogenomics in lung cancer: A review for the clinician
Thawani, R., McLane, M., Beig, N., Ghose, S., Prasanna, P., Velcheti, V., & Madabhushi, A. (2018) · Lung Cancer · doi:10.1016/j.lungcan.2017.10.015 · Cited by 450
Scapicchio, C., Gabelloni, M., Barucci, A., Cioni, D., Saba, L., & Neri, E. (2021) · La radiologia medica · doi:10.1007/s11547-021-01389-x · Cited by 435
Defining the biological basis of radiomic phenotypes in lung cancer
Grossmann, P., Stringfield, O., El-Hachem, N., Bui, M. M., Rios Velazquez, E., Parmar, C., Leijenaar, R. T., Haibe-Kains, B., Lambin, P., Gillies, R. J., et al. (2017) · eLife · doi:10.7554/elife.23421 · Cited by 365
Larue, R. T. H. M., Defraene, G., De Ruysscher, D., Lambin, P., & van Elmpt, W. (2017) · The British Journal of Radiology · doi:10.1259/bjr.20160665 · Cited by 363
A review in radiomics: Making personalized medicine a reality via routine imaging
Guiot, J., Vaidyanathan, A., Deprez, L., Zerka, F., Danthine, D., Frix, A., Lambin, P., Bottari, F., Tsoutzidis, N., Miraglio, B., et al. (2022) · Medicinal Research Reviews · doi:10.1002/med.21846 · Cited by 337
Beig, N., Khorrami, M., Alilou, M., Prasanna, P., Braman, N., Orooji, M., Rakshit, S., Bera, K., Rajiah, P., Ginsberg, J., et al. (2019) · Radiology · doi:10.1148/radiol.2018180910 · Cited by 328
A radiogenomic dataset of non-small cell lung cancer
Bakr, S., Gevaert, O., Echegaray, S., Ayers, K., Zhou, M., Shafiq, M., Zheng, H., Benson, J. A., Zhang, W., Leung, A. N. C., et al. (2018) · Scientific Data · doi:10.1038/sdata.2018.202 · Cited by 311
Radiogenomics: bridging imaging and genomics
Bodalal, Z., Trebeschi, S., Nguyen-Kim, T. D. L., Schats, W., & Beets-Tan, R. (2019) · Abdominal Radiology · doi:10.1007/s00261-019-02028-w · Cited by 310
Choe, J., Lee, S. M., Do, K.-H., Lee, G., Lee, J.-G., Lee, S. M., & Seo, J. B. (2019) · Radiology · doi:10.1148/radiol.2019181960 · Cited by 296
Lee, G., Lee, H. Y., Park, H., Schiebler, M. L., van Beek, E. J. R., Ohno, Y., Seo, J. B., & Leung, A. (2017) · European Journal of Radiology · doi:10.1016/j.ejrad.2016.09.005 · Cited by 282
Peerlings, J., Woodruff, H. C., Winfield, J. M., Ibrahim, A., Van Beers, B. E., Heerschap, A., Jackson, A., Wildberger, J. E., Mottaghy, F. M., DeSouza, N. M., et al. (2019) · Scientific Reports · doi:10.1038/s41598-019-41344-5 · Cited by 266
Larue, R. T. H. M., van Timmeren, J. E., de Jong, E. E. C., Feliciani, G., Leijenaar, R. T. H., Schreurs, W. M. J., Sosef, M. N., Raat, F. H. P. J., van der Zande, F. H. R., Das, M., et al. (2017) · Acta Oncologica · doi:10.1080/0284186x.2017.1351624 · Cited by 262
Deep learning classification of lung cancer histology using CT images
Chaunzwa, T. L., Hosny, A., Xu, Y., Shafer, A., Diao, N., Lanuti, M., Christiani, D. C., Mak, R. H., & Aerts, H. J. W. L. (2021) · Scientific Reports · doi:10.1038/s41598-021-84630-x · Cited by 257
Identification of non-small cell lung cancer sensitive to systemic cancer therapies using radiomics
Dercle, L., Fronheiser, M., Lu, L., Du, S., Hayes, W., Leung, D. K., Roy, A., Wilkerson, J., Guo, P., Fojo, A. T., et al. (2020) · Clinical Cancer Research · doi:10.1158/1078-0432.ccr-19-2942 · Cited by 177
Self-citationUnderstanding Sources of Variation to Improve the Reproducibility of Radiomics
Zhao, B. (2021) · Frontiers in Oncology · doi:10.3389/fonc.2021.633176 · Cited by 158
Self-citationKaissis, G., Ziegelmayer, S., Lohöfer, F., Steiger, K., Algül, H., Muckenhuber, A., Yen, H.-Y., Rummeny, E., Friess, H., Schmid, R., et al. (2019) · PLOS ONE · doi:10.1371/journal.pone.0218642 · Cited by 76
Yamashita, R., Perrin, T., Chakraborty, J., Chou, J. F., Horvat, N., Koszalka, M. A., Midya, A., Gonen, M., Allen, P., Jarnagin, W. R., et al. (2020) · European Radiology · doi:10.1007/s00330-019-06381-8 · Cited by 76
Radiogenomics for Precision Medicine With a Big Data Analytics Perspective
Panayides, A. S., Pattichis, M. S., Leandrou, S., Pitris, C., Constantinidou, A., & Pattichis, C. S. (2019) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2018.2879381 · Cited by 66
Lu, L., Ahmed, F. S., Akin, O., Luk, L., Guo, X., Yang, H., Yoon, J., Hakimi, A. A., Schwartz, L. H., & Zhao, B. (2021) · Frontiers in Oncology · doi:10.3389/fonc.2021.638185 · Cited by 31
Self-citation
Metadata
- Authors
- Zhao, B., Tan, Y., Tsai, W.-Y., Qi, J., Xie, C., Lu, L., & Schwartz, L. H.
- Venue
- Scientific Reports
- Published
- 2016-03-24
- Volume
- 6
- Issue
- 1
- Pages
- 23428-23428
- ISSN
- 2045-2322 (ISSN-L)
- OpenAlex
- W2327203407
- Cited by
- 509
- References
- 28
- FWCI
- 44.68
- Type
- journal article
- Language
- en
- Open access
- gold
- Licence
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.