High quality machine‐robust image features: Identification in nonsmall cell lung cancer computed tomography images
Open accessThe DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Abstract
Purpose: For nonsmall cell lung cancer (NSCLC) patients, quantitative image features extracted from computed tomography (CT) images can be used to improve tumor diagnosis, staging, and response assessment. For these findings to be clinically applied, image features need to have high intra and intermachine reproducibility. The objective of this study is to identify CT image features that are reproducible, nonredundant, and informative across multiple machines. Methods: Noncontrast‐enhanced, test‐retest CT image pairs were obtained from 56 NSCLC patients imaged on three CT machines from two institutions. Two machines (“M1” and “M2”) used cine 4D‐CT and one machine (“M3”) used breath‐hold helical 3D‐CT. Gross tumor volumes (GTVs) were semiautonomously segmented then pruned by removing voxels with CT numbers less than a prescribed Hounsfield unit (HU) cutoff. Three hundred and twenty eight quantitative image features were extracted from each pruned GTV based on its geometry, intensity histogram, absolute gradient image, co‐occurrence matrix, and run‐length matrix. For each machine, features with concordance correlation coefficient values greater than 0.90 were considered reproducible. The Dice similarity coefficient (DSC) and the Jaccard index (JI) were used to quantify reproducible feature set agreement between machines. Multimachine reproducible feature sets were created by taking the intersection of individual machine reproducible feature sets. Redundant features were removed through hierarchical clustering based on the average correlation between features across multiple machines. Results: For all image types, GTV pruning was found to negatively affect reproducibility (reported results use no HU cutoff). The reproducible feature percentage was highest for average images (M1 = 90.5%, M2 = 94.5%, M1∩M2 = 86.3%), intermediate for end‐exhale images (M1 = 75.0%, M2 = 71.0%, M1∩M2 = 52.1%), and lowest for breath‐hold images (M3 = 61.0%). Between M1 and M2, the reproducible feature sets generated from end‐exhale images were relatively machine‐sensitive (DSC = 0.71, JI = 0.55), and the reproducible feature sets generated from average images were relatively machine‐insensitive (DSC = 0.90, JI = 0.87). Histograms of feature pair correlation distances indicated that feature redundancy was machine‐sensitive and image type sensitive. After hierarchical clustering, 38 features, 28 features, and 33 features were found to be reproducible and nonredundant for M1∩M2 (average images), M1∩M2 (end‐exhale images), and M3, respectively. When blinded to the presence of test‐retest images, hierarchical clustering showed that the selected features were informative by correctly pairing 55 out of 56 test‐retest images using only their reproducible, nonredundant feature set values. Conclusions: Image feature reproducibility and redundancy depended on both the CT machine and the CT image type. For each image type, the authors found a set of cross‐machine reproducible, nonredundant, and informative image features that would be useful for future image‐based models. Compared to end‐exhale 4D‐CT and breath‐hold 3D‐CT, average 4D‐CT derived image features showed superior multimachine reproducibility and are the best candidates for clinical correlation.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A concordance correlation coefficient to evaluate reproducibility.
Lin, L. I.-K. (1989) · Biometrics · doi:10.2307/2532051 · Cited by 8,785
Radiomics: extracting more information from medical images using advanced feature analysis
Lambin, P., Rios-Velazquez, E., Leijenaar, R., Carvalho, S., van Stiphout, R. G. P. M., Granton, P., Zegers, C. M. L., Gillies, R., Boellard, R., Dekker, A., et al. (2012) · European Journal of Cancer · doi:10.1016/j.ejca.2011.11.036 · Cited by 6,084
Radiomics: the process and the challenges
Kumar, V., Gu, Y., Basu, S., Berglund, A., Eschrich, S. A., Schabath, M. B., Forster, K., Aerts, H. J. W. L., Dekker, A., Fenstermacher, D., et al. (2012) · Magnetic Resonance Imaging · doi:10.1016/j.mri.2012.06.010 · Cited by 2,258
Assessment of tumor heterogeneity: an emerging imaging tool for clinical practice?
Davnall, F., Yip, C. S. P., Ljungqvist, G., Selmi, M., Ng, F., Sanghera, B., Ganeshan, B., Miles, K. A., Cook, G. J., & Goh, V. (2012) · Insights into Imaging · doi:10.1007/s13244-012-0196-6 · Cited by 878
Automated medical image segmentation techniques
Sharma, N., & Aggarwal, L. (2010) · Journal of Medical Physics · doi:10.4103/0971-6203.58777 · Cited by 754
Cited by
Radiomics in medical imaging—“how-to” guide and critical reflection
van Timmeren, J. E., Cester, D., Tanadini-Lang, S., Alkadhi, H., & Baessler, B. (2020) · Insights into Imaging · doi:10.1186/s13244-020-00887-2 · Cited by 1,316
Measuring Computed Tomography Scanner Variability of Radiomics Features
Mackin, D., Fave, X., Zhang, L., Fried, D., Yang, J., Taylor, B., Rodriguez-Rivera, E., Dodge, C., Jones, A. K., & Court, L. (2015) · Investigative Radiology · doi:10.1097/rli.0000000000000180 · Cited by 651
Self-citationBerenguer, R., Pastor-Juan, M. D. R., Canales-Vázquez, J., Castro-García, M., Villas, M. V., Mansilla Legorburo, F., & Sabater, S. (2018) · Radiology · doi:10.1148/radiol.2018172361 · Cited by 622
Robust Radiomics Feature Quantification Using Semiautomatic Volumetric Segmentation
Parmar, C., Rios Velazquez, E., Leijenaar, R., Jermoumi, M., Carvalho, S., Mak, R. H., Mitra, S., Shankar, B. U., Kikinis, R., Haibe-Kains, B., et al. (2014) · PLoS ONE · doi:10.1371/journal.pone.0102107 · Cited by 611
Intrinsic dependencies of CT radiomic features on voxel size and number of gray levels
Shafiq‐ul‐Hassan, M., Zhang, G. G., Latifi, K., Ullah, G., Hunt, D. C., Balagurunathan, Y., Abdalah, M. A., Schabath, M. B., Goldgof, D. G., Mackin, D., et al. (2017) · Medical Physics · doi:10.1002/mp.12123 · Cited by 585
Self-citationReproducibility of radiomics for deciphering tumor phenotype with imaging
Zhao, B., Tan, Y., Tsai, W.-Y., Qi, J., Xie, C., Lu, L., & Schwartz, L. H. (2016) · Scientific Reports · doi:10.1038/srep23428 · Cited by 509
Radiomic Machine-Learning Classifiers for Prognostic Biomarkers of Head and Neck Cancer
Parmar, C., Grossmann, P., Rietveld, D., Rietbergen, M. M., Lambin, P., & Aerts, H. J. W. L. (2015) · Frontiers in Oncology · doi:10.3389/fonc.2015.00272 · Cited by 407
Park, J. E., Park, S. Y., Kim, H. J., & Kim, H. S. (2019) · Korean Journal of Radiology · doi:10.3348/kjr.2018.0070 · Cited by 388
Radiomics: from qualitative to quantitative imaging
Rogers, W., Thulasi Seetha, S., Refaee, T. A. G., Lieverse, R. I. Y., Granzier, R. W. Y., Ibrahim, A., Keek, S. A., Sanduleanu, S., Primakov, S. P., Beuque, M. P. L., et al. (2020) · The British Journal of Radiology · doi:10.1259/bjr.20190948 · Cited by 372
ibex: An open infrastructure software platform to facilitate collaborative work in radiomics
Zhang, L., Fried, D. V., Fave, X. J., Hunter, L. A., Yang, J., & Court, L. E. (2015) · Medical Physics · doi:10.1118/1.4908210 · Cited by 316
Self-citationHe, L., Huang, Y., Ma, Z., Liang, C., Liang, C., & Liu, Z. (2016) · Scientific Reports · doi:10.1038/srep34921 · Cited by 256
Understanding Sources of Variation to Improve the Reproducibility of Radiomics
Zhao, B. (2021) · Frontiers in Oncology · doi:10.3389/fonc.2021.633176 · Cited by 158
Metadata
- Authors
- Hunter, L. A., Krafft, S., Stingo, F., Choi, H., Martel, M. K., Kry, S. F., & Court, L. E.
- Venue
- Medical Physics
- Publisher
- Wiley
- Published
- 2013-12
- Volume
- 40
- Issue
- 12
- Pages
- 121916-121916
- ISSN
- 0094-2405 (ISSN-L)
- 2473-4209
- OpenAlex
- W2159375002
- Cited by
- 117
- References
- 5
- FWCI
- 6.42
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.