Making Radiomics More Reproducible across Scanner and Imaging Protocol Variations: A Review of Harmonization Methods
Open accessThe DOI matches its Crossref record; 30 of the 30 references it cites were found in a registry or an index.
Abstract
Radiomics converts medical images into mineable data via a high-throughput extraction of quantitative features used for clinical decision support. However, these radiomic features are susceptible to variation across scanners, acquisition protocols, and reconstruction settings. Various investigations have assessed the reproducibility and validation of radiomic features across these discrepancies. In this narrative review, we combine systematic keyword searches with prior domain knowledge to discuss various harmonization solutions to make the radiomic features more reproducible across various scanners and protocol settings. Different harmonization solutions are discussed and divided into two main categories: image domain and feature domain. The image domain category comprises methods such as the standardization of image acquisition, post-processing of raw sensor-level image data, data augmentation techniques, and style transfer. The feature domain category consists of methods such as the identification of reproducible features and normalization techniques such as statistical normalization, intensity harmonization, ComBat and its derivatives, and normalization using deep learning. We also reflect upon the importance of deep learning solutions for addressing variability across multi-centric radiomic studies especially using generative adversarial networks (GANs), neural style transfer (NST) techniques, or a combination of both. We cover a broader range of methods especially GANs and NST methods in more detail than previous reviews.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Unpaired image-to-image translation using cycle-consistent adversarial networks
Zhu, J.-Y., Park, T., Isola, P., & Efros, A. A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.244 · Cited by 23,602
Image-to-Image Translation with Conditional Adversarial Networks
Isola, P., Zhu, J.-Y., Zhou, T., & Efros, A. A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.632 · Cited by 22,930
Perceptual Losses for Real-Time Style Transfer and Super-Resolution
Johnson, J., Alahi, A., & Fei-Fei, L. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46475-6_43 · Cited by 11,283
A concordance correlation coefficient to evaluate reproducibility.
Lin, L. I.-K. (1989) · Biometrics · doi:10.2307/2532051 · Cited by 8,785
Adjusting batch effects in microarray expression data using empirical Bayes methods
Johnson, W. E., Li, C., & Rabinovic, A. (2007) · Biostatistics · doi:10.1093/biostatistics/kxj037 · Cited by 8,658
Radiomics: Images Are More than Pictures, They Are Data
Gillies, R. J., Kinahan, P. E., & Hricak, H. (2016) · Radiology · doi:10.1148/radiol.2015151169 · Cited by 8,421
Review of deep learning: concepts, CNN architectures, challenges, applications, future directions
Alzubaidi, L., Zhang, J., Humaidi, A. J., Al-Dujaili, A., Duan, Y., Al-Shamma, O., Santamaría, J., Fadhel, M. A., Al-Amidie, M., & Farhan, L. (2021) · Journal of Big Data · doi:10.1186/s40537-021-00444-8 · Cited by 7,906
Radiomics: extracting more information from medical images using advanced feature analysis
Lambin, P., Rios-Velazquez, E., Leijenaar, R., Carvalho, S., van Stiphout, R. G. P. M., Granton, P., Zegers, C. M. L., Gillies, R., Boellard, R., Dekker, A., et al. (2012) · European Journal of Cancer · doi:10.1016/j.ejca.2011.11.036 · Cited by 6,084
Self-citationImage Style Transfer Using Convolutional Neural Networks
Gatys, L. A., Ecker, A. S., & Bethge, M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.265 · Cited by 6,025
Radiomics: the bridge between medical imaging and personalized medicine
Lambin, P., Leijenaar, R. T. H., Deist, T. M., Peerlings, J., de Jong, E. E. C., van Timmeren, J., Sanduleanu, S., Larue, R. T. H. M., Even, A. J. G., Jochems, A., et al. (2017) · Nature Reviews Clinical Oncology · doi:10.1038/nrclinonc.2017.141 · Cited by 5,688
Self-citationDecoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach
Aerts, H. J. W. L., Velazquez, E. R., Leijenaar, R. T. H., Parmar, C., Grossmann, P., Carvalho, S., Bussink, J., Monshouwer, R., Haibe-Kains, B., Rietveld, D., et al. (2014) · Nature Communications · doi:10.1038/ncomms5006 · Cited by 5,132 · cited 3×
“Studies have shown the effects of image acquisition parameters on the reproducibility of radiomic features [21,24–27]. the image domain (Section 3) are performed on the whole image (raw before feature Many studies [1,28–31] have also explored the discriminative power of radiomic features.…”
BackgroundMethodSelf-citationArbitrary Style Transfer in Real-Time with Adaptive Instance Normalization
Huang, X., & Belongie, S. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.167 · Cited by 5,076
Zwanenburg, A., Vallières, M., Abdalah, M. A., Aerts, H. J. W. L., Andrearczyk, V., Apte, A., Ashrafinia, S., Bakas, S., Beukinga, R. J., Boellaard, R., et al. (2020) · Radiology · doi:10.1148/radiol.2020191145 · Cited by 4,177 · cited 1×
“Different feature extraction algorithms and image processing techniques also influence the feature variation and have been addressed by the image biomarker standardization initiative [39].”
BackgroundSelf-citationDeep CORAL: Correlation Alignment for Deep Domain Adaptation
Sun, B., & Saenko, K. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-49409-8_35 · Cited by 3,757 · cited 4×
“…that attempts to align the second-order statistics of two distributions by applying a linear transformation. have the same distribution and Sun and Saenko [182] used correlation alignment that attempts to align the second-order statistics of two distributions by applying a linear transformation.”
MethodInfluential(2012) · MPG.PuRe (Max Planck Society) · doi:10.5555/2188385.2188410 · Cited by 3,603
FDG PET/CT: EANM procedure guidelines for tumour imaging: version 2.0
Boellaard, R., Delgado-Bolton, R., Oyen, W. J. G., Giammarile, F., Tatsch, K., Eschner, W., Verzijlbergen, F. J., Barrington, S. F., Pike, L. C., Weber, W. A., et al. (2015) · European Journal of Nuclear Medicine and Molecular Imaging · doi:10.1007/s00259-014-2961-x · Cited by 3,303 · cited 1×
“For standardization of PET imaging, the European Association of Nuclear Medicine (EANM) [45] launched the EARL (EANM Research Ltd.) program covering areas such as scan acquisition, processing of images, and image interpretation.”
MethodAdapting visual category models to new domains
Saenko, K., Kulis, B., Fritz, M., & Darrell, T. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15561-1_16 · Cited by 2,847 · cited 1×
“[181] obtains an average accuracy of 0.908 while [182] got an average accuracy of 0.72, both using [181] obtains an average accuracy of 0.908 while [182] got an average accuracy of 0.72, both using the Office dataset [184]. the Office dataset [184].”
MethodInfluentialRadiomics: the process and the challenges
Kumar, V., Gu, Y., Basu, S., Berglund, A., Eschrich, S. A., Schabath, M. B., Forster, K., Aerts, H. J. W. L., Dekker, A., Fenstermacher, D., et al. (2012) · Magnetic Resonance Imaging · doi:10.1016/j.mri.2012.06.010 · Cited by 2,258 · cited 1×
“The radiomic hypothesis postulates that the quantitative study of medical image data can provide complementary knowledge in a quick and reproducible manner to support clinicians in their decision-making process, assisted by automated or semi-automated software [5,6].”
BackgroundSelf-citationArjovsky, M., Chintala, S., & Bottou, L. (2017) · arXiv · doi:10.48550/arxiv.1701.07875 · Cited by 2,225
Return of Frustratingly Easy Domain Adaptation
Sun, B., Feng, J., & Saenko, K. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10306 · Cited by 2,004 · cited 1×
“…by using a two-stream CNN architecture (one in the source domain with synthetic images and the other in the target domain with real images) with unshared weights and the DeepCORAL [183] architecture, respectively. images) with unshared weights and the DeepCORAL [183] architecture, respectively.”
MethodLearning to Discover Cross-Domain Relations with Generative Adversarial Networks
Kim, T., Cha, M., Kim, H., Lee, J. K., & Kim, J. (2017) · arXiv · doi:10.48550/arxiv.1703.05192 · Cited by 1,716
Miotto, R., Li, L., Kidd, B. A., & Dudley, J. T. (2016) · Scientific Reports · doi:10.1038/srep26094 · Cited by 1,710
Harmonization of multi-site diffusion tensor imaging data
Fortin, J.-P., Parker, D., Tunç, B., Watanabe, T., Elliott, M. A., Ruparel, K., Roalf, D. R., Satterthwaite, T. D., Gur, R. C., Gur, R. E., et al. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2017.08.047 · Cited by 1,261 · cited 1×
“The application of ComBat on radiomic features was first introduced by Fortin et al. [169].”
MethodQuantitative Imaging in Cancer Evolution and Ecology
Gatenby, R. A., Grove, O., & Gillies, R. J. (2013) · Radiology · doi:10.1148/radiol.13122697 · Cited by 515
Super‐resolution musculoskeletal MRI using deep learning
Chaudhari, A. S., Fang, Z., Kogan, F., Wood, J., Stevens, K. J., Gibbons, E. K., Lee, J. H., Gold, G. E., & Hargreaves, B. A. (2018) · Magnetic Resonance in Medicine · doi:10.1002/mrm.27178 · Cited by 406
Wibmer, A., Hricak, H., Gondo, T., Matsumoto, K., Veeraraghavan, H., Fehr, D., Zheng, J., Goldman, D., Moskowitz, C., Fine, S. W., et al. (2015) · European Radiology · doi:10.1007/s00330-015-3701-8 · Cited by 391
Papadimitroulas, P., Brocki, L., Christopher Chung, N., Marchadour, W., Vermet, F., Gaubert, L., Eleftheriadis, V., Plachouris, D., Visvikis, D., Kagadis, G. C., et al. (2021) · Physica Medica · doi:10.1016/j.ejmp.2021.03.009 · Cited by 226
Deep learning-based unlearning of dataset bias for mri harmonisation and confound removal
Dinsdale, N. K., Jenkinson, M., & Namburete, A. I. L. (2021) · NeuroImage · doi:10.1016/j.neuroimage.2020.117689 · Cited by 178 · cited 2×
“Adapting the same framework as proposed in [186], Dinsdale et al. [187] utilized an iterative update approach that aimed to generate scanner-invariant (i.e., harmonized features) representations of MRI neuroimages while evaluating the main task (segmentation), thus decreasing the influence of…”
MethodLearning from scanners: Bias reduction and feature correction in radiomics
Zhovannik, I., Bussink, J., Traverso, A., Shi, Z., Kalendralis, P., Wee, L., Dekker, A., Fijten, R., & Monshouwer, R. (2019) · Clinical and Translational Radiation Oncology · doi:10.1016/j.ctro.2019.07.003 · Cited by 75 · cited 2×
“Similar studies have been conducted in the past where the reproducibility of CT radiomic features was investigated using phantom data [25–27, 146 Considering MRI, a recent study using radiomics to investigate the reproducibility of features across several MRI scanners and scanning protocol…”
BackgroundMethod
Cited by
Must-have Qualities of Clinical Research on Artificial Intelligence and Machine Learning
Koçak, B., Cuocolo, R., Santos, D. P. D., Stanzione, A., & Ugga, L. (2023) · Balkan Medical Journal · doi:10.4274/balkanmedj.galenos.2022.2022-11-51 · Cited by 64
Moving towards vertically integrated artificial intelligence development
Zhang, J., Budhdeo, S., William, W., Cerrato, P., Shuaib, H., Sood, H., Ashrafian, H., Halamka, J., & Teo, J. T. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00690-x · Cited by 54
Ben Ahmed, K., Hall, L. O., Goldgof, D. B., & Fogarty, R. (2022) · IEEE Access · doi:10.1109/access.2022.3193700 · Cited by 12
Metadata
- Authors
- Mali, S. A., Ibrahim, A., Woodruff, H. C., Andrearczyk, V., Müller, H., Primakov, S., Salahuddin, Z., Chatterjee, A., & Lambin, P.
- Venue
- Journal of Personalized Medicine
- Publisher
- MDPI AG
- Published
- 2021-08-27
- Volume
- 11
- Issue
- 9
- Pages
- 842
- ISSN
- 2075-4426 (ISSN-L)
- OpenAlex
- W3193956823
- Cited by
- 207
- References
- 30
- FWCI
- 14.09
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.