CoChat
Sign Up
Cited by 41Radiomics and Machine Learning in Medical ImagingArtificial Intelligence in Healthcare and EducationMedical Imaging Techniques and Applications

Radiology and multi-scale data integration for precision oncology

Open access
Paverd, Hania, Zormpas-Petridis, Konstantinos, Clayton, Hannah, Burge, Sarah, Crispin-Ortuzar, Mireia·2024·npj Precision Oncology
DOI 10.1038/s41698-024-00656-0PMC PMC11282284
Open this paper in CoChat — free account

Abstract

In this Perspective paper we explore the potential of integrating radiological imaging with other data types, a critical yet underdeveloped area in comparison to the fusion of other multi-omic data. Radiological images provide a comprehensive, three-dimensional view of cancer, capturing features that would be missed by biopsies or other data modalities. This paper explores the complexities and challenges of incorporating medical imaging into data integration models, in the context of precision oncology. We present the different categories of imaging-omics integration and discuss recent progress, highlighting the opportunities that arise from bringing together spatial data on different scales.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s41698-024-00656-0

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • For cross-scale biomedical imaging, integration can often be achieved using shallow learning methods, though challenges remain in accurately registering and aligning cross-scale images.

    “For cross-scale biomedical imaging, integration can often be achieved using shallow learning methods, though challenges remain in accurately registering and aligning cross-scale images [176].”

    From Challenges in AI-driven Biomedical Multimodal Data Fusion and Analysis (2025) · cites this paper as [176]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

No citation lists for this paper

No citation lists for this paper yet — checked again daily.

References 82 · Cited by 41

Research works printed but not linked: 74

  1. [2]Mobadersany, P. Predicting cancer outcomes from histology and genomics using convolutional networks. Proc. Natl. Acad. Sci. 115, E2970–E2979 (2018). 29531073 10.1073/pnas.1717139115 PMC5879673 · DOI · Paper page
  2. [3]Sammut, S.-J. Multi-omic machine learning predictor of breast cancer therapy response. Nature 601, 623–629 (2022). 34875674 10.1038/s41586-021-04278-5 PMC8791834 · DOI · Paper page
  3. [4]Aloj, L. The emerging role of cell surface receptor and protein binding radiopharmaceuticals in cancer diagnostics and therapy. Nucl. Med. Biol. 92, 53–64 (2021). 32563612 10.1016/j.nucmedbio.2020.06.005 · DOI
  4. [5]Sourbron, S. Technical aspects of MR perfusion. Eur. J. Radiol. 76, 304–313 (2010). 20363574 10.1016/j.ejrad.2010.02.017 · DOI
  5. [6]Venkatesh, S. K., Yin, M. & Ehman, R. L. Magnetic resonance elastography of liver: technique, analysis, and clinical applications. J. Magn. Reson. Imaging 37, 544–555 (2013). 23423795 10.1002/jmri.23731 PMC3579218 · DOI
  6. [7]Hagmann, P. Understanding diffusion MR imaging techniques: from scalar diffusion-weighted imaging to diffusion tensor imaging and beyond. Radiographics 26, S205–S223 (2006). 17050517 10.1148/rg.26si065510 · DOI
  7. [8]Lipkova, J. Artificial intelligence for multimodal data integration in oncology. Cancer Cell 40, 1095–1110 (2022). 36220072 10.1016/j.ccell.2022.09.012 PMC10655164 · DOI · Paper page
  8. [9]Boehm, K. M., Khosravi, P., Vanguri, R., Gao, J. & Shah, S. P. Harnessing multimodal data integration to advance precision oncology. Nat. Rev. Cancer 22, 114–126 (2022). 34663944 10.1038/s41568-021-00408-3 PMC8810682 · DOI · Paper page
  9. [10]Barnum, G., Talukder, S. & Yue, Y. On the benefits of early fusion in multimodal representation learning. arXiv preprint arXiv:2011.07191 (2020). · arXiv
  10. [11]Gadzicki, K., Khamsehashari, R. & Zetzsche, C. Early vs late fusion in multimodal convolutional neural networks. In 2020 IEEE 23rd international conference on information fusion (FUSION), 1–6 (IEEE, 2020).
  11. [12]Hemker, K., Simidjievski, N. & Jamnik, M. HEALNet – Hybrid Multi-Modal Fusion for Heterogeneous Biomedical Data. arXiv preprint arXiv:2311.09115 (2023). · arXiv
  12. [13]Van Timmeren, J. E., Cester, D., Tanadini-Lang, S., Alkadhi, H. & Baessler, B. Radiomics in medical imaging—“how-to” guide and critical reflection. Insights into Imaging 11, 1–16 (2020). 32785796 PMC7423816
  13. [14]Mei, X. RadImageNet: an open radiologic deep learning research dataset for effective transfer learning. Radiology: Artif. Intell. 4, e210315 (2022). PMC9530758 36204533
  14. [15]Sun, W., Zheng, B. & Qian, W. Automatic feature learning using multichannel ROI based on deep structured algorithms for computerized lung cancer diagnosis. Computers Biol. Med. 89, 530–539 (2017). 10.1016/j.compbiomed.2017.04.006 28473055 · DOI
  15. [16]Paul, R. Deep feature transfer learning in combination with traditional features predicts survival among patients with lung adenocarcinoma. Tomography 2, 388–395 (2016). 28066809 10.18383/j.tom.2016.00211 PMC5218828 · DOI
  16. [17]Zeng, Q. Development and validation of a predictive model combining clinical, radiomics, and deep transfer learning features for lymph node metastasis in early gastric cancer. Front. Med. 9, 986437 (2022). 10.3389/fmed.2022.986437 PMC9573999 36262277 · DOI
  17. [18]Lao, J. A deep learning-based radiomics model for prediction of survival in glioblastoma multiforme. Sci. Rep. 7, 10353 (2017). 28871110 10.1038/s41598-017-10649-8 PMC5583361 · DOI · Paper page
  18. [19]Van Opbroek, A., Ikram, M. A., Vernooij, M. W. & De Bruijne, M. Transfer learning improves supervised image segmentation across imaging protocols. IEEE Trans. Med. Imaging 34, 1018–1030 (2014). 25376036 10.1109/TMI.2014.2366792 · DOI
  19. [20]Karimi, D., Warfield, S. K. & Gholipour, A. Transfer learning in medical image segmentation: New insights from analysis of the dynamics of model parameters and learned representations. Artif. Intell. Med. 116, 102078 (2021). 34020754 10.1016/j.artmed.2021.102078 PMC8164174 · DOI
  20. [21]Zoetmulder, R., Gavves, E., Caan, M. & Marquering, H. Domain-and task-specific transfer learning for medical segmentation tasks. Computer Methods Prog. Biomedicine 214, 106539 (2022). 10.1016/j.cmpb.2021.106539 34875512 · DOI
  21. [22]Tang, Y. et al. Self-supervised pre-training of swin transformers for 3d medical image analysis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 20730–20740 (2022).
  22. [24]Delgado, A. & Guddati, A. K. Clinical endpoints in oncology-a primer. Am. J. Cancer Res. 11, 1121 (2021). 33948349 PMC8085844
  23. [25]Jaffe, C. C. Measures of response: RECIST, WHO, and new alternatives. J. Clin. Oncol. 24, 3245–3251 (2006). 16829648 10.1200/JCO.2006.06.5599 · DOI
  24. [26]Wang, R. Development of a novel combined nomogram model integrating deep learning-pathomics, radiomics and immunoscore to predict postoperative outcome of colorectal cancer lung metastasis patients. J. Hematol. Oncol. 15, 11 (2022). 35073937 10.1186/s13045-022-01225-3 PMC8785554 · DOI
  25. [28]Feng, L. Development and validation of a radiopathomics model to predict pathological complete response to neoadjuvant chemoradiotherapy in locally advanced rectal cancer: a multicentre observational study. Lancet Digital Health 4, e8–e17 (2022). 34952679 10.1016/S2589-7500(21)00215-6 · DOI · Paper page
  26. [29]Wang, S. Mining whole-lung information by artificial intelligence for predicting EGFR genotype and targeted therapy response in lung cancer: a multicohort study. Lancet Digital Health 4, e309–e319 (2022). 35341713 10.1016/S2589-7500(22)00024-3 · DOI
  27. [30]Cardoso, M. R. Metabolomics by nmr combined with machine learning to predict neoadjuvant chemotherapy response for breast cancer. Cancers 14, 5055 (2022). 36291837 10.3390/cancers14205055 PMC9600495 · DOI
  28. [31]Zhou, H. Machine learning reveals multimodal mri patterns predictive of isocitrate dehydrogenase and 1p/19q status in diffuse low-and high-grade gliomas. J. Neuro-Oncol. 142, 299–307 (2019). 10.1007/s11060-019-03096-0 PMC6510979 30661193 · DOI
  29. [32]Zeng, H. Integrative radiogenomics analysis for predicting molecular features and survival in clear cell renal cell carcinoma. Aging (Albany NY) 13, 9960 (2021). 33795526 10.18632/aging.202752 PMC8064160 · DOI
  30. [33]Yi, X. Incorporating SULF1 polymorphisms in a pretreatment CT-based radiomic model for predicting platinum resistance in ovarian cancer treatment. Biomed. Pharmacother. 133, 111013 (2021). 33227705 10.1016/j.biopha.2020.111013 · DOI
  31. [34]Veeraraghavan, H. Integrated multi-tumor radio-genomic marker of outcomes in patients with high serous ovarian carcinoma. Cancers 12, 3403 (2020). 33212885 10.3390/cancers12113403 PMC7698381 · DOI
  32. [36]Hoivik, E. A. A radiogenomics application for prognostic profiling of endometrial cancer. Commun. Biol. 4, 1363 (2021). 34873276 10.1038/s42003-021-02894-5 PMC8648740 · DOI
  33. [37]Liang, H. Renal enhanced CT images reveal the tandem mechanism between tumor cells and immunocytes based on bulk/single-cell RNA sequencing. Funct. Integr. Genomics 23, 88 (2023). 36933049 10.1007/s10142-023-01011-5 · DOI
  34. [38]Zhang, W. Development and validation of mri-based deep learning models for prediction of microsatellite instability in rectal cancer. Cancer Med. 10, 4164–4173 (2021). 33963688 10.1002/cam4.3957 PMC8209621 · DOI
  35. [39]Vanguri, R. S. Multimodal integration of radiology, pathology and genomics for prediction of response to PD-(L)1 blockade in patients with non-small cell lung cancer. Nat. Cancer 3, 1151–1164 (2022). 36038778 10.1038/s43018-022-00416-8 PMC9586871 · DOI · Paper page
  36. [40]Li, Z., Jiang, Y., Lu, M., Li, R. & Xia, Y. Survival prediction via hierarchical multimodal co-attention transformer: A computational histology-radiology solution. IEEE Transactions on Medical Imaging (2023). 37030860
  37. [41]Cook, D. Next generation immuno-oncology tumor profiling using a rapid, non-invasive, computational biophysics biomarker in early-stage breast cancer. Front. Artif. Intell. 6, 1153083 (2023). 37138891 10.3389/frai.2023.1153083 PMC10149754 · DOI
  38. [43]Tomaszewski, M. R. & Gillies, R. J. The biological meaning of radiomic features. Radiology 298, 505–516 (2021). 33399513 10.1148/radiol.2021202553 PMC7924519 · DOI
  39. [44]Kather, J. N. Deep learning can predict microsatellite instability directly from histology in gastrointestinal cancer. Nat. Med. 25, 1054–1056 (2019). 31160815 10.1038/s41591-019-0462-y PMC7423299 · DOI · Paper page
  40. [45]Fremond, S. Interpretable deep learning model to predict the molecular classification of endometrial cancer from haematoxylin and eosin-stained whole-slide images: a combined analysis of the PORTEC randomised trials and clinical cohorts. Lancet Digital Health 5, e71–e82 (2023). 36496303 10.1016/S2589-7500(22)00210-2 · DOI
  41. [46]Lu, W. Slidegraph+: Whole slide image level graphs to predict HER2 status in breast cancer. Med. Image Anal. 80, 102486 (2022). 35640384 10.1016/j.media.2022.102486 · DOI
  42. [47]Monjo, T., Koido, M., Nagasawa, S., Suzuki, Y. & Kamatani, Y. Efficient prediction of a spatial transcriptomics profile better characterizes breast cancer tissue sections without costly experimentation. Sci. Rep. 12, 4133 (2022). 35260632 10.1038/s41598-022-07685-4 PMC8904587 · DOI
  43. [48]Levy-Jurgenson, A., Tekpli, X., Kristensen, V. N. & Yakhini, Z. Spatial transcriptomics inferred from pathology whole-slide images links tumor heterogeneity to survival in breast and lung cancer. Sci. Rep. 10, 18802 (2020). 33139755 10.1038/s41598-020-75708-z PMC7606448 · DOI · Paper page
  44. [49]He, B. Integrating spatial gene expression and breast tumour morphology via deep learning. Nat. Biomed. Eng. 4, 827–834 (2020). 32572199 10.1038/s41551-020-0578-x · DOI · Paper page
  45. [50]Li, H. Quantitative MRI radiomics in the prediction of molecular classifications of breast cancer subtypes in the TCGA/TCIA data set. npj Breast Cancer 2, 16012 (2016). 27853751 10.1038/npjbcancer.2016.12 PMC5108580 · DOI
  46. [51]Yang, L. Can CT-based radiomics signature predict KRAS/NRAS/BRAF mutations in colorectal cancer? Eur. Radiol. 28, 2058–2067 (2018). 29335867 10.1007/s00330-017-5146-8 · DOI
  47. [52]Chen, X. Reliable gene mutation prediction in clear cell renal cell carcinoma through multi-classifier multi-objective radiogenomics model. Phys. Med. Biol. 63, 215008 (2018). 30277889 10.1088/1361-6560/aae5cd PMC6240911 · DOI
  48. [53]O’Connor, J. P. Imaging biomarker roadmap for cancer studies. Nat. Rev. Clin. Oncol. 14, 169–186 (2017). 27725679 10.1038/nrclinonc.2016.162 PMC5378302 · DOI · Paper page
  49. [54]Bailey, C. VERDICT MRI validation in fresh and fixed prostate specimens using patient-specific moulds for histological and MR alignment. NMR Biomed. 32, e4073 (2019). 30779863 10.1002/nbm.4073 PMC6519204 · DOI
  50. [55]Wu, H. H. A system using patient-specific 3D-printed molds to spatially align in vivo MRI with ex vivo MRI and whole-mount histopathology for prostate cancer research. J. Magn. Reson. Imaging 49, 270–279 (2019). 30069968 10.1002/jmri.26189 · DOI
  51. [56]Delgado-Ortet, M. Lesion-specific 3D-printed moulds for image-guided tissue multi-sampling of ovarian tumours: A prospective pilot study. Front. Oncol. 13, 1085874 (2023). 36860310 10.3389/fonc.2023.1085874 PMC9969130 · DOI
  52. [57]Crispin-Ortuzar, M. Three-dimensional printed molds for image-guided surgical biopsies: an open source computational platform. JCO Clin. Cancer Inform. 4, 736–748 (2020). 32804543 10.1200/CCI.20.00026 PMC7469624 · DOI
  53. [58]Jardim-Perassi, B. V. Multiparametric MRI and coregistered histology identify tumor habitats in breast cancer mouse models. Cancer Res. 79, 3952–3964 (2019). 31186232 10.1158/0008-5472.CAN-19-0213 PMC6677627 · DOI
  54. [59]Zormpas-Petridis, K. Noninvasive MRI native T1 mapping detects response to MYCN-targeted therapies in the Th-MYCN model of neuroblastoma. Cancer Res. 80, 3424–3435 (2020). 32595135 10.1158/0008-5472.CAN-20-0133 · DOI
  55. [60]Mikhail, A. S. Liver-specific 3D sectioning molds for correlating in vivo CT and MRI with tumor histopathology in woodchucks (marmota monax). PLoS One 15, e0230794 (2020). 32214365 10.1371/journal.pone.0230794 PMC7098627 · DOI
  56. [62]AbdulJabbar, K. Geospatial immune variability illuminates differential evolution of lung adenocarcinoma. Nat. Med. 26, 1054–1062 (2020). 32461698 10.1038/s41591-020-0900-x PMC7610840 · DOI
  57. [63]Martin-Gonzalez, P. Integrative radiogenomics for virtual biopsy and treatment monitoring in ovarian cancer. Insights Into Imaging 11, 1–10 (2020). 32804260 10.1186/s13244-020-00895-2 PMC7431480 · DOI
  58. [64]Jiménez-Sánchez, A. Heterogeneous tumor-immune microenvironments among differentially growing metastases in an ovarian cancer patient. Cell 170, 927–938 (2017). 28841418 10.1016/j.cell.2017.07.025 PMC5589211 · DOI
  59. [65]De Feyter, H. et al. Deuterium metabolic imaging (dmi) for mri-based 3d mapping of metabolism in vivo, Sci. Adv., 4, eaat7314 (2018). PMC6105304 30140744
  60. [66]Wu, J. Intratumoral spatial heterogeneity at perfusion mr imaging predicts recurrence-free survival in locally advanced breast cancer treated with neoadjuvant chemotherapy. Radiology 288, 26–35 (2018). 29714680 10.1148/radiol.2018172462 PMC6029132 · DOI
  61. [67]Beer, L. Ultrasound-guided targeted biopsies of CT-based radiomic tumour habitats: technical development and initial experience in metastatic ovarian cancer. Eur. Radiol. 31, 3765–3772 (2021). 33315123 10.1007/s00330-020-07560-8 PMC8128813 · DOI
  62. [69]Weigelt, B. et al. Radiogenomics analysis of intratumor heterogeneity in a patient with high-grade serous ovarian cancer. JCO Precision Oncology 3, 1–9 (2019). 10.1200/PO.18.00410 PMC7446483 32914032 · DOI
  63. [70]Bathelt, F. The usage of OHDSI OMOP–a scoping review. Proceedings of the German Medical Data Sciences (GMDS) 95–95 (2021). 34545824
  64. [71]Mustra, M., Delac, K. & Grgic, M. Overview of the DICOM standard. In 2008 50th International Symposium ELMAR, 1, 39–44 (IEEE, 2008).
  65. [72]Herrmann, M. D. Implementing the DICOM standard for digital pathology. J. Pathol. Inform. 9, 37 (2018). 30533276 10.4103/jpi.jpi_42_18 PMC6236926 · DOI
  66. [73]Jha, A. Repeatability and reproducibility study of radiomic features on a phantom and human cohort. Sci. Rep. 11, 2055 (2021). 33479392 10.1038/s41598-021-81526-8 PMC7820018 · DOI
  67. [74]Gauriau, R. Using DICOM metadata for radiological image series categorization: a feasibility study on large clinical brain MRI datasets. J. digital imaging 33, 747–762 (2020). 10.1007/s10278-019-00308-x PMC7256138 31950302 · DOI
  68. [75]Kushida, C. A. Strategies for de-identification and anonymization of electronic health record data for use in multicenter research studies. Med. Care 50, S82 (2012). 22692265 10.1097/MLR.0b013e3182585355 PMC6502465 · DOI
  69. [76]Sousa, S. & Kern, R. How to keep text private? A systematic review of deep learning methods for privacy-preserving natural language processing. Artif. Intell. Rev. 56, 1427–1492 (2023). 10.1007/s10462-022-10204-6 · DOI
  70. [77]Sun, Z. Evaluating GPT-4 on impressions generation in radiology reports. Radiology 307, e231259 (2023). 37367439 10.1148/radiol.231259 PMC10534271 · DOI
  71. [79]Fink, M. A. Potential of ChatGPT and GPT-4 for data mining of free-text CT reports on lung cancer. Radiology 308, e231362 (2023). 37724963 10.1148/radiol.231362 · DOI
  72. [81]Aryanto, K., Oudkerk, M. & Van Ooijen, P. Free DICOM de-identification tools in clinical research: functioning and safety of patient privacy. Eur. Radiol. 25, 3685–3695 (2015). 26037716 10.1007/s00330-015-3794-0 PMC4636522 · DOI
  73. [83]Forsberg, D., Rosipko, B., Sunshine, J. L. & Ros, P. R. State of integration between PACS and other IT systems: a national survey of academic radiology departments. J. Am. Coll. Radiol. 13, 812–818 (2016). 27026579 10.1016/j.jacr.2016.01.018 · DOI
  74. [84]Mongan, J. & Avrin, D. Impact of PACS-EMR integration on radiologist usage of the EMR. J. Digital Imaging 31, 611–614 (2018). 10.1007/s10278-018-0077-8 PMC6148806 29696473 · DOI

Authors
Paverd, H., Zormpas-Petridis, K., Clayton, H., Burge, S., & Crispin-Ortuzar, M.
Affiliations
University of Cambridge (GB)
ORCID
Venue
npj Precision Oncology
Publisher
Springer Science and Business Media LLC
Published
2024-07-26
Volume
8
Issue
1
Pages
158-158
ISSN
2397-768X (ISSN-L)
DOI
10.1038/s41698-024-00656-0
PMC
PMC11282284
OpenAlex
W4401011476
Cited by
41
References
82
Citations by year
2026: 21 · 2025: 18 · 2024: 2
FWCI
2.67
Type
journal article
Language
en
Open access
Licence
Locations
  • npj Precision Oncology (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Apollo (University of Cambridge) (submittedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • PubliCatt (Università Cattolica del Sacro Cuore) (submittedVersion, other-oa)
Funders
  • AstraZeneca
  • Mark Foundation For Cancer Research (RG95043)
  • Cancer Research UK (100012)
  • National Institute for Health and Care Research ((NIHR203312, NIHR206092)
  • Department of Health and Social Care
  • Imperial Experimental Cancer Medicine Centre
  • NIHR BioResource
  • Engineering and Physical Sciences Research Council (EP/P020259/1, EP/P020259)
  • NIHR Cambridge Biomedical Research Centre (NIHR 203312)
Topics
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Medical Imaging Techniques and Applications
Keywords
radiological imaging, multi-scale data integration, precision oncology, multi-omics integration, diagnostic imaging

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review