Cited by 49Radiomics and Machine Learning in Medical ImagingMachine Learning in HealthcareAI in cancer detection

RadioPathomics: Multimodal learning in non-small cell lung cancer for adaptive radiotherapy

Open access
Tortora, Matteo, Cordelli, Ermanno, Sicilia, Rosa, Nibid, Lorenzo, Ippolito, Edy, Perrone, Giuseppe, Ramella, Sara, Soda, Paolo·2023·IEEE Access

Abstract

Current practice in cancer treatment collects multimodal data, such as radiology images, histopathology slides, genomics and clinical data. The importance of these data sources taken individually has fostered the recent rise of radiomics and pathomics, i.e., the extraction of quantitative features from radiology and histopathology images collected to predict clinical outcomes or guide clinical decisions using artificial intelligence algorithms. Nevertheless, how to combine them into a single multimodal framework is still an open issue. In this work, we develop a multimodal late fusion approach that combines hand-crafted features computed from radiomics, pathomics and clinical data to predict radiotherapy treatment outcomes for non-small-cell lung cancer patients. Within this context, we investigate eight different late fusion rules and two patient-wise aggregation rules leveraging the richness of information given by CT images, whole-slide scans and clinical data. The experiments in leave-one-patient-out cross-validation on an in-house cohort of 33 patients show that the proposed fusion-based multimodal paradigm, with an AUC equal to 90.9%, outperforms each unimodal approach, suggesting that data integration can advance precision medicine. The results also show that late fusion favourably compares against early fusion, another commonly used multimodal approach. As a further contribution, we explore the chance to use a deep learning framework against hand-crafted features. In our scenario characterised by different modalities and a limited amount of data, as it may happen in different areas of cancer research, the results show that the latter is still a viable and effective option for extracting relevant information with respect to the former.

Integrity signals

Registry

Crossref · 10.1109/access.2023.3275126

Cites

49 of 49 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • A key issue with prior multimodal AI applications using handcrafted features was that difficulties requiring particular domain expertise were multiplied, as expert clinicians would also need to be involved in the feature design phase.

    “A key issue with these approaches was that the difficulties requiring particular domain expertise are multiplied, as expert clinicians would also need to be involved in the feature design phase (Vaidya et al., 2020; Tortora et al., 2023).”

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Authors
Tortora, M., Cordelli, E., Sicilia, R., Nibid, L., Ippolito, E., Perrone, G., Ramella, S., & Soda, P.
Affiliations
Università Campus Bio-Medico (IT)
Venue
IEEE Access
Published
2023
Volume
11
Pages
47563-47578
ISSN
2169-3536 (ISSN-L)
OpenAlex
W4380980908
Cited by
49
References
49
Citations by year
2026: 12 · 2025: 21 · 2024: 11 · 2023: 4 · 2022: 1
FWCI
7.95
Type
journal article
Language
en
Open access
Locations
  • IEEE Access (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion)
  • DiVA at Umeå University (Umeå University) (submittedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • CINECA IRIS Institutial Research Information System (University of Genoa) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • Università Campus Bio-Medico di Roma (PE0000013-FAIR)
  • Regione Lazio (PO FSE 2014-2020)
Topics
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
  • Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
Keywords
non-small cell lung cancer, adaptive radiotherapy, multimodal fusion, decision fusion, radiomics, pathomics, histopathology, CT images, clinical data, treatment outcome prediction, precision medicine, handcrafted features, deep learning model comparison

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review