CoChat
Sign Up
Cited by 265AI in cancer detectionRadiomics and Machine Learning in Medical ImagingGenetic factors in colorectal cancer

The DOI matches its Crossref record; 30 of the 30 references it cites were found in a registry or an index.

Image-based consensus molecular subtype (imCMS) classification of colorectal cancer using deep learning

Open access
Sirinukunwattana, Korsuk, Domingo, Enric, Richman, Susan D, Redmond, Keara L, Blake, Andrew, Verrill, Clare, Leedham, Simon J, Chatzipli, Aikaterini, Hardy, Claire, Whalley, Celina M, Wu, Chieh-hsi, Beggs, Andrew D, McDermott, Ultan, Dunne, Philip D, Meade, Angela, Walker, Steven M, Murray, Graeme I, Samuel, Leslie, Seymour, Matthew, Tomlinson, Ian, Quirke, Phil, Maughan, Timothy, Rittscher, Jens, Koelzer, Viktor H·2021·Gut
DOI 10.1136/gutjnl-2019-319866PMC PMC7873419

Published version of a preprint

Open this paper in CoChat — free accountOpen PDF

Abstract

Objective: Complex phenotypes captured on histological slides represent the biological processes at play in individual cancers, but the link to underlying molecular classification has not been clarified or systematised. In colorectal cancer (CRC), histological grading is a poor predictor of disease progression, and consensus molecular subtypes (CMSs) cannot be distinguished without gene expression profiling. We hypothesise that image analysis is a cost-effective tool to associate complex features of tissue organisation with molecular and outcome data and to resolve unclassifiable or heterogeneous cases. In this study, we present an image-based approach to predict CRC CMS from standard H&E sections using deep learning. Design: Training and evaluation of a neural network were performed using a total of n=1206 tissue sections with comprehensive multi-omic data from three independent datasets (training on FOCUS trial, n=278 patients; test on rectal cancer biopsies, GRAMPIAN cohort, n=144 patients; and The Cancer Genome Atlas (TCGA), n=430 patients). Ground truth CMS calls were ascertained by matching random forest and single sample predictions from CMS classifier. Results: Image-based CMS (imCMS) accurately classified slides in unseen datasets from TCGA (n=431 slides, AUC)=0.84) and rectal cancer biopsies (n=265 slides, AUC=0.85). imCMS spatially resolved intratumoural heterogeneity and provided secondary calls correlating with bioinformatic prediction from molecular data. imCMS classified samples previously unclassifiable by RNA expression profiling, reproduced the expected correlations with genomic and epigenetic alterations and showed similar prognostic associations as transcriptomic CMS. Conclusion: This study shows that a prediction of RNA expression classifiers can be made from H&E images, opening the door to simple, cheap and reliable biological stratification within routine workflows.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1136/gutjnl-2019-319866

Cites

30 of 30 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 Data, study design and imCMS classification framework. Three independent datasets (FOCUS, TCGA and GRAMPIAN) were used in this study. (A) The distribution of the samples stratified by the CMS calls in each dataset. (B) The FOCUS dataset was primarily used for learning the imCMS discriminative model, while the TCGA and GRAMPIAN datasets were used for testing. (C) Training of the imCMS discriminative model based on the domain adversarial approach. Image tiles were extracted from annotated tumour regions. Tiles from the FOCUS cohort were categorised by CMS class of the original slide and were used to train the model to predict the imCMS classes on unseen datasets. Tiles from the TCGA and GRAMPIAN cohorts were unlabelled and were used together with those from the FOCUS cohort in the cohort (domain) prediction. Domain adversarial training forced the cohort classifier to perform poorly, which in turn encouraged the model to learn indiscriminative features across datasets. Five distinct models were produced. (D) At the inference time, the ensemble of the learnt models predicts the imCMS class for each of the image tiles extracted from annotated tumour regions of a slide. A slide is assigned to the imCMS class with the maximum prediction score (ie, highest number of tiles in the slide). imCMS, image-based consensus molecular subtype; TCGA, The Cancer Genome Atlas.
Figure 1 Data, study design and imCMS classification framework. Three independent datasets (FOCUS, TCGA and GRAMPIAN) were used in this study. (A) The distribution of the samples stratified by the CMS calls in each dataset. (B) The FOCUS dataset was primarily used for learning the imCMS discriminative model, while the TCGA and GRAMPIAN datasets were used for testing. (C) Training of the imCMS discriminative model based on the domain adversarial approach. Image tiles were extracted from annotated tumour regions. Tiles from the FOCUS cohort were categorised by CMS class of the original slide and were used to train the model to predict the imCMS classes on unseen datasets. Tiles from the TCGA and GRAMPIAN cohorts were unlabelled and were used together with those from the FOCUS cohort in the cohort (domain) prediction. Domain adversarial training forced the cohort classifier to perform poorly, which in turn encouraged the model to learn indiscriminative features across datasets. Five distinct models were produced. (D) At the inference time, the ensemble of the learnt models predicts the imCMS class for each of the image tiles extracted from annotated tumour regions of a slide. A slide is assigned to the imCMS class with the maximum prediction score (ie, highest number of tiles in the slide). imCMS, image-based consensus molecular subtype; TCGA, The Cancer Genome Atlas.CC BY · PubMed Central
Figure 2 Image-based consensus molecular subtype (imCMS) classification. (A) Receiver operating curves of the imCMS classifier, optimised by the domain adversarial approach, on the FOCUS (n slides=510, 3×), TCGA (n slides=431, 3×) and GRAMPIAN cohorts (n slides=265, 12×). (B) Correspondences between CMS and imCMS classes in different datasets. All samples labelled as unclassified by RNA-based CMS calls were reclassified by imCMS. (C) Examples of image tiles with high prediction confidence for each imCMS class in FOCUS. Histological patterns associated with imCMS1 are mucin and lymphocytic infiltration. In imCMS2, evident cribriform growth patterns and comedo-like necrosis are observed, while imCMS3 is characterised by ectatic, mucin-filled glandular structures in combination with a minor component showing papillary and cribriform morphology. imCMS4 is predominantly associated with infiltrative CRC growth pattern, a prominent desmoplastic stromal reaction and frequent presence of single cell invasion (tumour budding). Scale bar ~1 mm. (D) Molecular associations of the CMS classified samples (black) and the CMS unclassified samples that have been classified by imCMS (grey). The molecular profiles of reclassified samples are largely consistent with those of the classified CMS samples. Statistically significant differences (p<0.05) are marked with a red asterisk. AUC, area under the curve; TCGA, The Cancer Genome Atlas.
Figure 2 Image-based consensus molecular subtype (imCMS) classification. (A) Receiver operating curves of the imCMS classifier, optimised by the domain adversarial approach, on the FOCUS (n slides=510, 3×), TCGA (n slides=431, 3×) and GRAMPIAN cohorts (n slides=265, 12×). (B) Correspondences between CMS and imCMS classes in different datasets. All samples labelled as unclassified by RNA-based CMS calls were reclassified by imCMS. (C) Examples of image tiles with high prediction confidence for each imCMS class in FOCUS. Histological patterns associated with imCMS1 are mucin and lymphocytic infiltration. In imCMS2, evident cribriform growth patterns and comedo-like necrosis are observed, while imCMS3 is characterised by ectatic, mucin-filled glandular structures in combination with a minor component showing papillary and cribriform morphology. imCMS4 is predominantly associated with infiltrative CRC growth pattern, a prominent desmoplastic stromal reaction and frequent presence of single cell invasion (tumour budding). Scale bar ~1 mm. (D) Molecular associations of the CMS classified samples (black) and the CMS unclassified samples that have been classified by imCMS (grey). The molecular profiles of reclassified samples are largely consistent with those of the classified CMS samples. Statistically significant differences (p<0.05) are marked with a red asterisk. AUC, area under the curve; TCGA, The Cancer Genome Atlas.CC BY · PubMed Central
Figure 3 Intratumoural heterogeneity of the imCMS molecular subtypes. (A) Visualisation of the regional classification of the imCMS classifier. imCMS classification of a tumour sample can exhibit uniform results (left) or a degree of variation in the predicted imCMS class and the level of confidence (right). The colour overlay indicates the imCMS classes and the opacity reflects the classification confidence. (B) Heterogeneity of the CMS and imCMS classification scores. Each bar represents classification scores of a sample, and samples are sorted by the entropy of the prediction scores from the molecular-based random forest CMS classifier. (C) Heterogeneity of the CMS classification. A secondary CMS call was derived by relaxing the classification threshold of the random forest CMS classifier.11 (D) Cosine similarity between the imCMS and CMS prediction scores, stratified by the primary and secondary CMS calls. The levels of similarity were compared against those produced by a random classifier. Statistical analysis was performed using Wilcoxon rank-sum test, adjusted for the false discovery rate. P value <0.05 was considered statistically significant. n indicates the number of patients. Note that two diagnostic slides (serial sections) were available for the majority of cases in the FOCUS and GRAMPIAN cohorts. In cases where two slides were available, the analyses for each slide were performed separately. Panels (B) and (D) report the results for the first slide. The matched results for the second slide are provided in online supplementary figure S10. imCMS predictions represent the calls made by the domain adversarially trained imCMS classifier. imCMS, image-based consensus molecular subtype; TCGA, The Cancer Genome Atlas.
Figure 3 Intratumoural heterogeneity of the imCMS molecular subtypes. (A) Visualisation of the regional classification of the imCMS classifier. imCMS classification of a tumour sample can exhibit uniform results (left) or a degree of variation in the predicted imCMS class and the level of confidence (right). The colour overlay indicates the imCMS classes and the opacity reflects the classification confidence. (B) Heterogeneity of the CMS and imCMS classification scores. Each bar represents classification scores of a sample, and samples are sorted by the entropy of the prediction scores from the molecular-based random forest CMS classifier. (C) Heterogeneity of the CMS classification. A secondary CMS call was derived by relaxing the classification threshold of the random forest CMS classifier.11 (D) Cosine similarity between the imCMS and CMS prediction scores, stratified by the primary and secondary CMS calls. The levels of similarity were compared against those produced by a random classifier. Statistical analysis was performed using Wilcoxon rank-sum test, adjusted for the false discovery rate. P value <0.05 was considered statistically significant. n indicates the number of patients. Note that two diagnostic slides (serial sections) were available for the majority of cases in the FOCUS and GRAMPIAN cohorts. In cases where two slides were available, the analyses for each slide were performed separately. Panels (B) and (D) report the results for the first slide. The matched results for the second slide are provided in online supplementary figure S10. imCMS predictions represent the calls made by the domain adversarially trained imCMS classifier. imCMS, image-based consensus molecular subtype; TCGA, The Cancer Genome Atlas.CC BY · PubMed Central
Figure 4 Prognostic associations of the image-based consensus molecular subtypes (imCMSs). Overall survival (OS) outcomes of the FOCUS cohort (n=278 patients, (A)) and TCGA cohort (n=395 patients, (B)), progression-free interval (PFI) outcome of the TCGA cohort (n=395, (C)) and relapse-free survival (RFS) outcome (n=83, (D)) as stratified by the transcriptional-based CMS classification and imCMS classification produced by the domain adversarially trained imCMS classifier. Kaplan–Meier estimator was used to estimate the survival probability, and pairwise log-rank test and univariate Cox proportional hazards regression were performed between CMS groups and imCMS groups. HRs and 95% CI for pairwise comparisons were reported. Test results with p value<0.05 were considered statistically significant. TCGA, The Cancer Genome Atlas.
Figure 4 Prognostic associations of the image-based consensus molecular subtypes (imCMSs). Overall survival (OS) outcomes of the FOCUS cohort (n=278 patients, (A)) and TCGA cohort (n=395 patients, (B)), progression-free interval (PFI) outcome of the TCGA cohort (n=395, (C)) and relapse-free survival (RFS) outcome (n=83, (D)) as stratified by the transcriptional-based CMS classification and imCMS classification produced by the domain adversarially trained imCMS classifier. Kaplan–Meier estimator was used to estimate the survival probability, and pairwise log-rank test and univariate Cox proportional hazards regression were performed between CMS groups and imCMS groups. HRs and 95% CI for pairwise comparisons were reported. Test results with p value<0.05 were considered statistically significant. TCGA, The Cancer Genome Atlas.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 30 references

Research works printed but not linked: 1

  1. Popovici, 2018, 30(th) European congress of pathology: recognising molecular subtypes of colon cancer from virtual slides, Virchows Arch, 473, 1

Cited by

Sorted by citations

Items 201–265 of 265

Related

See all

Metadata

Authors
Sirinukunwattana, K., Domingo, E., Richman, S. D., Redmond, K. L., Blake, A., Verrill, C., Leedham, S. J., Chatzipli, A., Hardy, C., Whalley, C. M., Wu, C.-H., Beggs, A. D., McDermott, U., Dunne, P. D., Meade, A., Walker, S. M., Murray, G. I., Samuel, L., Seymour, M., Tomlinson, I., Quirke, P., Maughan, T., Rittscher, J., & Koelzer, V. H.
Affiliations
  • University of Oxford (GB)
  • University of Leeds (GB)
  • Queen's University Belfast (GB)
  • Centre for Human Genetics (GB)
  • Wellcome Sanger Institute (GB)
  • University of Birmingham (GB)
  • MRC Clinical Trials Unit at UCL (GB)
  • Almac (United Kingdom) (GB)
  • University of Aberdeen (GB)
  • Aberdeen Royal Infirmary (GB)
  • Department of Oncology, Leeds Institute of Cancer and Pathology, Leeds, UK
  • Edinburgh Cancer Research (GB)
  • Department of Pathology and Tumour Biology, Leeds Institute of Cancer and Pathology, Leeds, UK
  • CRUK/MRC Oxford Institute for Radiation Oncology, University of Oxford, Oxford, UK
  • University of Zurich (CH)
Corresponding author
Koelzer, Viktor H
ORCID
Venue
Gut
Publisher
BMJ
Published
2020-07-20
Volume
70
Issue
3
Pages
544-554
ISSN
  • 0017-5749 (ISSN-L)
  • 1468-3288
DOI
10.1136/gutjnl-2019-319866
PMC
PMC7873419
OpenAlex
W3043602140
Semantic Scholar
9531fb91f6876c57e3cf9bcb77ec79cc6f2a84d3
Cited by
264
References
30
Citations by year
2026: 35 · 2025: 40 · 2024: 63 · 2023: 37 · 2022: 57 · 2021: 28 · 2020: 4
FWCI
12.37
Type
journal article
Language
en
Open access
Licence
Locations
  • Gut (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Research Portal (Queen's University Belfast) (publishedVersion, cc-by)
  • ePrints Soton (University of Southampton) (acceptedVersion)
  • Zurich Open Repository and Archive (University of Zurich) (acceptedVersion, cc-by)
  • Aberdeen University Research Archive (Aberdeen University) (submittedVersion)
  • UCL Discovery (University College London) (submittedVersion)
  • University of Birmingham Research Portal (University of Birmingham) (submittedVersion)
  • University of Birmingham Research Portal (University of Birmingham) (submittedVersion, cc-by)
  • Edinburgh Research Explorer (University of Edinburgh) (submittedVersion, cc-by)
  • Zurich Open Repository and Archive (University of Zurich) (submittedVersion, cc-by)
  • Zurich Open Repository and Archive (University of Zurich)
  • Greater South Information System
Funders
  • National Institute for Health Research (NIHR) Oxford Biomedical Research Centre
  • Engineering and Physical Sciences Research Council (EP/M013774/1)
  • Medical Research Council UK
  • Cancer Research UK
  • Promedica Foundation (F-87701-41-01)
  • Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung (P2SKP3_168322/1)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
  • Health Sciences › Medicine › Pathology and Forensic Medicine › Genetic factors in colorectal cancer
Keywords
colorectal cancer, consensus molecular subtypes, image classification, H&E histology, deep learning, intratumoral heterogeneity, rectal cancer, histopathology, molecular classification, prognostic stratification, epigenetic modifications
MeSH terms
Deep Learning, Biopsy, Humans, Phenotype, Predictive Value of Tests, Prognosis, RNA, Biomarkers, Tumor, Colorectal Neoplasms, Gene Expression Regulation, Neoplastic, Disease Progression, Gene Expression Profiling, Consensus, Neoplasm Grading, Datasets as Topic

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review