CoChat

Predicting cancer outcomes from histology and genomics using convolutional networks

Open access
Mobadersany, Pooya, Yousefi, Safoora, Amgad, Mohamed, Gutman, David A., Barnholtz-Sloan, Jill S., Velázquez Vega, José E., Brat, Daniel J., Cooper, Lee A. D. · 2018 · Proceedings of the National Academy of Sciences

Published version of a preprint

The DOI matches its Crossref record; 53 of the 53 references it cites were found in a registry or an index; 4 of the 13 sources its claims cite were checked for support.

Abstract

Significance: Predicting the expected outcome of patients diagnosed with cancer is a critical step in treatment. Advances in genomic and imaging technologies provide physicians with vast amounts of data, yet prognostication remains largely subjective, leading to suboptimal clinical management. We developed a computational approach based on deep learning to predict the overall survival of patients diagnosed with brain tumors from microscopic images of tissue biopsies and genomic biomarkers. This method uses adaptive feedback to simultaneously learn the visual patterns and molecular biomarkers associated with patient outcomes. Our approach surpasses the prognostic accuracy of human experts using the current clinical standard for classifying brain tumors and presents an innovative approach for objective, accurate, and integrated prediction of patient outcomes.

Overview

CoChat Summary

TL;DR

The paper presents survival convolutional neural networks (SCNNs) and genomic SCNNs (GSCNNs) that predict overall survival in diffuse gliomas from histology and genomic biomarkers. SCNN performance was comparable to manual histologic grading or molecular subtyping, and GSCNN integrating IDH and 1p/19q improved median c index to 0.801, outperforming the WHO subtype-grade baseline.

Key findings

  • In 769 TCGA gliomas (1,061 H&E whole-slide images), SCNN achieved a median c index of 0.754, comparable to manual histologic grade (0.745, P = 0.307) and molecular subtype (0.746, P = 4.68e-2), but slightly below the combined WHO subtype-plus-grade baseline (0.774, P = 2.61e-3).
  • Sampling and risk filtering improved SCNN: without these techniques, median c index dropped to 0.696 (P = 6.55e-4).
  • GSCNN using histology plus IDH mutation status and 1p/19q codeletion improved median c index from 0.754 to 0.801 and significantly outperformed the WHO subtype-grade baseline (P = 1.06e-2); superficial integration performed worse (0.785, P = 4.68e-2).
  • Multivariable Cox regression found SCNN risk significant (HR 3.05, P = 2.71e-12) and GSCNN risk significant (HR 8.83, P = 9.69e-12); heat maps highlighted microvascular proliferation, cellularity, and nuclear pleomorphism, and also assigned high risk to sparsely infiltrated or edematous regions.

Methods

  • Retrospective TCGA Lower-Grade Glioma and Glioblastoma cohort: 769 patients and 1,061 H&E-stained whole-slide images; manually identified regions of interest, 1,024 × 1,024-pixel crops at 20× magnification, and 256 × 256-pixel high-power fields.
  • SCNN combined a 19-layer VGG-style convolutional network with a Cox proportional hazards layer; training used random HPF sampling per epoch, image augmentation, minibatches of 14 HPFs, adagrad, and dropout, while testing used nine HPFs per region, median risk per region, second-highest region risk, and model averaging over epochs 96–100.
  • Evaluation used Monte Carlo cross-validation with 15 random patient-level 80/20 training/testing splits, Harrell’s c index, Wilcoxon signed rank tests, Cox regression, and Kaplan–Meier analysis.

Limitations

  • Stated: only a small portion of each slide was used for training and prediction; ROI selection required expert guidance; risk-category thresholds were subjectively defined; heat map interpretation was subjective; validation in other diseases and larger datasets for enhanced GSCNN models are needed.
  • Evident from the text: the study is based on a single retrospective TCGA cohort, with no external independent validation reported, and manual ROI selection may limit scalability and generalizability.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1073/pnas.1717139115

Cites

53 of 53 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Fig. 1. The SCNN model. The SCNN combines deep learning CNNs with traditional survival models to learn survival-related patterns from histology images. (A) Large whole-slide images are generated by digitizing H&E-stained glass slides. (B) A web-based viewer is used to manually identify representative ROIs in the image. (C) HPFs are sampled from these regions and used to train a neural network to predict patient survival. The SCNN consists of (i) convolutional layers that learn visual patterns related to survival using convolution and pooling operations, (ii) fully connected layers that provide additional nonlinear transformations of extracted image features, and (iii) a Cox proportional hazards layer that models time-to-event data, like overall survival or time to progression. Predictions are compared with patient outcomes to adaptively train the network weights that interconnect the layers.
Fig. 1. The SCNN model. The SCNN combines deep learning CNNs with traditional survival models to learn survival-related patterns from histology images. (A) Large whole-slide images are generated by digitizing H&E-stained glass slides. (B) A web-based viewer is used to manually identify representative ROIs in the image. (C) HPFs are sampled from these regions and used to train a neural network to predict patient survival. The SCNN consists of (i) convolutional layers that learn visual patterns related to survival using convolution and pooling operations, (ii) fully connected layers that provide additional nonlinear transformations of extracted image features, and (iii) a Cox proportional hazards layer that models time-to-event data, like overall survival or time to progression. Predictions are compared with patient outcomes to adaptively train the network weights that interconnect the layers.CC BY-NC-ND · PubMed Central
Fig. 2. SCNN uses image sampling and filtering to improve the robustness of training and prediction. (A) During training, a single 256 × 256-pixel HPF is sampled from each region, producing multiple HPFs per patient. Each HPF is subjected to a series of random transformations and is then used as an independent sample to update the network weights. New HPFs are sampled at each training epoch (one training pass through all patients). (B) When predicting the outcome of a newly diagnosed patient, nine HPFs are sampled from each ROI, and a risk is predicted for each field. The median HPF risk is calculated in each region, these median risks are then sorted, and the second highest value is selected as the patient risk. This sampling and filtering framework was designed to deal with tissue heterogeneity by emulating manual histologic evaluation, where prognostication is typically based on the most malignant region observed within a heterogeneous sample. Predictions based on the highest risk and the second highest risk had equal performance on average in our experiments, but the maximum risk produced some outliers with poor prediction accuracy.
Fig. 2. SCNN uses image sampling and filtering to improve the robustness of training and prediction. (A) During training, a single 256 × 256-pixel HPF is sampled from each region, producing multiple HPFs per patient. Each HPF is subjected to a series of random transformations and is then used as an independent sample to update the network weights. New HPFs are sampled at each training epoch (one training pass through all patients). (B) When predicting the outcome of a newly diagnosed patient, nine HPFs are sampled from each ROI, and a risk is predicted for each field. The median HPF risk is calculated in each region, these median risks are then sorted, and the second highest value is selected as the patient risk. This sampling and filtering framework was designed to deal with tissue heterogeneity by emulating manual histologic evaluation, where prognostication is typically based on the most malignant region observed within a heterogeneous sample. Predictions based on the highest risk and the second highest risk had equal performance on average in our experiments, but the maximum risk produced some outliers with poor prediction accuracy.CC BY-NC-ND · PubMed Central
Fig. 3. Prognostication criteria for diffuse gliomas. (A) Prognosis in the diffuse gliomas is determined by genomic classification and manual histologic grading. Diffuse gliomas are first classified into one of three molecular subtypes based on IDH1/IDH2 mutations and the codeletion of chromosomes 1p and 19q. Grade is then determined within each subtype using histologic characteristics. Subtypes with an astrocytic lineage are split by IDH mutation status, and the combination of 1p/19q codeletion and IDH mutation defines an oligodendroglioma. These lineages have histologic differences; however, histologic evaluation is not a reliable predictor of molecular subtype (37). Histologic criteria used for grading range from nuclear morphology to higher-level patterns, like necrosis or the presence of abnormal microvascular structures. (B) Comparison of the prognostic accuracy of SCNN models with that of baseline models based on molecular subtype or molecular subtype and histologic grade. Models were evaluated over 15 independent training/testing sets with randomized patient assignments and with/without training and testing sampling. (C) The risks predicted by the SCNN models correlate with both histologic grade and molecular subtype, decreasing with grade and generally trending with the clinical aggressiveness of genomic subtypes. (D) Kaplan–Meier plots comparing manual histologic grading and SCNN predictions. Risk categories (low, intermediate, high) were generated by thresholding SCNN risks. N/A, not applicable.
Fig. 3. Prognostication criteria for diffuse gliomas. (A) Prognosis in the diffuse gliomas is determined by genomic classification and manual histologic grading. Diffuse gliomas are first classified into one of three molecular subtypes based on IDH1/IDH2 mutations and the codeletion of chromosomes 1p and 19q. Grade is then determined within each subtype using histologic characteristics. Subtypes with an astrocytic lineage are split by IDH mutation status, and the combination of 1p/19q codeletion and IDH mutation defines an oligodendroglioma. These lineages have histologic differences; however, histologic evaluation is not a reliable predictor of molecular subtype (37). Histologic criteria used for grading range from nuclear morphology to higher-level patterns, like necrosis or the presence of abnormal microvascular structures. (B) Comparison of the prognostic accuracy of SCNN models with that of baseline models based on molecular subtype or molecular subtype and histologic grade. Models were evaluated over 15 independent training/testing sets with randomized patient assignments and with/without training and testing sampling. (C) The risks predicted by the SCNN models correlate with both histologic grade and molecular subtype, decreasing with grade and generally trending with the clinical aggressiveness of genomic subtypes. (D) Kaplan–Meier plots comparing manual histologic grading and SCNN predictions. Risk categories (low, intermediate, high) were generated by thresholding SCNN risks. N/A, not applicable.CC BY-NC-ND · PubMed Central
Fig. 4. GSCNN models integrate genomic and imaging data for improved performance. (A) A hybrid architecture was developed to combine histology image and genomic data to make integrated predictions of patient survival. These models incorporate genomic variables as inputs to their fully connected layers. Here, we show the incorporation of genomic variables for gliomas; however, any number of genomic or proteomic measurements can be similarly used. (B) The GSCNN models significantly outperform SCNN models as well as the WHO paradigm based on genomic subtype and histologic grading.
Fig. 4. GSCNN models integrate genomic and imaging data for improved performance. (A) A hybrid architecture was developed to combine histology image and genomic data to make integrated predictions of patient survival. These models incorporate genomic variables as inputs to their fully connected layers. Here, we show the incorporation of genomic variables for gliomas; however, any number of genomic or proteomic measurements can be similarly used. (B) The GSCNN models significantly outperform SCNN models as well as the WHO paradigm based on genomic subtype and histologic grading.CC BY-NC-ND · PubMed Central
Fig. 5. Visualizing risk with whole-slide SCNN heat maps. We performed SCNN predictions exhaustively within whole-slide images to generate heat map overlays of the risks that SCNN associates with different histologic patterns. Red indicates relatively higher risk, and blue indicates lower risk (the scale for each slide is different). (Top) In TCGA-DB-5273, SCNN clearly and specifically predicts high risks for regions of early microvascular proliferation (region 1) and also, higher risks with increasing tumor infiltration and cell density (region 2 vs. 3). (Middle) In TCGA-S9-A7J0, SCNN can appropriately discriminate between normal cortex (region 1 in Bottom) and adjacent regions infiltrated by tumor (region 1 in Top). Highly cellular regions containing prominent microvascular structures (region 3) are again assigned higher risks than lower-density regions of tumor (region 2). Interestingly, low-density infiltrate in the cortex was associated with high risk (region 1 in Top). (Bottom) In TCGA-TM-A84G, SCNN assigns high risks to edematous regions (region 1 in Bottom) that are adjacent to tumor (region 1 in Top).
Fig. 5. Visualizing risk with whole-slide SCNN heat maps. We performed SCNN predictions exhaustively within whole-slide images to generate heat map overlays of the risks that SCNN associates with different histologic patterns. Red indicates relatively higher risk, and blue indicates lower risk (the scale for each slide is different). (Top) In TCGA-DB-5273, SCNN clearly and specifically predicts high risks for regions of early microvascular proliferation (region 1) and also, higher risks with increasing tumor infiltration and cell density (region 2 vs. 3). (Middle) In TCGA-S9-A7J0, SCNN can appropriately discriminate between normal cortex (region 1 in Bottom) and adjacent regions infiltrated by tumor (region 1 in Top). Highly cellular regions containing prominent microvascular structures (region 3) are again assigned higher risks than lower-density regions of tumor (region 2). Interestingly, low-density infiltrate in the cortex was associated with high risk (region 1 in Top). (Bottom) In TCGA-TM-A84G, SCNN assigns high risks to edematous regions (region 1 in Bottom) that are adjacent to tumor (region 1 in Top).CC BY-NC-ND · PubMed Central

Verification

Claims the paper makes

4 Valid · 0 Not valid · 8 Not checked · 87 Own results and interpretations

52 more claims

Create a free account to see all 99 claims and to verify the 9 references still open.

See all claims — free account
  • It may state two separate claims as one. 26
  • Probably not a substantial claim. 21

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

1 of 1

Citations

Cites

Show all 53 references

Cited by

See all 1,084

Metadata

Authors
Mobadersany, P., Yousefi, S., Amgad, M., Gutman, D. A., Barnholtz-Sloan, J. S., Velázquez Vega, J. E., Brat, D. J., & Cooper, L. A. D.
Affiliations
  • Emory University (US)
  • Case Comprehensive Cancer Center, Case Western Reserve University School of Medicine, Cleveland, OH 44106;
  • Department of Pathology and Laboratory Medicine, Emory University School of Medicine, Atlanta, GA 30322;
  • Northwestern University (US)
  • Georgia Institute of Technology (US)
Corresponding author
Cooper, Lee A. D.
Venue
Proceedings of the National Academy of Sciences
Published
2018-03-12
Volume
115
Issue
13
Pages
E2970-E2979
ISSN
  • 0027-8424 (ISSN-L)
  • 1091-6490
OpenAlex
W2761668583
Cited by
1,084
References
52
Citations by year
2026: 120 · 2025: 126 · 2024: 157 · 2023: 124 · 2022: 172 · 2021: 157 · 2020: 130 · 2019: 84 · 2018: 13
FWCI
54.74
Type
journal article
Language
en
Open access
Locations
  • Proceedings of the National Academy of Sciences (publishedVersion, cc-by-nc-nd)
  • bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion)
  • PubMed Central (publishedVersion, cc-by-nc-nd)
  • PubMed (publishedVersion)
Funders
  • HHS | NIH | U.S. National Library of Medicine (K22LM011576)
  • HHS | NIH | National Cancer Institute (U24CA194362)
  • National Brain Tumor Society (Not issued)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
Keywords
cancer prognosis, digital pathology, glioma, histological images, genomic biomarkers, time-to-event prediction, overall survival, tumor heterogeneity, microvascular proliferation, heat map visualization, precision medicine
MeSH terms
Algorithms, Brain Neoplasms, Glioma, Histological Techniques, Humans, Image Processing, Computer-Assisted, Prognosis, Neural Networks, Computer, Genomics, Precision Medicine

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-06, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-06 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (qwen3.8-omni-flash).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Predicting cancer outcomes from histology and genomics using convolutional networks. https://app.cochat.ai/papers/10.1073/pnas.1717139115
The paper
Mobadersany, P., Yousefi, S., Amgad, M., Gutman, D. A., Barnholtz-Sloan, J. S., Velázquez Vega, J. E., Brat, D. J., & Cooper, L. A. D. (2018). Predicting cancer outcomes from histology and genomics using convolutional networks. Proceedings of the National Academy of Sciences. https://doi.org/10.1073/pnas.1717139115

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.