CoChat
Sign Up
Cited by 136AI in cancer detectionRadiomics and Machine Learning in Medical ImagingGenetic factors in colorectal cancer

Histopathology images predict multi-omics aberrations and prognoses in colorectal cancer patients

Open access
Tsai, Pei-Chen, Lee, Tsung-Hua, Kuo, Kun-Chi, Su, Fang-Yi, Lee, Tsung-Lu Michael, Marostica, Eliana, Ugai, Tomotaka, Zhao, Melissa +13 more·2023·Nature Communications
DOI 10.1038/s41467-023-37179-4PMC PMC10102208
Open this paper in CoChat — free accountOpen PDF

Abstract

Histopathologic assessment is indispensable for diagnosing colorectal cancer (CRC). However, manual evaluation of the diseased tissues under the microscope cannot reliably inform patient prognosis or genomic variations crucial for treatment selections. To address these challenges, we develop the Multi-omics Multi-cohort Assessment (MOMA) platform, an explainable machine learning approach, to systematically identify and interpret the relationship between patients’ histologic patterns, multi-omics, and clinical profiles in three large patient cohorts (n = 1888). MOMA successfully predicts the overall survival, disease-free survival (log-rank test P-value<0.05), and copy number alterations of CRC patients. In addition, our approaches identify interpretable pathology patterns predictive of gene expression profiles, microsatellite instability status, and clinically actionable genetic alterations. We show that MOMA models are generalizable to multiple patient populations with different demographic compositions and pathology images collected from distinctive digitization methods. Our machine learning approaches provide clinically actionable predictions that could inform treatments for colorectal cancer patients.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s41467-023-37179-4

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Some deep learning approaches have focused on detecting molecular alterations in image data, while most existing studies have centered on inferring spatial transcriptome or bulk transcriptome information from histopathology images.

    “While some deep learning approaches have focused on detecting molecular alterations in image data [132], most existing studies have centered on inferring spatial transcriptome or bulk transcriptome information from histopathology images [133–136].”

    From Challenges in AI-driven Biomedical Multimodal Data Fusion and Analysis (2025) · cites this paper as [132]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

References 60 · Cited by 136

Research works printed but not linked: 51

  1. [1]Siegel RL Miller KD Fuchs HE Jemal A Cancer Statistics, 2021 CA Cancer J. Clin. 2021 71 7 33 10.3322/caac.21654 33433946 · DOI · Paper page
  2. [2]Benson AB Colon Cancer, Version 2.2021, NCCN Clinical Practice Guidelines in Oncology J. Natl Compr. Canc. Netw. 2021 19 329 359 10.6004/jnccn.2021.0012 33724754 · DOI
  3. [3]Otálora S Atzori M Andrearczyk V Khan A Müller H Staining Invariant Features for Improving Generalization of Deep Convolutional Neural Networks in Computational Pathology Front Bioeng. Biotechnol. 2019 7 198 10.3389/fbioe.2019.00198 31508414 PMC6716536 · DOI
  4. [5]Coudray N Classification and mutation prediction from non-small cell lung cancer histopathology images using deep learning Nat. Med. 2018 24 1559 1567 10.1038/s41591-018-0177-5 30224757 PMC9847512 · DOI · Paper page
  5. [6]Han Z Breast Cancer Multi-classification from Histopathological Images with Structured Deep Learning Model Sci. Rep. 2017 7 4172 10.1038/s41598-017-04075-z 28646155 PMC5482871 · DOI
  6. [7]Yu K-H Deciphering serous ovarian carcinoma histopathology and platinum response by convolutional neural networks BMC Med. 2020 18 236 10.1186/s12916-020-01684-w 32807164 PMC7433108 · DOI
  7. [8]Marostica E Development of a Histopathology Informatics Pipeline for Classification and Prediction of Clinical Outcomes in Subtypes of Renal Cell Carcinoma Clin. Cancer Res. 2021 27 2868 2878 10.1158/1078-0432.CCR-20-4119 33722896 · DOI
  8. [9]Campanella G Clinical-grade computational pathology using weakly supervised deep learning on whole slide images Nat. Med. 2019 25 1301 1309 10.1038/s41591-019-0508-1 31308507 PMC7418463 · DOI · Paper page
  9. [10]Wulczyn E Interpretable survival prediction for colorectal cancer using deep learning NPJ Digit Med. 2021 4 71 10.1038/s41746-021-00427-2 33875798 PMC8055695 · DOI · Paper page
  10. [11]Chuang W-Y Identification of nodal micrometastasis in colorectal cancer using deep learning on annotation-free whole-slide images Mod. Pathol. 2021 34 1901 1911 10.1038/s41379-021-00838-2 34103664 PMC8443445 · DOI
  11. [13]Ektefaie Y Integrative multiomics-histopathology analysis for breast cancer classification NPJ Breast Cancer. 2021 7 147 10.1038/s41523-021-00357-y 34845230 PMC8630188 · DOI
  12. [14]Jang H-J Lee A Kang J Song IH Lee SH Prediction of clinically actionable genetic alterations from colorectal cancer histopathology images using deep learning World J. Gastroenterol. 2020 26 6207 6223 10.3748/wjg.v26.i40.6207 33177794 PMC7596644 · DOI
  13. [15]Yu K-H Classifying non-small cell lung cancer types and transcriptomic subtypes using convolutional neural networks J. Am. Med. Inform. Assoc. 2020 27 757 769 10.1093/jamia/ocz230 32364237 PMC7309263 · DOI
  14. [16]Yu, K.-H. et al. Predicting non-small cell lung cancer prognosis by fully automated microscopic pathology image features. Nat. Commun.7, 12474 (2016). PMC4990706 27527408
  15. [17]Bychkov D Deep learning based tissue analysis predicts outcome in colorectal cancer Sci. Rep. 2018 8 3395 10.1038/s41598-018-21758-3 29467373 PMC5821847 · DOI · Paper page
  16. [18]Yamashita, R. et al. Deep learning model for the prediction of microsatellite instability in colorectal cancer: a diagnostic study. Lancet Oncol. 22, 132–141 (2021). 33387492
  17. [20]Tizhoosh HR Pantanowitz L Artificial Intelligence and Digital Pathology: Challenges and Opportunities J. Pathol. Inform. 2018 9 38 10.4103/jpi.jpi_53_18 30607305 PMC6289004 · DOI · Paper page
  18. [21]Turki H Taieb MAH Aouicha MB Developing intuitive and explainable algorithms through inspiration from human physiology and computational biology Brief. Bioinforma. 2021 22 bbab081 10.1093/bib/bbab081 33866351 · DOI
  19. [22]Stacke K Eilertsen G Unger J Lundstrom C Measuring Domain Shift for Deep Learning in Histopathology IEEE J. Biomed. Health Inf. 2021 25 325 336 10.1109/JBHI.2020.3032060 33085623 · DOI
  20. [23]Ducreux M Molecular targeted therapy of BRAF-mutant colorectal cancer Ther. Adv. Med. Oncol. 2019 11 1758835919856494 10.1177/1758835919856494 31244912 PMC6582307 · DOI
  21. [24]Juo YY Prognostic value of CpG island methylator phenotype among colorectal cancer patients: a systematic review and meta-analysis Ann. Oncol. 2014 25 2314 2327 10.1093/annonc/mdu149 24718889 PMC4239805 · DOI
  22. [25]Guinney J The consensus molecular subtypes of colorectal cancer Nat. Med. 2015 21 1350 1356 10.1038/nm.3967 26457759 PMC4636487 · DOI · Paper page
  23. [26]Phipps AI Association between molecular subtypes of colorectal cancer and patient survival Gastroenterology 2015 148 77 87.e2 10.1053/j.gastro.2014.09.038 25280443 PMC4274235 · DOI
  24. [27]Yu K-H Snyder M Omics Profiling in Precision Oncology Mol. Cell. Proteom. 2016 15 2525 2536 10.1074/mcp.O116.059253 PMC4974334 27099341 · DOI
  25. [28]Fu Y Pan-cancer computational histopathology reveals mutations, tumor composition and prognosis Nat. Cancer. 2020 1 800 810 10.1038/s43018-020-0085-8 35122049 · DOI
  26. [29]Zhang H Genomic analysis and selective small molecule inhibition identifies BCL-X(L) as a critical survival factor in a subset of colorectal cancer Mol. Cancer. 2015 14 126 10.1186/s12943-015-0397-y 26134786 PMC4487849 · DOI
  27. [30]Cho S-Y A Novel Combination Treatment Targeting BCL-X and MCL1 for -mutated and -amplified Colorectal Cancers Mol. Cancer Ther. 2017 16 2178 2190 10.1158/1535-7163.MCT-16-0735 28611106 · DOI
  28. [31]Mendelaar PAJ Whole genome sequencing of metastatic colorectal cancer reveals prior treatment effects and specific metastasis features Nat. Commun. 2021 12 574 10.1038/s41467-020-20887-6 33495476 PMC7835235 · DOI
  29. [32]Yu K-H Association of Omics Features with Histopathology Patterns in Lung Adenocarcinoma Cell Syst. 2017 5 620 627.e3 10.1016/j.cels.2017.10.014 29153840 PMC5746468 · DOI
  30. [33]Thrift AP Kanwal F El-Serag HB Prediction Models for Gastrointestinal and Liver Diseases: Too Many Developed, Too Few Validated Clin. Gastroenterol. Hepatol. 2016 14 1678 1680 10.1016/j.cgh.2016.08.026 27574756 · DOI
  31. [34]Yu K-H Reproducible Machine Learning Methods for Lung Cancer Detection Using Computed Tomography Images: Algorithm Development and Validation J. Med. Internet Res. 2020 22 e16709 10.2196/16709 32755895 PMC7439139 · DOI
  32. [35]Roberts K Biomedical informatics advancing the national health agenda: the AMIA 2015 year-in-review in clinical and consumer informatics J. Am. Med. Inform. Assoc. 2017 24 e185 e190 10.1093/jamia/ocw103 27497798 PMC6080724 · DOI · Paper page
  33. [36]Yu K-H Beam AL Kohane IS Artificial intelligence in healthcare Nat. Biomed. Eng. 2018 2 719 731 10.1038/s41551-018-0305-z 31015651 · DOI · Paper page
  34. [38]Zhou Y Single-Cell Multiomics Sequencing Reveals Prevalent Genomic Alterations in Tumor Stromal Cells of Human Colorectal Cancer Cancer Cell. 2020 38 818 828.e5 10.1016/j.ccell.2020.09.015 33096021 · DOI
  35. [39]Ganesh K Immunotherapy in colorectal cancer: rationale, challenges and potential Nat. Rev. Gastroenterol. Hepatol. 2019 16 361 375 10.1038/s41575-019-0126-x 30886395 PMC7295073 · DOI · Paper page
  36. [40]Sooro MA Zhang N Zhang P Targeting EGFR-mediated autophagy as a potential strategy for cancer therapy Int. J. Cancer. 2018 143 2116 2125 10.1002/ijc.31398 29574749 · DOI
  37. [41]Tan P Myeloid loss of Beclin 1 promotes PD-L1hi precursor B cell lymphoma development J. Clin. Invest. 2019 129 5261 5277 10.1172/JCI127721 31503548 PMC6877338 · DOI
  38. [42]Chen, T., Kornblith, S., Norouzi, M. & Hinton, G. A Simple Framework for Contrastive Learning of Visual Representations. In Proceedings of the 37th International Conference on Machine Learning (eds. Iii, H. D. & Singh, A.) vol. 119 1597–1607 (PMLR, 2020).
  39. [44]Dehaene, O. et al. Self-Supervision Closes the Gap Between Weak and Strong Supervision in Histology. arXiv. Preprint at 10.48550/arXiv.2012.03583 (2020). · DOI
  40. [45]He, K., Fan, H., Wu, Y., Xie, S. & Girshick, R. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 9729–9738 (2020).
  41. [47]Black A PLCO: Evolution of an Epidemiologic Resource and Opportunities for Future Studies Rev. Recent Clin. Trials. 2015 10 238 245 10.2174/157488711003150928130654 26435289 PMC6444921 · DOI
  42. [49]Bao Y Origin, Methods, and Evolution of the Three Nurses’ Health Studies Am. J. Public Health. 2016 106 1573 1581 10.2105/AJPH.2016.303338 27459450 PMC4981810 · DOI
  43. [50]Väyrynen JP Prognostic Significance of Immune Cell Populations Identified by Machine Learning in Colorectal Cancer Using Routine Hematoxylin and Eosin-Stained Sections Clin. Cancer Res. 2020 26 4326 4338 10.1158/1078-0432.CCR-20-0071 32439699 PMC7442724 · DOI
  44. [51]Taylor AM Genomic and Functional Approaches to Understanding Cancer Aneuploidy Cancer Cell. 2018 33 676 689.e3 10.1016/j.ccell.2018.03.007 29622463 PMC6028190 · DOI
  45. [52]Liu L Utility of inverse probability weighting in molecular pathological epidemiology Eur. J. Epidemiol. 2018 33 381 392 10.1007/s10654-017-0346-8 29264788 PMC5948129 · DOI
  46. [55]Blonska M Agarwal NK Vega F Shaping of the tumor microenvironment: Stromal cells and vessels Semin. Cancer Biol. 2015 34 3 13 10.1016/j.semcancer.2015.03.002 25794825 PMC6374506 · DOI
  47. [58]Dietterich TG Lathrop RH Lozano-Pérez T Solving the multiple instance problem with axis-parallel rectangles Artif. Intell. 1997 89 31 71 10.1016/S0004-3702(96)00034-3 · DOI · Paper page
  48. [59]Zhang, G. et al. Multi-instance learning for skin biopsy image features recognition. In 2012 IEEE International Conference on Bioinformatics and Biomedicine 1–6 (2012).
  49. [60]Lu MY Data-efficient and weakly supervised computational pathology on whole-slide images Nat. Biomed. Eng. 2021 5 555 570 10.1038/s41551-020-00682-w 33649564 PMC8711640 · DOI · Paper page
  50. [61]Lee Y-J Mangasarian OL SSVM: A Smooth Support Vector Machine for Classification Comput. Optim. Appl. 2001 20 5 22 10.1023/A:1011215321374 · DOI
  51. [62]Carroll KJ On the use and utility of the Weibull model in the analysis of survival data Control. Clin. Trials. 2003 24 682 701 10.1016/S0197-2456(03)00072-2 14662274 · DOI

Authors
Tsai, P.-C., Lee, T.-H., Kuo, K.-C., Su, F.-Y., Lee, T.-L. M., Marostica, E., Ugai, T., Zhao, M., Lau, M. C., Väyrynen, J. P., Giannakis, M., Takashima, Y., Kahaki, S. M., Wu, K., Song, M., Meyerhardt, J. A., Chan, A. T., Chiang, J.-H., Nowak, J., Ogino, S., & Yu, K.-H.
Affiliations
  • Harvard University (US)
  • National Cheng Kung University (TW)
  • Brigham and Women's Hospital (US)
  • Oulu University Hospital (FI)
  • Dana-Farber Cancer Institute (US)
  • Broad Institute (US)
Corresponding author
  • Chiang, Jung-Hsien
  • Yu, Kun-Hsing
ORCID
Venue
Nature Communications
Publisher
Springer Science and Business Media LLC
Published
2023-04-13
Volume
14
Issue
1
Pages
2102-2102
ISSN
2041-1723 (ISSN-L)
DOI
10.1038/s41467-023-37179-4
PMC
PMC10102208
OpenAlex
W4365447146
Cited by
136
References
60
Citations by year
2026: 34 · 2025: 41 · 2024: 42 · 2023: 19
FWCI
12.82
Type
journal article
Language
en
Open access
Licence
Locations
  • Nature Communications (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • U.S. Department of Health & Human Services | NIH | National Institute of General Medical Sciences (R35GM142879)
  • Google (Google Research Scholar Award)
  • Harvard Medical School (Blavatnik Center for Computational Biomedicine Award)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
  • Health Sciences › Medicine › Pathology and Forensic Medicine › Genetic factors in colorectal cancer
Keywords
colorectal cancer, histopathological images, multi-omics, outcome prediction, overall survival, disease-free survival, copy number alterations, microsatellite instability, gene expression profiles, actionable genomic alterations, explainable machine learning, model generalization, digital pathology
MeSH terms
Multiomics, Humans, Mutation, Colorectal Neoplasms, Disease-Free Survival, Microsatellite Instability

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review