Predicting endometrial cancer subtypes and molecular features from histopathology images using multi-resolution deep learning models
Open accessPublished version of a preprint
Abstract
Summary The determination of endometrial carcinoma histological subtypes, molecular subtypes, and mutation status is a critical diagnostic process that directly affects patients’ prognosis and treatment options. Compared to the histopathological approach, however, the availability of molecular subtyping is limited as it can only be accurately obtained by genomic sequencing, which may be cost prohibitive. Here, we implemented a customized multi-resolution deep convolutional neural network, Panoptes, that predicts not only the histological subtypes, but also molecular subtypes and 18 common gene mutations based on digitized H&E stained pathological images. The model achieved high accuracy and generalized well on independent datasets. Our results suggest that Panoptes has potential clinical application of helping pathologists determine molecular subtypes and mutations of endometrial carcinoma without sequencing. Significance Recently, molecular subtyping and mutation status are increasingly utilized in clinical practice as they offer better-informed prognosis and the possibility of individualized therapies for endometrial carcinoma patients. Taking advantage of the multi-resolution nature of the whole slide digital histopathology images, our Panoptes models integrate features of different magnification and make accurate predictions of histological subtypes, molecular subtypes, and key mutations in much faster workflows compared to conventional sequencing-based analyses. Feature extraction and visualization revealed that the model relied on human-interpretable patterns. Overall, our multi-resolution deep learning model is capable of assisting pathologists determine molecular subtypes of endometrial carcinoma, which can potentially accelerate diagnosis process.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
An increase in performance might be seen when using multiresolution models, as has been shown with promising results for endometrial carcinoma subtype classification and tissue segmentation in breast cancer.
“An increase in performance might also be seen when using multiresolution models, as has been shown with promising results for endometrial carcinoma subtype classification (Hong et al, 2021) and tissue segmentation in breast cancer (Van Rijthoven et al, 2021).”
From Deep Learning–Based Classification of Early-Stage Mycosis Fungoides and Benign Inflammatory Dermatoses on H&E-Stained Whole-Slide Images: A Retrospective, Proof-of-Concept Study (2025) · cites this paper as Hong et al, 2021
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Cerami, E., Gao, J., Dogrusoz, U., Gross, B. E., Sumer, S. O., Aksoy, B. A., Jacobsen, A., Byrne, C. J., Heuer, M. L., Larsson, E., et al. (2012) · Cancer Discovery · doi:10.1158/2159-8290.cd-12-0095 · Cited by 17,455 · see citing works
Siegel, R. L., Miller, K. D., & Jemal, A. (2016) · CA: A Cancer Journal for Clinicians · doi:10.3322/caac.21332 · Cited by 16,003 · see citing works
Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning
Szegedy, C., Ioffe, S., Vanhoucke, V., & Alemi, A. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.11231 · Cited by 4,636 · see citing works
Kather, J. N., Pearson, A. T., Halama, N., Jäger, D., Krause, J., Loosen, S. H., Marx, A., Boor, P., Tacke, F., Neumann, U. P., et al. (2019) · Nature Medicine · doi:10.1038/s41591-019-0462-y · Cited by 1,458 · see citing works
Image analysis and machine learning in digital pathology: challenges and opportunities
Madabhushi, A., & Lee, G. (2016) · Medical Image Analysis · doi:10.1016/j.media.2016.06.037 · Cited by 1,093 · see citing works
Machine Learning Methods for Histopathological Image Analysis
Komura, D., & Ishikawa, S. (2018) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2018.01.001 · Cited by 937 · see citing works
Proteogenomic Characterization Reveals Therapeutic Vulnerabilities in Lung Adenocarcinoma
Gillette, M. A., Satpathy, S., Cao, S., Dhanasekaran, S. M., Vasaikar, S. V., Krug, K., Petralia, F., Li, Y., Liang, W.-W., Reva, B., et al. (2020) · Cell · doi:10.1016/j.cell.2020.06.013 · Cited by 820 · see citing works
Self-citationProteogenomic and metabolomic characterization of human glioblastoma
Wang, L.-B., Karpova, A., Gritsenko, M. A., Kyle, J. E., Cao, S., Li, Y., Rykunov, D., Colaprico, A., Rothstein, J. H., Hong, R., et al. (2021) · Cancer Cell · doi:10.1016/j.ccell.2021.01.006 · Cited by 661 · see citing works
Self-citationProteogenomic Characterization of Endometrial Carcinoma
Dou, Y., Kawaler, E. A., Cui Zhou, D., Gritsenko, M. A., Huang, C., Blumenberg, L., Karpova, A., Petyuk, V. A., Savage, S. R., Satpathy, S., et al. (2020) · Cell · doi:10.1016/j.cell.2020.01.026 · Cited by 485 · see citing works
Self-citationMurali, R., Davidson, B., Fadare, O., Carlson, J. A., Crum, C. P., Gilks, C. B., Irving, J. A., Malpica, A., Matias-Guiu, X., McCluggage, W. G., et al. (2019) · International Journal of Gynecological Pathology · doi:10.1097/pgp.0000000000000491 · Cited by 278 · see citing works
Flagged first, then by citations
Research works printed but not linked: 26
- [1]Kandoth C. Schultz N. Cherniack A.D. Akbani R. Liu Y. Shen H. Robertson A.G. Pashtan I. Shen R. Benz C.C. Cancer Genome Atlas Research Network Integrated genomic characterization of endometrial carcinoma Nature 497 2013 67 73 23636398 PMC3704730
- [3]Amant F. Moerman P. Neven P. Timmerman D. Van Limbergen E. Vergote I. Endometrial cancer Lancet 366 2005 491 505 16084259
- [4]Morice P. Leary A. Creutzberg C. Abu-Rustum N. Darai E. Endometrial cancer Lancet 387 2016 1094 1108 26354523
- [5]Burke W.M. Orr J. Leitao M. Salom E. Gehrig P. Olawaiye A.B. Brewer M. Boruta D. Villella J. Herzog T. Abu Shahin F. SGO Clinical Practice Endometrial Cancer Working Group Society of Gynecologic Oncology Clinical Practice Committee Endometrial cancer: a review and current management strategies: part I Gynecol. Oncol. 134 2014 385 392 24905773
- [6]Burke W.M. Orr J. Leitao M. Salom E. Gehrig P. Olawaiye A.B. Brewer M. Boruta D. Herzog T.J. Shahin F.A. SGO Clinical Practice Endometrial Cancer Working Group Society of Gynecologic Oncology Clinical Practice Committee Endometrial cancer: a review and current management strategies: part II Gynecol. Oncol. 134 2014 393 402 24929052
- [8]Bokhman J.V. Two pathogenetic types of endometrial carcinoma Gynecol. Oncol. 15 1983 10 17 6822361
- [9]Murali R. Soslow R.A. Weigelt B. Classification of endometrial carcinoma: more than two types Lancet Oncol. 15 2014 e268 e278 24872110
- [10]Frumovitz M. Singh D.K. Meyer L. Smith D.H. Wertheim I. Resnik E. Bodurka D.C. Predictors of final histology in patients with endometrial cancer Gynecol. Oncol. 95 2004 463 468 15581947
- [11]Darvishian F. Hummer A.J. Thaler H.T. Bhargava R. Linkov I. Asher M. Soslow R.A. Serous endometrial cancers that mimic endometrioid adenocarcinomas: a clinicopathologic and immunohistochemical study of a group of problematic cases Am. J. Surg. Pathol. 28 2004 1568 1578 15577675
- [12]Murray S.K. Young R.H. Scully R.E. Uterine Endometrioid Carcinoma with Small Nonvillous Papillae: An Analysis of 26 Cases of a Favorable-Prognosis Tumor To Be Distinguished from Serous Carcinoma Int. J. Surg. Pathol. 8 2000 279 289 11494005
- [14]Brinton L.A. Felix A.S. McMeekin D.S. Creasman W.T. Sherman M.E. Mutch D. Cohn D.E. Walker J.L. Moore R.G. Downs L.S. Etiologic heterogeneity in endometrial cancer: evidence from a Gynecologic Oncology Group trial Gynecol. Oncol. 129 2013 277 284 23485770 PMC4006113
- [15]Zannoni G.F. Vellone V.G. Arena V. Prisco M.G. Scambia G. Carbone A. Gallo D. Does high-grade endometrioid carcinoma (grade 3 FIGO) belong to type I or type II endometrial cancer? A clinical-pathological and immunohistochemical study Virchows Arch. 457 2010 27 34 20552366
- [16]Bell D.W. Novel genetic targets in endometrial cancer Expert Opin. Ther. Targets 18 2014 725 730 24750045 PMC4128487
- [19]Louis D.N. Feldman M. Carter A.B. Dighe A.S. Pfeifer J.D. Bry L. Almeida J.S. Saltz J. Braun J. Tomaszewski J.E. Computational Pathology: A Path Ahead Arch. Pathol. Lab. Med. 140 2016 41 50 26098131 PMC4996078
- [21]Nawaz S. Yuan Y. Computational pathology: exploring the spatial dimension of tumor ecology Cancer Lett. 380 2016 296 303 26592351
- [22]Cooper L.A. Demicco E.G. Saltz J.H. Powell R.T. Rao A. Lazar A.J. PanCancer insights from The Cancer Genome Atlas: the pathologist’s perspective J. Pathol. 244 2018 512 524 29288495 PMC6240356
- [23]Coudray N. Ocampo P.S. Sakellaropoulos T. Narula N. Snuderl M. Fenyö D. Moreira A.L. Razavian N. Tsirigos A. Classification and mutation prediction from non-small cell lung cancer histopathology images using deep learning Nat. Med. 24 2018 1559 1567 30224757 PMC9847512
- [27]Kim R.H. Nomikou S. Dawood Z. Jour G. Donnelly D. Moran U. Weber J.S. Razavian N. Snuderl M. Shapiro R. A Deep Learning Approach for Rapid Mutational Screening in Melanoma bioRxiv 2019 10.1101/610311 · DOI
- [28]Hong R. Liu W. Fenyö D. Predicting and Visualizing STK11 Mutation in Lung Adenocarcinoma Histopathology Slides Using Deep Learning bioRxiv 2020 10.1101/2020.02.20.956557 · DOI
- [29]Kather J.N. Heij L.R. Grabsch H.I. Loeffler C. Echle A. Muti H.S. Krause J. Niehues J.M. Sommer K.A.J. Bankhead P. Pan-cancer image-based detection of clinically actionable genetic alterations Nat. Cancer 1 2020 789 799 33763651 PMC7610412
- [30]Fu Y. Jung A.W. Torne R.V. Gonzalez S. Vöhringer H. Shmatko A. Yates L.R. Jimenez-Linan M. Moore L. Gerstung M. Pan-cancer computational histopathology reveals mutations, tumor composition and prognosis Nat. Cancer 1 2020 800 810 35122049
- [32]Shia J. Black D. Hummer A.J. Boyd J. Soslow R.A. Routinely assessed morphological features correlate with microsatellite instability status in endometrial cancer Hum. Pathol. 39 2008 116 125 17949789
- [33]Yamashita H. Nakayama K. Ishikawa M. Nakamura K. Ishibashi T. Sanuki K. Ono R. Sasamori H. Minamoto T. Iida K. Microsatellite instability is a biomarker for immune checkpoint inhibitors in endometrial cancer Oncotarget 9 2017 5652 5664 29464025 PMC5814165
- [35]Gao J. Aksoy B.A. Dogrusoz U. Dresdner G. Gross B. Sumer S.O. Sun Y. Jacobsen A. Sinha R. Larsson E. Integrative analysis of complex cancer genomics and clinical profiles using the cBioPortal Sci. Signal. 6 2013 pl1 23550210 PMC4160307
- [36]Szegedy C. Liu W. Jia Y. Sermanet P. Reed S. Anguelov D. Erhan D. Vanhoucke V. Rabinovich A. Going Deeper With Convolutions arXiv 2015 1409.4842v1 http://arxiv.org/abs/1409.4842v1 · arXiv
- [37]Vahadane A. Peng T. Sethi A. Albarqouni S. Wang L. Baust M. Steiger K. Schlitter A.M. Esposito I. Navab N. Structure-Preserving Color Normalization and Sparse Stain Separation for Histological Images IEEE Trans. Med. Imaging 35 2016 1962 1971 27164577
Cited by
Sorted by citations
A whole-slide foundation model for digital pathology from real-world data
Xu, H., Usuyama, N., Bagga, J., Zhang, S., Rao, R., Naumann, T., Wong, C., Gero, Z., González, J., Gu, Y., et al. (2024) · Nature · doi:10.1038/s41586-024-07441-w · Cited by 921 · see citing works
Artificial intelligence in histopathology: enhancing cancer research and clinical oncology
Shmatko, A., Ghaffari Laleh, N., Gerstung, M., & Kather, J. N. (2022) · Nature Cancer · doi:10.1038/s43018-022-00436-4 · Cited by 446 · see citing works
Doeleman, T., Brussee, S., Hondelink, L. M., Westerbeek, D. W. F., Sequeira, A. M., Valkema, P. A., Jansen, P. M., He, J., Vermeer, M. H., Quint, K. D., et al. (2025) · Journal of Investigative Dermatology · doi:10.1016/j.jid.2024.07.036 · Cited by 7 · see citing works
- Authors
- Hong, R., Liu, W., DeLair, D., Razavian, N., & Fenyö, D.
- Affiliations
- New York University (US)
- Corresponding author
- Fenyö, David
- Venue
- Cell Reports Medicine
- Publisher
- Elsevier BV
- Published
- 2021-09
- Volume
- 2
- Issue
- 9
- Pages
- 100400
- ISSN
- 2666-3791 (ISSN-L)
- PMC
- PMC8484685
- OpenAlex
- W3201041614
- Cited by
- 154
- References
- 10
- Citations by year
- 2026: 23 · 2025: 39 · 2024: 32 · 2023: 30 · 2022: 25 · 2021: 4 · 2020: 1
- FWCI
- 8.04
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Cell Reports Medicine (publishedVersion, cc-by)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
- Europe PMC (PubMed Central) (publishedVersion, other-oa)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- National Institutes of Health
- National Cancer Institute (U24CA210972)
- New York University
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
- Health Sciences › Medicine › Obstetrics and Gynecology › Endometrial and Cervical Cancer Treatments
- Keywords
- endometrial cancer, histological subtypes, molecular subtypes, genetic mutations, H&E-stained images, histopathology image analysis, digital pathology
- MeSH terms
- Deep Learning, Algorithms, Female, Humans, ROC Curve, Endometrial Neoplasms, Area Under Curve, Imaging, Three-Dimensional
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
