Mapping the landscape of histomorphological cancer phenotypes using self-supervised learning on unannotated pathology slides
Open accessThe DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
Abstract
Cancer diagnosis and management depend upon the extraction of complex information from microscopy images by pathologists, which requires time-consuming expert interpretation prone to human bias. Supervised deep learning approaches have proven powerful, but are inherently limited by the cost and quality of annotations used for training. Therefore, we present Histomorphological Phenotype Learning, a self-supervised methodology requiring no labels and operating via the automatic discovery of discriminatory features in image tiles. Tiles are grouped into morphologically similar clusters which constitute an atlas of histomorphological phenotypes (HP-Atlas), revealing trajectories from benign to malignant tissue via inflammatory and reactive phenotypes. These clusters have distinct features which can be identified using orthogonal methods, linking histologic, molecular and clinical phenotypes. Applied to lung cancer, we show that they align closely with patient survival, with histopathologically recognised tumor types and growth patterns, and with transcriptomic measures of immunophenotype. These properties are maintained in a multi-cancer study.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482 · cited 1×
“P-values were adjusted for multiple comparisons through the Benjamini/Hochberg method for false discovery rate 95 .”
MethodWang, X., Girshick, R., Gupta, A., & He, K. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00813 · Cited by 11,596 · cited 1×
“In our work, we used a CNN composed ofseveral ResNet layers 80 and a self-attention layer 81,82 .”
UMAP: Uniform Manifold Approximation and Projection
McInnes, L., Healy, J., Saul, N., & Großberger, L. (2018) · Journal of Open Source Software · doi:10.21105/joss.00861 · Cited by 9,826 · cited 1×
“Next, we performed dimensionality reduction on the remaining ~411,000 tile representation vectors using the Uniform Manifold Approximation and Projection (UMAP) method 51 (Fig.”
MethodSCANPY: large-scale single-cell gene expression data analysis
Wolf, F. A., Angerer, P., & Theis, F. J. (2018) · Genome Biology · doi:10.1186/s13059-017-1382-0 · Cited by 8,881
The Immune Landscape of Cancer
Thorsson, V., Gibbs, D. L., Brown, S. D., Wolf, D., Bortone, D. S., Ou Yang, T.-H., Porta-Pardo, E., Gao, G. F., Plaisier, C. L., Eddy, J. A., et al. (2018) · Immunity · doi:10.1016/j.immuni.2018.03.023 · Cited by 5,870
Self-citationFrom Louvain to Leiden: guaranteeing well-connected communities
Traag, V. A., Waltman, L., & van Eck, N. J. (2019) · Scientific Reports · doi:10.1038/s41598-019-41695-z · Cited by 5,635 · cited 3×
“Leiden community detection 48 has been shown to be a successful algorithm for discovering well-connected communities in graph structures.”
MethodInfluentialBootstrap Your Own Latent: A New Approach to Self-Supervised Learning
Grill, J.-B., Strub, F., Altché, F., Tallec, C., Richemond, P. H., Buchatskaya, E., Doersch, C., Pires, B. A., Guo, Z. D., Azar, M. G., et al. (2020) · arXiv · doi:10.48550/arxiv.2006.07733 · Cited by 3,846 · cited 1×
“…this method has proven to be on par or to outperform other self-supervised methods, while eliminating the need for higher computational requirements such as large batches 45 or architecture asymmetries 46,47 , thus providing competitive state-of-the-art results with signi fi cantly fewer resources.”
Coudray, N., Ocampo, P. S., Sakellaropoulos, T., Narula, N., Snuderl, M., Fenyö, D., Moreira, A. L., Razavian, N., & Tsirigos, A. (2018) · Nature Medicine · doi:10.1038/s41591-018-0177-5 · Cited by 2,930
Self-citationClinical-grade computational pathology using weakly supervised deep learning on whole slide images
Campanella, G., Hanna, M. G., Geneslaw, L., Miraflor, A., Werneck Krauss Silva, V., Busam, K. J., Brogi, E., Reuter, V. E., Klimstra, D. S., & Fuchs, T. J. (2019) · Nature Medicine · doi:10.1038/s41591-019-0508-1 · Cited by 2,799
Research works printed but not linked: 67
- Moreira, A. L. et al. A grading system for invasive pulmonary adenocarcinoma: A proposal from the international association for the study of lung cancer pathology committee. J. Thorac. Oncol. 15, 1599–1610 (2020). · DOI · Paper page
- Almendro, V., Marusyk, A. & Polyak, K. Cellular heterogeneity and molecular evolution in cancer. Annu. Rev. Pathol. Mech. Dis. 8, 277–302 (2013). · DOI · Paper page
- de Sousa, V. M. L. & Carvalho, L. Heterogeneity in lung cancer. Pathobiology 85, 96–107 (2018). · DOI · Paper page
- Andrion, A. et al. Malignant mesothelioma of the pleura: interobserver variability. J. Clin. Pathol. 48, 856–860 (1995). · DOI
- Kujan, O. et al. Why oral histopathology suffers inter-observer variability on grading oral epithelial dysplasia: an attempt to understand the sources of variation. Oral. Oncol. 43, 224–231 (2007). · DOI · Paper page
- Warth, A. et al. Interobserver variability in the application of the novel iaslc/ats/ers classification for pulmonary adenocarcinomas. Eur. Respirat. J. 40, 1221–1227 (2012). · DOI
- Ozkan, T. A. et al. Interobserver variability in gleason histological grading of prostate cancer. Scand. J. Urol. 50, 420–424 (2016). · DOI · Paper page
- Kather, J. N. et al. Pan-cancer image-based detection of clinically actionable genetic alterations. Nat. Cancer 1, 789–799 (2020). · DOI · Paper page
- Fu, Y. et al. Pan-cancer computational histopathology reveals mutations, tumor composition and prognosis. Nat. Cancer 1, 800–810 (2020). · DOI · Paper page
- Johannet, P. et al. Using machine learning algorithms to predict immunotherapy response in patients with advanced melanoma. Clin. Cancer Res. 27, 131–140 (2021). · DOI · Paper page
- Burkart, N. & Huber, M. F. A survey on the explainability of supervised machine learning. J. Artif. Intell. Res. 70, 245–317 (2021). · DOI · Paper page
- Tarvainen, A. & Valpola, H. Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. (eds uyon, I., Von Luxburg, U., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S. & Garnett, R.) In Advances in Neural Information Processing Systems 30 (NIPS, 2017).
- Miyato, T., Maeda, S.-i, Koyama, M. & Ishii, S. Virtual adversarial training: a regularization method for supervised and semi-supervised learning. IEEE Trans. Pattern Anal. Mach. Intell. 41, 1979–1993 (2018). · DOI · Paper page
- Yalniz, I. Z., Jégou, H., Chen, K., Paluri, M. & Mahajan, D. Billion-scale semi-supervised learning for image classification. arXiv preprint arXiv:1905.00546 (2019). · arXiv
- Xu, Y. et al. Weakly supervised histopathology cancer image segmentation and classification. Med. image Anal. 18, 591–604 (2014). · DOI · Paper page
- Koohbanani, N. A., Unnikrishnan, B., Khurram, S. A., Krishnaswamy, P. & Rajpoot, N. Self-path: Self-supervision for classification of pathology images with limited annotations. IEEE Trans. Med. Imaging (2021). · DOI · Paper page
- Yu, G.et al. Accurate recognition of colorectal cancer with semi-supervised deep learning on pathological images. bioRxiv 2020–07 (2021). · DOI
- Sun, C. et al. Deep learning-based classification of liver cancer histopathology images using only global labels. IEEE J. Biomed. Health Inform. 24, 1643–1651 (2020). · DOI
- Ilse, M., Tomczak, J. & Welling, M. Attention-based deep multiple instance learning. In Dy, J. & Krause, A. (eds.) Proceedings of the 35th International Conference on Machine Learning, vol. 80 of Proceedings of Machine Learning Research, 2127–2136 (2018).
- Yao, J., Zhu, X., Jonnagaddala, J., Hawkins, N. & Huang, J. Whole slide images based cancer survival prediction using attention guided deep multiple instance learning networks. Med. Image Anal. 65, 101789 (2020). · DOI · Paper page
- Dehaene, O., Camara, A., Moindrot, O., de Lavergne, A. & Courtiol, P. Self-supervision closes the gap between weak and strong supervision in histology. arXiv preprint at arXiv:2012.03583 (2020). · arXiv
- Chen, Z., Li, X., Yang, M., Zhang, H. & Xu, X. S. Optimize deep learning models for prediction of gene mutations using unsupervised clustering. J Pathol Clin Res 9, 3–17 (2023). · DOI
- Quiros, A. C. et al. Adversarial learning of cancer tissue representations. In International Conference on Medical Image Computing and Computer-Assisted Intervention (Springer, 2021).
- Kim, J. et al. Unsupervised discovery of tissue architecture in multiplexed imaging. Nat. Methods 1–9 (2022). · DOI
- Lafarge, M. W., Pluim, J. P. & Veta, M. Orientation-disentangled unsupervised representation learning for computational pathology. arXiv preprint arXiv:2008.11673 (2020). · arXiv
- Sahasrabudhe, M. et al. Self-supervised nuclei segmentation in histopathological images using attention. In International Conference on Medical Image Computing and Computer-Assisted Intervention, 393–402 (Springer, 2020). · DOI
- Reinhard, E., Adhikhmin, M., Gooch, B. & Shirley, P. Color transfer between images. IEEE Comput. Graph. Appl. 21, 34–41 (2001). · DOI · Paper page
- Zbontar, J., Jing, L., Misra, I., LeCun, Y. & Deny, S. Barlow twins: Self-supervised learning via redundancy reduction. arXiv preprint at arXiv:2103.03230 (2021). · arXiv
- Chen, T., Kornblith, S., Norouzi, M. & Hinton, G. A simple framework for contrastive learning of visual representations. In III, H. D. & Singh, A. (eds.) Proceedings of the 37th International Conference on Machine Learning, vol. 119 of Proceedings of Machine Learning Research, 1597–1607 (PMLR, 2020). http://proceedings.mlr.press/v119/chen20j.html.
- He, K., Fan, H., Wu, Y., Xie, S. & Girshick, R. Momentum contrast for unsupervised visual representation learning. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (2020). · DOI · Paper page
- Gueréndel, C., Arnold, P. & Torben-Nielsen, B. Creating small but meaningful representations of digital pathology images. In MICCAI Workshop on Computational Pathology, 206–215 (PMLR, 2021).
- Thunnissen, E. et al. Reproducibility of histopathological subtypes and invasion in pulmonary adenocarcinoma. an international interobserver study. Mod. Pathol. 25, 1574–1583 (2012). · DOI · Paper page
- Rokutan-Kurata, M. et al. Discohesive growth pattern (disco-p) as an unfavorable prognostic factor in lung adenocarcinoma: an analysis of 1062 japanese patients with resected lung adenocarcinoma. Mod. Pathol. 33, 1722–1731 (2020). · doi:10.1038/s41379-020-0537-9
- Graham, S. et al. Hover-net: Simultaneous segmentation and classification of nuclei in multi-tissue histology images. Med. Image Anal. 58, 101563 (2019). · DOI · Paper page
- Toccaceli, P. & Gammerman, A. Combination of inductive mondrian conformal predictors. Mach. Learn. 108, 489–510 (2019). · DOI
- Yi, M. et al. TGF-β: A novel predictor and target for anti-PD-1/PD-L1 therapy. Front. Immunol. 13, 1061394 (2022). · DOI
- Katsuno, Y. & Derynck, R. Epithelial plasticity, epithelial-mesenchymal transition, and the TGF-β family. Dev. Cell 56, 726–746 (2021). · DOI · Paper page
- Gonzalez-Ericsson, P. I. et al. The path to a better biomarker: application of a risk management framework for the implementation of pd-l1 and tils as immuno-oncology biomarkers in breast cancer clinical trials and daily practice. J. Pathol. 250, 667–684 (2020). · DOI · Paper page
- Hou, W., Yi, C. & Zhu, H. Predictive biomarkers of colon cancer immunotherapy: Present and future. Frontiers in Immunology13 (2022). · DOI
- Hendry, S. et al. Assessing tumor infiltrating lymphocytes in solid tumors: a practical review for pathologists and proposal for a standardized method from the international immuno-oncology biomarkers working group: Part 2: Tils in melanoma, gastrointestinal tract carcinomas, non-small cell lung carcinoma and mesothelioma, endometrial and ovarian carcinomas, squamous cell carcinoma of the head and neck, genitourinary carcinomas, and primary brain tumors. Adv. Anat. Pathol. 24, 311 (2017). · DOI · Paper page
- AbdulJabbar, K. et al. Geospatial immune variability illuminates differential evolution of lung adenocarcinoma. Nat. Med. 26, 1054–1062 (2020). · DOI · Paper page
- Luo, X. et al. Comprehensive computational pathological image analysis predicts lung cancer prognosis. J. Thorac. Oncol. 12, 501–509 (2017). · DOI · Paper page
- Lu, C. et al. A prognostic model for overall survival of patients with early-stage non-small cell lung cancer: a multicentre, retrospective study. Lancet Digit. Health 2, e594–e606 (2020). · DOI
- Wang, S. et al. Comprehensive analysis of lung cancer pathology images to discover tumor shape and boundary features that predict survival outcome. Sci. Rep. 8, 10393 (2018). · DOI · Paper page
- Sun, Z. et al. Histologic grade is an independent prognostic factor for survival in non-small cell lung cancer: an analysis of 5018 hospital- and 712 population-based cases. J. Thorac. Cardiovasc. Surg. 131, 1014–1020 (2006). · DOI
- Sica, G. et al. A grading system of lung adenocarcinomas based on histologic pattern is predictive of disease recurrence in stage i tumors. Am. J. Surg. Pathol. 34 https://journals.lww.com/ajsp/Fulltext/2010/08000/A_Grading_System_of_Lung_Adenocarcinomas_Based_on.9.aspx (2010). · DOI · Paper page
- Oiwa, H. et al. Clinicopathological, gene expression and genetic features of stage I lung adenocarcinoma with necrosis. Lung Cancer 159, 74–83 (2021). · DOI
- Schäfer, M. & Werner, S. Cancer as an overhealing wound: an old hypothesis revisited. Nat. Rev. Mol. Cell Biol. 9, 628–638 (2008). · DOI · Paper page
- Rybinski, B., Franco-Barraza, J. & Cukierman, E. The wound healing, chronic fibrosis, and cancer progression triad. Physiol. Genom. 46, 223–244 (2014). · DOI · Paper page
- Chen, C.-L. et al. An annotation-free whole-slide training approach to pathological classification of lung cancer types using deep learning. Nat. Commun. 12, 1193 (2021). · DOI · Paper page
- Xu, X. et al. A weak supervision-based framework for automatic lung cancer classification on whole slide image. In 2020 42nd Annual International Conference of the IEEE Engineering in Medicine Biology Society (EMBC), 1372–1375 (2020). · DOI
- Hui, D. et al. Prognostication in advanced cancer: update and directions for future research. Support. Care Cancer 27, 1973–1984 (2019). · DOI · Paper page
- Hui, D., Mo, L. & Paiva, C. E. The importance of prognostication: impact of prognostic predictions, disclosures, awareness, and acceptance on patient outcomes. Curr. Treat. options Oncol. 22, 1–14 (2021). · DOI
- Amano, K. et al. The accuracy of physicians’ clinical predictions of survival in patients with advanced cancer. J. Pain. Sympt. Manag. 50, 139–146 (2015). · DOI
- Dosovitskiy, A., Springenberg, J. T., Riedmiller, M. & Brox, T. Discriminative unsupervised feature learning with convolutional neural networks. In Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N. & Weinberger, K. Q. (eds.) Advances in Neural Information Processing Systems, vol. 27 (Curran Associates, Inc., 2014). https://proceedings.neurips.cc/paper/2014/file/07563a3fe3bbe7e3ba84431ad9d055af-Paper.pdf.
- Patacchiola, M. & Storkey, A. J. Self-supervised relational reasoning for representation learning. In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M. F. & Lin, H. (eds.) Advances in Neural Information Processing Systems, vol. 33, 4003–4014 (Curran Associates, Inc., 2020). https://proceedings.neurips.cc/paper/2020/file/29539ed932d32f1c56324cded92c07c2-Paper.pdf.
- Zhang, H., Goodfellow, I., Metaxas, D. & Odena, A. Self-attention generative adversarial networks. In Chaudhuri, K. & Salakhutdinov, R. (eds.) Proceedings of the 36th International Conference on Machine Learning, vol. 97 of Proceedings of Machine Learning Research, 7354–7363 (PMLR, 2019).
- Miyato, T., Kataoka, T., Koyama, M. & Yoshida, Y. Spectral normalization for generative adversarial networks. arXiv preprint arXiv:1802.05957 (2018). · arXiv
- Blondel, V. D., Guillaume, J.-L., Lambiotte, R. & Lefebvre, E. Fast unfolding of communities in large networks. J. Stat. Mech. Theory Exp. 2008, P10008 (2008). · DOI · Paper page
- McGregor, D., Palarea-Albaladejo, J., Dall, P., Hron, K. & Chastin, S. Cox regression survival analysis with compositional covariates: Application to modelling mortality risk from 24-h physical activity patterns. Stat. Methods Med. Res. 29, 1447–1465 (2020). · DOI
- Martín-Fernández, J. A., Barceló-Vidal, C. & Pawlowsky-Glahn, V. Dealing with zeros and missing values in compositional data sets using nonparametric imputation. Math. Geol. 35, 253–278 (2003). · DOI · Paper page
- Cule, E., Vineis, P. & De Iorio, M. Significance testing in ridge regression for genetic data. BMC Bioinforma. 12, 372 (2011). · DOI · Paper page
- Goeman, J. J. L1 penalized estimation in the cox proportional hazards model. Biom. J. 52, 70–84 (2010). · DOI · Paper page
- Lockhart, R., Taylor, J., Tibshirani, R. J. & Tibshirani, R. A significance test for the lasso. Ann. Stat. 42 https://doi.org/10.1214/2F13-aos1175 (2014) · doi:10.1214/2f13-aos1175
- Davidson-Pilon, C. lifelines: survival analysis in python. J. Open Source Softw. 4, 1317 (2019). · DOI · Paper page
- Virtanen, P. et al. SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python. Nat. Methods 17, 261–272 (2020). · DOI · Paper page
- Seabold, S. & Perktold, J. statsmodels: Econometric and statistical modeling with python. In 9th Python in Science Conference (2010). · DOI · Paper page
Cited by
Artificial intelligence in drug development
Zhang, K., Yang, X., Wang, Y., Yu, Y., Huang, N., Li, G., Li, X., Wu, J. C., & Yang, S. (2025) · Nature Medicine · doi:10.1038/s41591-024-03434-4 · Cited by 498
Self-citationMorphological Prototyping for Unsupervised Slide Representation Learning in Computational Pathology
Song, A. H., Chen, R. J., Ding, T., Williamson, D. F. K., Jaume, G., & Mahmood, F. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01099 · Cited by 79
Liu, B., Polack, M., Coudray, N., Claudio Quiros, A., Sakellaropoulos, T., Le, H., Karimkhan, A., Crobach, A. S. L. P., van Krieken, J. H. J. M., Yuan, K., et al. (2025) · Nature Communications · doi:10.1038/s41467-025-57541-y · Cited by 25
Self-citationCurrent and future applications of artificial intelligence in lung cancer and mesothelioma
Roche, J. J., Seyedshahi, F., Rakovic, K., Thu, A. W., Le Quesne, J., & Blyth, K. G. (2025) · Thorax · doi:10.1136/thorax-2024-222054 · Cited by 11
Metadata
- Authors
- Claudio Quiros, A., Coudray, N., Yeaton, A., Yang, X., Liu, B., Le, H., Chiriboga, L., Karimkhan, A., Narula, N., Moore, D. A., Park, C. Y., Pass, H., Moreira, A. L., Le Quesne, J., Tsirigos, A., & Yuan, K.
- Venue
- Nature Communications
- Published
- 2024-06-11
- Volume
- 15
- Issue
- 1
- Pages
- 4596-4596
- ISSN
- 2041-1723 (ISSN-L)
- OpenAlex
- W4399544575
- Cited by
- 71
- References
- 28
- FWCI
- 10.57
- Type
- journal article
- Language
- en
- Open access
- gold
- Licence
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Mapping the landscape of histomorphological cancer phenotypes using self-supervised learning on unannotated pathology slides. https://app.cochat.ai/papers/10.1038/s41467-024-48666-7
- The paper
- Claudio Quiros, A., Coudray, N., Yeaton, A., Yang, X., Liu, B., Le, H., Chiriboga, L., Karimkhan, A., Narula, N., Moore, D. A., Park, C. Y., Pass, H., Moreira, A. L., Le Quesne, J., Tsirigos, A., & Yuan, K. (2024). Mapping the landscape of histomorphological cancer phenotypes using self-supervised learning on unannotated pathology slides. Nature Communications. https://doi.org/10.1038/s41467-024-48666-7
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.