CoChat

Mapping the landscape of histomorphological cancer phenotypes using self-supervised learning on unannotated pathology slides

Open access
Claudio Quiros, Adalberto, Coudray, Nicolas, Yeaton, Anna, Yang, Xinyu, Liu, Bojing, Le, Hortense, Chiriboga, Luis, Karimkhan, Afreen, Narula, Navneet, Moore, David A., Park, Christopher Y., Pass, Harvey, Moreira, Andre L., Le Quesne, John, Tsirigos, Aristotelis, Yuan, Ke · 2024 · Nature Communications
Cited by 71

The DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.

Abstract

Cancer diagnosis and management depend upon the extraction of complex information from microscopy images by pathologists, which requires time-consuming expert interpretation prone to human bias. Supervised deep learning approaches have proven powerful, but are inherently limited by the cost and quality of annotations used for training. Therefore, we present Histomorphological Phenotype Learning, a self-supervised methodology requiring no labels and operating via the automatic discovery of discriminatory features in image tiles. Tiles are grouped into morphologically similar clusters which constitute an atlas of histomorphological phenotypes (HP-Atlas), revealing trajectories from benign to malignant tissue via inflammatory and reactive phenotypes. These clusters have distinct features which can be identified using orthogonal methods, linking histologic, molecular and clinical phenotypes. Applied to lung cancer, we show that they align closely with patient survival, with histopathologically recognised tumor types and growth patterns, and with transcriptomic measures of immunophenotype. These properties are maintained in a multi-cancer study.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s41467-024-48666-7

Cites

28 of 28 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 28 references

Research works printed but not linked: 67

  1. Moreira, A. L. et al. A grading system for invasive pulmonary adenocarcinoma: A proposal from the international association for the study of lung cancer pathology committee. J. Thorac. Oncol. 15, 1599–1610 (2020). · DOI · Paper page
  2. Almendro, V., Marusyk, A. & Polyak, K. Cellular heterogeneity and molecular evolution in cancer. Annu. Rev. Pathol. Mech. Dis. 8, 277–302 (2013). · DOI · Paper page
  3. de Sousa, V. M. L. & Carvalho, L. Heterogeneity in lung cancer. Pathobiology 85, 96–107 (2018). · DOI · Paper page
  4. Andrion, A. et al. Malignant mesothelioma of the pleura: interobserver variability. J. Clin. Pathol. 48, 856–860 (1995). · DOI
  5. Kujan, O. et al. Why oral histopathology suffers inter-observer variability on grading oral epithelial dysplasia: an attempt to understand the sources of variation. Oral. Oncol. 43, 224–231 (2007). · DOI · Paper page
  6. Warth, A. et al. Interobserver variability in the application of the novel iaslc/ats/ers classification for pulmonary adenocarcinomas. Eur. Respirat. J. 40, 1221–1227 (2012). · DOI
  7. Ozkan, T. A. et al. Interobserver variability in gleason histological grading of prostate cancer. Scand. J. Urol. 50, 420–424 (2016). · DOI · Paper page
  8. Kather, J. N. et al. Pan-cancer image-based detection of clinically actionable genetic alterations. Nat. Cancer 1, 789–799 (2020). · DOI · Paper page
  9. Fu, Y. et al. Pan-cancer computational histopathology reveals mutations, tumor composition and prognosis. Nat. Cancer 1, 800–810 (2020). · DOI · Paper page
  10. Johannet, P. et al. Using machine learning algorithms to predict immunotherapy response in patients with advanced melanoma. Clin. Cancer Res. 27, 131–140 (2021). · DOI · Paper page
  11. Burkart, N. & Huber, M. F. A survey on the explainability of supervised machine learning. J. Artif. Intell. Res. 70, 245–317 (2021). · DOI · Paper page
  12. Tarvainen, A. & Valpola, H. Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. (eds uyon, I., Von Luxburg, U., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S. & Garnett, R.) In Advances in Neural Information Processing Systems 30 (NIPS, 2017).
  13. Miyato, T., Maeda, S.-i, Koyama, M. & Ishii, S. Virtual adversarial training: a regularization method for supervised and semi-supervised learning. IEEE Trans. Pattern Anal. Mach. Intell. 41, 1979–1993 (2018). · DOI · Paper page
  14. Yalniz, I. Z., Jégou, H., Chen, K., Paluri, M. & Mahajan, D. Billion-scale semi-supervised learning for image classification. arXiv preprint arXiv:1905.00546 (2019). · arXiv
  15. Xu, Y. et al. Weakly supervised histopathology cancer image segmentation and classification. Med. image Anal. 18, 591–604 (2014). · DOI · Paper page
  16. Koohbanani, N. A., Unnikrishnan, B., Khurram, S. A., Krishnaswamy, P. & Rajpoot, N. Self-path: Self-supervision for classification of pathology images with limited annotations. IEEE Trans. Med. Imaging (2021). · DOI · Paper page
  17. Yu, G.et al. Accurate recognition of colorectal cancer with semi-supervised deep learning on pathological images. bioRxiv 2020–07 (2021). · DOI
  18. Sun, C. et al. Deep learning-based classification of liver cancer histopathology images using only global labels. IEEE J. Biomed. Health Inform. 24, 1643–1651 (2020). · DOI
  19. Ilse, M., Tomczak, J. & Welling, M. Attention-based deep multiple instance learning. In Dy, J. & Krause, A. (eds.) Proceedings of the 35th International Conference on Machine Learning, vol. 80 of Proceedings of Machine Learning Research, 2127–2136 (2018).
  20. Yao, J., Zhu, X., Jonnagaddala, J., Hawkins, N. & Huang, J. Whole slide images based cancer survival prediction using attention guided deep multiple instance learning networks. Med. Image Anal. 65, 101789 (2020). · DOI · Paper page
  21. Dehaene, O., Camara, A., Moindrot, O., de Lavergne, A. & Courtiol, P. Self-supervision closes the gap between weak and strong supervision in histology. arXiv preprint at arXiv:2012.03583 (2020). · arXiv
  22. Chen, Z., Li, X., Yang, M., Zhang, H. & Xu, X. S. Optimize deep learning models for prediction of gene mutations using unsupervised clustering. J Pathol Clin Res 9, 3–17 (2023). · DOI
  23. Quiros, A. C. et al. Adversarial learning of cancer tissue representations. In International Conference on Medical Image Computing and Computer-Assisted Intervention (Springer, 2021).
  24. Kim, J. et al. Unsupervised discovery of tissue architecture in multiplexed imaging. Nat. Methods 1–9 (2022). · DOI
  25. Lafarge, M. W., Pluim, J. P. & Veta, M. Orientation-disentangled unsupervised representation learning for computational pathology. arXiv preprint arXiv:2008.11673 (2020). · arXiv
  26. Sahasrabudhe, M. et al. Self-supervised nuclei segmentation in histopathological images using attention. In International Conference on Medical Image Computing and Computer-Assisted Intervention, 393–402 (Springer, 2020). · DOI
  27. Reinhard, E., Adhikhmin, M., Gooch, B. & Shirley, P. Color transfer between images. IEEE Comput. Graph. Appl. 21, 34–41 (2001). · DOI · Paper page
  28. Zbontar, J., Jing, L., Misra, I., LeCun, Y. & Deny, S. Barlow twins: Self-supervised learning via redundancy reduction. arXiv preprint at arXiv:2103.03230 (2021). · arXiv
  29. Chen, T., Kornblith, S., Norouzi, M. & Hinton, G. A simple framework for contrastive learning of visual representations. In III, H. D. & Singh, A. (eds.) Proceedings of the 37th International Conference on Machine Learning, vol. 119 of Proceedings of Machine Learning Research, 1597–1607 (PMLR, 2020). http://proceedings.mlr.press/v119/chen20j.html.
  30. He, K., Fan, H., Wu, Y., Xie, S. & Girshick, R. Momentum contrast for unsupervised visual representation learning. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (2020). · DOI · Paper page
  31. Gueréndel, C., Arnold, P. & Torben-Nielsen, B. Creating small but meaningful representations of digital pathology images. In MICCAI Workshop on Computational Pathology, 206–215 (PMLR, 2021).
  32. Thunnissen, E. et al. Reproducibility of histopathological subtypes and invasion in pulmonary adenocarcinoma. an international interobserver study. Mod. Pathol. 25, 1574–1583 (2012). · DOI · Paper page
  33. Rokutan-Kurata, M. et al. Discohesive growth pattern (disco-p) as an unfavorable prognostic factor in lung adenocarcinoma: an analysis of 1062 japanese patients with resected lung adenocarcinoma. Mod. Pathol. 33, 1722–1731 (2020). · doi:10.1038/s41379-020-0537-9
  34. Graham, S. et al. Hover-net: Simultaneous segmentation and classification of nuclei in multi-tissue histology images. Med. Image Anal. 58, 101563 (2019). · DOI · Paper page
  35. Toccaceli, P. & Gammerman, A. Combination of inductive mondrian conformal predictors. Mach. Learn. 108, 489–510 (2019). · DOI
  36. Yi, M. et al. TGF-β: A novel predictor and target for anti-PD-1/PD-L1 therapy. Front. Immunol. 13, 1061394 (2022). · DOI
  37. Katsuno, Y. & Derynck, R. Epithelial plasticity, epithelial-mesenchymal transition, and the TGF-β family. Dev. Cell 56, 726–746 (2021). · DOI · Paper page
  38. Gonzalez-Ericsson, P. I. et al. The path to a better biomarker: application of a risk management framework for the implementation of pd-l1 and tils as immuno-oncology biomarkers in breast cancer clinical trials and daily practice. J. Pathol. 250, 667–684 (2020). · DOI · Paper page
  39. Hou, W., Yi, C. & Zhu, H. Predictive biomarkers of colon cancer immunotherapy: Present and future. Frontiers in Immunology13 (2022). · DOI
  40. Hendry, S. et al. Assessing tumor infiltrating lymphocytes in solid tumors: a practical review for pathologists and proposal for a standardized method from the international immuno-oncology biomarkers working group: Part 2: Tils in melanoma, gastrointestinal tract carcinomas, non-small cell lung carcinoma and mesothelioma, endometrial and ovarian carcinomas, squamous cell carcinoma of the head and neck, genitourinary carcinomas, and primary brain tumors. Adv. Anat. Pathol. 24, 311 (2017). · DOI · Paper page
  41. AbdulJabbar, K. et al. Geospatial immune variability illuminates differential evolution of lung adenocarcinoma. Nat. Med. 26, 1054–1062 (2020). · DOI · Paper page
  42. Luo, X. et al. Comprehensive computational pathological image analysis predicts lung cancer prognosis. J. Thorac. Oncol. 12, 501–509 (2017). · DOI · Paper page
  43. Lu, C. et al. A prognostic model for overall survival of patients with early-stage non-small cell lung cancer: a multicentre, retrospective study. Lancet Digit. Health 2, e594–e606 (2020). · DOI
  44. Wang, S. et al. Comprehensive analysis of lung cancer pathology images to discover tumor shape and boundary features that predict survival outcome. Sci. Rep. 8, 10393 (2018). · DOI · Paper page
  45. Sun, Z. et al. Histologic grade is an independent prognostic factor for survival in non-small cell lung cancer: an analysis of 5018 hospital- and 712 population-based cases. J. Thorac. Cardiovasc. Surg. 131, 1014–1020 (2006). · DOI
  46. Sica, G. et al. A grading system of lung adenocarcinomas based on histologic pattern is predictive of disease recurrence in stage i tumors. Am. J. Surg. Pathol. 34 https://journals.lww.com/ajsp/Fulltext/2010/08000/A_Grading_System_of_Lung_Adenocarcinomas_Based_on.9.aspx (2010). · DOI · Paper page
  47. Oiwa, H. et al. Clinicopathological, gene expression and genetic features of stage I lung adenocarcinoma with necrosis. Lung Cancer 159, 74–83 (2021). · DOI
  48. Schäfer, M. & Werner, S. Cancer as an overhealing wound: an old hypothesis revisited. Nat. Rev. Mol. Cell Biol. 9, 628–638 (2008). · DOI · Paper page
  49. Rybinski, B., Franco-Barraza, J. & Cukierman, E. The wound healing, chronic fibrosis, and cancer progression triad. Physiol. Genom. 46, 223–244 (2014). · DOI · Paper page
  50. Chen, C.-L. et al. An annotation-free whole-slide training approach to pathological classification of lung cancer types using deep learning. Nat. Commun. 12, 1193 (2021). · DOI · Paper page
  51. Xu, X. et al. A weak supervision-based framework for automatic lung cancer classification on whole slide image. In 2020 42nd Annual International Conference of the IEEE Engineering in Medicine Biology Society (EMBC), 1372–1375 (2020). · DOI
  52. Hui, D. et al. Prognostication in advanced cancer: update and directions for future research. Support. Care Cancer 27, 1973–1984 (2019). · DOI · Paper page
  53. Hui, D., Mo, L. & Paiva, C. E. The importance of prognostication: impact of prognostic predictions, disclosures, awareness, and acceptance on patient outcomes. Curr. Treat. options Oncol. 22, 1–14 (2021). · DOI
  54. Amano, K. et al. The accuracy of physicians’ clinical predictions of survival in patients with advanced cancer. J. Pain. Sympt. Manag. 50, 139–146 (2015). · DOI
  55. Dosovitskiy, A., Springenberg, J. T., Riedmiller, M. & Brox, T. Discriminative unsupervised feature learning with convolutional neural networks. In Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N. & Weinberger, K. Q. (eds.) Advances in Neural Information Processing Systems, vol. 27 (Curran Associates, Inc., 2014). https://proceedings.neurips.cc/paper/2014/file/07563a3fe3bbe7e3ba84431ad9d055af-Paper.pdf.
  56. Patacchiola, M. & Storkey, A. J. Self-supervised relational reasoning for representation learning. In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M. F. & Lin, H. (eds.) Advances in Neural Information Processing Systems, vol. 33, 4003–4014 (Curran Associates, Inc., 2020). https://proceedings.neurips.cc/paper/2020/file/29539ed932d32f1c56324cded92c07c2-Paper.pdf.
  57. Zhang, H., Goodfellow, I., Metaxas, D. & Odena, A. Self-attention generative adversarial networks. In Chaudhuri, K. & Salakhutdinov, R. (eds.) Proceedings of the 36th International Conference on Machine Learning, vol. 97 of Proceedings of Machine Learning Research, 7354–7363 (PMLR, 2019).
  58. Miyato, T., Kataoka, T., Koyama, M. & Yoshida, Y. Spectral normalization for generative adversarial networks. arXiv preprint arXiv:1802.05957 (2018). · arXiv
  59. Blondel, V. D., Guillaume, J.-L., Lambiotte, R. & Lefebvre, E. Fast unfolding of communities in large networks. J. Stat. Mech. Theory Exp. 2008, P10008 (2008). · DOI · Paper page
  60. McGregor, D., Palarea-Albaladejo, J., Dall, P., Hron, K. & Chastin, S. Cox regression survival analysis with compositional covariates: Application to modelling mortality risk from 24-h physical activity patterns. Stat. Methods Med. Res. 29, 1447–1465 (2020). · DOI
  61. Martín-Fernández, J. A., Barceló-Vidal, C. & Pawlowsky-Glahn, V. Dealing with zeros and missing values in compositional data sets using nonparametric imputation. Math. Geol. 35, 253–278 (2003). · DOI · Paper page
  62. Cule, E., Vineis, P. & De Iorio, M. Significance testing in ridge regression for genetic data. BMC Bioinforma. 12, 372 (2011). · DOI · Paper page
  63. Goeman, J. J. L1 penalized estimation in the cox proportional hazards model. Biom. J. 52, 70–84 (2010). · DOI · Paper page
  64. Lockhart, R., Taylor, J., Tibshirani, R. J. & Tibshirani, R. A significance test for the lasso. Ann. Stat. 42 https://doi.org/10.1214/2F13-aos1175 (2014) · doi:10.1214/2f13-aos1175
  65. Davidson-Pilon, C. lifelines: survival analysis in python. J. Open Source Softw. 4, 1317 (2019). · DOI · Paper page
  66. Virtanen, P. et al. SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python. Nat. Methods 17, 261–272 (2020). · DOI · Paper page
  67. Seabold, S. & Perktold, J. statsmodels: Econometric and statistical modeling with python. In 9th Python in Science Conference (2010). · DOI · Paper page

Cited by

See all 71

Metadata

Authors
Claudio Quiros, A., Coudray, N., Yeaton, A., Yang, X., Liu, B., Le, H., Chiriboga, L., Karimkhan, A., Narula, N., Moore, D. A., Park, C. Y., Pass, H., Moreira, A. L., Le Quesne, J., Tsirigos, A., & Yuan, K.
Venue
Nature Communications
Published
2024-06-11
Volume
15
Issue
1
Pages
4596-4596
ISSN
2041-1723 (ISSN-L)
OpenAlex
W4399544575
Cited by
71
References
28
FWCI
10.57
Type
journal article
Language
en
Open access
gold
Licence

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Mapping the landscape of histomorphological cancer phenotypes using self-supervised learning on unannotated pathology slides. https://app.cochat.ai/papers/10.1038/s41467-024-48666-7
The paper
Claudio Quiros, A., Coudray, N., Yeaton, A., Yang, X., Liu, B., Le, H., Chiriboga, L., Karimkhan, A., Narula, N., Moore, D. A., Park, C. Y., Pass, H., Moreira, A. L., Le Quesne, J., Tsirigos, A., & Yuan, K. (2024). Mapping the landscape of histomorphological cancer phenotypes using self-supervised learning on unannotated pathology slides. Nature Communications. https://doi.org/10.1038/s41467-024-48666-7

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.