CoChat
Sign Up
Cited by 75AI in cancer detectionCancer Immunotherapy and BiomarkersRadiomics and Machine Learning in Medical Imaging

Using deep learning to predict anti-PD-1 response in melanoma and lung cancer patients from histopathology images

Open access
Hu, Jing, Cui, Chuanliang, Yang, Wenxian, Huang, Lihong, Yu, Rongshan, Liu, Siyang, Kong, Yan·2021·Translational Oncology
DOI 10.1016/j.tranon.2020.100921PMC PMC7595938
Open this paper in CoChat — free accountOpen PDF

Abstract

BACKGROUND: Recent studies showed that immune-checkpoint blockade (ICB) has significantly improved clinical outcomes of melanoma and lung cancer patients. However, only a small subset of patients can benefit from ICB. Deep learning has been successfully implemented in complementary clinical diagnosis. The aim of this study is to demonstrate the potential of deep learning to facilitate the prediction of anti-PD-1 response from H&E images directly. METHODS: In this study, 190 H&E slides of melanoma were segmented into 256 × 256 tiles which were used as the training set for the convolutional neural network (CNN). Additional 54 melanoma and 55 lung cancer H&E slides were collected as independent testing sets. FINDINGS: An AUC of 0.778(95% CI: 63.8%-90.5%) was achieved for 54 melanoma testing samples with 15(65.2%) responders and 23(74.2%) non-responders correctly classified. We also obtained an AUC of 0.645(95% CI: 49.4%-78.4%) for 55 lung cancer samples. INTERPRETATION: To our knowledge, this is the first study of using deep learning to determine patients' anti-PD-1 response from H&E slides directly. Our CNN model achieved the state-of-the-art performance and has the potential to screen ICB beneficial patients in routine clinical practice.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.tranon.2020.100921

Cites

3 of 3 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Hu et al. developed a supervised deep learning model that used extracted hidden features from histopathology images to predict anti-PD-1 response in melanoma and lung cancer.

    “Hu et al. developed a supervised deep learning model that used extracted hidden features from histopathology images to predict anti-PD-1 response in melanoma and lung cancer.”

    From Predicting the tumor microenvironment composition and immunotherapy response in non-small cell lung cancer from digital histopathology images (2024) · cites this paper as [65]

  • A main caveat of Hu et al.'s approach was the limited interpretability of their hidden features.

    “However, a main caveat of their approach was the limited interpretability of their hidden features65.”

    From Predicting the tumor microenvironment composition and immunotherapy response in non-small cell lung cancer from digital histopathology images (2024) · cites this paper as [65]

    The Methods describe how the model extracts and reduces features, but do not assess whether those features are interpretable. The Discussion’s Limitations section identifies other caveats, including training labels inferred from IFNG scores and the need to test generalizability; a search for interpretability and hidden-feature limitations found no supporting passage. Page numbers are unavailable in our copy. (LLM, conf=0.74)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 28

  1. [1]Rodriguez-Cerdeira C. Carnero Gregorio M. Lopez-Barcenas A. Advances in Immunotherapy for Melanoma: a Comprehensive Review Mediators Inflamm. 2017 2017 3264217 PMC5564072 28848246
  2. [2]Chi Z. Li S. Sheng X. Clinical presentation, histology, and prognoses of malignant melanoma in ethnic Chinese: a study of 522 consecutive cases BMC Cancer 11 2011 85 21349197 PMC3056833
  3. [3]Binnewies M. Roberts E.W. Kersten K. Understanding the tumor immune microenvironment (TIME) for effective therapy Nat. Med. 24 5 2018 541 550 29686425 PMC5998822
  4. [4]Si L. Zhang X. Shu Y. A phase Ib study of pembrolizumab as second-line therapy for chinese patients with advanced or metastatic melanoma (KEYNOTE-151) Transl. Oncol. 12 6 2019 828 835 30981094 PMC6458446
  5. [6]Barroso-Sousa R. Jain E. Cohen O. Prevalence and mutational determinants of high tumor mutation burden in breast cancer Ann. Oncol. 31 3 2020 387 394 32067680
  6. [7]Cohen R. Pudlarz T. Garcia-Larnicol M.L. [Localized MSI/dMMR gastric cancer patients, perioperative immunotherapy instead of chemotherapy: the GERCOR NEONIPIGA phase II study is opened to recruitment] Bull. Cancer 2020 32057467
  7. [8]Gainor J.F. Rizvi H. Jimenez Aguilar E. Clinical activity of programmed cell death 1 (PD-1) blockade in never, light, and heavy smokers with non-small-cell lung cancer and PD-L1 expression >/=50 Ann. Oncol. 31 3 2020 404 411 32067682 PMC7545963
  8. [9]Kather J.N. Pearson A.T. Halama N. Deep learning can predict microsatellite instability directly from histology in gastrointestinal cancer Nat. Med. 25 7 2019 1054 1056 31160815 PMC7423299
  9. [10]Xu H., Park S., Lee S.H., Hwang T.H. Using transfer learning on whole slide images to predict tumor mutational burden in bladder cancer patients. bioRxiv. 2019:554527.
  10. [12]Goode A. Gilbert B. Harkes J. Jukic D. Satyanarayanan M OpenSlide: a vendor-neutral software foundation for digital pathology J. Pathol. Inform. 4 2013 27 24244884 PMC3815078
  11. [13]Tam A. Barker J. Rubin D A method for normalizing pathology images to improve feature extraction for quantitative pathology Med. Phys. 43 1 2016 528 26745946
  12. [14]Riaz F. Hassan A. Pimentel-Nunes P. Libnio E.J.L.D. Tavares Coimbra M How well can the fusion of Gabor filters and local binary patterns help in identifying gastric lesions? Conf. Proc. IEEE Eng. Med. Biol. Soc. 2016 2016 1204 1207 28268541
  13. [15]Frey B.J. Dueck D. Clustering by passing messages between data points Science 315 5814 2007 972 976 17218491
  14. [16]Chollet F. Xception: deep learning with depthwise separable convolutions. In. arXiv e-prints2016.
  15. [17]Saltz J. Gupta R. Hou L. Spatial organization and molecular correlation of tumor-infiltrating lymphocytes using deep learning on pathology images Cell Rep. 23 1 2018 181 193 e187 29617659 PMC5943714
  16. [19]Szegedy C., Vanhoucke V., Ioffe S., Shlens J., Wojna Z. Rethinking the inception architecture for computer vision. arXiv e-prints. 2015. https://ui.adsabs.harvard.edu/abs/2015arXiv151200567S. Accessed 1 December 2015.
  17. [20]Simonyan K., Zisserman A. Very deep convolutional networks for large-scale image recognition. arXiv e-prints. 2014. https://ui.adsabs.harvard.edu/abs/2014arXiv1409.1556S. Accessed 1 September 2014.
  18. [21]Zoph B., Vasudevan V., Shlens J., Le Q.V. Learning transferable architectures for scalable image recognition. arXiv e-prints. 2017. https://ui.adsabs.harvard.edu/abs/2017arXiv170707012Z. Accessed 1 July 2017.
  19. [22]Howard A.G., Zhu M., Chen B., et al. MobileNets: efficient convolutional neural networks for mobile vision applications. arXiv e-prints. 2017. https://ui.adsabs.harvard.edu/abs/2017arXiv170404861H. Accessed 1 April 2017.
  20. [23]Hayward N.K. Wilmott J.S. Waddell N. Whole-genome landscapes of major melanoma subtypes Nature 545 7653 2017 175 180 28467829
  21. [24]Campanella G. Hanna M.G. Geneslaw L. Clinical-grade computational pathology using weakly supervised deep learning on whole slide images Nat. Med. 25 8 2019 1301 1309 31308507 PMC7418463
  22. [25]Coudray N. Ocampo P.S. Sakellaropoulos T. Classification and mutation prediction from non-small cell lung cancer histopathology images using deep learning Nat. Med. 24 10 2018 1559 1567 30224757 PMC9847512
  23. [27]Higgs B.W. Morehouse C.A. Streicher K. Interferon gamma messenger rna signature in tumor biopsies predicts outcomes in patients with non-small cell lung carcinoma or urothelial cancer treated with durvalumab Clin. Cancer Res. 24 16 2018 3857 3866 29716923
  24. [28]Cristescu R. Mogg R. Ayers M. Pan-tumor genomic biomarkers for PD-1 checkpoint blockade-based immunotherapy Science 362 6411 2018 PMC6718162 30309915
  25. [29]Kong Y., Xu C., Cui C., et al. Ratio of the interferon-<em>γ</em>signature to the immunosuppression signature predicts anti-PD-1 therapy response in melanoma. bioRxiv. 2020:2020.2004.2018.047852. PMC7862369 33542239
  26. [30]Jiang P. Gu S. Pan D. Signatures of T cell dysfunction and exclusion predict cancer immunotherapy response Nat. Med. 24 10 2018 1550 1558 30127393 PMC6487502
  27. [31]Auslander N. Zhang G. Lee J.S. Robust prediction of response to immune checkpoint blockade therapy in metastatic melanoma Nat. Med. 24 10 2018 1545 1549 30127394 PMC6693632
  28. [32]Rosenthal R. Cadieux E.L. Salgado R. Neoantigen-directed immune escape in lung cancer evolution Nature 567 7749 2019 479 485 30894752 PMC6954100

Cited by

Sorted by citations

Authors
Hu, J., Cui, C., Yang, W., Huang, L., Yu, R., Liu, S., & Kong, Y.
Affiliations
  • Xiamen University (CN)
  • Peking University Cancer Hospital (CN)
Corresponding author
Kong, Yan
ORCID
Venue
Translational Oncology
Publisher
Elsevier BV
Published
2021-01
Volume
14
Issue
1
Pages
100921
ISSN
1936-5233 (ISSN-L)
DOI
10.1016/j.tranon.2020.100921
PMC
PMC7595938
OpenAlex
W3094646398
Cited by
75
References
3
Citations by year
2026: 9 · 2025: 15 · 2024: 21 · 2023: 14 · 2022: 15 · 2021: 1
FWCI
0.26
Type
journal article
Language
en
Open access
Licence
Locations
  • Translational Oncology (publishedVersion, cc-by-nc-nd)
  • PubMed Central (publishedVersion, cc-by-nc-nd)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • Beijing Municipal Natural Science Foundation (Z191100006619006)
  • National Natural Science Foundation of China (81972557, 81772912, 7202022)
  • National Key Research and Development Program of China
  • Natural Science Foundation of Beijing Municipality (2019YFA0904404)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
  • Health Sciences › Medicine › Oncology › Cancer Immunotherapy and Biomarkers
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
Keywords
immune checkpoint blockade, melanoma, lung cancer, convolutional neural networks, predictive biomarkers, immunotherapy selection

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review