CoChat
Sign Up
Cited by 22AI in cancer detectionCancer Immunotherapy and BiomarkersRadiomics and Machine Learning in Medical Imaging

Weakly Supervised Deep Learning Predicts Immunotherapy Response in Solid Tumors Based on PD-L1 Expression

Open access
Ligero, Marta, Serna, Garazi, El Nahhas, Omar S.M., Sansano, Irene, Mauchanski, Siarhei, Viaplana, Cristina, Calderaro, Julien, Toledo, Rodrigo A. +10 more·2024·Cancer Research Communications
DOI 10.1158/2767-9764.crc-23-0287PMC PMC10782919

Published version of a preprint

Open this paper in CoChat — free accountOpen PDF

Abstract

Programmed death-ligand 1 (PD-L1) IHC is the most commonly used biomarker for immunotherapy response. However, quantification of PD-L1 status in pathology slides is challenging. Neither manual quantification nor a computer-based mimicking of manual readouts is perfectly reproducible, and the predictive performance of both approaches regarding immunotherapy response is limited. In this study, we developed a deep learning (DL) method to predict PD-L1 status directly from raw IHC image data, without explicit intermediary steps such as cell detection or pigment quantification. We trained the weakly supervised model on PD-L1–stained slides from the non–small cell lung cancer (NSCLC)-Memorial Sloan Kettering (MSK) cohort (N = 233) and validated it on the pan-cancer-Vall d'Hebron Institute of Oncology (VHIO) cohort (N = 108). We also investigated the performance of the model to predict response to immune checkpoint inhibitors (ICI) in terms of progression-free survival. In the pan-cancer-VHIO cohort, the performance was compared with tumor proportion score (TPS) and combined positive score (CPS). The DL model showed good performance in predicting PD-L1 expression (TPS ≥ 1%) in both NSCLC-MSK and pan-cancer-VHIO cohort (AUC 0.88 ± 0.06 and 0.80 ± 0.03, respectively). The predicted PD-L1 status showed an improved association with response to ICIs [HR: 1.5 (95% confidence interval: 1–2.3), P = 0.049] compared with TPS [HR: 1.4 (0.96–2.2), P = 0.082] and CPS [HR: 1.2 (0.79–1.9), P = 0.386]. Notably, our explainability analysis showed that the model does not just look at the amount of brown pigment in the IHC slides, but also considers morphologic factors such as lymphocyte conglomerates. Overall, end-to-end weakly supervised DL shows potential for improving patient stratification for cancer immunotherapy by analyzing PD-L1 IHC, holistically integrating morphology and PD-L1 staining intensity. Significance: The weakly supervised DL model to predict PD-L1 status from raw IHC data, integrating tumor staining intensity and morphology, enables enhanced patient stratification in cancer immunotherapy compared with traditional pathologist assessment.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1158/2767-9764.crc-23-0287

Cites

4 of 4 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Numerous deep learning studies for predicting ICI responses in NSCLC from H&E images are primarily focused on refining PD-L1 quantification.

    “While numerous deep learning studies have emerged for predicting ICI responses in NSCLC from H&E images, they are primarily focused on refining PD-L1 quantification56–58.”

    From Predicting the tumor microenvironment composition and immunotherapy response in non-small cell lung cancer from digital histopathology images (2024) · cites this paper as [58]

    The Introduction (page unavailable) describes prior deep-learning studies that predict PD-L1 status from H&E images, supporting that part of the claim. It does not establish that numerous such studies predict ICI response in NSCLC. The Methods and Results (pages unavailable) specify that this paper’s own response-related model uses PD-L1–stained IHC images, not H&E images. (LLM, conf=0.85)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 36

  1. [1]Reck M, Rodríguez-Abreu D, Robinson AG, Hui R, Csőszi T, Fülöp A,. Pembrolizumab versus chemotherapy for PD-L1-positive non-small-cell lung cancer. N Engl J Med 2016; 375: 1823– 33. 27718847
  2. [2]Ribas A, Puzanov I, Dummer R, Schadendorf D, Hamid O, Robert C,. Pembrolizumab versus investigator-choice chemotherapy for ipilimumab-refractory melanoma (KEYNOTE-002): a randomised, controlled, phase 2 trial. Lancet Oncol 2015; 16: 908– 18. 26115796 PMC9004487
  3. [3]Bellmunt J, de Wit R, Vaughn DJ, Fradet Y, Lee J-L, Fong L,. Pembrolizumab as second-line therapy for advanced urothelial carcinoma. N Engl J Med 2017; 376: 1015– 26. 28212060 PMC5635424
  4. [4]Roach C, Zhang N, Corigliano E, Jansson M, Toland G, Ponto G,. Development of a companion diagnostic PD-L1 immunohistochemistry assay for pembrolizumab therapy in non–small-cell lung cancer. Appl Immunohistochem Mol Morphol 2016; 24: 392– 7. 27333219 PMC4957959
  5. [5]Cyprian FS, Akhtar S, Gatalica Z, Vranic S. Targeted immunotherapy with a checkpoint inhibitor in combination with chemotherapy: a new clinical paradigm in the treatment of triple-negative breast cancer. Bosn J Basic Med Sci 2019; 19: 227– 33. 30915922 PMC6716092
  6. [7]Brunnström H, Johansson A, Westbom-Fremer S, Backman M, Djureinovic D, Patthey A,. PD-L1 immunohistochemistry in clinical diagnostics of lung cancer: inter-pathologist variability is higher than assay variability. Mod Pathol 2017; 30: 1411– 21. 28664936
  7. [8]Chang S, Park HK, Choi Y-L, Jang SJ. Interobserver reproducibility of PD-L1 biomarker in non-small cell lung cancer: a multi-institutional study by 27 pathologists. J Pathol Transl Med 2019; 53: 347– 53. 31656061 PMC6877442
  8. [9]Cooper WA, Russell PA, Cherian M, Duhig EE, Godbolt D, Jessup PJ,. Intra-and interobserver reproducibility assessment of PD-L1 biomarker in non–small cell lung cancerreproducibility of PD-L1 biomarker assessment in NSCLC. Clin Cancer Res 2017; 23: 4569– 77. 28420726
  9. [10]Rimm DL, Han G, Taube JM, Yi ES, Bridge JA, Flieder DB,. A prospective, multi-institutional, pathologist-based assessment of 4 immunohistochemistry assays for PD-L1 expression in non-small cell lung cancer. JAMA Oncol 2017; 3: 1051– 8. 28278348 PMC5650234
  10. [11]Inge LJ, Dennis E. Development and applications of computer image analysis algorithms for scoring of PD-L1 immunohistochemistry. Immunooncol Technol 2020; 6: 2– 8. 35757235 PMC9216464
  11. [12]Hondelink LM, Hüyük M, Postmus PE, Smit VTHBM, Blom S, von der Thüsen JH,. Development and validation of a supervised deep learning algorithm for automated whole-slide programmed death-ligand 1 tumour proportion score assessment in non-small cell lung cancer. Histopathology 2022; 80: 635– 47. 34786761 PMC9299490
  12. [13]Puladi B, Ooms M, Kintsler S, Houschyar KS, Steib F, Modabber A,. Automated PD-L1 scoring using artificial intelligence in head and neck squamous cell carcinoma. Cancers 2021; 13: 4409. 34503218 PMC8431396
  13. [14]Wu J, Liu C, Liu X, Sun W, Li L, Gao N,. Artificial intelligence-assisted system for precision diagnosis of PD-L1 expression in non-small cell lung cancer. Mod Pathol 2022; 35: 403– 11. 34518630
  14. [15]Wang X, Wang L, Bu H, Zhang N, Yue M, Jia Z,. How can artificial intelligence models assist PD-L1 expression scoring in breast cancer: results of multi-institutional ring studies. NPJ Breast Cancer 2021; 7: 61. 34039982 PMC8155065
  15. [16]Naso JR, Povshedna T, Wang G, Banyi N, MacAulay C, Ionescu DN,. Automated PD-L1 scoring for non-small cell lung carcinoma using open-source software. Pathol Oncol Res 2021; 27: 609717. 34257575 PMC8262183
  16. [17]Kapil A, Meier A, Steele K, Rebelatto M, Nekolla K, Haragan A,. Domain adaptation-based deep learning for automated tumor cell (TC) scoring and survival analysis on PD-L1 stained tissue images. IEEE Trans Med Imaging 2021; 40: 2513– 23. 34003747
  17. [18]Shamai G, Livne A, Polónia A, Sabo E, Cretu A, Bar-Sela G,. Deep learning-based image analysis predicts PD-L1 status from H&E-stained histopathology images in breast cancer. Nat Commun 2022; 13: 6753. 36347854 PMC9643479
  18. [20]Wang C, Ma J, Shao J, Zhang S, Li J, Yan J,. Non-invasive measurement using deep learning algorithm based on multi-source features fusion to predict PD-L1 expression and survival in NSCLC. Front Immunol 2022; 13: 828560. 35464416 PMC9022118
  19. [21]Kather JN, Pearson AT, Halama N, Jäger D, Krause J, Loosen SH,. Deep learning can predict microsatellite instability directly from histology in gastrointestinal cancer. Nat Med 2019; 25: 1054– 6. 31160815 PMC7423299
  20. [22]Ghaffari Laleh N, Muti HS, Loeffler CML, Echle A, Saldanha OL, Mahmood F,. Benchmarking weakly-supervised deep learning pipelines for whole slide classification in computational pathology. Med Image Anal 2022; 79: 102474. 35588568
  21. [23]Cifci D, Foersch S, Kather JN. Artificial intelligence to identify genetic alterations in conventional histopathology. J Pathol 2022; 257: 430– 44. 35342954
  22. [24]Schrammen PL, Laleh NG, Echle A, Truhn D, Schulz V, Brinker TJ,. Weakly supervised annotation-free cancer detection and prediction of genotype in routine histopathology. J Pathol 2022; 256: 50– 60. 34561876
  23. [26]Saillard C, Dubois R, Tchita O, Loiseau N, Garcia T, Adriansen A,. Validation of MSIntuit, as an AI-based pre-screening tool for MSI detection from colorectal cancer histology slides. Nat Commun 2023; 14: 6695. 37932267 PMC10628260
  24. [27]Kleppe A, Skrede O-J, De Raedt S, Hveem TS, Askautrud HA, Jacobsen JE,. A clinical decision support system optimising adjuvant chemotherapy for colorectal cancers by integrating deep learning and pathological staging markers: a development and validation study. Lancet Oncol 2022; 23: 1221– 32. 35964620
  25. [28]Vanguri RS, Luo J, Aukerman AT, Egger JV, Fong CJ, Horvat N,. Multimodal integration of radiology, pathology and genomics for prediction of response to PD-(L)1 blockade in patients with non-small cell lung cancer. Nat Cancer 2022; 3: 1151– 64. 36038778 PMC9586871
  26. [29]El Nahhas OSM, Loeffler CML, Carrero ZI, van Treeck M, Kolbinger FR, Hewitt KJ,. Regression-based Deep-Learning predicts molecular biomarkers from pathology slides. arXiv [cs.CV]; 2023. Available from: https://arxiv.org/abs/2304.05153. · arXiv
  27. [30]Loeffler CML, El Nahhas OSM, Muti HS, Seibel T, Cifci D, van Treeck M,. Direct prediction of homologous recombination deficiency from routine histology in ten different tumor types with attention-based Multiple Instance Learning: a development and validation study. medRxiv [Preprint]. 2023. PMC11462727 39379982
  28. [31]Canny J. A computational approach to edge detection. IEEE Trans Pattern Anal Mach Intell 1986; 8: 679– 98. 21869365
  29. [32]Saldanha OL, Loeffler CML, Niehues JM, van Treeck M, Seraphin TP, Hewitt KJ,. Self-supervised attention-based deep learning for pan-cancer mutation prediction from histopathology. NPJ Precis Oncol 2023; 7: 35. 36977919 PMC10050159
  30. [33]Wagner SJ, Reisenbüchler D, West NP, Niehues JM, Zhu J, Foersch S,. Transformer-based biomarker prediction from colorectal cancer histology: a large-scale multicentric study. Cancer Cell 2023; 41: 1650– 61. 37652006 PMC10507381
  31. [35]Ilse M, Tomczak J, Welling M. Attention-based deep multiple instance learning. In: Dy J, Krause A, editors. Proceedings of the 35th International Conference on Machine Learning, PMLR; 10–15 Jul 2018. p. 2127– 36.
  32. [36]Davis AA, Patel VG. The role of PD-L1 expression as a predictive biomarker: an analysis of all US food and drug administration (FDA) approvals of immune checkpoint inhibitors. J Immunother Cancer 2019; 7: 278. 31655605 PMC6815032
  33. [37]Kluger HM, Zito CR, Turcu G, Baine MK, Zhang H, Adeniran A,. PD-L1 studies across tumor types, its differential expression and predictive value in patients treated with immune checkpoint inhibitors. Clin Cancer Res 2017; 23: 4270– 9. 28223273 PMC5540774
  34. [38]Kintsler S, Cassataro MA, Drosch M, Holenya P, Knuechel R, Braunschweig T. Expression of programmed death ligand (PD-L1) in different tumors. Comparison of several current available antibody clones and antibody profiling. Ann Diagn Pathol 2019; 41: 24– 37. 31132649
  35. [39]Hagi T, Kurokawa Y, Kawabata R, Omori T, Matsuyama J, Fujitani K,. Multicentre biomarker cohort study on the efficacy of nivolumab treatment for gastric cancer. Br J Cancer 2020; 123: 965– 72. 32616848 PMC7492241
  36. [40]Cohen EEW, Bell RB, Bifulco CB, Burtness B, Gillison ML, Harrington KJ,. The society for immunotherapy of cancer consensus statement on immunotherapy for the treatment of squamous cell carcinoma of the head and neck (HNSCC). J Immunother Cancer 2019; 7: 184. 31307547 PMC6632213

Cited by

Sorted by citations

Authors
Ligero, M., Serna, G., El Nahhas, O. S. M., Sansano, I., Mauchanski, S., Viaplana, C., Calderaro, J., Toledo, R. A., Dienstmann, R., Vanguri, R. S., Sauter, J. L., Sanchez-Vega, F., Shah, S. P., Ramón y Cajal, S., Garralda, E., Nuciforo, P., Perez-Lopez, R., & Kather, J. N.
Affiliations
  • Vall d'Hebron Institute of Oncology (ES)
  • 3Else Kroener Fresenius Center for Digital Health, Technical University Dresden, Dresden, Germany.
  • Vall d'Hebron Hospital Universitari (ES)
  • 6Assistance Publique-Hôpitaux de Paris, Département de Pathologie, CHU Henri Mondor, Créteil, France.
  • Children's Hospital of Philadelphia (US)
  • Memorial Sloan Kettering Cancer Center (US)
  • 11Computational Oncology, Memorial Sloan Kettering Cancer Center, New York, New York.
  • 4Pathology Department, Vall d'Hebron University Hospital (VHUH), Barcelona, Spain.
  • 1Radiomics Group, Vall d'Hebron Institute of Oncology (VHIO), Barcelona, Spain.
  • University of Leeds (GB)
Corresponding author
Kather, Jakob Nikolas
ORCID
Venue
Cancer Research Communications
Publisher
American Association for Cancer Research (AACR)
Published
2024-01-11
Volume
4
Issue
1
Pages
92-102
ISSN
2767-9764 (ISSN-L)
DOI
10.1158/2767-9764.crc-23-0287
PMC
PMC10782919
OpenAlex
W4390061188
Cited by
22
References
4
Citations by year
2026: 6 · 2025: 10 · 2024: 6
FWCI
2.07
Type
journal article
Language
en
Open access
Licence
Locations
  • Cancer Research Communications (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
Funders
  • Bundesministerium für Gesundheit (ZMVI1-2520DAT111)
  • Deutsche Krebshilfe (#70113864)
  • Bundesministerium für Bildung und Forschung (01KD2104C)
  • 'la Caixa' Foundation
  • CRIS Cancer Foundation (TALENT19-05)
  • MEC | Instituto de Salud Carlos III (PI18/01395, PI21/01019)
  • NIHR | National Institute for Health and Care Research Applied Research Collaboration Oxford and Thames Valley (NIHR213331)
  • Fundación Fero
  • Prostate Cancer Foundation (18YOUN19)
  • PERIS
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
  • Health Sciences › Medicine › Oncology › Cancer Immunotherapy and Biomarkers
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
Keywords
PD-L1 expression, immunotherapy response prediction, weakly supervised deep learning, immunohistochemistry image analysis, immune checkpoint inhibitors, non-small cell lung cancer, solid tumors, tumor proportion score, combined positive score, model explainability, patient stratification, progression-free survival, pan-cancer cohort
MeSH terms
Deep Learning, Carcinoma, Non-Small-Cell Lung, Humans, Immunotherapy, Lung Neoplasms, B7-H1 Antigen

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review