The DOI matches its Crossref record; 87 of the 87 references it cites were found in a registry or an index; 30 of the 39 sources its claims cite were checked for support.
Predicting the tumor microenvironment composition and immunotherapy response in non-small cell lung cancer from digital histopathology images
Open accessPublished version of a preprint
Abstract
HistoTME predicts 29 BostonGene TME signatures from H&E and uses them to predict immunotherapy response in NSCLC patients treated with pembrolizumab.
Overview
CoChat Summary
## TL;DR
HistoTME is a weakly supervised deep learning method that uses H&E whole-slide histopathology images to infer 30 tumor microenvironment (TME) signatures in NSCLC and to predict immunotherapy response. It achieved an average Pearson correlation of 0.50 on independent tumor cohorts and an AUROC of 0.75 for first-line ICI response prediction in an external clinical cohort of 652 patients.
## Key findings
- The final HistoTME variant (multi-task AB-MIL + UNI) achieved an average Pearson correlation coefficient of 0.50 against ground truth on the CPTAC-NSCLC validation cohort.
- On an external SUNY cohort with serial IHC, HistoTME-predicted signatures correlated with measured cell abundances at Pearson 0.60 for T cells, 0.48 for B cells, and 0.41 for macrophages; Spearman correlations were 0.60, 0.54, and 0.49, respectively.
- Unsupervised clustering of predicted signatures separated patients into 2 TME subtypes, described as Immune-Inflamed and Immune-Desert.
- For immunotherapy response prediction, interaction-based features improved test-set performance to AUROC 0.68 overall and AUROC 0.75 [95% CI: 0.61-0.88] for patients receiving first-line ICI; using TME signatures alone gave AUROC 0.55 [95% CI: 0.41-0.69].
- In the SUNY ICI-treated cohort, the Immune-Inflamed vs Immune-Desert status was predictive of overall survival for first-line ICI patients (HR = 0.53 [95% CI: 0.36-0.78], log-rank p = 0.0012) and especially for first-line ICI + chemotherapy (HR = 0.39 [95% CI: 0.22-0.68], log-rank p = 0.00067).
## Methods
- Design: Weakly supervised deep learning model (HistoTME) trained to predict 30 cell type-specific TME signatures from H&E whole-slide images, then used for unsupervised clustering and supervised ICI response prediction.
- Data/sample: Trained on TCGA-NSCLC (865 patients; 955 WSIs), validated on CPTAC-NSCLC (333 patients; 1501 WSIs), tested on SUNY Upstate NSCLC with serial IHC (82 patients), and evaluated for clinical response on a retrospective SUNY cohort of 652 patients; 230 patients had response labels for the supervised response model.
- Main analysis: Tile extraction and feature learning with foundation models (CTransPath, RetCCL, UNI) plus AB-MIL; K-means clustering with silhouette-based selection of K = 2; response modeling with engineered pairwise interaction features (1740) selected by random forest and classified with XGBoost, evaluated by AUROC and survival analyses.
## Limitations
- The paper states that HistoTME cannot predict the spatial distribution of specific cell types or molecules because it is trained weakly supervised on bulk transcriptomics data.
- The paper states that there was no additional external validation dataset for ICI response beyond the clinical cohort used here.
- The work was retrospective and limited to NSCLC, so broader validation and extension to other cancer types are still needed.
- Only 3 TME signatures were validated with IHC because comparable markers were not available for the others.
- Evident from the text: predictive accuracy was limited for second- and subsequent-line ICI-treated patients, likely because most patients received first-line therapy and because H&E slides may not reflect later treatment-time TME changes.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures






Verification
Claims the paper makes
18 Valid · 3 Not valid · 4 Not checked · 72 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“However, both PDL-1 expression and TMB fail to encapsulate various tumor microenvironmental features influencing ICI responses15–17.”
§ Predicting the tumor microenvironment composition and immunotherapy response in non-small cell lung cancer from digital histopathology images > Introduction Lung cancer is the leading cause of cancer-related mortality globally, of which non-small cell lung cancer (NSCLC) is the most common histological subtype1. In recent years, immune checkpoint inhibitors (ICI) have radically transformed the prognosis of clinically advanced NSCLC. Although ICI-treatment is approved for the treatment of NSCLC patients in the localized and metastatic setting regardless of PD-L1 status2–4, many patients still fail to achieve clinically meaningful responses. Considering the high costs of treatment and potential immune related toxicities5,6, identifying biomarkers of anti-PD1/PDL1 treatment response is of critical importance. The use of PD-L1 as a biomarker to determine which patients respond to ICI-treatment is hindered by challenges associated with standardized quantification of PD-L1 expression7. Controversies such as responders with low PD-L1 expression levels and non-responders with high PD-L1 levels are very common8. For example, after selecting patients based on PD-L1 expression, response rates to treatment still vary widely, ranging from 17 – 49% in patients with Tumor Proportion Score (TPS) > 1%9. These discrepancies could be attributed to a high level of intratumor heterogeneity of PDL1 expression and the underlying complexity of tumor microenvironment10. The clinical utility of other biomarkers such as the Tumor Mutational Burden (TMB) has been explored in several different clinical trials11–14. However, both PDL-1 expression and TMB fail to encapsulate various tumor microenvironmental features influencing ICI responses15–17. Hence, there is a clinically unmet need for additional predictive biomarkers capturing both tumor and microenvironmental factors associated with ICI responses. In recent years, several new multiplex tissue imaging and spatial transcriptomics technologies have been used for profiling the tumor microenvironment (TME) of patients in unprecedented detail18–26. However, they are quite expensive, which does not allow them to be implemented for wider use in a clinical setting. Hematoxylin and Eosin (H&E)-stained pathology slides, on the other hand, are relatively cheap and easily accessible in any pathology lab. These slides hold a wealth of TME-related information that can be unlocked with the help of Artificial Intelligence (AI)27. One of the first large-scale attempts to characterize the TME of patients from H&E slides was the work of Saltz et al., who mapped the abundance and spatial distribution of tumor infiltrating lymphocytes (TILs) across 23 different cancer types28,29. Graham et al. developed Hover-Net30, a pan-cancer nuclei segmentation and classification neural network that enables single-cell quantification of tumor, stroma, and lymphocyte populations from H&E slides. More recently, Diao et al. developed a collection of supervised machine learning (ML) methods to quantify 607 human interpretable TME features from histopathology images31. While these approaches are extremely valuable, they are limited by availability of relevant pixel-level annotations from expert pathologists, which are time and resource consuming to generate. To overcome these limitations, several research groups have alternatively proposed the use of weakly-supervised deep learning models, which can be trained to perform various downstream computational pathology tasks such as tumor subtyping and prognosis32–37 without any region or pixel-level annotations. Building on these recent AI-based advances, we introduce HistoTME: a weakly supervised multi-task learning approach to infer the TME composition of patients from routinely collected pathology slides. Unlike previous approaches, HistoTME harnesses recently developed digital pathology foundation models to infer the expression of 30 distinct cell type-specific TME signatures, previously shown to be associated with immunotherapy responses38. HistoTME is trained in a weakly supervised fashion utilizing matched whole slide H&E and bulk transcriptomics data of 865 NSCLC patients from The Cancer Genome Atlas (TCGA), validated on matched whole slide H&E and bulk transcriptomic data of 333 NSCLC patients from the Clinical Proteomic Tumor Analysis Consortium (CPTAC) and tested on whole slide H&E and IHC data from 82 NSCLC patients that had complete surgical resection at SUNY Upstate Medical University. We further demonstrate the clinical utility of HistoTME predictions by retrospectively analyzing needle biopsy specimens and clinical outcome data from an additional 570 NSCLC patients from the SUNY Upstate cohort treated with either chemotherapy or immune checkpoint inhibitors. Importantly, we show that HistoTME AI scores complement low PD-L1 expression and can identify more patients responding to first line immune checkpoint inhibitor therapy. Taken together, HistoTME presents a versatile and accessible tool for unraveling the complex dynamics of TME, leading to improved risk stratification and management of NSCLC patients.
What the sources say
- Partly supportedcites [15] Tumeh, P. C. et al. PD-1 blockade induces responses by inhibiting adaptive immune resistance. Nature 515, 568–571 (2014). sourceExperimental — accuracy not yet measured
The paper’s Abstract and Extended Data Figures 4–5 link pembrolizumab response to CD8-cell location, PD-1/PD-L1 expression and T-cell repertoire features, supporting the point that PD-L1 expression alone does not capture every relevant microenvironmental feature. Our copy does not assess tumor mutational burden (TMB) or whether it fails to capture those features. (LLM, conf=0.87)
- Partly supportedcites [16] Taube, J. M. et al. Colocalization of inflammatory response with B7-h1 expression in human melanocytic lesions supports an adaptive resistance mechanism of immune escape. Sci. Transl. Med. 4, 127ra137 (2012). sourceExperimental — accuracy not yet measured
The Discussion (page unavailable) says clinical outcomes may depend on PD-L1 (B7-H1) expression alongside immune-infiltrate composition and persistent inflammation, and links an inflamed tumor microenvironment to immunotherapy response. This supports the point that PD-L1 expression alone does not capture all relevant features. The paper does not address tumor mutational burden or establish that TMB fails to capture features influencing checkpoint-inhibitor responses. (LLM, conf=0.85)
- Partly supportedcites [17] Liu, Y. T. et al. Immune cell PD-L1 colocalizes with macrophages and is associated with outcome in PD-1 pathway blockade therapy. Clin. Cancer Res. 26, 970–977 (2020). sourceExperimental — accuracy not yet measured
The Introduction and Discussion (page numbers unavailable) show that which cells express PD-L1 matters for anti-PD-1 therapy; the Discussion reports an association between macrophage PD-L1 and survival in treated patients. This supports the concern that tumor-cell PD-L1 expression alone misses relevant microenvironmental features. The paper does not assess TMB or establish that TMB fails to capture those features. (LLM, conf=0.85)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“This model effectively identifies two main clusters resembling the Immune-Inflamed and Immune-Desert phenotypes42–44”
§ Predicting the tumor microenvironment composition and immunotherapy response in non-small cell lung cancer from digital histopathology images > Results > HistoTME accurately infers the TME composition of NSCLC patients from histopathology images Of all versions of HistoTME that were explored, we observed that the version utilizing multi-task AB-MIL + the UNI foundation model produced the most accurate predictions when compared with the ground truth, achieving an average Pearson correlation coefficient of 0.50 (Fig. 1C, Supplementary Fig. 2A). The performance of single-task AB-MIL + UNI was slightly worse than the performance of multi-task AB-MIL + UNI. However, both single-task AB-MIL + UNI and multi-task AB-MIL + UNI significantly outperformed other versions of HistoTME for predicting antitumor and protumor immune signatures while displaying similar performance for other signature prediction tasks (Supplementary Fig. 3). Therefore, we settled on multitask AB-MIL + UNI as the final version of HistoTME. Having validated the accuracy of HistoTME on CPTAC-NSCLC data, we next tested HistoTME on whole slide H&E images of 82 NSCLC patients from SUNY Upstate Medical University, which had serial immunohistochemistry (IHC) performed on surgical resection specimens for immune cell panel: T cells (CD3, CD4, DC8), B cells (CD20) and Macrophage (CD163) markers using adjacent serial sections (Supplementary Fig. 1). Overall, we found that HistoTME predicted expression levels were correlated with the abundance of each cell type derived from IHC, achieving Pearson correlations of 0.60 [95% CI: 0.49-0.69] for T cells, 0.48 [95% CI: 0.21-0.67] for B cells, and 0.41 [95% CI: 0.28-0.53] for macrophages, and Spearman correlations of 0.60 [95% CI: 0.44-0.72] for T cells, 0.54 [95% CI: 0.34-0.71] for B cells, and 0.49 [95% CI: 0.30-0.65] for macrophages (Fig. 1D, Supplementary Fig. 2B). Spearman correlations were included due to the uncertain linear correlation between gene expression and IHC-measured protein abundance41. We then trained a simple unsupervised model on the TCGA + CPTAC cohorts to cluster NSCLC patients into distinct subgroups based on their predicted TME signatures (Fig. 2A, see Methods). This model effectively identifies two main clusters resembling the Immune-Inflamed and Immune-Desert phenotypes42–44(Fig. 2B, Supplementary Fig. 4). Subsequently, we applied this two-cluster model to the entire institutional SUNY Upstate cohort, including patients with core needle biopsies (N = 652 patients) (Fig. 2C). We next compared the distribution of various clinical characteristics of patients across model-predicted immune inflamed and immune desert clusters including age, smoking history, tissue type (Primary vs metastatic), specimen source (biopsy vs surgical specimen), disease stage, grade, PDL1 score and histopathologic subtype. Overall, we do not find any significant differences in the distribution of majority of the clinical features across the two subtypes with the exception of tumor stage, tissue type (primary vs metastatic) and PD-L1 scores (Supplementary Fig. 5). Upon examining the feature importance scores assigned to each TME signature by the trained two-class classification model (Fig. 2D), we found that the following signatures: T cell traffic, Antitumor Cytokines, MDSC(myeloid-derived suppressor cells), Co-activation molecules and Macrophage/Dendritic Cell Traffic were among the top 5 TME signatures driving the distinction between the Immune-Inflamed and Immune-Desert clusters. Interestingly, in line with these results, we noted substantial differences in the abundances of T cells, B cells and, macrophages, when observed through IHC, among patients predicted to be either in the Immune Inflamed or in the Immune Desert cluster (Fig. 2E). To understand the histopathological features influencing HistoTME predictions, we generated attention maps for all 652 patients within the SUNY Upstate cohort. In general, HistoTME attends to different areas of the TME to estimate the expression of antitumor, pro-tumor, angiogenesis/stroma and malignant cell signatures, with the exception of antitumor and protumor immune signatures (Supplementary Fig. 6). We additionally performed a qualitative review of four randomly chosen resection cases (2 Immune-Inflamed, 2 Immune-Desert) with the help of two board-certified pathologists (Figs. 3 and 4, Supplementary Figs. 7 and 8). Overall, for immune-inflamed cases, which have relatively high predicted expression levels of antitumor and protumor immune signatures, HistoTME assigns great attention to regions abundant with lymphocytes and the formation of lymphocytic aggregates around tumor-stroma boundaries. For, immune desert cases, which have relatively low predicted expression levels of antitumor and protumor immune signatures, HistoTME assigns great attention to regions containing solid areas of more pleomorphic cells, and in addition, dense fibrotic areas within and around the tumor periphery. Collectively, these results suggest that HistoTME effectively captures the TME composition of tumors from H&E slides.
What the sources say
- Partly supportedcites [42] Petitprez, F. et al. B cells are associated with survival and immunotherapy response in sarcoma. Nature 577, 556–560 (2020). sourceExperimental — accuracy not yet measured
The paper’s ‘functional orientation’ section describes an ‘immune desert’ class and an immune-rich class, consistent with the two phenotypes named in the claim. But ‘Establishing the immune classification of STS’ reports five sarcoma immune classes, not two main clusters; the paper therefore does not support that aspect of the description. (LLM, conf=0.85)
- Not checked — no full text of the sourcecites [43] Chen, D. S. & Mellman, I. Elements of cancer immunity and the cancer-immune set point. Nature 541, 321–330 (2017). source
- Could not verifycites [44] Cristescu, R. et al. Pan-tumor genomic biomarkers for PD-1 checkpoint blockade-based immunotherapy. Science 362, eaar3593 (2018). sourceExperimental — accuracy not yet measured
The ‘Stratification of additional genomic features by TMB and T cell–inflamed GEP’ section (page unavailable) describes groups defined by TMB and GEP status and K-means clustering of gene coexpression modules. It does not report an unsupervised model identifying two tumor clusters resembling Immune-Inflamed and Immune-Desert phenotypes (searched: ‘unsupervised’ and the phenotype names). (LLM, conf=0.73)
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Although ICI-treatment is approved for the treatment of NSCLC patients in the localized and metastatic setting regardless of PD-L1 status2–4, many patients still fail to achieve clinically meaningful responses.”
§ Predicting the tumor microenvironment composition and immunotherapy response in non-small cell lung cancer from digital histopathology images > Introduction Lung cancer is the leading cause of cancer-related mortality globally, of which non-small cell lung cancer (NSCLC) is the most common histological subtype1. In recent years, immune checkpoint inhibitors (ICI) have radically transformed the prognosis of clinically advanced NSCLC. Although ICI-treatment is approved for the treatment of NSCLC patients in the localized and metastatic setting regardless of PD-L1 status2–4, many patients still fail to achieve clinically meaningful responses. Considering the high costs of treatment and potential immune related toxicities5,6, identifying biomarkers of anti-PD1/PDL1 treatment response is of critical importance. The use of PD-L1 as a biomarker to determine which patients respond to ICI-treatment is hindered by challenges associated with standardized quantification of PD-L1 expression7. Controversies such as responders with low PD-L1 expression levels and non-responders with high PD-L1 levels are very common8. For example, after selecting patients based on PD-L1 expression, response rates to treatment still vary widely, ranging from 17 – 49% in patients with Tumor Proportion Score (TPS) > 1%9. These discrepancies could be attributed to a high level of intratumor heterogeneity of PDL1 expression and the underlying complexity of tumor microenvironment10. The clinical utility of other biomarkers such as the Tumor Mutational Burden (TMB) has been explored in several different clinical trials11–14. However, both PDL-1 expression and TMB fail to encapsulate various tumor microenvironmental features influencing ICI responses15–17. Hence, there is a clinically unmet need for additional predictive biomarkers capturing both tumor and microenvironmental factors associated with ICI responses. In recent years, several new multiplex tissue imaging and spatial transcriptomics technologies have been used for profiling the tumor microenvironment (TME) of patients in unprecedented detail18–26. However, they are quite expensive, which does not allow them to be implemented for wider use in a clinical setting. Hematoxylin and Eosin (H&E)-stained pathology slides, on the other hand, are relatively cheap and easily accessible in any pathology lab. These slides hold a wealth of TME-related information that can be unlocked with the help of Artificial Intelligence (AI)27. One of the first large-scale attempts to characterize the TME of patients from H&E slides was the work of Saltz et al., who mapped the abundance and spatial distribution of tumor infiltrating lymphocytes (TILs) across 23 different cancer types28,29. Graham et al. developed Hover-Net30, a pan-cancer nuclei segmentation and classification neural network that enables single-cell quantification of tumor, stroma, and lymphocyte populations from H&E slides. More recently, Diao et al. developed a collection of supervised machine learning (ML) methods to quantify 607 human interpretable TME features from histopathology images31. While these approaches are extremely valuable, they are limited by availability of relevant pixel-level annotations from expert pathologists, which are time and resource consuming to generate. To overcome these limitations, several research groups have alternatively proposed the use of weakly-supervised deep learning models, which can be trained to perform various downstream computational pathology tasks such as tumor subtyping and prognosis32–37 without any region or pixel-level annotations. Building on these recent AI-based advances, we introduce HistoTME: a weakly supervised multi-task learning approach to infer the TME composition of patients from routinely collected pathology slides. Unlike previous approaches, HistoTME harnesses recently developed digital pathology foundation models to infer the expression of 30 distinct cell type-specific TME signatures, previously shown to be associated with immunotherapy responses38. HistoTME is trained in a weakly supervised fashion utilizing matched whole slide H&E and bulk transcriptomics data of 865 NSCLC patients from The Cancer Genome Atlas (TCGA), validated on matched whole slide H&E and bulk transcriptomic data of 333 NSCLC patients from the Clinical Proteomic Tumor Analysis Consortium (CPTAC) and tested on whole slide H&E and IHC data from 82 NSCLC patients that had complete surgical resection at SUNY Upstate Medical University. We further demonstrate the clinical utility of HistoTME predictions by retrospectively analyzing needle biopsy specimens and clinical outcome data from an additional 570 NSCLC patients from the SUNY Upstate cohort treated with either chemotherapy or immune checkpoint inhibitors. Importantly, we show that HistoTME AI scores complement low PD-L1 expression and can identify more patients responding to first line immune checkpoint inhibitor therapy. Taken together, HistoTME presents a versatile and accessible tool for unraveling the complex dynamics of TME, leading to improved risk stratification and management of NSCLC patients.
What the sources say
- Partly supportedcites [2] Wakelee, H. et al. Perioperative pembrolizumab for early-stage non-small-cell lung cancer. N. Engl. J. Med 389, 491–503 (2023). sourceExperimental — accuracy not yet measured
The Results (Efficacy) report major pathological response in only 30.2% of participants receiving perioperative pembrolizumab plus chemotherapy, supporting the point that many did not achieve that response. But the study concerns resectable NSCLC and also reports event-free survival benefit among participants without a major pathological response; it does not establish that those patients lacked any clinically meaningful benefit from ICI treatment generally. (LLM, conf=0.83)
- Partly supportedcites [3] Spigel, D. R. et al. Five-year survival outcomes from the PACIFIC trial: durvalumab after chemoradiotherapy in stage III non-small-cell lung cancer. J. Clin. Oncol. 40, 1301–1311 (2022). sourceExperimental — accuracy not yet measured
The Results, under “Antitumor Response” (page not provided), report a 29.8% objective response rate with durvalumab, supporting the point that many patients did not have an objective response. The trial studied durvalumab after chemoradiotherapy in stage III NSCLC; it does not establish the broader claim about clinically meaningful responses to ICI treatment across NSCLC settings. (LLM, conf=0.82)
- Supportedcites [4] Garassino, M. C. et al. Pembrolizumab plus pemetrexed and platinum in nonsquamous non-small-cell lung cancer: 5 Year outcomes from the phase 3 KEYNOTE-189 study. J. Clin. Oncol. 41, 1992–1998 (2023). source
Appendix Table A2 (page not provided) reports a 48.3% objective response rate among patients receiving pembrolizumab plus chemotherapy. Thus, many patients in this NSCLC trial did not achieve an objective tumor response, supporting the citing sentence’s broad point. (LLM, conf=0.88)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“The use of PD-L1 as a biomarker to determine which patients respond to ICI-treatment is hindered by challenges associated with standardized quantification of PD-L1 expression7.”
§ Predicting the tumor microenvironment composition and immunotherapy response in non-small cell lung cancer from digital histopathology images > Introduction Lung cancer is the leading cause of cancer-related mortality globally, of which non-small cell lung cancer (NSCLC) is the most common histological subtype1. In recent years, immune checkpoint inhibitors (ICI) have radically transformed the prognosis of clinically advanced NSCLC. Although ICI-treatment is approved for the treatment of NSCLC patients in the localized and metastatic setting regardless of PD-L1 status2–4, many patients still fail to achieve clinically meaningful responses. Considering the high costs of treatment and potential immune related toxicities5,6, identifying biomarkers of anti-PD1/PDL1 treatment response is of critical importance. The use of PD-L1 as a biomarker to determine which patients respond to ICI-treatment is hindered by challenges associated with standardized quantification of PD-L1 expression7. Controversies such as responders with low PD-L1 expression levels and non-responders with high PD-L1 levels are very common8. For example, after selecting patients based on PD-L1 expression, response rates to treatment still vary widely, ranging from 17 – 49% in patients with Tumor Proportion Score (TPS) > 1%9. These discrepancies could be attributed to a high level of intratumor heterogeneity of PDL1 expression and the underlying complexity of tumor microenvironment10. The clinical utility of other biomarkers such as the Tumor Mutational Burden (TMB) has been explored in several different clinical trials11–14. However, both PDL-1 expression and TMB fail to encapsulate various tumor microenvironmental features influencing ICI responses15–17. Hence, there is a clinically unmet need for additional predictive biomarkers capturing both tumor and microenvironmental factors associated with ICI responses. In recent years, several new multiplex tissue imaging and spatial transcriptomics technologies have been used for profiling the tumor microenvironment (TME) of patients in unprecedented detail18–26. However, they are quite expensive, which does not allow them to be implemented for wider use in a clinical setting. Hematoxylin and Eosin (H&E)-stained pathology slides, on the other hand, are relatively cheap and easily accessible in any pathology lab. These slides hold a wealth of TME-related information that can be unlocked with the help of Artificial Intelligence (AI)27. One of the first large-scale attempts to characterize the TME of patients from H&E slides was the work of Saltz et al., who mapped the abundance and spatial distribution of tumor infiltrating lymphocytes (TILs) across 23 different cancer types28,29. Graham et al. developed Hover-Net30, a pan-cancer nuclei segmentation and classification neural network that enables single-cell quantification of tumor, stroma, and lymphocyte populations from H&E slides. More recently, Diao et al. developed a collection of supervised machine learning (ML) methods to quantify 607 human interpretable TME features from histopathology images31. While these approaches are extremely valuable, they are limited by availability of relevant pixel-level annotations from expert pathologists, which are time and resource consuming to generate. To overcome these limitations, several research groups have alternatively proposed the use of weakly-supervised deep learning models, which can be trained to perform various downstream computational pathology tasks such as tumor subtyping and prognosis32–37 without any region or pixel-level annotations. Building on these recent AI-based advances, we introduce HistoTME: a weakly supervised multi-task learning approach to infer the TME composition of patients from routinely collected pathology slides. Unlike previous approaches, HistoTME harnesses recently developed digital pathology foundation models to infer the expression of 30 distinct cell type-specific TME signatures, previously shown to be associated with immunotherapy responses38. HistoTME is trained in a weakly supervised fashion utilizing matched whole slide H&E and bulk transcriptomics data of 865 NSCLC patients from The Cancer Genome Atlas (TCGA), validated on matched whole slide H&E and bulk transcriptomic data of 333 NSCLC patients from the Clinical Proteomic Tumor Analysis Consortium (CPTAC) and tested on whole slide H&E and IHC data from 82 NSCLC patients that had complete surgical resection at SUNY Upstate Medical University. We further demonstrate the clinical utility of HistoTME predictions by retrospectively analyzing needle biopsy specimens and clinical outcome data from an additional 570 NSCLC patients from the SUNY Upstate cohort treated with either chemotherapy or immune checkpoint inhibitors. Importantly, we show that HistoTME AI scores complement low PD-L1 expression and can identify more patients responding to first line immune checkpoint inhibitor therapy. Taken together, HistoTME presents a versatile and accessible tool for unraveling the complex dynamics of TME, leading to improved risk stratification and management of NSCLC patients.
What the sources say
- Supportedcites [7] Rimm, D. L. et al. A prospective, multi-institutional, pathologist-based assessment of 4 immunohistochemistry assays for PD-L1 expression in non-small cell lung dancer. JAMA Oncol. 3, 1051–1058 (2017). source
The Introduction describes drug-specific PD-L1 assays with different antibodies, protocols, and thresholds as a challenge for diagnostic testing. The Discussion reports differing staining results across assays and poor agreement among pathologists when scoring immune-cell expression. The paper does not test treatment response directly, but it supports the claim that quantification challenges hinder the biomarker’s use for that purpose. (LLM, conf=0.90)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“These slides hold a wealth of TME-related information that can be unlocked with the help of Artificial Intelligence (AI)27.”
§ Predicting the tumor microenvironment composition and immunotherapy response in non-small cell lung cancer from digital histopathology images > Introduction Lung cancer is the leading cause of cancer-related mortality globally, of which non-small cell lung cancer (NSCLC) is the most common histological subtype1. In recent years, immune checkpoint inhibitors (ICI) have radically transformed the prognosis of clinically advanced NSCLC. Although ICI-treatment is approved for the treatment of NSCLC patients in the localized and metastatic setting regardless of PD-L1 status2–4, many patients still fail to achieve clinically meaningful responses. Considering the high costs of treatment and potential immune related toxicities5,6, identifying biomarkers of anti-PD1/PDL1 treatment response is of critical importance. The use of PD-L1 as a biomarker to determine which patients respond to ICI-treatment is hindered by challenges associated with standardized quantification of PD-L1 expression7. Controversies such as responders with low PD-L1 expression levels and non-responders with high PD-L1 levels are very common8. For example, after selecting patients based on PD-L1 expression, response rates to treatment still vary widely, ranging from 17 – 49% in patients with Tumor Proportion Score (TPS) > 1%9. These discrepancies could be attributed to a high level of intratumor heterogeneity of PDL1 expression and the underlying complexity of tumor microenvironment10. The clinical utility of other biomarkers such as the Tumor Mutational Burden (TMB) has been explored in several different clinical trials11–14. However, both PDL-1 expression and TMB fail to encapsulate various tumor microenvironmental features influencing ICI responses15–17. Hence, there is a clinically unmet need for additional predictive biomarkers capturing both tumor and microenvironmental factors associated with ICI responses. In recent years, several new multiplex tissue imaging and spatial transcriptomics technologies have been used for profiling the tumor microenvironment (TME) of patients in unprecedented detail18–26. However, they are quite expensive, which does not allow them to be implemented for wider use in a clinical setting. Hematoxylin and Eosin (H&E)-stained pathology slides, on the other hand, are relatively cheap and easily accessible in any pathology lab. These slides hold a wealth of TME-related information that can be unlocked with the help of Artificial Intelligence (AI)27. One of the first large-scale attempts to characterize the TME of patients from H&E slides was the work of Saltz et al., who mapped the abundance and spatial distribution of tumor infiltrating lymphocytes (TILs) across 23 different cancer types28,29. Graham et al. developed Hover-Net30, a pan-cancer nuclei segmentation and classification neural network that enables single-cell quantification of tumor, stroma, and lymphocyte populations from H&E slides. More recently, Diao et al. developed a collection of supervised machine learning (ML) methods to quantify 607 human interpretable TME features from histopathology images31. While these approaches are extremely valuable, they are limited by availability of relevant pixel-level annotations from expert pathologists, which are time and resource consuming to generate. To overcome these limitations, several research groups have alternatively proposed the use of weakly-supervised deep learning models, which can be trained to perform various downstream computational pathology tasks such as tumor subtyping and prognosis32–37 without any region or pixel-level annotations. Building on these recent AI-based advances, we introduce HistoTME: a weakly supervised multi-task learning approach to infer the TME composition of patients from routinely collected pathology slides. Unlike previous approaches, HistoTME harnesses recently developed digital pathology foundation models to infer the expression of 30 distinct cell type-specific TME signatures, previously shown to be associated with immunotherapy responses38. HistoTME is trained in a weakly supervised fashion utilizing matched whole slide H&E and bulk transcriptomics data of 865 NSCLC patients from The Cancer Genome Atlas (TCGA), validated on matched whole slide H&E and bulk transcriptomic data of 333 NSCLC patients from the Clinical Proteomic Tumor Analysis Consortium (CPTAC) and tested on whole slide H&E and IHC data from 82 NSCLC patients that had complete surgical resection at SUNY Upstate Medical University. We further demonstrate the clinical utility of HistoTME predictions by retrospectively analyzing needle biopsy specimens and clinical outcome data from an additional 570 NSCLC patients from the SUNY Upstate cohort treated with either chemotherapy or immune checkpoint inhibitors. Importantly, we show that HistoTME AI scores complement low PD-L1 expression and can identify more patients responding to first line immune checkpoint inhibitor therapy. Taken together, HistoTME presents a versatile and accessible tool for unraveling the complex dynamics of TME, leading to improved risk stratification and management of NSCLC patients.
What the sources say
- Supportedcites [27] Wang, X. et al. Spatial interplay patterns of cancer nuclei and tumor-infiltrating lymphocytes (TILs) predict clinical benefit for immune checkpoint inhibitors. Sci. Adv. 8, eabn3966 (2022). source
The Methods describe analysis of digitized H&E slides, and the Introduction says the automated classifier extracts image biomarkers of immune–tumor interactions in the TME. The Discussion describes AI approaches for interrogating the TME in histology images. The supplied passages do not give page numbers. (LLM, conf=0.93)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
53 more claims · 8 with their cited sources checked
Create a free account to see all 97 claims and to verify the 2 references still open.
- Probably not a substantial claim. 17
- It may state two separate claims as one. 24
Citations
Cites
Scikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016
XGBoost: A Scalable Tree Boosting System
Chen, T., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939785 · Cited by 54,569 · cited 6×
“XGBoost learning rate was selected at 0.1, gamma was set to 0.1 to reduce overfitting, and the number of boosting rounds was set to 35 based on cross-validation.”
InfluentialA Threshold Selection Method from Gray-Level Histograms
Otsu, N. (1979) · IEEE Transactions on Systems, Man, and Cybernetics · doi:10.1109/tsmc.1979.4310076 · Cited by 43,398 · cited 1×
“All WSI in the experiments were first preprocessed to mask out the tumor tissue from background using RGB to HSV color transformation, median blurring and Otsu thresholding 51 .”
SciPy 1.0: fundamental algorithms for scientific computing in Python
Virtanen, P., Gommers, R., Oliphant, T. E., Haberland, M., Reddy, T., Cournapeau, D., Burovski, E., Peterson, P., Weckesser, W., Bright, J., et al. (2020) · Nature Methods · doi:10.1038/s41592-019-0686-2 · Cited by 38,430 · cited 1×
“Aftercompletionoftraining,thePearsoncorrelationmetricwasuti-lizedtoevaluateaccuracyofpredictionsofeachmodelontheindependent validationset(CPTAC-NSCLC).95%con fi dence intervals (CI) of the Pearson correlation metric were estimated through 1000 bootstrapping iterations using the SciPy package 73 .”
PyTorch: An Imperative Style, High-Performance Deep Learning Library
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al. (2019) · arXiv · doi:10.48550/arxiv.1912.01703 · Cited by 31,473 · cited 1×
“All AI models were developed using open source PyTorch version 2.1.0 55 .”
InfluentialA Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707 · cited 5×
“On performing SHAP analysis, we found that the total expression of coactivation molecules and the Th2 signature were the most predictive of ICI response.”
InfluentialDeLong, E. R., DeLong, D. M., & Clarke-Pearson, D. L. (1988) · Biometrics · doi:10.2307/2531595 · Cited by 22,649 · cited 1×
“95% Con fi dence intervals for AUROC were computed by the DeLong approach 79 .”
Silhouettes: A graphical aid to the interpretation and validation of cluster analysis
Rousseeuw, P. J. (1987) · Journal of Computational and Applied Mathematics · doi:10.1016/0377-0427(87)90125-7 · Cited by 21,392
Adverse Renal Effects of Immune Checkpoint Inhibitors: A Narrative Review
Wanchoo, R., Karam, S., Uppal, N. N., Barta, V. S., Deray, G., Devoe, C., Launay-Vacher, V., Jhaveri, K. D., & on behalf of Cancer and Kidney International Network Workgroup on Immune Checkpoint Inhibitors (2017) · American Journal of Nephrology · doi:10.1159/000455014 · Cited by 15,256
Gauriat, C.-M., Pencolé, Y., Ribot, P., & Brouillet, G. (2024) · HAL (Le Centre pour la Communication Scientifique Directe) · doi:10.4230/oasics.dx.2024.27 · Cited by 12,586
Research works printed but not linked: 2
- Ilse, M., Tomczak, J. & Welling, M. In International Conference on Machine Learning 2127–2136 (PMLR, 2024).
- Macenko, M. et al. 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro (Boston, MA, 2009).
Cited by
Artificial intelligence-based biomarkers for treatment decisions in oncology
Ligero, M., Nahhas, O. S. M. E., Aldea, M. D., & Kather, J. N. (2025) · Trends in cancer · doi:10.1016/j.trecan.2024.12.001 · Cited by 83
Wu, S., Zhuang, A., Huang, G., Zhao, Z., Zhan, W., Yu, L., Li, C., Yao, L., Zhou, Y., Guo, Y., et al. (2025) · Intelligent Oncology · doi:10.1016/j.intonc.2025.05.001 · Cited by 13
Targeting immune cell migration as therapy for inflammatory disease: a review
Song, B., Guo, W., He, Y., Yao, X., Sun, J., & Wang, S. (2025) · Frontiers in Immunology · doi:10.3389/fimmu.2025.1650760 · Cited by 16
Sekar, P. K. C., & Veerabathiran, R. (2025) · Clinical Immunology Communications · doi:10.1016/j.clicom.2025.12.002 · Cited by 8
Sardar, T. H., Naresh, P., Hassan, S. M., Kulkarni, P., Rajesh, T. M., & Shafeeq, B. M. A. (2025) · Scientific Reports · doi:10.1038/s41598-025-27748-6 · Cited by 7
Metadata
- Authors
- Patkar, S., Chen, A., Basnet, A., Bixby, A., Rajendran, R., Chernet, R., Faso, S., Kumar, P. A., Desai, D., El-Zammar, O., Curtiss, C., Carello, S. J., Nasr, M. R., Choyke, P., Harmon, S., Turkbey, B., & Jamaspishvili, T.
- Affiliations
- National Institutes of Health (US)
- SUNY Upstate Medical University (US)
- Corresponding author
- Patkar, Sushant
- Jamaspishvili, Tamara
- ORCID
- Venue
- npj Precision Oncology
- Published
- 2024-12-19
- Volume
- 8
- Issue
- 1
- Pages
- 280-280
- ISSN
- 2397-768X (ISSN-L)
- PMC
- PMC11659524
- OpenAlex
- W4405596115
- Cited by
- 41
- References
- 84
- Citations by year
- 2026: 27 · 2025: 14
- FWCI
- 6.04
- Type
- journal article
- Language
- en
- Update notices
- correction (2025-01-09)
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- npj Precision Oncology (publishedVersion, cc-by)
- medRxiv (acceptedVersion, cc-by-nc-nd)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- PubMed Central (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- National Institutes of Health
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
- Health Sciences › Medicine › Oncology › Cancer Immunotherapy and Biomarkers
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
- Keywords
- non-small cell lung cancer, tumor microenvironment, immune checkpoint inhibitors, immunotherapy response prediction, digital histopathology, whole slide images, weakly supervised deep learning, prognostic factors, PD-L1 expression, cancer immunotherapy
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Predicting the tumor microenvironment composition and immunotherapy response in non-small cell lung cancer from digital histopathology images. https://app.cochat.ai/papers/10.1038/s41698-024-00765-w
- The paper
- Patkar, S., Chen, A., Basnet, A., Bixby, A., Rajendran, R., Chernet, R., Faso, S., Kumar, P. A., Desai, D., El-Zammar, O., Curtiss, C., Carello, S. J., Nasr, M. R., Choyke, P., Harmon, S., Turkbey, B., & Jamaspishvili, T. (2024). Predicting the tumor microenvironment composition and immunotherapy response in non-small cell lung cancer from digital histopathology images. npj Precision Oncology. https://doi.org/10.1038/s41698-024-00765-w
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.