Deep Learning-Based Mapping of Tumor Infiltrating Lymphocytes in Whole Slide Images of 23 Types of Cancer
Open accessThe DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index.
Abstract
The role of tumor infiltrating lymphocytes (TILs) as a biomarker to predict disease progression and clinical outcomes has generated tremendous interest in translational cancer research. We present an updated and enhanced deep learning workflow to classify 50x50 um tiled image patches (100x100 pixels at 20x magnification) as TIL positive or negative based on the presence of 2 or more TILs in gigapixel whole slide images (WSIs) from the Cancer Genome Atlas (TCGA). This workflow generates TIL maps to study the abundance and spatial distribution of TILs in 23 different types of cancer. We trained three state-of-the-art, popular convolutional neural network (CNN) architectures (namely VGG16, Inception-V4, and ResNet-34) with a large volume of training data, which combined manual annotations from pathologists (strong annotations) and computer-generated labels from our previously reported first-generation TIL model for 13 cancer types (model-generated annotations). Specifically, this training dataset contains TIL positive and negative patches from cancers in additional organ sites and curated data to help improve algorithmic performance by decreasing known false positives and false negatives. Our new TIL workflow also incorporates automated thresholding to convert model predictions into binary classifications to generate TIL maps. The new TIL models all achieve better performance with improvements of up to 13% in accuracy and 15% in F-score. We report these new TIL models and a curated dataset of TIL maps, referred to as TIL-Maps-23, for 7983 WSIs spanning 23 types of cancer with complex and diverse visual appearances, which will be publicly available along with the code to evaluate performance. Code Available at:https://github.com/ShahiraAbousamra/til_classification.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
QuPath: Open source software for digital pathology image analysis
Bankhead, P., Loughrey, M. B., Fernández, J. A., Dombrowski, Y., McArt, D. G., Dunne, P. D., McQuaid, S., Gray, R. T., Murray, L. J., Coleman, H. G., et al. (2017) · Scientific Reports · doi:10.1038/s41598-017-17204-5 · Cited by 8,908 · cited 1×
“Acs et al. (27) developed a computerized TIL scoring method using QuPath software (28) to cluster melanoma cancer patients into those with favorable prognosis and those with poor prognosis.”
MethodType, Density, and Location of Immune Cells Within Human Colorectal Tumors Predict Clinical Outcome
Galon, J., Costes, A., Sanchez-Cabo, F., Kirilovsky, A., Mlecnik, B., Lagorce-Pagès, C., Tosolini, M., Camus, M., Berger, A., Wind, P., et al. (2006) · Science · doi:10.1126/science.1129139 · Cited by 6,516
The Immune Landscape of Cancer
Thorsson, V., Gibbs, D. L., Brown, S. D., Wolf, D., Bortone, D. S., Ou Yang, T.-H., Porta-Pardo, E., Gao, G. F., Plaisier, C. L., Eddy, J. A., et al. (2018) · Immunity · doi:10.1016/j.immuni.2018.03.023 · Cited by 5,870 · cited 1×
“These fi ndings have led to efforts to characterize the abundance and spatial distribution of TILs in cancer tissue samples to further our understanding of tumor immune interactions and help develop precision medicine applications in oncology (7 – 11).”
BackgroundSelf-citationThe immune contexture in human tumours: impact on clinical outcome
Fridman, W. H., Pagès, F., Sautès-Fridman, C., & Galon, J. (2012) · Nature reviews. Cancer · doi:10.1038/nrc3245 · Cited by 4,981 · cited 1×
“…that the host immune system is capable of controlling tumor growth through the activation of adaptive and innate immune surveillance mechanisms (44) and that the spatial context and nature of cellular heterogeneity of the tumor microenvironment are important in cancer prognosis (1, 4, 45, 46).”
BackgroundSaltz, J., Gupta, R., Hou, L., Kurc, T., Singh, P., Nguyen, V., Samaras, D., Shroyer, K. R., Zhao, T., Batiste, R., et al. (2018) · Cell Reports · doi:10.1016/j.celrep.2018.03.086 · Cited by 1,027
Self-citationMlecnik, B., Tosolini, M., Kirilovsky, A., Berger, A., Bindea, G., Meatchi, T., Bruneval, P., Trajanoski, Z., Fridman, W.-H., Pagès, F., et al. (2011) · Journal of Clinical Oncology · doi:10.1200/jco.2010.30.5425 · Cited by 1,009
Loi, S., Drubay, D., Adams, S., Pruneri, G., Francis, P. A., Lacroix-Triki, M., Joensuu, H., Dieci, M. V., Badve, S., Demaria, S., et al. (2019) · Journal of Clinical Oncology · doi:10.1200/jco.18.01010 · Cited by 893
Defining an Optimal Cut-Point Value in ROC Analysis: An Alternative Approach
Unal, I. (2017) · Computational and Mathematical Methods in Medicine · doi:10.1155/2017/3762651 · Cited by 789
Deep neural network models for computational histopathology: A survey
Srinidhi, C. L., Ciga, O., & Martel, A. L. (2021) · Medical Image Analysis · doi:10.1016/j.media.2020.101813 · Cited by 749
Deep learning in microscopy image analysis: a survey
Xing, F., Xie, Y., Su, H., Liu, F., & Yang, L. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2017.2766168 · Cited by 460
Saltz, J., Sharma, A., Iyer, G., Bremer, E., Wang, F., Jasniewski, A., DiPrima, T., Almeida, J. S., Gao, Y., Zhao, T., et al. (2017) · Cancer Research · doi:10.1158/0008-5472.can-17-0316 · Cited by 60 · cited 1×
“QuIP consists of multiple services, implemented as micro-services with software containers, and a set of Web-based applications that support viewing of WSIs, annotation of image regions and patches, and interactive viewing of model predictions as heatmaps overlaid on WSIs (41).”
MethodSelf-citation
Cited by
Towards a general-purpose foundation model for computational pathology
Chen, R. J., Ding, T., Lu, M. Y., Williamson, D. F. K., Jaume, G., Song, A. H., Chen, B., Zhang, A., Shao, D., Shaban, M., et al. (2024) · Nature Medicine · doi:10.1038/s41591-024-02857-3 · Cited by 1,560
A foundation model for clinical-grade computational pathology and rare cancers detection
Vorontsov, E., Bozkurt, A., Casson, A., Shaikovski, G., Zelechowski, M., Severson, K., Zimmermann, E., Hall, J., Tenenholtz, N., Fusi, N., et al. (2024) · Nature Medicine · doi:10.1038/s41591-024-03141-0 · Cited by 572
Patkar, S., Chen, A., Basnet, A., Bixby, A., Rajendran, R., Chernet, R., Faso, S., Kumar, P. A., Desai, D., El-Zammar, O., et al. (2024) · npj Precision Oncology · doi:10.1038/s41698-024-00765-w · Cited by 41
“Oneofthe fi rst large-scale attempts to characterize the TME of patients from H&E slides was the work of Saltz et al., who mapped the abundance and spatial distribution of tumor in fi ltrating lymphocytes (TILs) across 23 different cancer types 28,29 .”
Deep learning in digital pathology for personalized treatment plans of cancer patients
Wen, Z., Wang, S., Yang, D. M., Xie, Y., Chen, M., Bishop, J., & Xiao, G. (2023) · Seminars in Diagnostic Pathology · doi:10.1053/j.semdp.2023.02.003 · Cited by 34
Metadata
- Authors
- Abousamra, S., Gupta, R., Hou, L., Batiste, R., Zhao, T., Shankar, A., Rao, A., Chen, C., Samaras, D., Kurc, T., & Saltz, J.
- Venue
- Frontiers in Oncology
- Publisher
- Frontiers Media SA
- Published
- 2022-02-16
- Volume
- 11
- Pages
- 806603-806603
- ISSN
- 2234-943X (ISSN-L)
- OpenAlex
- W4213294068
- Cited by
- 84
- References
- 14
- FWCI
- 5.41
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.