Deep learning-based image analysis methods for brightfield-acquired multiplex immunohistochemistry images
Open accessThe DOI matches its Crossref record; 20 of the 20 references it cites were found in a registry or an index.
Abstract
Background: Multiplex immunohistochemistry (mIHC) permits the labeling of six or more distinct cell types within a single histologic tissue section. The classification of each cell type requires detection of uniquely colored chromogens localized to cells expressing biomarkers of interest. The most comprehensive and reproducible method to evaluate such slides is to employ digital pathology and image analysis pipelines to whole-slide images (WSIs). Our suite of deep learning tools quantitatively evaluates the expression of six biomarkers in mIHC WSIs. These methods address the current lack of readily available methods to evaluate more than four biomarkers and circumvent the need for specialized instrumentation to spectrally separate different colors. The use case application for our methods is a study that investigates tumor immune interactions in pancreatic ductal adenocarcinoma (PDAC) with a customized mIHC panel. Methods: Six different colored chromogens were utilized to label T-cells (CD3, CD4, CD8), B-cells (CD20), macrophages (CD16), and tumor cells (K17) in formalin-fixed paraffin-embedded (FFPE) PDAC tissue sections. We leveraged pathologist annotations to develop complementary deep learning-based methods: (1) ColorAE is a deep autoencoder which segments stained objects based on color; (2) U-Net is a convolutional neural network (CNN) trained to segment cells based on color, texture and shape; and (3) ensemble methods that employ both ColorAE and U-Net, collectively referred to as ColorAE:U-Net. We assessed the performance of our methods using: structural similarity and DICE score to evaluate segmentation results of ColorAE against traditional color deconvolution; F1 score, sensitivity, positive predictive value, and DICE score to evaluate the predictions from ColorAE, U-Net, and ColorAE:U-Net ensemble methods against pathologist-generated ground truth. We then used prediction results for spatial analysis (nearest neighbor). Results: We observed that (1) the performance of ColorAE is comparable to traditional color deconvolution for single-stain IHC images (note: traditional color deconvolution cannot be used for mIHC); (2) ColorAE and U-Net are complementary methods that detect six different classes of cells with comparable performance; (3) combinations of ColorAE and U-Net in ensemble methods outperform ColorAE and U-Net alone; and (4) ColorAE:U-Net ensemble methods can be employed for detailed analysis of the tumor microenvironment (TME). Summary: We developed a suite of scalable deep learning methods to analyze 6 distinctly labeled cell populations in mIHC WSIs. We evaluated our methods and found that they reliably detected and classified cells in the PDAC tumor microenvironment. We also utilized the ColorAE:U-Net ensemble method to analyze 3 mIHC WSIs with nearest neighbor spatial analysis. We demonstrate a proof of concept that these methods can be employed to quantitatively describe the spatial distribution of immune cells within the tumor microenvironment. These complementary deep learning methods are readily deployable for use in clinical research studies.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892
The Immune Landscape of Cancer
Thorsson, V., Gibbs, D. L., Brown, S. D., Wolf, D., Bortone, D. S., Ou Yang, T.-H., Porta-Pardo, E., Gao, G. F., Plaisier, C. L., Eddy, J. A., et al. (2018) · Immunity · doi:10.1016/j.immuni.2018.03.023 · Cited by 5,870
Self-citationThe immune contexture in human tumours: impact on clinical outcome
Fridman, W. H., Pagès, F., Sautès-Fridman, C., & Galon, J. (2012) · Nature reviews. Cancer · doi:10.1038/nrc3245 · Cited by 4,981
Innate and adaptive immune cells in the tumor microenvironment
Gajewski, T. F., Schreiber, H., & Fu, Y.-X. (2013) · Nature Immunology · doi:10.1038/ni.2703 · Cited by 4,148
Trained immunity: A program of innate immune memory in health and– disease
Netea, M. G., Joosten, L. A. B., Latz, E., Mills, K. H. G., Natoli, G., Stunnenberg, H. G., O’Neill, L. A. J., & Xavier, R. J. (2016) · Science · doi:10.1126/science.aaf1098 · Cited by 2,768
Immune checkpoint targeting in cancer therapy: toward combination strategies with curative potential
Sharma, P., & Allison, J. P. (2015) · Cell · doi:10.1016/j.cell.2015.03.030 · Cited by 2,345
PD-L1 expression as a predictive biomarker in cancer immunotherapy
Patel, S. P., & Kurzrock, R. (2015) · Molecular Cancer Therapeutics · doi:10.1158/1535-7163.mct-14-0983 · Cited by 2,291
Histopathological Image Analysis: A Review
Gurcan, M. N., Boucheron, L. E., Can, A., Madabhushi, A., Rajpoot, N. M., & Yener, B. (2009) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2009.2034865 · Cited by 2,052
Predictive biomarkers for checkpoint inhibitor-based immunotherapy
Gibney, G. T., Weiner, L. M., & Atkins, M. B. (2016) · The Lancet Oncology · doi:10.1016/s1470-2045(16)30406-5 · Cited by 1,635
Cited by
Artificial Intelligence in Cancer Research and Precision Medicine
Bhinder, B., Gilvary, C., Madhukar, N. S., & Elemento, O. (2021) · Cancer Discovery · doi:10.1158/2159-8290.cd-21-0090 · Cited by 781
Deep learning in cancer pathology: a new generation of clinical biomarkers
Echle, A., Rindtorff, N. T., Brinker, T. J., Luedde, T., Pearson, A. T., & Kather, J. N. (2021) · British Journal of Cancer · doi:10.1038/s41416-020-01122-x · Cited by 667
Artificial intelligence for digital and computational pathology
Song, A. H., Jaume, G., Williamson, D. F. K., Lu, M. Y., Vaidya, A., Miller, T. R., & Mahmood, F. (2023) · Nature Reviews Bioengineering · doi:10.1038/s44222-023-00096-8 · Cited by 356
The Role of Artificial Intelligence in Early Cancer Diagnosis
Hunter, B., Hindocha, S., & Lee, R. W. (2022) · Cancers · doi:10.3390/cancers14061524 · Cited by 332
Metadata
- Authors
- Fassler, D. J., Abousamra, S., Gupta, R., Chen, C., Zhao, M., Paredes, D., Batool, S. A., Knudsen, B. S., Escobar-Hoyos, L., Shroyer, K. R., Samaras, D., Kurc, T., & Saltz, J.
- Venue
- Diagnostic Pathology
- Published
- 2020-07-28
- Volume
- 15
- Issue
- 1
- Pages
- 100-100
- ISSN
- 1746-1596 (ISSN-L)
- OpenAlex
- W3046129945
- Cited by
- 78
- References
- 20
- FWCI
- 4.07
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Deep learning-based image analysis methods for brightfield-acquired multiplex immunohistochemistry images. https://app.cochat.ai/papers/10.1186/s13000-020-01003-0
- The paper
- Fassler, D. J., Abousamra, S., Gupta, R., Chen, C., Zhao, M., Paredes, D., Batool, S. A., Knudsen, B. S., Escobar-Hoyos, L., Shroyer, K. R., Samaras, D., Kurc, T., & Saltz, J. (2020). Deep learning-based image analysis methods for brightfield-acquired multiplex immunohistochemistry images. Diagnostic Pathology. https://doi.org/10.1186/s13000-020-01003-0
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.