Automatic discovery of image-based signatures for ipilimumab response prediction in malignant melanoma
Open accessThe DOI matches its Crossref record; 4 of the 4 references it cites were found in a registry or an index.
Abstract
In the context of precision medicine with immunotherapies there is an increasing need for companion diagnostic tests to identify potential therapy responders and avoid treatment coming along with severe adverse events for non-responders. Here, we present a retrospective case study to discover image-based signatures for developing a potential companion diagnostic test for ipilimumab (IPI) in malignant melanoma. Signature discovery is based on digital pathology and fully automatic quantitative image analysis using virtual multiplexing as well as machine learning and deep learning on whole-slide images. We systematically correlated the patient outcome data with potentially relevant local image features using a Tissue Phenomics approach with a sound cross validation procedure for reliable performance evaluation. Besides uni-variate models we also studied combinations of signatures in several multi-variate models. The most robust and best performing model was a decision tree model based on relative densities of CD8+ tumor infiltrating lymphocytes in the intra-tumoral infiltration region. Our results are well in agreement with observations described in previously published studies regarding the predictive value of the immune contexture, and thus, provide predictive potential for future development of a companion diagnostic test.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
The perceptron: A probabilistic model for information storage and organization in the brain.
Rosenblatt, F. (1958) · Psychological Review · doi:10.1037/h0042519 · Cited by 12,280
Predictive biomarkers for checkpoint inhibitor-based immunotherapy
Gibney, G. T., Weiner, L. M., & Atkins, M. B. (2016) · The Lancet Oncology · doi:10.1016/s1470-2045(16)30406-5 · Cited by 1,635
Hamid, O., Schmidt, H., Nissan, A., Ridolfi, L., Aamdal, S., Hansson, J., Guida, M., Hyams, D. M., Gómez, H., Bastholt, L., et al. (2011) · Journal of Translational Medicine · doi:10.1186/1479-5876-9-204 · Cited by 534
Research works printed but not linked: 28
- Larkin, J. et al. Combined Nivolumab and Ipilimumab or Monotherapy in Previously Untreated Melanoma. N. Engl. J. Med. 373, 23–34 (2015). · DOI · Paper page
- Teixidó, C., González-Cao, M., Karachaliou, N. & Rosell, R. Predictive factors for immunotherapy in melanoma. Ann. Transl. Med. 3, 208 (2015).
- Ladányi, A. Prognostic and predictive significance of immune cells infiltrating cutaneous melanoma. Pigment Cell Melanoma Res. 28, 490–500 (2015). · DOI · Paper page
- Espinosa, E. et al. Predictive factors of response to immunotherapy—a review from the Spanish Melanoma Group (GEM). Ann. Transl. Med. 5, 389 (2017). · DOI
- Ascierto, P. A., Kalos, M., Schaer, D. A., Callahan, M. K. & Wolchok, J. D. Biomarkers for Immunostimulatory Monoclonal Antibodies in Combination Strategies for Melanoma and Other Tumor Types. Clin. Cancer Res. 19, 1009 (2013). · DOI · Paper page
- Ji, R.-R. et al. An immune-active tumor microenvironment favors clinical response to ipilimumab. Cancer Immunol. Immunother. 61, 1019–1031 (2012). · DOI · Paper page
- Snyder, A. et al. Genetic Basis for Clinical Response to CTLA-4 Blockade in Melanoma. N. Engl. J. Med. 371, 2189–2199 (2014). · DOI · Paper page
- Galon, J. et al. Cancer classification using the Immunoscore: a worldwide task force. J. Transl. Med. 10, 205 (2012). · DOI · Paper page
- Capone, M. et al. Immunoscore: a new possible approach for melanoma classification. J. Immunother. Cancer 2, P193–P193 (2014). · doi:10.1186/2051-1426-2-s3-p193
- Bifulco, C. et al. MISIPI study: Melanoma ImmunoScore evaluation in patients treated with IPIlimumab. J. Transl. Med. 12, P11 (2014). · doi:10.1186/1479-5876-12-s1-p11
- Galon, J. et al. Immunoscore and Immunoprofiling in cancer: an update from the melanoma and immunotherapy bridge 2015. J. Transl. Med. 14, 273 (2016). · DOI · Paper page
- Madonna, G. et al. PD-L1 expression with immune-infiltrate evaluation and outcome prediction in melanoma patients treated with ipilimumab. OncoImmunology e1405206, https://doi.org/10.1080/2162402X.2017.1405206 (2017). · DOI · Paper page
- Harder, N. et al. Tissue Phenomics for prognostic biomarker discovery in low- and intermediate-risk prostate cancer. Sci. Rep. 8, 4470 (2018). · DOI
- Harder, N. et al. Optimal Patient Selection for Immunotherapy in Melanoma using Tissue Phenomics. In Proc. 6th European Post-Chicago Melanoma/Skin Cancer Meeting (2016).
- Brieu, N., Pauly, O., Zimmermann, J., Binnig, G. & Schmidt, G. Slide-specific models for segmentation of differently stained digital histopathology whole slide images. In Proc. SPIE Medical Imaging 2016: Image Processing (eds Styner, M. A. & Angelini, E. D.) 9784, 978410-978410–7 (SPIE, 2016). · DOI
- Criminisi, A., Shotton, J. & Bucciarelli, S. Decision Forests with Long-Range Spatial Context for Organ Localization in CT Volumes. In MICCAI workshop on Probabilistic Models for Medical Image Analysis (MICCAI-PMMIA) 69–80 (2009).
- Yigitsoy, M., Schmidt, G. & Binnig, G. Hierarchical patch-based co-registration of differently stained histopathology slides. in Medical Imaging 2017: Digital Pathology, Proc. SPIE (2017). · DOI
- Russakovsky, O. et al. ImageNet Large Scale Visual Recognition Challenge. Int. J. Comput. Vis. IJCV 115, 211–252 (2015). · DOI · Paper page
- Szegedy, C. et al. Going deeper with convolutions. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 1–9, https://doi.org/10.1109/CVPR.2015.7298594 (IEEE, 2015) · DOI · Paper page
- Hermitte, F. Biomarkers immune monitoring technology primer: Immunoscore® Colon. J. Immunother. Cancer 4, 57–57 (2016). · DOI
- Breiman, L., Friedman, J., Stone, C. J. & Olshen, R. A. Classification and Regression Trees. (Taylor & Francis Ltd, 1984).
- Vapnik, V. N. The nature of statistical learning theory. (Springer, 1995). · DOI · Paper page
- Cox, D. R. The Regression Analysis of Binary Sequences. J. R. Stat. Soc. Ser. B Methodol. 20, 215–242 (1958). · DOI · Paper page
- Jia, Y. et al. Caffe: Convolutional Architecture for Fast Feature Embedding. ArXiv Prepr. ArXiv14085093 (2014). · DOI · Paper page
- Pedregosa, F. et al. Scikit-learn: Machine Learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011).
- Franklin, C., Livingstone, E., Roesch, A., Schilling, B. & Schadendorf, D. Immunotherapy in melanoma: Recent advances and future directions. New Roads Melanoma Manag. 43, 604–611 (2017).
- Berthel, A. et al. Detailed resolution analysis reveals spatial T cell heterogeneity in the invasive margin of colorectal cancer liver metastases associated with improved survival. OncoImmunology 6, e1286436 (2017). · DOI
- Tumeh, P. C. et al. PD-1 blockade induces responses by inhibiting adaptive immune resistance. Nature 515, 568–571 (2014). · DOI · Paper page
Cited by
Deep learning in cancer pathology: a new generation of clinical biomarkers
Echle, A., Rindtorff, N. T., Brinker, T. J., Luedde, T., Pearson, A. T., & Kather, J. N. (2021) · British Journal of Cancer · doi:10.1038/s41416-020-01122-x · Cited by 667
“Notably, Harder and coworkers classi fi ed melanoma patients as responders and non-responders to ipilimumab, 51 and Madabhushi et al. demonstrated a concept for the prediction of response to immunotherapy in patients with NSCLC directly from H&E-stained images.”
BackgroundPan-cancer integrative histology-genomic analysis via multimodal deep learning
Chen, R. J., Lu, M. Y., Williamson, D. F. K., Chen, T. Y., Lipkova, J., Noor, Z., Shaban, M., Shady, M., Williams, M., Joo, B., et al. (2022) · Cancer Cell · doi:10.1016/j.ccell.2022.07.004 · Cited by 660
Artificial intelligence in histopathology: enhancing cancer research and clinical oncology
Shmatko, A., Ghaffari Laleh, N., Gerstung, M., & Kather, J. N. (2022) · Nature Cancer · doi:10.1038/s43018-022-00436-4 · Cited by 446
“In addition, genetic surrogate markers of immunotherapy response, such as gene expression signatures, have been predicted from pathology images with AI 65 .”
BackgroundArtificial intelligence for digital and computational pathology
Song, A. H., Jaume, G., Williamson, D. F. K., Lu, M. Y., Vaidya, A., Miller, T. R., & Mahmood, F. (2023) · Nature Reviews Bioengineering · doi:10.1038/s44222-023-00096-8 · Cited by 356
A survey on deep learning in medicine: Why, how and when?
Piccialli, F., Somma, V. D., Giampaolo, F., Cuomo, S., & Fortino, G. (2021) · Information Fusion · doi:10.1016/j.inffus.2020.09.006 · Cited by 348
Metadata
- Authors
- Harder, N., Schönmeyer, R., Nekolla, K., Meier, A., Brieu, N., Vanegas, C., Madonna, G., Capone, M., Botti, G., Ascierto, P. A., & Schmidt, G.
- Venue
- Scientific Reports
- Published
- 2019-05-15
- Volume
- 9
- Issue
- 1
- Pages
- 7449-7449
- ISSN
- 2045-2322 (ISSN-L)
- OpenAlex
- W2946404782
- Cited by
- 56
- References
- 4
- FWCI
- 2.88
- Type
- journal article
- Language
- en
- Open access
- gold
- Licence
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Automatic discovery of image-based signatures for ipilimumab response prediction in malignant melanoma. https://app.cochat.ai/papers/10.1038/s41598-019-43525-8
- The paper
- Harder, N., Schönmeyer, R., Nekolla, K., Meier, A., Brieu, N., Vanegas, C., Madonna, G., Capone, M., Botti, G., Ascierto, P. A., & Schmidt, G. (2019). Automatic discovery of image-based signatures for ipilimumab response prediction in malignant melanoma. Scientific Reports. https://doi.org/10.1038/s41598-019-43525-8
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.