Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Sign Up
Cited by 269

The DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.

Human-interpretable image features derived from densely mapped cancer pathology slides predict diverse molecular phenotypes

Open access
Diao, James A., Wang, Jason K., Chui, Wan Fung, Mountain, Victoria, Gullapally, Sai Chowdary, Srinivasan, Ramprakash, Mitchell, Richard N., Glass, Benjamin, Hoffman, Sara, Rao, Sudha K., Maheshwari, Chirag, Lahiri, Abhik, Prakash, Aaditya, McLoughlin, Ryan, Kerner, Jennifer K., Resnick, Murray B., Montalto, Michael C., Khosla, Aditya, Wapinski, Ilan N., Beck, Andrew H., Elliott, Hunter L., Taylor-Weiner, Amaro·2021·Nature Communications
DOI 10.1038/s41467-021-21896-9
Open this paper in CoChat — free account

Abstract

Computational methods have made substantial progress in improving the accuracy and throughput of pathology workflows for diagnostic, prognostic, and genomic prediction. Still, lack of interpretability remains a significant barrier to clinical integration. We present an approach for predicting clinically-relevant molecular phenotypes from whole-slide histopathology images using human-interpretable image features (HIFs). Our method leverages >1.6 million annotations from board-certified pathologists across >5700 samples to train deep learning models for cell and tissue classification that can exhaustively map whole-slide images at two and four micron-resolution. Cell- and tissue-type model outputs are combined into 607 HIFs that quantify specific and biologically-relevant characteristics across five cancer types. We demonstrate that these HIFs correlate with well-known markers of the tumor microenvironment and can predict diverse molecular signatures (AUROC 0.601–0.864), including expression of four immune checkpoint proteins and homologous recombination deficiency, with performance comparable to ‘black-box’ methods. Our HIF-based approach provides a comprehensive, quantitative, and interpretable window into the composition and spatial architecture of the tumor microenvironment.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s41467-021-21896-9

Cites

26 of 26 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 26 references

Research works printed but not linked: 65

  1. He, L., Rodney Long, L., Antani, S. & Thoma, G. R. Histology image analysis for carcinoma detection and grading. Comput. Methods Programs Biomed. 107, 538–556 (2012). · DOI · Paper page
  2. Sokolenko, A. P. & Imyanitov, E. N. Molecular diagnostics in clinical oncology. Front. Mol. Biosci. 5, 76 (2018). · DOI
  3. Yamaguchi, T. et al. Inter-observer agreement among pathologists in grading the pathological response to neoadjuvant chemotherapy in breast cancer. Breast Cancer 25, 118–125 (2018). · DOI
  4. Vestjens, J. H. M. J. et al. Relevant impact of central pathology review on nodal classification in individual breast cancer patients. Ann. Oncol. 23, 2561–2566 (2012). · DOI
  5. Thorstenson, S., Molin, J. & Lundström, C. Implementation of large-scale routine diagnostics using whole slide imaging in Sweden: digital pathology experiences 2006-2013. J. Pathol. Inform. 5, 14 (2014). · DOI · Paper page
  6. Filipp, F. V. Opportunities for artificial intelligence in advancing precision. Med. Curr. Genet. Med. Rep. 7, 208–213 (2019). · DOI
  7. Nagpal, K. et al. Development and validation of a deep learning algorithm for improving Gleason scoring of prostate cancer. NPJ Digital Med. 2, 48 (2019). · DOI · Paper page
  8. Kather, J. N. et al. Pan-cancer image-based detection of clinically actionable genetic alterations. Nat. Cancer https://doi.org/10.1038/s43018-020-0087-6 (2020). · DOI · Paper page
  9. Fu, Y. et al. Pan-cancer computational histopathology reveals mutations, tumor composition and prognosis. Nat. Cancer https://doi.org/10.1038/s43018-020-0085-8 (2020). · DOI · Paper page
  10. Schmauch, B. et al. A deep learning model to predict RNA-Seq expression of tumours from whole slide images. Nat. Commun. 11, 3877 (2020). · DOI · Paper page
  11. Wang, D., Khosla, A., Gargeya, R., Irshad, H. & Beck, A. H. Deep learning for identifying metastatic breast cancer. Preprint at arXiv https://arxiv.org/abs/1606.05718 (2016). · arXiv
  12. Zhang, Z. et al. Pathologist-level interpretable whole-slide cancer diagnosis with deep learning. Nat. Mach. Intell. 1, 236–245 (2019). · DOI · Paper page
  13. Campanella, G. et al. Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nat. Med. 25, 1301–1309 (2019). · DOI · Paper page
  14. Beck, A. H. et al. Systematic analysis of breast cancer morphology uncovers stromal features associated with survival. Sci. Transl. Med. 3, 108ra113 (2011). · DOI · Paper page
  15. Wang, S. et al. Comprehensive analysis of lung cancer pathology images to discover tumor shape and boundary features that predict survival outcome. Sci. Rep. 8, 10393 (2018). · DOI
  16. Lennon, F. E. et al. Lung cancer—a fractal viewpoint. Nat. Rev. Clin. Oncol. 12, 664–675 (2015). · DOI
  17. AbdulJabbar, K. et al. Geospatial immune variability illuminates differential evolution of lung adenocarcinoma. Nat. Med. https://doi.org/10.1038/s41591-020-0900-x (2020). · DOI · Paper page
  18. Saltz, J. et al. Spatial organization and molecular correlation of tumor-infiltrating lymphocytes using deep learning on pathology images. Cell Rep. 23, 181–193.e7 (2018). · DOI · Paper page
  19. Fridman, W. H. The immune microenvironment as a guide for cancer therapies. Oncoimmunology 1, 261–262 (2012). · DOI
  20. Galon, J. & Bruni, D. Approaches to treat immune hot, altered and cold tumours with combination immunotherapies. Nat. Rev. Drug Discov. 18, 197–218 (2019). · DOI · Paper page
  21. Zhang, J., Shi, Z., Xu, X., Yu, Z. & Mi, J. The influence of microenvironment on tumor immunotherapy. FEBS J. 286, 4160–4175 (2019). · DOI
  22. Peikari, M., Salama, S., Nofech-Mozes, S. & Martel, A. L. Automatic cellularity assessment from post-treated breast surgical specimens. Cytom. A 91, 1078–1087 (2017). · DOI
  23. Rosenberg, A. & Hirschberg, J. V-measure: a conditional entropy-based external cluster evaluation measure. In Proc. 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL) 410–420 (Association for Computational Linguistics, 2007).
  24. Dorrity, M. W., Saunders, L. M., Queitsch, C., Fields, S. & Trapnell, C. Dimensionality reduction by UMAP to visualize physical and genetic interactions. Nat. Commun. 11, 1537 (2020). · DOI · Paper page
  25. Hu, G. & Christman, J. W. Editorial: alveolar macrophages in lung inflammation and resolution. Front. Immunol. 10, 2275 (2019). · DOI · Paper page
  26. Wang, S.-S. et al. Tumor-infiltrating B cells: their role and application in anti-tumor immunity in lung cancer. Cell. Mol. Immunol. 16, 6–18 (2019). · DOI · Paper page
  27. Gottlin, E. B. et al. The association of intratumoral germinal centers with early-stage non-small cell lung cancer. J. Thorac. Oncol. 6, 1687–1690 (2011). · doi:10.1097/jto.0b013e3182217bec
  28. Newman, A. M. et al. Robust enumeration of cell subsets from tissue expression profiles. Nat. Methods 12, 453–457 (2015). · DOI · Paper page
  29. Poole, W., Gibbs, D. L., Shmulevich, I., Bernard, B. & Knijnenburg, T. A. Combining dependent P-values with an empirical adaptation of Brown’s method. Bioinformatics 32, i430–i436 (2016). · DOI
  30. Buffa, F. M., Harris, A. L., West, C. M. & Miller, C. J. Large meta-analysis of multiple cancers reveals a common, compact and highly prognostic hypoxia metagene. Br. J. Cancer 102, 428–435 (2010). · DOI · Paper page
  31. Bhandari, V. et al. Molecular landmarks of tumor hypoxia across cancer types. Nat. Genet. 51, 308–318 (2019). · DOI · Paper page
  32. Thorsson, V. et al. The immune landscape of cancer. Immunity 48, 812–830.e14 (2018). · DOI · Paper page
  33. Petrov Victor, V., Fagard Robert, H. & Lijnen Paul, J. Stimulation of collagen production by transforming growth factor-β1 during differentiation of cardiac fibroblasts to myofibroblasts. Hypertension 39, 258–263 (2002). · DOI · Paper page
  34. Maroni, D. & Davis, J. S. Transforming growth factor beta 1 stimulates profibrotic activities of luteal fibroblasts in cows. Biol. Reprod. 87, 127 (2012). · DOI
  35. Kieffer, Y. et al. Single-cell analysis reveals fibroblast clusters linked to immunotherapy resistance in cancer. Cancer Discov. 10, 1330–1351 (2020). · DOI · Paper page
  36. Costanza, B., Umelo, I. A., Bellier, J., Castronovo, V. & Turtoi, A. Stromal modulators of TGF-β in cancer. J. Clin. Med. Res. 6, 7 (2017). · DOI
  37. Foster, D. S., Jones, R. E., Ransom, R. C., Longaker, M. T. & Norton, J. A. The evolving relationship of wound healing and tumor stroma. JCI Insight 3, e99911 (2018). · DOI
  38. Newman, A. C., Nakatsu, M. N., Chou, W., Gershon, P. D. & Hughes, C. C. W. The requirement for fibroblasts in angiogenesis: fibroblast-derived matrix proteins are essential for endothelial cell lumen formation. Mol. Biol. Cell 22, 3791–3800 (2011). · DOI · Paper page
  39. Du Cheyne, C., Tay, H. & De Spiegelaere, W. The complex TIE between macrophages and angiogenesis. Anat. Histol. Embryol. 49, 585–596 (2020). · DOI
  40. Schoenhals, M. et al. Hypoxia favors the generation of human plasma cells. Cell Cycle 16, 1104–1117 (2017). · DOI
  41. Kitano, A. et al. Tumour-infiltrating lymphocytes are correlated with higher expression levels of PD-1 and PD-L1 in early breast cancer. ESMO Open 2, e000150 (2017). · DOI · Paper page
  42. Velcheti, V. et al. Programmed death ligand-1 expression in non-small cell lung cancer. Lab. Invest. 94, 107–116 (2014). · DOI · Paper page
  43. McLaughlin, J. et al. Quantitative assessment of the heterogeneity of PD-L1 expression in non-small-cell lung cancer. JAMA Oncol. 2, 46–54 (2016). · DOI · Paper page
  44. Reiniger, L. et al. Tumor necrosis correlates with PD-L1 and PD-1 expression in lung adenocarcinoma. Acta Oncol. 58, 1087–1094 (2019). · DOI
  45. Li, Z. et al. Cancer-associated fibroblasts promote PD-L1 expression in mice cancer cells via secreting CXCL5. Int. J. Cancer 145, 1946–1957 (2019). · DOI · Paper page
  46. Jackson, S. P. & Bartek, J. The DNA-damage response in human biology and disease. Nature 461, 1071–1078 (2009). · DOI · Paper page
  47. Legrand, A. J. et al. Persistent DNA strand breaks induce a CAF-like phenotype in normal fibroblasts. Oncotarget 9, 13666–13681 (2018). · DOI
  48. Blessin, N. C. et al. Patterns of TIGIT expression in lymphatic tissue, inflammation, and cancer. Dis. Markers 2019, 5160565 (2019). · DOI
  49. Bera, K., Schalper, K. A., Rimm, D. L., Velcheti, V. & Madabhushi, A. Artificial intelligence in digital pathology - new tools for diagnosis and precision oncology. Nat. Rev. Clin. Oncol. 16, 703–715 (2019). · DOI · Paper page
  50. Wang, X. et al. Prediction of recurrence in early stage non-small cell lung cancer using computer extracted nuclear features from digital H&E images. Sci. Rep. 7, 13543 (2017). · DOI
  51. Amgad, M. et al. Joint region and nucleus segmentation for characterization of tumor infiltrating lymphocytes in breast cancer. Proc. SPIE Int. Soc. Opt. Eng. 10956, 109560M (2019). · DOI
  52. Amgad, M. et al. Report on computational assessment of tumor infiltrating lymphocytes from the International Immuno-Oncology Biomarker Working Group. NPJ Breast Cancer 6, 16 (2020). · DOI
  53. Havel, J. J., Chowell, D. & Chan, T. A. The evolving landscape of biomarkers for checkpoint inhibitor immunotherapy. Nat. Rev. Cancer 19, 133–150 (2019). · DOI · Paper page
  54. Yélamos, J., Moreno-Lama, L., Jimeno, J. & Ali, S. O. Immunomodulatory roles of PARP-1 and PARP-2: impact on PARP-centered cancer therapies. Cancers 12, 392 (2020). · DOI
  55. Ragab, H. M., Samy, N., Afify, M., El Maksoud, N. A. & Shaaban, H. M. Assessment of Ki-67 as a potential biomarker in patients with breast cancer. J. Genet. Eng. Biotechnol. 16, 479–484 (2018). · DOI
  56. Davis, A. A. & Patel, V. G. The role of PD-L1 expression as a predictive biomarker: an analysis of all US Food and Drug Administration (FDA) approvals of immune checkpoint inhibitors. J. Immunother. Cancer 7, 278 (2019). · DOI · Paper page
  57. Li, S. et al. Cancer immunotherapy via targeted TGF-β signalling blockade in TH cells. Nature https://doi.org/10.1038/s41586-020-2850-3 (2020). · DOI
  58. Kalra, S. et al. Pan-cancer diagnostic consensus through searching archival histopathology images using artificial intelligence. NPJ Digital Med. 3, 31 (2020). · DOI
  59. Lu, S. et al. Comparison of biomarker modalities for predicting response to PD-1/PD-L1 checkpoint blockade: a systematic review and meta-analysis. JAMA Oncol. https://doi.org/10.1001/jamaoncol.2019.1549 (2019). · DOI · Paper page
  60. Selvaraju, R. R. et al. Grad-CAM: visual explanations from deep networks via gradient-based localization. Int. J. Comput. Vis. 128, 336–359 (2020). · DOI · Paper page
  61. Sadimin, E. T. & Foran, D. J. Pathology imaging informatics for clinical practice and investigative and translational research. North Am. J. Med. Sci. 5, 103–109 (2012). · DOI
  62. Benjamini, Y. & Hochberg, Y. Controlling the false discovery rate: a practical and powerful approach to multiple testing. J. R. Stat. Soc. Ser. B Stat. Methodol. 57, 289–300 (1995). · DOI · Paper page
  63. Knijnenburg, T. A. et al. Genomic and molecular landscape of DNA damage repair deficiency across The Cancer Genome Atlas. Cell Rep. 23, 239–254.e6 (2018). · DOI · Paper page
  64. McKinney, B. A., Reif, D. M., Ritchie, M. D. & Moore, J. H. Machine learning for detecting gene-gene interactions: a review. Appl. Bioinforma. 5, 77–88 (2006). · DOI · Paper page
  65. Carpenter, J. & Bithell, J. Bootstrap confidence intervals: when, which, what? A practical guide for medical statisticians. Stat. Med. 19, 1141–1164 (2000). · doi:10.1002/(sici)1097-0258(20000515)19:9<1141::aid-sim479>3.0.co;2-f

Cited by

Metadata

Authors
Diao, J. A., Wang, J. K., Chui, W. F., Mountain, V., Gullapally, S. C., Srinivasan, R., Mitchell, R. N., Glass, B., Hoffman, S., Rao, S. K., Maheshwari, C., Lahiri, A., Prakash, A., McLoughlin, R., Kerner, J. K., Resnick, M. B., Montalto, M. C., Khosla, A., Wapinski, I. N., Beck, A. H., Elliott, H. L., & Taylor-Weiner, A.
Venue
Nature Communications
Publisher
Springer Science and Business Media LLC
Published
2021-03-12
Volume
12
Issue
1
Pages
1613-1613
ISSN
2041-1723 (ISSN-L)
DOI
10.1038/s41467-021-21896-9
OpenAlex
W3133782060
Cited by
269
References
26
FWCI
14.73
Type
journal article
Language
en
Open access
gold
Licence

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.