A survey on graph-based deep learning for computational histopathology
Open accessThe DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
Abstract
With the remarkable success of representation learning for prediction problems, we have witnessed a rapid expansion of the use of machine learning and deep learning for the analysis of digital pathology and biopsy image patches. However, learning over patch-wise features using convolutional neural networks limits the ability of the model to capture global contextual information and comprehensively model tissue composition. The phenotypical and topological distribution of constituent histological entities play a critical role in tissue diagnosis. As such, graph data representations and deep learning have attracted significant attention for encoding tissue representations, and capturing intra- and inter- entity level interactions. In this review, we provide a conceptual grounding for graph analytics in digital pathology, including entity-graph construction and graph architectures, and present their current success for tumor localization and classification, tumor invasion and staging, image retrieval, and survival prediction. We provide an overview of these methods in a systematic manner organized by the graph representation of the input image, scale, and organ on which they operate. We also outline the limitations of existing techniques, and suggest potential future research directions in this domain.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.74 · Cited by 24,513
A survey on deep learning in medical image analysis
Litjens, G., Kooi, T., Bejnordi, B. E., Setio, A. A. A., Ciompi, F., Ghafoorian, M., van der Laak, J. A. W. M., van Ginneken, B., & Sánchez, C. I. (2017) · Medical Image Analysis · doi:10.1016/j.media.2017.07.005 · Cited by 15,332
Comprehensive molecular portraits of human breast tumours
The Cancer Genome Atlas Network (2012) · Nature · doi:10.1038/nature11412 · Cited by 12,129 · cited 1×
“The performance of this method is evaluated on the hematoxylin and eosin (H&E) stained WSI images from the TCGA-BRCA [94] dataset, which consist of 608 HER2 negative and 101 HER2 positive, and 452 PR positive and 256 PR negative samples.”
MethodVeličković, P., Cucurull, G., Casanova, A., Romero, A., Liò, P., & Bengio, Y. (2017) · arXiv · doi:10.48550/arxiv.1710.10903 · Cited by 12,107
A Comprehensive Survey on Graph Neural Networks
Wu, Z., Pan, S., Chen, F., Long, G., Zhang, C., & Yu, P. S. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2978386 · Cited by 11,000
Inductive Representation Learning on Large Graphs
Hamilton, W. L., Ying, R., & Leskovec, J. (2017) · arXiv · doi:10.48550/arxiv.1706.02216 · Cited by 10,639
The graph neural network model
Scarselli, F., Gori, M., Tsoi, A. C., Hagenbuchner, M., & Monfardini, G. (2009) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2008.2005605 · Cited by 9,582 · cited 1×
“Based on the original graph neural networks in [52], we introduce the most representative GNN variants that have been proposed for computational pathology.”
MethodRepresentation Learning with Contrastive Predictive Coding
Oord, A. V. D., Li, Y., & Vinyals, O. (2018) · arXiv · doi:10.48550/arxiv.1807.03748 · Cited by 6,242
How Powerful are Graph Neural Networks?
Xu, K., Hu, W., Leskovec, J., & Jegelka, S. (2018) · arXiv · doi:10.48550/arxiv.1810.00826 · Cited by 6,078
Shuman, D. I., Narang, S. K., Frossard, P., Ortega, A., & Vandergheynst, P. (2013) · IEEE Signal Processing Magazine · doi:10.1109/msp.2012.2235192 · Cited by 4,421 · cited 1×
“Commonly used methods to determine the entries, w ij , of W include Pearson correlation-based graph, the K-nearest neighbor (KNN) method, and the distance-based graph [48].”
MethodMutational landscape and significance across 12 major cancer types
Kandoth, C., McLellan, M. D., Vandin, F., Ye, K., Niu, B., Lu, C., Xie, M., Zhang, Q., McMichael, J. F., Wyczalkowski, M. A., et al. (2013) · Nature · doi:10.1038/nature12634 · Cited by 4,419
Grad-CAM++: Generalized Gradient-Based Visual Explanations for Deep Convolutional Networks
Chattopadhay, A., Sarkar, A., Howlader, P., & Balasubramanian, V. N. (2018) · 2018 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv.2018.00097 · Cited by 3,639
Clinical-grade computational pathology using weakly supervised deep learning on whole slide images
Campanella, G., Hanna, M. G., Geneslaw, L., Miraflor, A., Werneck Krauss Silva, V., Busam, K. J., Brogi, E., Reuter, V. E., Klimstra, D. S., & Fuchs, T. J. (2019) · Nature Medicine · doi:10.1038/s41591-019-0508-1 · Cited by 2,799
Data-efficient and weakly supervised computational pathology on whole-slide images
Lu, M. Y., Williamson, D. F. K., Chen, T. Y., Chen, R. J., Barbieri, M., & Mahmood, F. (2021) · Nature Biomedical Engineering · doi:10.1038/s41551-020-00682-w · Cited by 2,282
Artificial intelligence in digital pathology — new tools for diagnosis and precision oncology
Bera, K., Schalper, K. A., Rimm, D. L., Velcheti, V., & Madabhushi, A. (2019) · Nature Reviews Clinical Oncology · doi:10.1038/s41571-019-0252-y · Cited by 1,698 · cited 1×
“cess to massive digital datasets of tissue images, deep learning methods for tumor localization, survival prediction and cancer recurrence prediction have made substantial progress [78].”
BackgroundGraph Networks as a Universal Machine Learning Framework for Molecules and Crystals
Chen, C., Ye, W., Zuo, Y., Zheng, C., & Ong, S. P. (2019) · Chemistry of Materials · doi:10.1021/acs.chemmater.9b01294 · Cited by 1,552
Veličković, P., Fedus, W., Hamilton, W. L., Liò, P., Bengio, Y., & Hjelm, R. D. (2018) · arXiv · doi:10.48550/arxiv.1809.10341 · Cited by 1,510
Kather, J. N., Pearson, A. T., Halama, N., Jäger, D., Krause, J., Loosen, S. H., Marx, A., Boor, P., Tacke, F., Neumann, U. P., et al. (2019) · Nature Medicine · doi:10.1038/s41591-019-0462-y · Cited by 1,458 · cited 1×
“In addition to the above, extensive research over past years in deep learning [140]–[142] showed that a decision classifier based on Multiple Instance Learning (MIL) can boost the performance in classifying cancer by aggregating instance-level predictions.”
BackgroundTechniques for Interpretable Machine Learning
Du, M., Liu, N., & Hu, X. (2018) · arXiv · doi:10.48550/arxiv.1808.00033 · Cited by 1,261
Deep learning in histopathology: the path to the clinic
van der Laak, J., Litjens, G., & Ciompi, F. (2021) · Nature Medicine · doi:10.1038/s41591-021-01343-4 · Cited by 902
GNNExplainer: Generating Explanations for Graph Neural Networks
Ying, R., Bourgeois, D., You, J., Zitnik, M., & Leskovec, J. (2019) · arXiv · doi:10.48550/arxiv.1903.03894 · Cited by 850
Deep neural network models for computational histopathology: A survey
Srinidhi, C. L., Ciga, O., & Martel, A. L. (2021) · Medical Image Analysis · doi:10.1016/j.media.2020.101813 · Cited by 749
Chen, R. J., Lu, M. Y., Wang, J., Williamson, D. F. K., Rodig, S. J., Lindeman, N. I., & Mahmood, F. (2022) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2020.3021387 · Cited by 729
Explainable Deep Learning Models in Medical Image Analysis
Singh, A., Sengupta, S., & Lakshminarayanan, V. (2020) · Journal of Imaging · doi:10.3390/jimaging6060052 · Cited by 718 · cited 1×
“Thus, more work should be conducted to investigate how a pathologist could refine a graph model decision via a human-in-the-loop system [128], [129] Such approaches provide an important safety mechanism for detecting and correcting algorithmic errors that may occur.”
BackgroundGRAM: Graph-based Attention Model for Healthcare Representation Learning
Choi, E., Bahadori, M. T., Song, L., Stewart, W. F., & Sun, J. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098126 · Cited by 637 · cited 1×
“The hierarchical information inherent in medical ontologies naturally lends itself to creating a rich network of medical knowledge, and other data types such as symptoms and genomics [127].”
BackgroundExplainability Methods for Graph Convolutional Neural Networks
Pope, P. E., Kolouri, S., Rostami, M., Martin, C. E., & Hoffmann, H. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01103 · Cited by 553
Deep learning in microscopy image analysis: a survey
Xing, F., Xie, Y., Su, H., Liu, F., & Yang, L. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2017.2766168 · Cited by 460
Parameterized Explainer for Graph Neural Network
Luo, D., Cheng, W., Xu, D., Yu, W., Zong, B., Chen, H., & Zhang, X. (2020) · arXiv · doi:10.48550/arxiv.2011.04573 · Cited by 355
What Clinicians Want: Contextualizing Explainable Machine Learning for Clinical End Use
Tonekaboni, S., Joshi, S., McCradden, M. D., & Goldenberg, A. (2019) · arXiv · doi:10.48550/arxiv.1905.05134 · Cited by 318
CGC-Net: Cell Graph Convolutional Network for Grading of Colorectal Cancer Histology Images
Zhou, Y., Graham, S., Alemi Koohbanani, N., Shaban, M., Heng, P.-A., & Rajpoot, N. (2019) · 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW) · doi:10.1109/iccvw.2019.00050 · Cited by 225
Context-aware convolutional neural network for grading of colorectal cancer histology images
Shaban, M., Awan, R., Fraz, M. M., Azam, A., Tsang, Y.-W., Snead, D., & Rajpoot, N. M. (2020) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2020.2971006 · Cited by 179
Yottixel – An Image Search Engine for Large Archives of Histopathology Whole Slide Images
Kalra, S., Tizhoosh, H. R., Choi, C., Shah, S., Diamandis, P., Campbell, C. J. V., & Pantanowitz, L. (2020) · Medical Image Analysis · doi:10.1016/j.media.2020.101757 · Cited by 139 · cited 2×
“In this approach, a DenseNet is used to extract features from all important patches sampled from a segmented tissue using color thresholds [57].”
MethodRepresentation Learning of Histopathology Images using Graph Neural Networks
Adnan, M., Kalra, S., & Tizhoosh, H. R. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00502 · Cited by 109 · cited 7×
“Only one approach to date has computed the connections between nodes by using an adjacency learning layer in an end-to-end manner that considered the global context of all patches [43].”
BackgroundMethodInfluentialHistopathological Whole Slide Image Analysis Using Context-Based CBIR
Zheng, Y., Jiang, Z., Zhang, H., Xie, F., Ma, Y., Shi, H., & Zhao, Y. (2018) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2018.2796130 · Cited by 109
Histographs: graphs in histopathology
Anand, D., Gadiya, S., & Sethi, A. (2020) · Medical Imaging 2020: Digital Pathology · doi:10.1117/12.2550114 · Cited by 53 · cited 7×
“Recreated from [28]. mechanism that can be learned by gradient-based methods [82] can be formulated as, where h t is the output of a layer; and W , u w and b are trainable weights and bias.”
BackgroundMethodInfluentialChen, T., Wang, Z., Li, L., Li, D., Li, Z., Chang, X., Bian, P., Liang, G., Wang, Q., & Xie, T. (2025) · Schloss Dagstuhl – Leibniz-Zentrum für Informatik · doi:10.4230/lipics.ecoop.2025.6
Cited by
Artificial intelligence for multimodal data integration in oncology
Lipkova, J., Chen, R. J., Chen, B., Lu, M. Y., Barbieri, M., Shao, D., Vaidya, A. J., Chen, C., Zhuang, L., Williamson, D. F. K., et al. (2022) · Cancer Cell · doi:10.1016/j.ccell.2022.09.012 · Cited by 777
Artificial intelligence for digital and computational pathology
Song, A. H., Jaume, G., Williamson, D. F. K., Lu, M. Y., Vaidya, A., Miller, T. R., & Mahmood, F. (2023) · Nature Reviews Bioengineering · doi:10.1038/s44222-023-00096-8 · Cited by 356
Graph representation learning in biomedicine and healthcare
Li, M. M., Huang, K., & Zitnik, M. (2022) · Nature Biomedical Engineering · doi:10.1038/s41551-022-00942-x · Cited by 294
“Biomedically focused review articles have surveyed the use of GNNs for molecular generation 33,34 , single-cell biology 35 , drug discovery and drug repurposing 36–40 , and histopathology 41 .”
Graph-Based Deep Learning for Medical Diagnosis and Analysis: Past, Present and Future
Ahmedt-Aristizabal, D., Armin, M. A., Denman, S., Fookes, C., & Petersson, L. (2021) · Sensors · doi:10.3390/s21144758 · Cited by 272
Self-citationMultimodal data integration for oncology in the era of deep neural networks: a review
Waqas, A., Tripathi, A., Ramachandran, R. P., Stewart, P. A., & Rasool, G. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1408843 · Cited by 165
Metadata
- Authors
- Ahmedt-Aristizabal, D., Armin, M. A., Denman, S., Fookes, C., & Petersson, L.
- Venue
- Computerized Medical Imaging and Graphics
- Publisher
- Elsevier BV
- Published
- 2022-01
- Volume
- 95
- Pages
- 102027
- ISSN
- 0730-4862 (ISSN-L)
- 0895-6111
- OpenAlex
- W3175191608
- Cited by
- 147
- References
- 38
- FWCI
- 7.50
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.