Cited by 400AI in cancer detectionRadiomics and Machine Learning in Medical ImagingMultimodal Machine Learning Applications

The DOI matches its Crossref record; 73 of the 75 references it cites were found in a registry or an index.

Multimodal Co-Attention Transformer for Survival Prediction in Gigapixel Whole Slide Images

Chen, Richard J., Lu, Ming Y., Weng, Wei-Hung, Chen, Tiffany Y., Williamson, Drew FK., Manz, Trevor, Shady, Maha, Mahmood, Faisal·2021·2021 IEEE/CVF International Conference on Computer Vision (ICCV)

DOI 10.1109/iccv48922.2021.00398

Open this paper in CoChat — free account

Abstract

Survival outcome prediction is a challenging weakly-supervised and ordinal regression task in computational pathology that involves modeling complex interactions within the tumor microenvironment in gigapixel whole slide images (WSIs). Despite recent progress in formulating WSIs as bags for multiple instance learning (MIL), representation learning of entire WSIs remains an open and challenging problem, especially in overcoming: 1) the computational complexity of feature aggregation in large bags, and 2) the data heterogeneity gap in incorporating biological priors such as genomic measurements. In this work, we present a Multimodal Co-Attention Transformer (MCAT) framework that learns an interpretable, dense co-attention mapping between WSIs and genomic features formulated in an embedding space. Inspired by approaches in Visual Question Answering (VQA) that can attribute how word embed-dings attend to salient objects in an image when answering a question, MCAT learns how histology patches attend to genes when predicting patient survival. In addition to visualizing multimodal interactions, our co-attention trans-formation also reduces the space complexity of WSI bags, which enables the adaptation of Transformer layers as a general encoder backbone in MIL. We apply our proposed method on five different cancer datasets (4,730 WSIs, 67 million patches). Our experimental results demonstrate that the proposed method consistently achieves superior performance compared to the state-of-the-art methods.

Integrity signals

Registry

Crossref · 10.1109/iccv48922.2021.00398

Cites

73 of 73 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Attention Is All You Need
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
MizAR 60 for Mizar 50
Jakubův, J., Chvalovský, K., Goertzel, Z., et al. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,648
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Gene set enrichment analysis: A knowledge-based approach for interpreting genome-wide expression profiles
Subramanian, A., Tamayo, P., Mootha, V. K., Mukherjee, S., Ebert, B. L., Gillette, M. A., Paulovich, A., Pomeroy, S. L., Golub, T. R., Lander, E. S., et al. (2005) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0506580102 · Cited by 53,625
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472
Regression Models and Life-Tables
Cox, D. R. (1972) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1972.tb00899.x · Cited by 39,793
Regression Models and Life-Tables
Cox, D. R. (1992) · Springer Series in Statistics · doi:10.1007/978-1-4612-4380-9_37 · Cited by 26,297
Graph Attention NetworksPreprint
Veličković, P., Casanova, A., Liò, P., Cucurull, G., Romero, A., & Bengio, Y. (2017) · arXiv (Cornell University) · doi:10.17863/cam.48429 · Cited by 19,745
The 2016 World Health Organization Classification of Tumors of the Central Nervous System: a summary
Louis, D. N., Perry, A., Reifenberger, G., von Deimling, A., Figarella‐Branger, D., Cavenee, W. K., Ohgaki, H., Wiestler, O. D., Kleihues, P., & Ellison, D. W. (2016) · Acta Neuropathologica · doi:10.1007/s00401-016-1545-1 · Cited by 15,703

Show all 75 references

Research works printed but not linked: 1

zaheer, 2017, Deep sets, Advances in neural information processing systems

Sorted by citations

Items 101–200 of 400

MGCM: Multi-modal graph convolutional mamba for cancer survival prediction
Cui, J., Li, Y., Shen, D. G., & Wang, Y. (2025) · Pattern Recognition · doi:10.1016/j.patcog.2025.111991 · Cited by 20
Large-vocabulary forensic pathological analyses via prototypical cross-modal contrastive learning
Shen, C., Lian, C., Zhang, W., Wang, F., Zhang, J., Fan, S., Wei, X., Wang, G., Li, K., Mu, H., et al. (2025) · Nature Communications · doi:10.1038/s41467-025-62060-x · Cited by 18
Improving diagnosis and outcome prediction of gastric cancer via multimodal learning using whole slide pathological images and gene expression
Xie, Y., Sang, Q., Da, Q., Niu, G., Deng, S., Feng, H., Chen, Y., Li, Y., Liu, B., Yang, Y., et al. (2024) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2024.102871 · Cited by 16
Unsupervised mutual transformer learning for multi-gigapixel Whole Slide Image classification
Javed, S., Mahmood, A., Qaiser, T., Werghi, N., & Rajpoot, N. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103203 · Cited by 16
Rotation-Agnostic Image Representation Learning for Digital Pathology
Alfasly, S., Shafique, A., Nejat, P., Khan, J., Alsaafin, A., Alabtah, G., & Tizhoosh, H. R. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01110 · Cited by 21
Clinical utility of receptor status prediction in breast cancer and misdiagnosis identification using deep learning on hematoxylin and eosin-stained slides
Shamai, G., Schley, R., Cretu, A., Neoran, T., Sabo, E., Binenbaum, Y., Cohen, S., Goldman, T., Polónia, A., Drumea, K., et al. (2024) · Communications Medicine · doi:10.1038/s43856-024-00695-5 · Cited by 15
Cross-modal alignment and contrastive learning for enhanced cancer survival prediction
Li, T., Zhou, X., Xue, J., Zeng, L., Zhu, Q., Wang, R., Yu, H., & Xia, J. (2025) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2025.108633 · Cited by 15
Robust Multimodal Survival Prediction with Conditional Latent Differentiation Variational AutoEncoder
Zhou, J., Tang, J., Zuo, Y., Wan, P., Zhang, D., & Shao, W. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00971 · Cited by 21
Transformer-based personalized attention mechanism for medical images with clinical records
Takagi, Y., Hashimoto, N., Masuda, H., Miyoshi, H., Ohshima, K., Hontani, H., & Takeuchi, I. (2023) · Journal of Pathology Informatics · doi:10.1016/j.jpi.2022.100185 · Cited by 14
Comprehensive learning and adaptive teaching: Distilling multi-modal knowledge for pathological glioma grading
Xing, X., Zhu, M., Chen, Z., & Yuan, Y. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.102990 · Cited by 15
Point Transformer with Federated Learning for Predicting Breast Cancer HER2 Status from Hematoxylin and Eosin-Stained Whole Slide Images
Li, B., Liu, Z., Shao, L., Qiu, B., Bu, H., & Tian, J. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i4.28082 · Cited by 16
Tumor Micro-Environment Interactions Guided Graph Learning for Survival Analysis of Human Cancers from Whole-Slide Pathological Images
Shao, W., Shi, Y., Zhang, D., Zhou, J., & Wan, P. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01111 · Cited by 16
Survival Prediction for Gastric Cancer via Multimodal Learning of Whole Slide Images and Gene Expression
Xie, Y., Niu, G., Da, Q., Dai, W., & Yang, Y. (2022) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm55620.2022.9995480 · Cited by 15
Multimodal multi-instance evidence fusion neural networks for cancer survival prediction
Luo, H., Huang, J., Ju, H., Zhou, T., & Ding, W. (2025) · Scientific Reports · doi:10.1038/s41598-025-93770-3 · Cited by 20
Gradient modulated contrastive distillation of low-rank multi-modal knowledge for disease diagnosis
Xing, X., Chen, Z., Hou, Y., & Yuan, Y. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.102874 · Cited by 14
Histopathologic brain age estimation via multiple instance learning
Marx, G. A., Kauffman, J., McKenzie, A. T., Koenigsberg, D., McMillan, C. T., Morgello, S., Karlovich, E., Insausti, R., Richardson, T. E., Walker, J. M., et al. (2023) · Acta Neuropathologica · doi:10.1007/s00401-023-02636-3 · Cited by 19
Cross-modality attention-based multimodal fusion for non-small cell lung cancer (NSCLC) patient survival prediction
Deng, R., Shaikh, N., Shannon, G., & Nie, Y. (2024) · SPIE Medical Imaging: Digital and Computational Pathology · doi:10.1117/12.3006036 · Cited by 16
Adversarial attacks and adversarial robustness in computational pathologyPreprint
Laleh, N. G., Truhn, D., Veldhuizen, G. P., Han, T., van Treeck, M., Buelow, R. D., Langer, R., Dislich, B., Boor, P., Schulz, V., et al. (2022) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2022.03.15.484515 · Cited by 14
Multimodal fusion strategies for survival prediction in breast cancer: A comparative deep learning study
Sucre, A., Sánchez, X. C., Perez-Herrera, L. V., Vivanco, M. D., González, M. J. G., López‐Linares, K., Calvo, B., & Garín-Muga, A. (2025) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2025.10.038 · Cited by 17
Position Paper: Artificial Intelligence in Medical Image Analysis: Advances, Clinical Translation, and Emerging Frontiers
Panayides, A. S., Chen, H., Filipović, N., Geroski, T., Hou, J., Lekadir, K., Kostas Marias, A., Matsopoulos, G. K., Papanastasiou, G., Sarder, P., et al. (2025) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2025.3649496 · Cited by 14
MRI and RNA-seq fusion for prediction of pathological response to neoadjuvant chemotherapy in breast cancer
Li, H., Zhao, Y., Duan, J., Gu, J., Liu, Z., Zhang, H., Zhang, Y., & Li, Z. (2024) · Displays · doi:10.1016/j.displa.2024.102698 · Cited by 14
Histopathology and proteomics are synergistic for high-grade serous ovarian cancer platinum response prediction
Kilim, O., Olar, A., Biricz, A., Madaras, L., Pollner, P., Szállási, Z., Sztupinszki, Z. M., & Csabai, I. (2025) · npj Precision Oncology · doi:10.1038/s41698-025-00808-w · Cited by 13
Gene-Induced Multimodal Pre-training for Image-Omic Classification
Jin, T., Xie, X., Wan, R., Li, Q., & Wang, Y. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-43987-2_49 · Cited by 12
From Pixels to Prognosis: A Survey on AI-Driven Cancer Patient Survival Prediction Using Digital Histology Images
Parvaiz, A., Nasir, E. S., & Fraz, M. M. (2024) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-024-01049-2 · Cited by 12
Prognosis prediction based on liver histopathological image via graph deep learning and transformer
Zhang, J., Sun, Z., Wang, K., Wang, C., Cheng, S., Jiang, Y. Y., & Bai, Q. (2024) · Applied Soft Computing · doi:10.1016/j.asoc.2024.111653 · Cited by 14
Pathology-Knowledge Enhanced Multi-instance Prompt Learning for Few-Shot Whole Slide Image Classification
Qu, L., Yang, D., Huang, D., Guo, Q., Luo, R., Zhang, S., & Wang, X. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73247-8_12 · Cited by 16
HiViT: Hierarchical attention-based Transformer for multi-scale whole slide histopathological image classification
Yu, J., Li, S., Tan, L., Zhou, H., Li, J., & Li, J. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.127164 · Cited by 14
MM-ALT
Gu, X., Ou, L., Ong, D., & Wang, Y. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3548411 · Cited by 15
HGMSurvNet: A two-stage hypergraph learning network for multimodal cancer survival prediction
Ding, S., Li, L., Jin, G., Wang, J., Ying, S., & Shi, J. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103661 · Cited by 11
Attention-based multimodal fusion transformer for predicting the efficacy of neoadjuvant therapy in breast cancer: a cross-institutional retrospective study
Zhang, W., Zhang, S., You, J., Li, F., Wu, X., Lu, X., Lv, Q., Huang, J., Yi, Y., & Bu, H. (2025) · Breast Cancer Research · doi:10.1186/s13058-025-02181-9 · Cited by 13
Vision Transformer-Based Multilabel Survival Prediction for Oropharynx Cancer After Radiation Therapy
Chen, M., Wang, K., & Wang, J. (2023) · International Journal of Radiation Oncology*Biology*Physics · doi:10.1016/j.ijrobp.2023.10.022 · Cited by 11
NACNet: A histology context-aware transformer graph convolution network for predicting treatment response to neoadjuvant chemotherapy in Triple Negative Breast Cancer
Li, Q., Teodoro, G., Jiang, Y., & Kong, J. (2024) · Computerized Medical Imaging and Graphics · doi:10.1016/j.compmedimag.2024.102467 · Cited by 12
From Single-Cancer to Pan-Cancer Prognosis
Zhang, B., Li, S., Jian, J., Ren, X., Zhao, Z., Guo, L., Su, F., Meng, Z., & Zhao, Z. (2025) · American Journal Of Pathology · doi:10.1016/j.ajpath.2025.06.006 · Cited by 14
Cross-Attention-Based Saliency Inference for Predicting Cancer Metastasis on Whole Slide Images
Su, Z., Rezapour, M., Sajjad, U., Niu, S., Gurcan, M. N., & Niazi, M. K. K. (2024) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2024.3439499 · Cited by 10
Artificial Intelligence-based agents in chronic liver disease: transforming diagnostic and therapeutic workflows through clinical decision-making
Singh, Y., Hathaway, Q. A., Vera-Garcia, D. V., Povero, D., Salehi, S., Wei, Y., Horvat, N., Chaudhary, A., Kh, A. M., Keishing, V., et al. (2025) · npj gut and liver. · doi:10.1038/s44355-025-00049-5 · Cited by 10
Multi-modal Data Binding for Survival Analysis Modeling with Incomplete Data and Annotations
Qu, L., Huang, D., Zhang, S., & Wang, X. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72086-4_47 · Cited by 10
Transformers in pathological image analysis: A survey
Liu, L., Liu, Z., Xie, J., Zhang, H., Chang, J., & Qiao, H. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.111114 · Cited by 11
cDP-MIL: Robust Multiple Instance Learning via Cascaded Dirichlet Process
Chen, Y., Chan, T. H., Yin, G., Jiang, Y., & Yu, L. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72949-2_14 · Cited by 9
MSMMIL: Multi-scan Mamba-based Multiple Instance Learning for whole slide image classification
Zhong, H., Ding, M., Zhao, C., Zhang, Y., Wang, T., & Lei, B. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113871 · Cited by 10
Integrating images and genomics for multi-modal cancer survival analysis via mixture of experts
Zhang, W., Xu, W., Chen, T., Sakal, C., & Li, X. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103521 · Cited by 12
Self-citation
Recent Advances of Multimodal Continual Learning: A Comprehensive Survey
Yu, D., Zhang, X., Chen, Y., Liu, A., Zhang, Y., Yu, P. S., & King, I. (2026) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2026.3658485 · Cited by 52
MIF: Multi-Shot Interactive Fusion Model for Cancer Survival Prediction Using Pathological Image and Genomic Data
Shi, Y., Wang, M., Liu, H., Zhao, F., Li, A., & Chen, X. (2024) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2024.3363161 · Cited by 10
MoMA: Momentum contrastive learning with multi-head attention-based knowledge distillation for histopathology image analysis
Vuong, T. T. L., & Kwak, J. T. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103421 · Cited by 13
$$M^{2}$$Fusion: Bayesian-Based Multimodal Multi-level Fusion on Colorectal Cancer Microsatellite Instability Prediction
Liu, Q., Yao, J., Yao, L., Chen, X., Zhou, J., Lü, L., Zhang, L., Liu, Z., & Huo, Y. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-47425-5_12 · Cited by 7
CAF-AHGCN: context-aware attention fusion adaptive hypergraph convolutional network for human-interpretable prediction of gigapixel whole-slide image
Liang, M., Jiang, X. Y., Cao, J., Li, B., Wang, L., Chen, Q., Zhang, C., & Zhao, Y. (2024) · The Visual Computer · doi:10.1007/s00371-024-03269-7 · Cited by 8
Unleash the Power of State Space Model for Whole Slide Image With Local Aware Scanning and Importance Resampling
Huang, Y., Zhao, W., Fu, Y., Zhu, L., & Yu, L. (2024) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2024.3475587 · Cited by 10
Multimodal Cross-Task Interaction for Survival Analysis in Whole Slide Pathological Images
Jiang, S., Gan, Z., Cai, L., Wang, Y., & Zhang, Y. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72083-3_31 · Cited by 9
Multimodal Fusion Framework Based on Low-Rank Interaction for Tumor Prognostic Prediction
An, Y., Lan, R., Lin, H., Wang, Y., Deng, H., Liu, Z., Wang, L., Liu, Z., Lu, C. Y., Yang, H., et al. (2025) · IEEE Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbbio.2025.3578334 · Cited by 8
Hist2Cell: Deciphering fine-grained cellular architectures from histology images
Zhao, W., Liang, Z., Huang, X., Huang, Y., & Yu, L. (2026) · Cell Genomics · doi:10.1016/j.xgen.2025.101137 · Cited by 8
MuGI: Multi-Granularity Interactions of Heterogeneous Biomedical Data for Survival Prediction
Long, L., Cui, J., Zeng, P., Li, Y., Liu, Y., & Wang, Y. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72086-4_46 · Cited by 7
Diagnostic classification in toxicologic pathology using attention-guided weak supervision and whole slide image features: a pilot study in rat livers
Zehnder, P., Feng, J., Nguyen, T., Shen, P., Sullivan, R., Fuji, R. N., & Hu, F. (2025) · Scientific Reports · doi:10.1038/s41598-025-86615-6 · Cited by 7
Generating crossmodal gene expression from cancer histopathology improves multimodal AI predictions
Dey, S., Banerji, C. R. S., Basuchowdhuri, P., Saha, S. K., Parashar, D., & Chakraborti, T. (2025) · Nature Communications · doi:10.1038/s41467-025-66961-9 · Cited by 11
Multi-modal Pathological Pre-training via Masked Autoencoders for Breast Cancer Diagnosis
Lu, M., Wang, T., & Xia, Y. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-43987-2_44 · Cited by 8
Self-citation
Masked autoencoders with handcrafted feature predictions: Transformer for weakly supervised esophageal cancer classification
Bai, Y., Li, W., An, J., Xia, L., Chen, H., Zhao, G., & Gao, Z. (2023) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2023.107936 · Cited by 8
Path-Gptomic: A Balanced Multi-Modal Learning Framework For Survival Outcome Prediction
Wang, H., Yang, Y., Zhao, Z., Gu, P., Sapkota, N., & Chen, D. Z. (2024) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi56570.2024.10635171 · Cited by 7
SAMamba: Integrating State Space Model for Enhanced Multi-modal Survival Analysis
Zhang, W., Chen, T., Xu, W., & Li, X. (2024) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm62325.2024.10822595 · Cited by 8
Self-citation
A novel multimodal deep learning framework for predicting residual strength of corroded rectangular hollow-section columns
Zhang, Y., Zhang, L., Zhou, Y., Lincoln, R. L., Lincoln, R. L., Tong, J., & Shen, J. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.110554 · Cited by 7
SurvMamba: State Space Model with Multi-Grained Multi-Modal Interaction for Survival Prediction
Chen, Y., Xie, J., Lin, Y., Song, Y., Zhang, C., Yang, W., & Yu, R. (2025) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm66473.2025.11356727 · Cited by 16
Potential to Enhance Large Scale Molecular Assessments of Skin Photoaging through Virtual Inference of Spatial Transcriptomics from Routine Staining
Srinivasan, G., Davis, M., LeBoeuf, M. R., Fatemi, M., Azher, Z., Lu, Y., Diallo, A., Montivero, M. K. S., Kolling, F., Perrard, L., et al. (2023) · Pacific Symposium on Biocomputing (PSB) · doi:10.1142/9789811286421_0037 · Cited by 7
TAD-Graph: Enhancing Whole Slide Image Analysis via Task-Aware Subgraph Disentanglement
Wang, F., Xin, J., Zhao, W., Jiang, Y., Yeung, M. C. F., Wang, L., & Yu, L. (2025) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2025.3545680 · Cited by 6
Classification of Whole-Slide Pathology Images Based on State Space Models and Graph Neural Networks
Ding, F., Cai, C., Li, J., Liu, M., Jiao, Y., Wu, Z., & Xu, J. (2025) · Electronics · doi:10.3390/electronics14102056 · Cited by 6
The potential and challenges of large language model agent systems in chemical process simulation: from automated modeling to intelligent design
Du, W., & Yang, S. (2025) · Frontiers of Chemical Science and Engineering · doi:10.1007/s11705-025-2587-5 · Cited by 6
Disentangled and Interpretable Multimodal Attention Fusion for Cancer Survival Prediction
Eijpe, A., Lakbir, S., Cesur, M. E., Oliveira, S. P., Abeln, S., & Silva, W. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-05185-1_12 · Cited by 7
Multiple Instance Learning Framework with Masked Hard Instance Mining for Gigapixel Histopathology Image Analysis
Tang, W., Huang, S., Fang, H., Zhou, F., Liu, B., & Liu, Q. (2026) · International Journal of Computer Vision · doi:10.1007/s11263-025-02587-0 · Cited by 6
Multimodal exponential moving average-guided representation for integrated tumor-segmentation and survival prediction
Datta, A., Sarkar, S., Sharma, A. L., & Ghosal, P. (2026) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2026.113868 · Cited by 6
Assessing the performance of fully supervised and weakly supervised learning in breast cancer histopathology
Kang, H., Xu, Q., Chen, D., Ren, S., Xie, H. M., Wang, L., Gao, Y., Gong, M., & Chen, X. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.121575 · Cited by 7
Translating prognostic quantification of c-MYC and BCL2 from tissue microarrays to whole slide images in diffuse large B-cell lymphoma using deep learning
Tavolara, T. E., Niazi, M. K. K., Feldman, A. L., Jaye, D. L., Flowers, C. R., Cooper, L., & Gürcan, M. N. (2024) · Diagnostic Pathology · doi:10.1186/s13000-023-01425-6 · Cited by 6
Knowledge-Driven Subspace Fusion and Gradient Coordination for Multi-modal Learning
Zhang, Y., Wang, X., Meng, F., Tang, J. H., & Li, C. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72083-3_25 · Cited by 6
Joint modeling histology and molecular markers for cancer classification
Wang, X., Liu, H., Zhang, Y., Zhao, B., Duan, H., Hu, W., Mou, Y., Price, S., & Li, C. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103505 · Cited by 6
$$\mathrm {M^{2}F}$$: A Multi-modal and Multi-task Fusion Network for Glioma Diagnosis and Prognosis
Lu, Z., Lu, M., & Xia, Y. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-18814-5_1 · Cited by 6
Self-citation
An Explainable Coarse-to-Fine Survival Analysis Method on Multi-Center Whole Slide Images
Wang, H., Jiang, D., Zhang, H., Wang, Y., Yang, L., Kerr, D. J., & Zhang, Y. (2023) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2023.3285855 · Cited by 6
Improved Multimodal Fusion for Small Datasets with Auxiliary Supervision
Holste, G., van der Wal, D., Pinckaers, H., Yamashita, R., Mitani, A., & Esteva, A. (2023) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi53787.2023.10230356 · Cited by 6
HC-MAE: Hierarchical Cross-attention Masked Autoencoder Integrating Histopathological Images and Multi-omics for Cancer Survival Prediction
Wang, S., Hu, X., & Zhang, Q. (2023) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm58861.2023.10385635 · Cited by 6
Improving prediction of treatment response and prognosis in colorectal cancer with AI-based medical image analysis
Liu, X., Zhang, S., Shao, L., Sun, C., Li, B., Wei, W., Ying, Z., Liu, Z., & Tian, J. (2024) · The Innovation Medicine · doi:10.59717/j.xinn-med.2024.100069 · Cited by 8
Deep Learning Model for Predicting Lung Adenocarcinoma Recurrence from Whole Slide Images
Su, Z., Afzaal, U., Niu, S., de Toro, M. M., Xing, F., Ruiz, J., Gürcan, M. N., Li, W., & Niazi, M. K. K. (2024) · Cancers · doi:10.3390/cancers16173097 · Cited by 6
Explainable artificial intelligence for multi-modal cancer analysis: from genomics to immunology
Zhang, M., & Yin, L. (2026) · Critical Reviews in Oncology/Hematology · doi:10.1016/j.critrevonc.2025.105040 · Cited by 5
GenoPath-MCA: Multimodal masked cross-attention between genomics and pathology for survival prediction
Zhang, K., Dong, S., Shi, P., Hu, D., Gao, G., Yang, J., Gan, T., & Rao, N. (2026) · Computerized Medical Imaging and Graphics · doi:10.1016/j.compmedimag.2026.102699 · Cited by 6
GMMamba: Group Masking Mamba for Whole Slide Image Classification
Zheng, T., Yao, H., Jiang, K., Xiao, Y., & Zhao, S. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00926 · Cited by 5
PS3: A Multimodal Transformer Integrating Pathology Reports with Histology Images and Biological Pathways for Cancer Survival Prediction
Raza, M., Azam, A., Qaiser, T., & Rajpoot, N. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.02059 · Cited by 8
Agnostic-Specific Modality Learning for Cancer Survival Prediction From Multiple Data
Liu, H., Shi, Y., Xu, Y., Li, A., & Wang, M. (2024) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2024.3481310 · Cited by 6
Fine-tuned SegFormer for enhanced fetal head segmentation
Joudi, N. A. E., Lazaar, M., Delmotte, F., Allaoui, H., & Mahboub, O. (2024) · Procedia Computer Science · doi:10.1016/j.procs.2024.11.120 · Cited by 6
Confidence-Aware Adaptive Fusion Leaning of Imbalance Multi-Modal Data for Cancer Diagnosis and Prognosis
Zhang, Z., Wang, S., Huang, Y., Zheng, X., & Dong, S. (2025) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2025.3582626 · Cited by 5
Transformer-Based Multimodal Fusion for Survival Prediction by Integrating Whole Slide Images, Clinical, and Genomic Data
Chen, Y., Zhao, W., & Yu, L. (2023) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi53787.2023.10230804 · Cited by 5
Surrogate Biomarker Prediction from Whole-Slide Images for Evaluating Overall Survival in Lung Adenocarcinoma
Murchan, P., Baird, A., Broin, P. Ó., Sheils, O. M., & Finn, S. P. (2024) · Diagnostics · doi:10.3390/diagnostics14050462 · Cited by 5
Dual-space disentangled-multimodal network (DDM-net) for glioma diagnosis and prognosis with incomplete pathology and genomic data
Qiu, L., Zhao, L., Zhao, W., & Zhao, J. (2024) · Physics in Medicine and Biology · doi:10.1088/1361-6560/ad37ec · Cited by 5
Deep Learning-based Modeling for Preclinical Drug Safety AssessmentPreprint
Jaume, G., de Brot, S., Song, A. H., Williamson, D. F. K., Oldenburg, L., Zhang, A., Chen, R. J., Asín, J., Blatter, S., Dettwiler, M., et al. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.07.20.604430 · Cited by 5
Self-citation
Approximating vision transformers for edge: variational inference and mixed-precision for multi-modal data
Katare, D., Leroux, S., Janssen, M., & Ding, A. Y. (2025) · Computing · doi:10.1007/s00607-025-01427-w · Cited by 4
Prototype-Guided Cross-Modal Knowledge Enhancement for Adaptive Survival Prediction
Liu, F., Cai, L., Wang, Z., Fan, Z., Yu, J.-G., Chen, H., & Zhang, Y. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-04978-0_50 · Cited by 6
PTCMIL: Multiple Instance Learning via Prompt Token Clustering for Whole Slide Image Analysis
Zhao, B., Kim, S., Chen, H., Zhou, C., Gao, Z., Wang, G., & Li, X. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-05182-0_49 · Cited by 4
A fake news detection framework integrating multi-domain and multimodal features
Guo, L., Chen, Z., & Liu, X. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.131711 · Cited by 5
An Efficient Survival Prediction Framework Based on Multimodal Multi-Instance Fuzzy Optimal Transport
Luo, H., Huang, J., Zhou, T., Wu, E., & Ding, W. (2025) · IEEE Transactions on Fuzzy Systems · doi:10.1109/tfuzz.2025.3609687 · Cited by 5
TKA-MIL: Top-K Attention Multiple Instance Learning for Whole Slide Image Classification and Instance Probability Derivation
Yu, S., Chen, X., Cao, F., & Tian, T. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10888939 · Cited by 5
Knowledge-Driven Multiple Instance Learning With Hierarchical Cluster-Incorporated Aware Filtering for Larynx Pathological Grading
Li, C., Huang, P., Qin, J., & Luo, X. (2025) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2025.3609838 · Cited by 10
MurreNet: Modeling Holistic Multimodal Interactions Between Histopathology and Genomic Profiles for Survival Prediction
Liu, M., Cai, C., Li, J., Xu, P., Li, J., Ma, J., & Xu, J. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-05182-0_39 · Cited by 5
HSFSurv: A hybrid supervision framework at individual and feature levels for multimodal cancer survival analysis
Fu, B., He, J., Zhang, X., Peng, Y., Zhang, Z., Tang, Q., Liu, X., Cao, Y., & Wang, R. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103810 · Cited by 4
Deep learning-based multimodal pathogenomics integration for precision cancer prognosis
Feng, X., Song, G., Zhang, Y., Guo, L., Jiang, Y., Gong, W., Feng, Y., Xu, C., Yang, Y., & He, M. (2026) · Journal of Translational Medicine · doi:10.1186/s12967-026-07682-5 · Cited by 4
A multimodal data-based model for breast cancer diagnosis
Wang, H., Wei, L., Li, J., Liu, B., Fang, J., & Mooney, C. (2026) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2026.109288 · Cited by 5
Lightweight multimodal Cycle-Attention Transformer towards cancer diagnosis
Liu, S., Ma, X., Deng, S., Suo, Y., Zhang, J., & Ng, W. W. Y. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.124616 · Cited by 4
ContraMAE: Contrastive alignment masked autoencoder framework for cancer survival prediction
Wang, S., Lai, H., Wang, S., & Zhang, Q. (2024) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm62325.2024.10821720 · Cited by 4
Memory-Augmented Incomplete Multimodal Survival Prediction via Cross-Slide and Gene-Attentive Hypergraph Learning
Qu, M., Yang, G., Di, D., Gao, Y., Su, T., Song, Y., & Fan, L. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-05127-1_31 · Cited by 7

Authors: Chen, R. J., Lu, M. Y., Weng, W.-H., Chen, T. Y., Williamson, D. F., Manz, T., Shady, M., & Mahmood, F.
Affiliations: Broad Institute (US)
MIT Computer Science and Artificial Intelligence Laboratory (US)
Brigham and Women's Hospital (US)
ORCID: Chen, Richard J.
Lu, Ming Y.
Weng, Wei-Hung
Chen, Tiffany Y.
Williamson, Drew FK.
Manz, Trevor
Shady, Maha
Mahmood, Faisal
Venue: 2021 IEEE/CVF International Conference on Computer Vision (ICCV)
Publisher: IEEE
Published: 2021-10
Pages: 3995-4005
DOI: 10.1109/iccv48922.2021.00398
OpenAlex: W3203898052
Cited by: 398
References: 75
Citations by year: 2026: 106 · 2025: 121 · 2024: 94 · 2023: 55 · 2022: 22
FWCI: 19.71
Type: conference paper
Language: en
Open access: closed
Licence: doi.org/10.15223/policy-029 (stm-asf)
doi.org/10.15223/policy-037 (stm-asf)
Locations: IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
Topics: Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
Keywords: survival prediction, whole slide images, multimodal representation learning, multiple instance learning, genomic features, tumor microenvironment, visual question answering, ordinal regression, cancer

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Multimodal Co-Attention Transformer for Survival Prediction in Gigapixel Whole Slide Images

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks