Cited by 400AI in cancer detectionRadiomics and Machine Learning in Medical ImagingMultimodal Machine Learning Applications

The DOI matches its Crossref record; 73 of the 75 references it cites were found in a registry or an index.

Multimodal Co-Attention Transformer for Survival Prediction in Gigapixel Whole Slide Images

Chen, Richard J., Lu, Ming Y., Weng, Wei-Hung, Chen, Tiffany Y., Williamson, Drew FK., Manz, Trevor, Shady, Maha, Mahmood, Faisal·2021·2021 IEEE/CVF International Conference on Computer Vision (ICCV)

DOI 10.1109/iccv48922.2021.00398

Open this paper in CoChat — free account

Abstract

Survival outcome prediction is a challenging weakly-supervised and ordinal regression task in computational pathology that involves modeling complex interactions within the tumor microenvironment in gigapixel whole slide images (WSIs). Despite recent progress in formulating WSIs as bags for multiple instance learning (MIL), representation learning of entire WSIs remains an open and challenging problem, especially in overcoming: 1) the computational complexity of feature aggregation in large bags, and 2) the data heterogeneity gap in incorporating biological priors such as genomic measurements. In this work, we present a Multimodal Co-Attention Transformer (MCAT) framework that learns an interpretable, dense co-attention mapping between WSIs and genomic features formulated in an embedding space. Inspired by approaches in Visual Question Answering (VQA) that can attribute how word embed-dings attend to salient objects in an image when answering a question, MCAT learns how histology patches attend to genes when predicting patient survival. In addition to visualizing multimodal interactions, our co-attention trans-formation also reduces the space complexity of WSI bags, which enables the adaptation of Transformer layers as a general encoder backbone in MIL. We apply our proposed method on five different cancer datasets (4,730 WSIs, 67 million patches). Our experimental results demonstrate that the proposed method consistently achieves superior performance compared to the state-of-the-art methods.

Integrity signals

Registry

Crossref · 10.1109/iccv48922.2021.00398

Cites

73 of 73 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Attention Is All You Need
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
MizAR 60 for Mizar 50
Jakubův, J., Chvalovský, K., Goertzel, Z., et al. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,648
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Gene set enrichment analysis: A knowledge-based approach for interpreting genome-wide expression profiles
Subramanian, A., Tamayo, P., Mootha, V. K., Mukherjee, S., Ebert, B. L., Gillette, M. A., Paulovich, A., Pomeroy, S. L., Golub, T. R., Lander, E. S., et al. (2005) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0506580102 · Cited by 53,625
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472
Regression Models and Life-Tables
Cox, D. R. (1972) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1972.tb00899.x · Cited by 39,793
Regression Models and Life-Tables
Cox, D. R. (1992) · Springer Series in Statistics · doi:10.1007/978-1-4612-4380-9_37 · Cited by 26,297
Graph Attention NetworksPreprint
Veličković, P., Casanova, A., Liò, P., Cucurull, G., Romero, A., & Bengio, Y. (2017) · arXiv (Cornell University) · doi:10.17863/cam.48429 · Cited by 19,745
The 2016 World Health Organization Classification of Tumors of the Central Nervous System: a summary
Louis, D. N., Perry, A., Reifenberger, G., von Deimling, A., Figarella‐Branger, D., Cavenee, W. K., Ohgaki, H., Wiestler, O. D., Kleihues, P., & Ellison, D. W. (2016) · Acta Neuropathologica · doi:10.1007/s00401-016-1545-1 · Cited by 15,703

Show all 75 references

Research works printed but not linked: 1

zaheer, 2017, Deep sets, Advances in neural information processing systems

Sorted by citations

Items 201–300 of 400

Improving Colorectal Cancer Screening and Risk Assessment through Predictive Modeling on Medical Images and Records
Jiang, S., Robinson, C. M., Anderson, J. C., Hisey, W., Butterly, L. F., Suriawinata, A. A., & Hassanpour, S. (2025) · American Journal Of Pathology · doi:10.1016/j.ajpath.2025.09.016 · Cited by 4
Maximum Mean Discrepancy Kernels for Predictive and Prognostic Modeling of Whole Slide Images
Keller, P., Dawood, M., & Minhas, F. (2023) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi53787.2023.10230578 · Cited by 4
FOAA: Flattened Outer Arithmetic Attention for Multimodal Tumor Classification
Alwazzan, O., Patras, I., & Slabaugh, G. (2024) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi56570.2024.10635901 · Cited by 4
Histopathology-based Protein Multiplex Generation using Deep LearningPreprint
Andani, S., Chen, B., Ficek, J., Heinke, S., Casanova, R., Hild, B. F., Sobottka, B., Bodenmiller, B., Koelzer, V. H., & Rätsch, G. (2024) · medRxiv · doi:10.1101/2024.01.26.24301803 · Cited by 4
Deep fusion model integrating whole slide images and preoperative MRIs for survival prediction in IDH wild-type glioblastoma
Guo, X., Lai, M., Zhang, J., Wang, L., Huang, H., Huang, L., Li, H., Cai, L., & Liang, J. (2025) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2025.108936 · Cited by 3
Toward clinical translation of AI-Led drug discovery in endometrial cancer
Nguyen, P. K., Luong, T., Bui, X., & Le, N. Q. K. (2025) · Expert Review of Anticancer Therapy · doi:10.1080/14737140.2025.2557602 · Cited by 3
Integrating histopathology and genomic data: a comparative study of fusion methods for breast cancer survival prediction
Akbari, Y., Abdullakutty, F. C., Maadeed, S. A., Bouridane, A., & Hamoudi, R. (2025) · Complex & Intelligent Systems · doi:10.1007/s40747-025-02133-y · Cited by 3
SSR-Mam2MIL: Spatial sequence reordering Mamba2 based multiple instance learning for computational pathology
Zhang, H., Xu, H., & Lv, R. (2025) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2025.109063 · Cited by 3
Prior knowledge-guided multimodal deep learning system for biomarker exploration and prognosis prediction of urothelial carcinoma
He, Q., Tan, H., Xiao, B., Tan, Y., Peng, X., Peng, C., Yue, X., Jiang, L., Cao, Y., Lv, F. J., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-02232-7 · Cited by 3
A deep learning radiopathomic signature predicts recurrence risk of hepatocellular carcinoma after hepatectomy
Wang, G., Chen, W., Liang, Z., Ding, F., Cui, J., Shen, Z., Lin, F., Li, X., Wang, X., & Zhao, X. (2026) · Communications Biology · doi:10.1038/s42003-026-09571-5 · Cited by 3
Integrating whole-slide images and transcriptomic data for survival analysis using multimodal attention networks
Shao, C.-F., Zhao, Y.-S., Chen, Y.-S., Duan, J.-X., Zhang, Z.-Y., Wang, R., Liang, D., Chen, D.-Y., & Li, Z. (2026) · Engineering applications of artificial intelligence · doi:10.1016/j.engappai.2026.114461 · Cited by 3
On the differences between CNNs and vision transformers for COVID-19 diagnosis using CT and chest x-ray mono- and multimodality
El-Ateif, S., Idri, A., & Fernández‐Alemán, J. L. (2024) · Data Technologies and Applications · doi:10.1108/dta-01-2023-0005 · Cited by 3
MVKTrans: Multi-View Knowledge Transfer for Robust Multiomics Classification
Cong, S., Sang, Z., Liu, H., Luo, H., Wang, X., Liang, H., Hao, J., & Yao, X. (2024) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm62325.2024.10822642 · Cited by 3
Deep metric loss for multimodal learning
Moon, S., & Lee, H. (2025) · Machine Learning · doi:10.1007/s10994-024-06709-6 · Cited by 3
Robust Multimodal Learning for Ophthalmic Disease Grading via Disentangled Representation
Wang, X., Wang, Y., Liang, S., Tang, F., Liu, C., Hu, M., Hu, C., He, J., Ge, Z., & Razzak, I. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-04984-1_43 · Cited by 3
DSCENet: Dynamic Screening and Clinical-Enhanced Multimodal Fusion for MPNs Subtype Classification
Zhang, Y., Qi, Y., Qi, X., Wei, Y., & Yang, G. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72083-3_7 · Cited by 3
Deep Learning-Based Survival Prediction for Breast Cancer Using Multi-Modal Data
Ning, T., Kui, X., & Ma, B. (2024) · International Conference on Computer Science and Network Technology (ICCSNT) · doi:10.1109/iccsnt62291.2024.10776668 · Cited by 3
Integrated Multi-Omics and Whole Slide Images for Survival Prediction in Glioblastoma Using Multiple Instance Learning and Co-Attention
Hao, J., Liu, Y., Mo, Z., Liu, X., Sun, H., & Li, J. (2024) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc53108.2024.10782321 · Cited by 3
MMCAF : A Survival Status Prediction Method Based on Cross‐Attention Fusion of Multimodal Colorectal Cancer Data
Tan, X., Wu, D., Wang, H., Zhao, Z., Ge, Y., & Hu, S. (2025) · International Journal of Imaging Systems and Technology · doi:10.1002/ima.70051 · Cited by 3
AdaMHF: Adaptive Multimodal Hierarchical Fusion for Survival Prediction
Zhang, S., Lin, X., Zhang, R., Bai, Y., Xu, Y. H., Tan, T., Zheng, X., & Yu, Z. (2025) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme59968.2025.11209546 · Cited by 6
Multimodal Survival Ensemble Network: Integrating Genomic and Histopathological Insights for Enhanced Cancer Prognosis
Zhou, C., Wang, H., Li, X., Liu, W., & Liu, Z. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10447524 · Cited by 3
Multi-Scale Co-Attention Transformer Model to Integrate Radiology, Histology, and Genomics: Application to Survival Prediction in Glioblastoma
Krebs, O. K., & Tiwari, P. (2024) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi56570.2024.10635763 · Cited by 2
Dual-stream cross-modal fusion alignment network for survival analysis
Song, J., Hao, Y., Zhao, S., Zhang, P., Feng, Q., Dai, Q., & Duan, X. (2025) · Briefings in Bioinformatics · doi:10.1093/bib/bbaf103 · Cited by 2
Multi-modal Knowledge Decomposition Based Online Distillation for Biomarker Prediction in Breast Cancer Histopathology
Zhang, Q., Hao, X., Chen, Q., Xu, R., Cong, F., Lu, C., & Xu, H. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-05182-0_35 · Cited by 2
TMSE: Tri-Modal Survival Estimation with Context-Aware Tissue Prototype and Attention-Entropy Interaction
Zhang, R., Fang, M., Liu, S., Wang, Z., Tian, J., & Dong, D. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-04927-8_61 · Cited by 2
Clinically aligned multi-modal image-text model for pan-cancer prognosis prediction
Lee, J., Leiby, J. S., Takemaru, L., Huang, Y., Noh, M., Kim, J., Lee, B., Lee, M. E., Oldridge, D. A., Eun, Y., et al. (2025) · BioData Mining · doi:10.1186/s13040-025-00495-0 · Cited by 2
DSEM: Gene pathway and histology dual-stage enhanced fusion model for survival prediction
Wang, M., Wang, N., Han, S., Xue, Y., & Zhang, C. (2025) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2025.109288 · Cited by 2
LCM-Net: LLM-Driven Cross-Modality MoE Feature Fusion Network for Cancer Survival Analysis
Yang, S., Zhou, H., Yang, Y., Wang, W., Chen, S., Yang, G., Fu, H., & Zhu, L. (2026) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2026.3657518 · Cited by 2
Decoding the ERS–CAF immunoregulatory axis via multimodal AI and its pan-cancer prognostic and therapeutic predictive value
Zheng, B., Xia, C., Tang, M., Huang, W., Zheng, B., Niu, H.-Q., Li, J., Zhang, T., Zhou, H., & Zou, M. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-02388-w · Cited by 2
Privacy-preserving multimodal fusion for Alzheimer’s staging: A federated vision transformer framework with explainable AI
Ali, M. B. G., & Smiti, A. (2026) · Computerized Medical Imaging and Graphics · doi:10.1016/j.compmedimag.2026.102730 · Cited by 2
Comparison of Digital Histology AI Models with Low-Dimensional Genomic and Clinical Models in Survival Modeling for Prostate Cancer
McLoughlin, A., YinHo, H., Zhao, X., Hakansson, A. K., Moradi, A., Xu, Q. J., & Liu, Y. (2025) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw69036.2025.00097 · Cited by 2
Development and validation of a multimodal AI-agent system for prognosis analysis of bladder urothelial carcinoma
He, Q., Tan, H., Xiao, B., Peng, X., Peng, C., Tan, Y., Liu, Y., Cao, Y., Lv, F. J., Zhao, W., et al. (2026) · npj Precision Oncology · doi:10.1038/s41698-026-01415-z · Cited by 2
A Genomics-Guided Multimodal Contrastive Learning Framework for Clinically Significant Prostate Cancer Risk Stratification with Missing Clinical Data
Abdullah, A., Shahid, M., Ather, M. A., Fatima, Z., Sanchéz-Mejorada, C. G., Torres-Ruiz, M. J., Quintero, R., Mata, F., & Zagal-Flores, R. (2026) · Cancers · doi:10.3390/cancers18121952 · Cited by 2
Closing the Translational Gap: Closed‐Loop AI Discovery Frameworks for Experimental Validation and Clinical Implementation in Cancer Therapeutics
Varol, T. Ö., & Varol, M. (2026) · Cancer Medicine · doi:10.1002/cam4.72193 · Cited by 2
CMIB: A Cross Modal Interaction with Information Bottleneck Framework for Multimodal Survival Analysis
Zhouyou, L., Lifan, L., Cui, J., Deng, X., Yuanjun, L., Hao, Y., & Yan, W. (2024) · IEEE Conference on Cybernetics and Intelligent Systems · doi:10.1109/cis-ram61939.2024.10673062 · Cited by 2
Semantics-Aware Attention Guidance for Diagnosing Whole Slide Images
Liu, K., Wu, W., Elmore, J. G., & Shapiro, L. G. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72086-4_56 · Cited by 3
A survival prediction network based on multi-scale hypergraph enhancement and cross-modal refinement
Yang, C., Liang, Y., Qin, F., Cao, Y., Wang, P., Fan, J., & Wei, B. (2025) · Journal of King Saud University - Computer and Information Sciences · doi:10.1007/s44443-025-00202-3 · Cited by 2
Vision Intelligence Assisted Lung Function Estimation Based on Transformer Encoder–Decoder Network With Invertible Modeling
Chen, L., Lu, D., Zhai, J., Cai, K., Wang, L., & Zhang, Z. (2024) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2023.3348428 · Cited by 2
Survival Prediction via Deep Attention-Based Multiple-Instance Learning Networks with Instance Sampling
Tarkhan, A., Nguyen, T. K., Simon, N., & Dai, J. (2024) · Proceedings of the AAAI Symposium Series · doi:10.1609/aaaiss.v2i1.27717 · Cited by 2
Investigating Morphologic Correlates of Driver Gene Mutation Heterogeneity via Deep Learning
Song, A. H., Williamson, D. F. K., & Mahmood, F. (2022) · Cancer Research · doi:10.1158/0008-5472.can-22-2040 · Cited by 2
Self-citation
Histology and spatial transcriptomic integration revealed infiltration zone with specific cell composition as a prognostic hotspot in glioblastomaPreprint
Fidon, L., Lubrano, M., Hoffmann, C., Baena, E., Thiriez, C., Cano, L., Cornish, A. J., Madissoon, E., Ducret, V., Esposito, C., et al. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.10.08.681087 · Cited by 2
Research on Image Segmentation Algorithm Based on Multimodal Hierarchical Attention Mechanism and Genetic Neural Network
Wang, D., & Ma, L. (2022) · Computational Intelligence and Neuroscience · doi:10.1155/2022/9980928 · Cited by 1
Erklärbare Künstliche Intelligenz in der Pathologie
Klauschen, F., Dippel, J., Keyl, P., Jurmeister, P. S., Bockmayr, M. L., Möck, A., Buchstab, O., Alber, M., Ruff, L., Montavon, G., et al. (2024) · Die Pathologie · doi:10.1007/s00292-024-01308-7 · Cited by 1
AI-Driven Integration of Histological and Genomic Data for Understanding Breast Cancer Survival
Perez-Herrera, L. V., Sucre, A., Sánchez, X. C., García-González, M. J., de Celis, G., Garín-Muga, A., Vivanco, M. D., Alonso, E., & López‐Linares, K. (2024) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi56570.2024.10635774 · Cited by 1
Multimodal survival analysis using optimal transport matching and global-local feature fusion
Sun, B., Peng, Y., & Ge, Y. (2025) · Digital Signal Processing · doi:10.1016/j.dsp.2025.105119 · Cited by 1
Co-attention-based Multi-modal Feature Alignment for Cancer Analysis
Fan, S., Li, C., He, X., & Song, C. (2024) · International Conference on Image Processing, Computer Vision and Machine Learning (ICICML) · doi:10.1109/icicml63543.2024.10957808 · Cited by 1
Drop and Align: Fusing Pathology and Genomics via Graph Learning for Cancer Survival Prediction
Zhang, Z., Zhao, Y., Duan, J., Liu, Y., Liang, D., & Li, Z. (2025) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi60581.2025.10980720 · Cited by 1
M3T-CKD: A multi-modal multi-teacher contrastive knowledge distillation framework for survival prediction of patients with carotid atherosclerosis
Jia, N., Jia, T., Zhang, Z., Zhang, Y., Xing, L., & Liu, S. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.107811 · Cited by 1
Uncertainty-Aware Survival Analysis With Dirichlet Distribution for Multi-Scale Pathology and Genomics
Jiang, S., Cai, L., Gan, Z., Wang, Y., Tang, G., & Zhang, Y. (2025) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2025.3601892 · Cited by 2
KPVG: Knowledge-prompted vision-genomics model for cancer survival prediction in whole slide images
Shi, J., Li, C., Zheng, Y., & Fu, H. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103660 · Cited by 2
Enhancing WSI-Based Survival Analysis with Report-Auxiliary Self-distillation
Wang, Z., Liu, H., Wang, Z., Li, D., Cen, M., Magnier, B., Li, L., & Wang, L. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-05182-0_20 · Cited by 2
Learning Explainable Imaging-Genetics Associations Related to a Neurological Disorder
Wang, J., Jacokes, Z., Horn, J. D. V., Schatz, M. C., Pelphrey, K. A., & Venkataraman, A. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-05141-7_34 · Cited by 2
Reliable Multimodal Cancer Survival Prediction With Confidence-Aware Risk Modeling
Xie, X., Wang, Y., Zhao, Z., Yang, Q., Huang, L., Zhou, F., & Wang, Y. (2025) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2025.3613406 · Cited by 1
MMGFFN: A Multimodal Multi-Granularity Feature Fusion Network for Cancer Survival Prediction
Luo, H., Ding, W., Huang, J., & Diao, J. (2025) · International Conference on Computer Communication and Artificial Intelligence (CCAI) · doi:10.1109/ccai65422.2025.11189601 · Cited by 1
VGAT: A Cancer Survival Analysis Framework Transitioning from Generative Visual Question Answering to Genomic Reconstruction
Chen, Z., Han, M., Zhang, X., Ma, S., Liu, T., Wei, X., & Zhang, L. (2025) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme59968.2025.11209080 · Cited by 1
A Multimodal Fusion Framework Employing Full-Stage Attention for Survival Prediction
Liu, A., Guo, J., Wang, Y., Liu, J., Tian, H., Qin, T., Xu, H., & Wei, H. (2025) · Lecture notes in computer science · doi:10.1007/978-981-95-3052-6_15 · Cited by 1
ECP-KD: Efficient Computational Pathology Heterogeneous Model Fusion Using Knowledge Distillation
Ni, T., Zhu, H., Han, Y., Lai, C., Chen, J., & Pan, Y. (2025) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc58623.2025.11254751 · Cited by 1
Beyond NLL: Pathwise Cross-Entropy Loss for Discriminative and Calibrated Event-Time Survival Prediction
Long, J., Li, J. P., Kers, J., & Verbeek, F. J. (2026) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2026.3650827 · Cited by 1
DABSP: Integrating histologic and transcriptomic data for pan-cancer survival outcome prediction through a dual-directional attention based multimodal data fusion framework
Liu, Y., Hu, B., Hao, X., Huang, J., & Huang, J. (2026) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2026.109527 · Cited by 1
MoMIL: Multi-order enhanced multiple instance learning for computational pathology
Zhang, Y., Zhang, X., Wang, J., Liang, B., Yang, Y., & Tong, C. (2026) · Image and Vision Computing · doi:10.1016/j.imavis.2026.105918 · Cited by 1
ME-Mamba: Multi-Expert Mamba with efficient knowledge capture and fusion for multimodal survival analysis
Zhang, C., Qu, L., Liu, X., & Song, Z. (2026) · Computerized Medical Imaging and Graphics · doi:10.1016/j.compmedimag.2026.102733 · Cited by 1
Towards Robust Multimodal Representation: A Unified Approach with Adaptive Experts and Alignment
Moradinasab, N., Sengupta, S., Liu, J., Syed, S., & Brown, D. E. (2025) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw69036.2025.00130 · Cited by 1
Pseudo-label-free instance screening of non-tumor regions in whole slide images for improved classification and survival prediction
Zhao, Y., Ding, H., Zhang, X., Zhao, X., Zhao, S., Yang, P., Wang, D., Wang, S., & Xing, X. (2026) · Computerized Medical Imaging and Graphics · doi:10.1016/j.compmedimag.2026.102738 · Cited by 1
Decouple, Reorganize, and Fuse: A Multimodal Framework for Cancer Survival Prediction
Wang, H., Ying, H., Xu, Y., Qiu, Q., Zhang, C., Chen, D. Z., Sun, Y., & Wu, J. (2026) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2026.3668773 · Cited by 2
A Contrastive Variational AutoEncoder for NSCLC Survival Prediction with Missing Modalities
Zanitti, M., Miskovic, V., Trovò, F., Pedrocchi, A., Shen, M., Tun, Y. K., Prelaj, A., & Kosta, S. (2025) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata66926.2025.11401431 · Cited by 1
Survival prediction for bladder cancer using multimodal data with quantum neural networks and transformer architectures
Qin, Z., Zhou, H., Hu, Y., Xiong, X., Lu, J., Su, H., & He, J. (2026) · Scientific Reports · doi:10.1038/s41598-026-42047-4 · Cited by 1
End-to-End Multimodal Multiple Instance Learning for Cancer Histopathology Classification with Dual-Attention Fusion
Shirae, S., Debsarkar, S. S., Kawanaka, H., Aronow, B. J., & Prasath, V. B. S. (2026) · Journal of Medical Systems · doi:10.1007/s10916-026-02379-0 · Cited by 1
RSCO-Net: A refined single candidate optimizer for multimodal colorectal cancer postoperative recurrence prediction
Wei, Z., Ye, K., Hou, Y., Li, Y. M., Meng, L., & Xu, A. (2026) · Alexandria Engineering Journal · doi:10.1016/j.aej.2026.04.044 · Cited by 1
Beyond attention heatmaps: How to get better explanations for multiple instance learning models in histopathology
Idaji, M. J., Hense, J., Neuhauser, T., Krause, A., Luo, Y., Eberle, O., Schnake, T., Ciernik, L., Jafari, F. R., Vahidimajd, R., et al. (2026) · Medical Image Analysis · doi:10.1016/j.media.2026.104148 · Cited by 2
Positive Semi-definite Group-Aware Instance Disentangled Learning for WSI Representation
Li, C., Bozorgtabar, B., Ping, Y., Wong, C.-K., Huang, P., & Qin, J. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-38224-5_38 · Cited by 1
CrossLinkNet: An Explainable and Trustworthy AI Framework for Whole-Slide Images Segmentation
Xiao, P., Zhong, Q., Chen, J., Wu, D., Qin, Z., & Zhou, E. (2024) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2024.049791 · Cited by 1
Attention-Enhanced Guided Multimodal and Semi-Supervised Networks for Visual Acuity (VA) Prediction after Anti-VEGF Therapy
Wang, Y., Wang, Y., Liu, X., Cui, W., Jin, P., Cheng, Y., & Jia, G. (2024) · Electronics · doi:10.3390/electronics13183701 · Cited by 1
PG-MLIF: Multimodal Low-Rank Interaction Fusion Framework Integrating Pathological Images and Genomic Data for Cancer Prognosis Prediction
Pan, X., An, Y., Lan, R., Liu, Z., Liu, Z., Lu, C. Y., & Yang, H. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72384-1_33 · Cited by 2
A Scaled Mask Attention-based Hybrid model for Survival Prediction
Junayed, M. S., & Nabavi, S. (2024) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm62325.2024.10822154 · Cited by 1
Pathology-Genomic Cross Attention network for Cancer Survival Prediction from Whole Slide Image
Lin, P., & Chen, X. (2024) · International Conference on Image Processing, Multimedia Technology and Machine Learning (IPMML) · doi:10.1145/3722405.3722407 · Cited by 1
CoC: Chain-of-Cancer Based on Cross-Modal Autoregressive Traction for Survival Prediction
Zhou, H., Yang, S., Yang, S., Qin, J., Chen, L., & Zhu, L. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-05182-0_9 · Cited by 1
Interpretable Deep Learning Predicts the Molecular Endometrial Cancer Classification from H&E Images: A Combined Analysis of the Portec Randomized Clinical TrialsPreprint
Volinsky-Fremond, S., Andani, S., Wolf, J. B., Dijkstra, J., Melsbach, S., Jobsen, J. J., Brinkhuis, M., Roothaan, S. M., Jurgenliemk-Schulz, I. M., Lutgens, L. C. H. W., et al. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4144537 · Cited by 1
STEP: Deciphering Spatial Atlas at Single-Cell Level with Whole-Transcriptome CoveragePreprint
Hu, Z., Zhu, Z., Cai, L., Zhan, Y., Yan, X., Chen, J., Sun, B., Du, S., Jiang, S., Wang, H., et al. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.11.22.624797 · Cited by 1
A Context Aware Lung Cancer Survival Prediction Network by Using Whole Slide Images
Liu, X., Wang, Y., & Luo, Y. (2023) · Communications in computer and information science · doi:10.1007/978-981-99-8141-0_28
HECTOR: multimodal deep learning predicts recurrence risk in endometrial cancerPreprint
Volinsky-Fremond, S., Horeweg, N., Andani, S., Wolf, J. B., Lafarge, M. W., de Kroon, C. D., Ørtoft, G., Høgdall, E. V. S., Dijkstra, J., Jobsen, J. J., et al. (2023) · medRxiv · doi:10.1101/2023.11.27.23298994
Multimodal Diverse Granularity Fusion Network based on US and CT Images for Lymph Node Metastasis Prediction of Thyroid CarcinomaPreprint
Li, G., Yao, J., Peng, C., Hu, Y., Zhao, S., Feng, X., Yang, J., Xu, D., Li, X., Sha, C., et al. (2023) · medRxiv · doi:10.1101/2023.12.25.23300117
Erklärbare Künstliche Intelligenz in der Pathologie
Klauschen, F., Dippel, J., Keyl, P., Jurmeister, P. S., Bockmayr, M. L., Möck, A., Buchstab, O., Alber, M., Ruff, L., Montavon, G., et al. (2024) · Forum · doi:10.1007/s12312-024-01337-6
Benchmarking Vision Encoders for Survival Analysis using Histopathological ImagesPreprint
Nizami, A., & Halder, A. (2024) · medRxiv · doi:10.1101/2024.08.23.24312362
Unified Modeling Enhanced Multimodal Learning for Precision Neuro-Oncology
Yi, H., Wang, X., Li, K., & Li, C. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73360-4_1
A Multimodal Adaptive Cooperative Learning Framework for Cancer Survival Risk Prediction
Yang, Z., Liu, H., & Wang, X. (2024) · International Workshop on Multimedia Computing for Health and Medicine (MCHM) · doi:10.1145/3688868.3689195
Multi-stain modelling of histopathology slides for breast cancer prognosis predictionPreprint
Sharma, A., Gustafsson, F., Hartman, J., & Rantalainen, M. (2024) · medRxiv · doi:10.1101/2024.11.10.24317066 · Cited by 1
Opportunistic AI for enhanced cardiovascular disease risk stratification using abdominal CT scans
Rehman, A., Kim, J.-W., Lee, H., Chang, J., & Park, S. M. (2025) · Computerized Medical Imaging and Graphics · doi:10.1016/j.compmedimag.2025.102493
Effective latent hierarchical feature fusion in multiple instance learning for Whole Slide Image classification
Lan, Q., Wu, Y., Ding, W., & Yuan, J. (2025) · Applied Soft Computing · doi:10.1016/j.asoc.2025.113191
TAFL: Task-Agnostic Feature Learner for Efficient Adaptation to Unseen Clinical Tasks Based on Whole-Slide Histopathological Images
Zuo, Y., Wang, L., Liu, J., Feng, S., Wang, Y., Zhu, Q., Shao, W., & Zhang, D. (2025) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2025.3575801
Multi-Scale Hierarchical Context-Aware Survival Prediction Network based on whole slide images
Song, J., Hao, Y., Zhai, H., Zhao, S., Tao, N. P., Huang, S., & Duan, X. (2025) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2025.108159 · Cited by 2
Clinical Stage Prompt Induced Multi-Modal Prognosis
Jin, T., Xie, X., Li, Q., Li, X., & Wang, Y. (2025) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2025.3588836
AMKD: Adaptive Multi-modality Knowledge Distillation for Pathological Survival Analysis
Xu, Y., Cai, L., Wang, Y., Cheng, H., Liu, F., Wang, R., & Zhang, Y. (2025) · Lecture notes in computer science · doi:10.1007/978-981-95-0033-8_35
Whole Slide Images Based Cancer Survival Prediction Using Multi-task Learning
Liu, L., Lv, Y., & Zhang, B. (2025) · Lecture notes in computer science · doi:10.1007/978-981-95-0030-7_28
A Survival Prediction Model Integrating Hierarchical Pathological Image and Pathway Features
Xu, X., Peng, W., Dai, W., Fu, X., Liu, L., & Liu, L. (2025) · Lecture notes in computer science · doi:10.1007/978-981-95-0695-8_9
FD-CSS: Nested Cross-Modal Fusion for Survival Prediction
Li, M., Chen, L., & Wu, C. (2025) · IEEE International Conference on Image Processing and Computer Applications (ICIPCA) · doi:10.1109/icipca65645.2025.11138236
Explainable Classifier for Malignant Lymphoma Subtyping via Cell Graph and Image Fusion
Nishiyama, D., Miyoshi, H., Hashimoto, N., Ohshima, K., Hontani, H., Takeuchi, I., & Sakuma, J. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-05162-2_31
Bipartite Patient-Modality Graph Learning with Event-Conditional Modelling of Censoring for Cancer Survival Prediction
Yue, H., Kuang, H., Liu, J., Li, J., Wang, L., He, M., & Wang, J. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-05162-2_10
MOSCARD - Multimodal Opportunistic Screening for Cardiovascular Adverse Events with Causal Reasoning and De-confounding
Pi, J., Farina, J., Lahiri, R., Jeong, J., Gurudu, A., Park, H., Chao, C., Ayoub, C., Arsanjani, R., & Banerjee, I. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-04984-1_32
Thread the Needle: Genomics-Guided Prompt-Bridged Attention Model for Survival Prediction of Glioma Based on MRI Images
Zhong, Y., Zheng, X., Shen, X., Wang, J., Zhao, L., Song, Z., & Zhang, Z. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-04981-0_59
Lymphoma Prognosis with Lesion-Anatomy Context Fusion and Attention-Based Multi-lesion Aggregation
Zhang, S., Zhang, J., Qiu, L., Liu, W., Jin, D., Lu, L., Yang, S., & Yan, K. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-04927-8_31

Authors: Chen, R. J., Lu, M. Y., Weng, W.-H., Chen, T. Y., Williamson, D. F., Manz, T., Shady, M., & Mahmood, F.
Affiliations: Broad Institute (US)
MIT Computer Science and Artificial Intelligence Laboratory (US)
Brigham and Women's Hospital (US)
ORCID: Chen, Richard J.
Lu, Ming Y.
Weng, Wei-Hung
Chen, Tiffany Y.
Williamson, Drew FK.
Manz, Trevor
Shady, Maha
Mahmood, Faisal
Venue: 2021 IEEE/CVF International Conference on Computer Vision (ICCV)
Publisher: IEEE
Published: 2021-10
Pages: 3995-4005
DOI: 10.1109/iccv48922.2021.00398
OpenAlex: W3203898052
Cited by: 398
References: 75
Citations by year: 2026: 106 · 2025: 121 · 2024: 94 · 2023: 55 · 2022: 22
FWCI: 19.71
Type: conference paper
Language: en
Open access: closed
Licence: doi.org/10.15223/policy-029 (stm-asf)
doi.org/10.15223/policy-037 (stm-asf)
Locations: IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
Topics: Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
Keywords: survival prediction, whole slide images, multimodal representation learning, multiple instance learning, genomic features, tumor microenvironment, visual question answering, ordinal regression, cancer

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Multimodal Co-Attention Transformer for Survival Prediction in Gigapixel Whole Slide Images

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks