Cited by 400AI in cancer detectionRadiomics and Machine Learning in Medical ImagingMultimodal Machine Learning Applications

The DOI matches its Crossref record; 73 of the 75 references it cites were found in a registry or an index.

Multimodal Co-Attention Transformer for Survival Prediction in Gigapixel Whole Slide Images

Chen, Richard J., Lu, Ming Y., Weng, Wei-Hung, Chen, Tiffany Y., Williamson, Drew FK., Manz, Trevor, Shady, Maha, Mahmood, Faisal·2021·2021 IEEE/CVF International Conference on Computer Vision (ICCV)

DOI 10.1109/iccv48922.2021.00398

Open this paper in CoChat — free account

Abstract

Survival outcome prediction is a challenging weakly-supervised and ordinal regression task in computational pathology that involves modeling complex interactions within the tumor microenvironment in gigapixel whole slide images (WSIs). Despite recent progress in formulating WSIs as bags for multiple instance learning (MIL), representation learning of entire WSIs remains an open and challenging problem, especially in overcoming: 1) the computational complexity of feature aggregation in large bags, and 2) the data heterogeneity gap in incorporating biological priors such as genomic measurements. In this work, we present a Multimodal Co-Attention Transformer (MCAT) framework that learns an interpretable, dense co-attention mapping between WSIs and genomic features formulated in an embedding space. Inspired by approaches in Visual Question Answering (VQA) that can attribute how word embed-dings attend to salient objects in an image when answering a question, MCAT learns how histology patches attend to genes when predicting patient survival. In addition to visualizing multimodal interactions, our co-attention trans-formation also reduces the space complexity of WSI bags, which enables the adaptation of Transformer layers as a general encoder backbone in MIL. We apply our proposed method on five different cancer datasets (4,730 WSIs, 67 million patches). Our experimental results demonstrate that the proposed method consistently achieves superior performance compared to the state-of-the-art methods.

Integrity signals

Registry

Crossref · 10.1109/iccv48922.2021.00398

Cites

73 of 73 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Attention Is All You Need
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
MizAR 60 for Mizar 50
Jakubův, J., Chvalovský, K., Goertzel, Z., et al. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,648
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Gene set enrichment analysis: A knowledge-based approach for interpreting genome-wide expression profiles
Subramanian, A., Tamayo, P., Mootha, V. K., Mukherjee, S., Ebert, B. L., Gillette, M. A., Paulovich, A., Pomeroy, S. L., Golub, T. R., Lander, E. S., et al. (2005) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0506580102 · Cited by 53,625
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472
Regression Models and Life-Tables
Cox, D. R. (1972) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1972.tb00899.x · Cited by 39,793
Regression Models and Life-Tables
Cox, D. R. (1992) · Springer Series in Statistics · doi:10.1007/978-1-4612-4380-9_37 · Cited by 26,297
Graph Attention NetworksPreprint
Veličković, P., Casanova, A., Liò, P., Cucurull, G., Romero, A., & Bengio, Y. (2017) · arXiv (Cornell University) · doi:10.17863/cam.48429 · Cited by 19,745
The 2016 World Health Organization Classification of Tumors of the Central Nervous System: a summary
Louis, D. N., Perry, A., Reifenberger, G., von Deimling, A., Figarella‐Branger, D., Cavenee, W. K., Ohgaki, H., Wiestler, O. D., Kleihues, P., & Ellison, D. W. (2016) · Acta Neuropathologica · doi:10.1007/s00401-016-1545-1 · Cited by 15,703

Show all 75 references

Research works printed but not linked: 1

zaheer, 2017, Deep sets, Advances in neural information processing systems

Sorted by citations

Items 301–400 of 400

Interpretable Cancer Survival Prediction by Fusing Semantic Labelling of Cell Types and Whole Slide Images
Chen, J., Liu, P., Chen, C., Su, Y., Wang, J., Chen, C., Ai, X., & Lv, X. (2025) · Interdisciplinary Sciences Computational Life Sciences · doi:10.1007/s12539-025-00744-0
M2SOPT: Multi-scale Multi-modal Survival Outcome Prediction Transformer
Su, Y., & Wang, P. (2025) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn64981.2025.11227617
SIB-MIL: Sparsity-Induced Bayesian Neural Network for Robust Multiple Instance Learning on Whole Slide Image Analysis
Chen, Y., Chan, T. H., Chen, J., Li, L., Yin, G., & Yu, L. (2025) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2025.3638243
MPT-MIL: Multimodal Aware Prompt Tuning for Prediction of Cancer Survival
Zhang, R., Fang, M., Liu, S., Wang, Z., Xie, X., Ma, X., Tian, J., & Dong, D. (2025) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2025.3640398
Multimodal Nested Attention Network for Lymph Node Metastasis Prediction of Thyroid Carcinoma
Li, G., Yao, J., Peng, C., Hu, Y., Feng, X., Yang, J., Gao, X., Xu, D., Li, X., Sha, C., et al. (2025) · Big Data Mining and Analytics · doi:10.26599/bdma.2025.9020046
Fine-Grained Interaction and Task-Driven Fusion of Histology Images and Genomics for Multimodal Cancer Survival Prediction
Chen, X., & Huang, J. (2025) · ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics (ACM-BCB) · doi:10.1145/3765612.3767199
MADSurv: An Uncertainty-Aware Framework for Multimodal Cancer Survival Analysis
Zhang, E., Raigir, V. S., Poellabauer, C., Kohut, A., Templeton, J. M., Mondal, A. M., & Weintraub, L. (2025) · ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics (ACM-BCB) · doi:10.1145/3765612.3767227
SG-MuRCL: Smoothed Graph-Enhanced Multi-Instance Contrastive Learning for Robust Whole-Slide Image Classification
Ziabari, S. S. M., Lin, B. Y., Husaini, Y. N. A., & Alsahag, A. M. M. (2026) · Information · doi:10.3390/info17010037 · Cited by 1
ODS-Net: Orthogonal Disentangled Synergy Network for Multimodal Cancer Survival Prognosis
Zhang, W., Yu, Y., Wu, X., Zhang, J., Mu, C., & Liu, S. H. (2025) · International Conference on Mobility, Sensing and Networking (MSN) · doi:10.1109/msn69125.2025.00057
Dual-stream Co-attention based Cross-modal Alignment Fusion for Survival Analysis
Fan, S., Huang, S., & Gong, Y. (2025) · International Conference on Image Processing, Computer Vision and Machine Learning (ICICML) · doi:10.1109/icicml67980.2025.11333463
iMCN: information compression-based multimodal confidence-guided fusion network for cancer survival prediction
Lyu, C., Zhao, L., Xie, Y., Zhao, W., Zhou, Y., Ting, H.-N., Zhang, P.-M., & Zhao, J. (2026) · Biomedical Physics & Engineering Express · doi:10.1088/2057-1976/ae3b46 · Cited by 1
A Multi-branch Independent Masking and Dirichlet-Based Fusion Method for Multiple Instance Learning on Whole-Slide Images
Tan, K., Li, Z., Chen, X., Zhu, H., Fan, X., & Kong, J. (2026) · Lecture notes in computer science · doi:10.1007/978-981-95-5634-2_13
Cross-Attention Transformer with Textual Prompt Learning for Survival Prediction
Wang, P. (2026) · Lecture notes in computer science · doi:10.1007/978-981-95-5631-1_21
HMSP: Hierarchical Multimodal Semantic Fusion of Pathology-Text-Genomic Representations for Robust Survival Prediction
Yang, J., Hu, J., & He, X. (2025) · IEEE International Conference on Medical Artificial Intelligence (MedAI) · doi:10.1109/medai67139.2025.00013
A Flow Model with Low-Rank Transformers for Incomplete Multimodal Survival Analysis
Yin, Y., Shou, Y., Dai, Z., Peng, Y., Meng, T., Ai, W., & Li, K. (2025) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm66473.2025.11356098
Genomics-Aware Multimodal Self-Supervised Learning for Cancer Survival Prediction
Sun, K., Guo, Y., He, Y., Liu, Z., Cui, J., & Li, S. (2025) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm66473.2025.11356896
Multimodal Survival Prediction Framework with Contrastive Learning Guided by Pathological Reports
Wang, Y., Shi, J., Sun, D., Liu, B., Wu, Z., & Zheng, Y. (2025) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm66473.2025.11356311
A Multimodal Survival Prediction Fusion Framework Based on Prototype Learning
Wang, N., Wang, M., Yuan, C., Han, S., Ym, X., & Li, J. (2025) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm66473.2025.11357145
A Hybrid Framework for Multimodal Cancer Survival Prediction
Peng, H., Qu, A., & Zeng, Y. (2025) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm66473.2025.11356286
Multimodal Transformer-Based Classification of Breast Cancer Laterality
Mujawar, S., Shaikh, A., & Chaware, A. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-17840-4_4
Frequency-Enhanced and Hypergraph-Aware Cross-Modal Attention for Cancer Survival Prediction
Cao, Y., Liang, Y., Yang, C., Qin, F., Fan, J., Wang, P., Huang, X., & Wei, B. (2025) · International Conference on Algorithms, Computing and Artificial Intelligence (ACAI) · doi:10.1109/acai68217.2025.11406691
Bridging the gap between performance and interpretability: An explainable disentangled multimodal framework for cancer survival prediction
Eijpe, A., Lakbir, S., Cesur, M. E., Oliveira, S. P., Chatzimparmpas, A., Abeln, S., & Silva, W. (2026) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2026.103504
ProtoDecNet: A Prototype-Based Representation Decoupling Network for Multimodal Cancer Survival Prediction
Li, X., Song, J., Zhang, Y., & Tong, T. (2025) · International Conference on Electronic Information Engineering and Computer Communication (EIECC) · doi:10.1109/eiecc67963.2025.11409628
Multi-scale Multimodal Representation for Enhanced Survival Prediction in Computational Pathology
Hou, Q., Sun, Y., Ling, J., Lu, M., & Yao, S. (2026) · Interdisciplinary Sciences Computational Life Sciences · doi:10.1007/s12539-026-00816-9
Self-citation
CD-Former: A Cross-Modal Dual-Interaction Transformer With Whole-Slide Image Pyramids and Genomics for Survival Prediction
Long, L., Peng, X., Cui, J., Liu, B., Wu, X., Zhang, D., & Wang, Y. (2026) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2026.3672108
Deep Generative Model of Macrophage Immune Response for Hepato-intestinal Tumor Therapy Optimization
Zhou, L., Tan, Y., Lv, W., Lin, K., Li, F., Ouyang, W., & Zhang, D. (2026) · Cyborg and Bionic Systems · doi:10.34133/cbsystems.0559
VRWKV-MSA: Multi-Modal Survival Analysis Using VRWKV with Optimal Transport Alignment
Zeng, Y., Ouyang, L., & Zhang, X. (2025) · International Conference on Big Data and Data Mining (BDDM) · doi:10.1109/bddm68348.2025.11442002
Dynamic multimodal survival prediction in multiple myeloma integrating gene expression, longitudinal laboratories, and treatment historyPreprint
Jia, S., Lysenko, A., Boroevich, K. A., Sharma, A. K., & Tsunoda, T. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.03.30.715136
StableMIL: Entropy-Stabilized Attention-Based Multiple Instance Learning for Morphologically Variable Whole Slide Images
Lu, Y., Qi, M., Fu, Y., Xiao, Z., Shao, W., Tian, J., & Mu, W. (2026) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2026.3682009
Representation learning of multimodal and longitudinal data for renal transplantation monitoring
Milecki, L. (2024) · theses.fr (ABES) · doi:10.70675/8478bb39za27fz4524z8e86z5df6c7d2fc00 · Cited by 1
MPRSurv: Multi-perspective prompted ranking for vision-language survival analysis on whole slide images
Zhang, R., Liu, S., Fang, M., Zhao, S., Guo, W., Wang, Z., Feng, X., Zhang, X.-Y., Xie, X., Tian, J., et al. (2026) · Pattern Recognition · doi:10.1016/j.patcog.2026.113695
SPACT: A clustering-driven multi-modal framework for survival prediction using genomic and histopathology data
Öğülmüş, F. E., Gafarov, S., Almalıoğlu, Y., Özdemir, B. H., Atılgan, A. O., Demir, D., Özen, Ö., Keles, G. E., Kahveci, T., & Turan, M. (2026) · Medical Image Analysis · doi:10.1016/j.media.2026.104078
M2OTCA: Multiple-magnification optimal transport-based cross-attention learning for whole slide image classification
Zhu, Z., Meijering, E. H. W., & Wang, L. S. (2026) · Medical Image Analysis · doi:10.1016/j.media.2026.104082
SemaMIL: Semantic-Aware Multiple Instance Learning with Retrieval-Guided State Space Modeling for Whole Slide Images
Gan, L., Wu, X., Wang, Z., Qu, L., Wu, S., Zhang, J., & Sun, X. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11464264
OT-AMF: An Optimal Transport-Based Adaptive Multi-Scale Fusion Framework for Cancer Survival Prediction
Lin, Z., Yang, Z., Wu, S., Li, J., Zhang, H., & Wang, R. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11460638
Multimodal Fusion of 3D CT and Pathological Images for Gastric Cancer Recurrence Prediction
Cao, L., Tian, M., Li, J., Chen, Z., Wang, X., Zhong, S., Wu, G., Tang, Z., & Yu, J. (2026) · Cancer Management and Research · doi:10.2147/cmar.s563640
Anticipating Cancer Onset: An Edge Optimized Transformer for Personalized, Early Breast Cancer Risk Assessment
Patil, S. B., & Patil, R. V. (2026) · International Conference on Emerging Smart Computing and Informatics (ESCI) · doi:10.1109/esci68015.2026.11493280
Modaltune: Fine-Tuning Slide-Level Foundation Models with Multi-Modal Information for Multi-Task Learning in Digital Pathology
Ramanathan, V., Xu, T., Pati, P., Ahmed, F., Goubran, M., & Martel, A. L. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.02217 · Cited by 3
CD-AHF: A causal-driven adaptive hierarchical fusion framework for multimodal cancer prognosis
Tang, Q., Zhao, Q., & Zhao, H. (2026) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2026.110391
Multimodal Fusion for Breast Cancer Prognosis: Integrating Frozen Swin-T Backbones with Mutual-Guided Attention
Sharma, B., & Dey, B. (2026) · IEEE International Conference on AI Engineering and Innovations (AIEI) · doi:10.1109/aiei69164.2026.11497098
PaRaChute: Pathology-Radiology Cross-Modal Fusion for Missing-Modality-Robust Survival Prediction
Caforio, P., Poles, I., & Santambrogio, M. D. (2026) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61042.2026.00077
BRIDGE: A Multi-organ Histo-ST Foundation Model Enables Virtual Spatial Transcriptomics for Enhanced Few-shot Cancer DiagnosisPreprint
Liang, Z., Zhao, W., Wang, F., Chen, G., Huang, Y., & Yu, L. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.05.05.722971
A Semantically-Structured Representation Learning Framework for Head and Neck Cancer Survival Prediction
Chanchal, M., & Swapna, T. (2026) · International Conference on Inventive Computation Technologies (ICICT) · doi:10.1109/icict68280.2026.11511108
MoHD: Multi-mOdal survival prediction through Hierarchical Decoupling of whole-slide image pyramids and genomics
Long, L., Li, Y., Zhang, D., & Wang, Y. (2026) · Medical Image Analysis · doi:10.1016/j.media.2026.104098
Dual-path Asymptotic Multimodal Fusion Network for cancer survival prediction
Ge, Y., Peng, Y., & Sun, B. (2026) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2026.110636
Uncertainty Quantification for Multimodal Generation in Cancer Ai
Dey, S., Wood, R., Basuchowdhuri, P., Saha, S. K., & Chakraborti, T. (2026) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi61048.2026.11515703
MCAMamba: A multimodal method with bidirectional cross-attention and state space model for cancer survival prediction
Cui, H., Liu, X., Ding, W., & Wang, W. (2026) · Science Progress · doi:10.1177/00368504261456869
Hypernetwork-Mediated Multimodal Fusion for Breast Cancer Molecular Subtype Classification Using Histopathology and Genomics
Abdullakutty, F. C., Akbari, Y., Al-Maadeed, S. A., Bouridane, A., Hamoudi, R., & Kunhoth, S. (2026) · IEEE International Conference on Computing and Machine Intelligence (ICMI) · doi:10.1109/icmi68585.2026.11539914
Comparative Study of Imprint Extraction Methods with Vision Transformer–Based Feature Fusion for Pill Image Recognition
My, L. N. T., Vo, T., & Hoang, V. T. (2026) · International Conference on Computational Intelligence Approaches and Applications (ICCIAA) · doi:10.1109/icciaa68481.2026.11543734
ChronoSurv: A Clinical Pathway-Guided Graph Framework for Multimodal Survival Analysis
Miccinilli, H., & Piazza, T. D. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-38172-9_13
Evidential Fusion Network for Multimodal Survival Prediction Under Missing Modalities
Xing, Y., Mo, H., Wang, Z., Huang, L., & Feng, M. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-38172-9_23
HERO: Hypothesis-Driven Evidence Retrieval from Omics for Multi-task Breast Cancer Analysis
Li, X., & Su, R. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-38470-6_49
MMDiffuzzy: Fuzzy memory guided diffusion for uncertainty-aware multimodal fusion in WSI analysis
Zhou, T., Ding, W., Chen, Y., Huang, J., & Luo, H. (2026) · Pattern Recognition · doi:10.1016/j.patcog.2026.114394
Causality-Guided Diffusion and Fusion of incomplete multi-modal data for robust survival prognosis
Huang, Y., Zheng, X., & Dong, S. (2026) · Medical Image Analysis · doi:10.1016/j.media.2026.104207
SAGEAgent: A Self-evolving Agent for Cost-Aware Modality Acquisition in Multimodal Survival Prediction
Qu, C., Cui, C., Lu, Z., ZHU, J., Yao, T., Guo, J., Xiong, J., Zhu, Y., Yang, Y., Landman, B. A., et al. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-38059-3_51
Interpretation Before Integration: LLM-Guided Multimodal Completion and Fusion Network for Survival Analysis With Incomplete Data
Ling, F., Zeng, H., Li, M., Huang, G., Yuan, X., Pun, C., Liao, X., Liang, J.-L., Li, H., & Yang, Q. (2026) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2025.3631651
FusionODE: Biochemical Function Prediction from Microscopy Images via Multimodal Continuous-Time Learning
Zhou, S., Long, J., Miao, Q., Niyogisubizo, J., Zhou, R., Hu, Q., Ma, M., & Wei, Y. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3719-7_30
A multiple instance learning framework with instance identification and supervised contrastive learning for WSI classification
Yuan, L., Hu, G., Qin, N., & Zhao, L. (2026) · Multimedia Systems · doi:10.1007/s00530-026-02515-3
MORM: Multi-omics-Guided Region Mining and Cross-Modal Interaction for Multimodal Survival Prediction
Li, Y., Xu, X., Luo, J., Liang, Q., & Peng, S. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3716-6_13
ProFound-Surv: Empowering Multimodal Cancer Survival Analysis with Pathology Foundation Models via Adaptive Feature Alignment
Wang, Y., Huang, C., Guo, Q., Zhou, W., Chen, Y., 杨照安, Fu, Y., & Yang, Y. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3485-1_10
AdaSurvMamba: Dynamic Fusion and Semantic Scanning for Multimodal Survival Analysis
Zhong, J., Liu, T., Yue, B., Li, J., Piao, Y., Zhang, M., Liu, L., Jiang, J., Ji, W., & Ҫirkinoğlu, H. O. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-38172-9_2
GLMFN: Global-Local multimodal fusion network with contrastive learning for cancer subtype classification
Dong, R., Gao, Y., Liu, J., Lv, E., Ding, Y., Wen, P., & Ge, S. (2026) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2026.111062
Slide-Omics: An Interpretable Bi-Directional Attention Framework for Integrating Multi-Omics and Pathology in Cancer Survival Analysis
Green, C., Momennsab, A., Nguyen, M., Reidel, E., Narayan, S., Parsa, S. P., & Kosaraju, S. (2026) · ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics (ACM-BCB) · doi:10.1145/3807503.3819461
Multimodal AI in high-grade serous ovarian cancer: integrated prediction and clinical decision-making
Pelissier-Combescure, M., Villemin, J.-P., Bonzom, D., Berger, C., Estoup-Streiff, C., Lakhman, Y., Woitek, R., Colombo, P., Lin, Z., Li, H., et al. (2026) · Frontiers in Oncology · doi:10.3389/fonc.2026.1874918
Bidirectional Cross-Modal Attention Gating for Multimodal Estrogen Receptor Status Classification in Breast Cancer
Azam, M. T., Mohamed, W., Ali, K., Aboudessouki, A., Balaha, H. M., El-Melegy, M., Ali, A., Ghazal, M., Khalil, A., Gondim, D. D., et al. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-31930-2_11
Enhancing multimodal survival prediction: tri-modal learning with clinical knowledge integration via state space models
YiJiang, D., Yuanwei, J., & Zhang, W. (2026) · Frontiers in Oncology · doi:10.3389/fonc.2026.1825286
CIGTSurv: Clinical Information Guided Tri-Modal Survival Prediction with Local Prototype Association and Global Feature Alignment
Dai, J., Zhang, Q., Zhou, W., Xu, M. D., Liu, J., Zhang, J., & Xu, H. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-38172-9_14
FORESEE: Multimodal and multiview representation learning for robust cancer survival prediction
Pan, L., Nie, C., Wang, X., & Peng, S. (2026) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2026.111196
HDMoE: A Hierarchical Decoupling-Fusion Mixture-of-Experts Framework for Multimodal Cancer Survival Prediction
Wang, H., Ying, H., Xu, Y., Zheng, Q., Wang, J., Zhang, C., Sun, Y. H., & Wu, J. (2026) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3770855.3818968
MA2MU-Net++: A Mahalanobis distance and Ackley function based multi-modal U-Net++ model for breast nuclei segmentation in histopathology images
Majumder, S., Roy, A., Викторович, Г. В., Kaplun, D., & Sarkar, R. (2026) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2026.111160
Chemotherapy Response Prediction in Non-Small Cell Lung Cancer through Cross-Modal Attention Bridge Network (CMABN)
Pooja, P., & Hameed, H. H. S. (2026) · Engineering Technology & Applied Science Research · doi:10.48084/etasr.19002
Adversarial Evaluation of Multi-Organ Lassification in Computational Pathology with CNNS and Foundation Models
Ziaullah, K., Mozumder, M. A. I., & Kim, H. C. (2026) · International Conference on Electrical, Computer and Energy Technologies (ICECET) · doi:10.1109/icecet65726.2026.11632529
Patch-Level Cross-Modal Learning for Multimodal Estrogen Receptor Status Classification in Breast Cancer Histopathology
Azam, M. T., Mohamed, W., Ali, K., Aboudessouki, A., Balaha, H. M., El-Melegy, M. T., Ali, A., Ghazal, M., Khalil, A., Gondim, D. D., et al. (2026) · Proceedings - International Conference on Image Processing · doi:10.1109/icip61757.2026.11630321
A Vision-Language Framework for Multimodal Retrieval in Glioma Histopathology
Jha, S. K., Rana, A., Singh, J., Beriwal, R., Jana, S., Mathur, M., Akbar, A., Suri, V. S., & Sethi, T. (2026) · Lecture notes in electrical engineering · doi:10.1007/978-981-95-7425-4_49
Multi-dimensional Pathological Feature Enhancement for Cross-Modal Survival Prediction in Lung Adenocarcinoma
Ji, Z., Chen, Q., Shi, X., TIAN, M., Gao, H., Wan, S., Yin, Y., Zhou, X., & Huang, S. (2026) · International Conference on Computer Vision, Image and Deep Learning (CVIDL) · doi:10.1109/cvidl70130.2026.11637865
Multimodal graph neural networks for integrating heterogeneous medical data in clinical decision making
Reddy, K. A. N., Lakshmi, L., Prasad, G. S. C., Priyadharshini, M., Murugesh, V., & Tilaye, G. F. (2026) · Discover Computing · doi:10.1007/s10791-026-10489-z
An Uncertainty-Guided Evidential Deep Learning Framework for Reliability-Aware Multimodal Fusion in Cancer Prognosis
Huang, Y., Yuan, Y., Ye, W., & Ming, W. (2026) · Mathematics · doi:10.3390/math14173059
Prognostic saliency-driven hypergraph neural network for survival prediction via vision foundation model
Li, S., Shang, H., Zhou, J., Lei, M., Zheng, F., Wang, J., Yang, D., Bao, W., & Wang, Y. (2026) · Medical Image Analysis · doi:10.1016/j.media.2026.104286
Neighbor-constrained attention for multiple instance learning in whole-slide histopathology
Li, M., & Li, W. (2026) · Medical & Biological Engineering & Computing · doi:10.1007/s11517-026-03627-8
SeekFlow: Synergizing Radiology and Pathology Foundation Models for Precision Oncology via Knowledge-Guided Evidence Flow
Huang, P., Huang, Y., Chen, Y., Yeung, M., Jiang, Y., & Yu, L. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37432-5_13
Scaling Whole-Slide Pathology Foundation Model Pretraining with Billions Off-the-Shelf Tokens
Li, H., Shui, Z., Zhu, C., & Yang, L. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37335-9_21
Multimodal Artificial Intelligence in Lung Cancer: From Data Integration to Precision Oncology
Chakrabarti, T., Mansour, A. G., Wu, X., Arias-Romero, J., Mambetsariev, I., Chang, N., Santos, S. D., Mirzapoiazova, T., Fricke, J., Kim, J., et al. (2026) · Cancers · doi:10.3390/cancers18182953
Morphology Predicts Grade, Transcriptomics Predicts Nodal Status: Task-Dependent Modality Contributions in Multimodal Prostate Cancer Classification
Moon, C. E., Song, H.-J., & Kim, Y. (2026) · Journal of Imaging · doi:10.3390/jimaging12090446
SYNPRED: A Synergistic Approach to Multimodal Learning for Clinical Prediction
Janíčková, I., Tan, Y. Y., Helbich, T. H., Seeböck, P., Marco, J. D., Bagó-Horváth, Z., Heber, U., Spiegel, T., & Langs, G. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-38172-9_56
Knowledge-Enhanced Representation Learning with Retrieval-Augmented Multimodal Fusion for Survival Prediction
Zhang, Z., Moghadam, P. A., Maneshgar, B., Mirabadi, A. K., Alabi, N., Farahani, H., & Bashashati, A. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-38172-9_34
Angular-Constrained Hyperbolic Learning for Hierarchical Multimodal Survival Prediction
Yang, H., Zhang, Q., Li, Q., & Wang, Y. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-38172-9_4
Retrieval-Based Mixture-of-Experts for Patient-Specific Cancer Survival Prediction with Incomplete Multimodal Data
Lim, M., Kang, B., Jung, H. S., Jo, E., Heo, K.-S., Liu, M., & Kam, T. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-38172-9_51
Di-PACT: Discrete Prognosis via Aligned Co-Topology of Pathways and Tumor Microenvironment
Cai, Z., Li, F., Cai, Q., & Zhang, S. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-38172-9_19
Case-Specific Priors-Guided Multimodal Fusion for Prognosis Prediction in Head and Neck Cancer
Lin, Y., Ye, S., Wu, S., Zheng, D., & Meng, M. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-38172-9_11
RAM-Missing: Retrieval-Augmented Missing-Aware Fusion for Robust Lung Cancer Subtyping
Li, F., Cai, Q., Zhang, S., & Li, J. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-38098-2_55
Imbalance-Aware Distributional Alignment of Heterogeneous Modalities for HER2 Status Prediction
Huang, X., Tang, W., Wu, Y., Cai, J., Liu, Q., Wu, W., Guo, Y., & Zhang, J. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-38072-2_24
A Generalizable Multimodal Model for Treatment‐Stratified Risk and Survival Assessment under Real‐World Constraints: A Multi‐Center Study of Colorectal Cancer
Cao, C., Ji, Z., He, C., Wang, J., Luo, X., Liu, J., Jing, X., Yan, F., Chen, Y., Dong, Z., et al. (2026) · Advanced Science · doi:10.1002/advs.78019 · Cited by 1
HyM2Surv: Hypergraph learning for cross-magnification and microbiome-attentive multimodal survival prediction.
Zhang, F.-Y., Hu, K.-M., Sun, C.-M., Liu, X.-F., Wei, L.-Y., & Su, R. (2026) · Medical Image Analysis · doi:10.1016/j.media.2026.104343
Toward integrated multimodal artificial intelligence in bone marrow examination: current state and open challenges
Lee, S. Y. (2026) · Blood Research · doi:10.1007/s44313-026-00177-8
DSIB: Towards Robust Multimodal Survival Prediction via Discrete Self-Distillation Information Bottleneck
Tian, Y., & Bai, R. (2026) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn69854.2026.11729258
A Controlled Synthetic-Feature Evaluation of Cross-Modal Attention and Shared-Specific Feature Disentanglement for Multi-Sequence Glioma Grading
Zhang, Y.-L. (2026) · 2026 8th International Conference on Artificial Intelligence Technologies and Applications (ICAITA) · doi:10.1109/icaita71179.2026.11737004
Artificial Intelligence-Driven Multimodal Integration of Genomic, Pathological, and Clinical Data in Breast Cancer: A Narrative Review of Clinical Applications and Translational Challenges
Taslimi, J., & Minagar, A. (2026) · Archives of Breast Cancer · doi:10.32768/abc.4829103847-562
Learning discriminative prototypes via Variational Information Bottleneck for multimodal cancer survival prediction
Tan, Z., Cao, R., 魏丕敬, Su, Y., & Zheng, C. (2026) · Medical Image Analysis · doi:10.1016/j.media.2026.104361
A comprehensive review of attention-based multimodal deep learning techniques for lung cancer diagnosis
Payagalage, R., Senanayake, N., & Yapa, P. (2026) · Discover Artificial Intelligence · doi:10.1007/s44163-026-02496-2
Federated Modality Aware Network for Robust Glioblastoma Survival Prediction
Gulati, R. I. S., Keerthika, P., Gulecha, S., & Baghel, T. (2026) · International Conference on Intelligent Cyber Physical Systems and Internet of Things (ICoICI) · doi:10.1109/icoici69824.2026.11724339

Authors: Chen, R. J., Lu, M. Y., Weng, W.-H., Chen, T. Y., Williamson, D. F., Manz, T., Shady, M., & Mahmood, F.
Affiliations: Broad Institute (US)
MIT Computer Science and Artificial Intelligence Laboratory (US)
Brigham and Women's Hospital (US)
ORCID: Chen, Richard J.
Lu, Ming Y.
Weng, Wei-Hung
Chen, Tiffany Y.
Williamson, Drew FK.
Manz, Trevor
Shady, Maha
Mahmood, Faisal
Venue: 2021 IEEE/CVF International Conference on Computer Vision (ICCV)
Publisher: IEEE
Published: 2021-10
Pages: 3995-4005
DOI: 10.1109/iccv48922.2021.00398
OpenAlex: W3203898052
Cited by: 398
References: 75
Citations by year: 2026: 106 · 2025: 121 · 2024: 94 · 2023: 55 · 2022: 22
FWCI: 19.71
Type: conference paper
Language: en
Open access: closed
Licence: doi.org/10.15223/policy-029 (stm-asf)
doi.org/10.15223/policy-037 (stm-asf)
Locations: IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
Topics: Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
Keywords: survival prediction, whole slide images, multimodal representation learning, multiple instance learning, genomic features, tumor microenvironment, visual question answering, ordinal regression, cancer

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Multimodal Co-Attention Transformer for Survival Prediction in Gigapixel Whole Slide Images

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks