Multimodal Co-Attention Transformer for Survival Prediction in Gigapixel Whole Slide Images
Abstract
Survival outcome prediction is a challenging weakly-supervised and ordinal regression task in computational pathology that involves modeling complex interactions within the tumor microenvironment in gigapixel whole slide images (WSIs). Despite recent progress in formulating WSIs as bags for multiple instance learning (MIL), representation learning of entire WSIs remains an open and challenging problem, especially in overcoming: 1) the computational complexity of feature aggregation in large bags, and 2) the data heterogeneity gap in incorporating biological priors such as genomic measurements. In this work, we present a Multimodal Co-Attention Transformer (MCAT) framework that learns an interpretable, dense co-attention mapping between WSIs and genomic features formulated in an embedding space. Inspired by approaches in Visual Question Answering (VQA) that can attribute how word embed-dings attend to salient objects in an image when answering a question, MCAT learns how histology patches attend to genes when predicting patient survival. In addition to visualizing multimodal interactions, our co-attention trans-formation also reduces the space complexity of WSI bags, which enables the adaptation of Transformer layers as a general encoder backbone in MIL. We apply our proposed method on five different cancer datasets (4,730 WSIs, 67 million patches). Our experimental results demonstrate that the proposed method consistently achieves superior performance compared to the state-of-the-art methods.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the comparison between the FSM and MCAT [6] models across the TCGA and CPTAC cohorts, the FSM model consistently demonstrated superior performance in predicting survival outcomes for both LUAD and LUSC cases.
“In the comparison between the FSM and MCAT [6] models across the TCGA and CPTAC cohorts, the FSM model consistently demonstrated superior performance in predicting survival outcomes for both LUAD and LUSC cases.”
From Graph attention-based fusion of pathology images and gene expression for prediction of cancer survival (2024) · cites this paper as [6]
TAH is applicable to models employing attention-weighted pooling of instances, while CoAttn suits multimodal models that fuse WSI and genomic features using co-attention.
“TAH is applicable to models employing attention-weighted pooling of instances, while CoAttn suits multimodal models that fuse WSI and genomic features using co-attention.”
From Graph attention-based fusion of pathology images and gene expression for prediction of cancer survival (2024) · cites this paper as [6]
The work is unique in mixing node and edge embeddings along with fusion of bulk gene expression embeddings to learn a multimodal topographic mapping to predict survival.
“While other researchers have previously explored the development of mid-level-fusion and mixer architectures [6], [7] as well as the use of graph-based representations of WSIs [8], our work is unique in mixing node and edge embeddings along with fusion of bulk gene expression embeddings to learn a multimodal topographic mapping to predict survival.”
From Graph attention-based fusion of pathology images and gene expression for prediction of cancer survival (2024) · cites this paper as [6]
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
Jakubův, J., Chvalovský, K., Goertzel, Z., et al. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,648 · see citing works
Subramanian, A., Tamayo, P., Mootha, V. K., Mukherjee, S., Ebert, B. L., Gillette, M. A., Paulovich, A., Pomeroy, S. L., Golub, T. R., Lander, E. S., et al. (2005) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0506580102 · Cited by 53,625 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
Regression Models and Life-Tables
Cox, D. R. (1972) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1972.tb00899.x · Cited by 39,793 · see citing works
Regression Models and Life-Tables
Cox, D. R. (1992) · Springer Series in Statistics · doi:10.1007/978-1-4612-4380-9_37 · Cited by 26,297 · see citing works
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 21,793 · see citing works
The 2016 World Health Organization Classification of Tumors of the Central Nervous System: a summary
Louis, D. N., Perry, A., Reifenberger, G., von Deimling, A., Figarella‐Branger, D., Cavenee, W. K., Ohgaki, H., Wiestler, O. D., Kleihues, P., & Ellison, D. W. (2016) · Acta Neuropathologica · doi:10.1007/s00401-016-1545-1 · Cited by 15,703 · see citing works
The Molecular Signatures Database Hallmark Gene Set Collection
Liberzon, A., Birger, C., Thorvaldsdóttir, H., Ghandi, M., Mesirov, J. P., & Tamayo, P. (2015) · Cell Systems · doi:10.1016/j.cels.2015.12.004 · Cited by 13,807 · see citing works
Veličković, P., Cucurull, G., Casanova, A., Romero, A., Liò, P., & Bengio, Y. (2017) · arXiv · doi:10.48550/arxiv.1710.10903 · Cited by 12,107
Graph Attention NetworksPreprint
Veličković, P., Casanova, A., Liò, P., Cucurull, G., Romero, A., & Bengio, Y. (2017) · arXiv (Cornell University) · doi:10.17863/cam.48429 · Cited by 12,099 · see citing works
Multimodal Machine Learning: A Survey and Taxonomy
Baltrusaitis, T., Ahuja, C., & Morency, L.-P. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2798607 · Cited by 5,018 · see citing works
Breslow, N. E., Cox, D. R., & Oakes, D. (1985) · Biometrics · doi:10.2307/2530888 · Cited by 4,382 · see citing works
Axiomatic attribution for deep networks
Sundararajan, M., Taly, A., & Yan, Q. (2017) · arXiv · doi:10.48550/arxiv.1703.01365 · Cited by 4,087 · see citing works
Ehteshami Bejnordi, B., Veta, M., Johannes van Diest, P., van Ginneken, B., Karssemeijer, N., Litjens, G., van der Laak, J. A. W. M., and the CAMELYON16 Consortium, Hermsen, M., Manson, Q. F., et al. (2017) · JAMA · doi:10.1001/jama.2017.14585 · Cited by 3,293 · cited 1× · see citing works
“…occupies a tiny proportion of the total image ( i.e. - a needle-in-a-haystack problem such as differentiating normal tissue vs micro-metastases) [4, 3, 5, 45], or 2) spanning the entire composition of a WSI and dependent on the interactions of its components ( i.e. - a fine-grained visual…”
BackgroundClinical-grade computational pathology using weakly supervised deep learning on whole slide images
Campanella, G., Hanna, M. G., Geneslaw, L., Miraflor, A., Werneck Krauss Silva, V., Busam, K. J., Brogi, E., Reuter, V. E., Klimstra, D. S., & Fuchs, T. J. (2019) · Nature Medicine · doi:10.1038/s41591-019-0508-1 · Cited by 2,799 · see citing works
(2011) · Cited by 2,752 · see citing works
Data-efficient and weakly supervised computational pathology on whole-slide images
Lu, M. Y., Williamson, D. F. K., Chen, T. Y., Chen, R. J., Barbieri, M., & Mahmood, F. (2021) · Nature Biomedical Engineering · doi:10.1038/s41551-020-00682-w · Cited by 2,282 · see citing works
Self-citationTensor Fusion Network for Multimodal Sentiment Analysis
Zadeh, A., Chen, M., Poria, S., Cambria, E., & Morency, L.-P. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1115 · Cited by 2,102 · see citing works
Self-Normalizing Neural NetworksPreprint
Klambauer, G., Unterthiner, T., Mayr, A., & Hochreiter, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.02515 · Cited by 2,031 · see citing works
Katzman, J. L., Shaham, U., Cloninger, A., Bates, J., Jiang, T., & Kluger, Y. (2018) · BMC Medical Research Methodology · doi:10.1186/s12874-018-0482-1 · Cited by 1,995 · see citing works
Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Grounding
Fukui, A., Park, D. H., Yang, D., Rohrbach, A., Darrell, T., & Rohrbach, M. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1044 · Cited by 1,582 · see citing works
Hover-Net: Simultaneous segmentation and classification of nuclei in multi-tissue histology images
Graham, S., Vu, Q. D., Raza, S. E. A., Azam, A. S., Tsang, Y. W., Kwak, J. T., & Rajpoot, N. (2019) · Medical Image Analysis · doi:10.1016/j.media.2019.101563 · Cited by 1,445 · see citing works
Hierarchical Question-Image Co-Attention for Visual Question AnsweringPreprint
Lu, J., Yang, J., Batra, D., & Parikh, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.00061 · Cited by 1,303 · see citing works
Attention-based Deep Multiple Instance LearningPreprint
Ilse, M., Tomczak, J. M., & Welling, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.04712 · Cited by 1,235 · see citing works
Predicting cancer outcomes from histology and genomics using convolutional networks
Mobadersany, P., Yousefi, S., Amgad, M., Gutman, D. A., Barnholtz-Sloan, J. S., Velázquez Vega, J. E., Brat, D. J., & Cooper, L. A. D. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1717139115 · Cited by 1,084 · see citing works
Saltz, J., Gupta, R., Hou, L., Kurc, T., Singh, P., Nguyen, V., Samaras, D., Shroyer, K. R., Zhao, T., Batiste, R., et al. (2018) · Cell Reports · doi:10.1016/j.celrep.2018.03.086 · Cited by 1,027 · see citing works
Deep Modular Co-Attention Networks for Visual Question Answering
Zhou, Y., Yu, J., Cui, Y., Tao, D., & Tian, Q. C. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00644 · Cited by 1,013 · see citing works
Stand-Alone Self-Attention in Vision Models
Ramachandran, P., Parmar, N., Vaswani, A., Bello, I., Levskaya, A., & Shlens, J. (2019) · arXiv · doi:10.48550/arxiv.1906.05909 · Cited by 977 · see citing works
Patch-Based Convolutional Neural Network for Whole Slide Tissue Image Classification
Hou, L., Samaras, D., Kurç, T., Gao, Y., Davis, J., & Saltz, J. H. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.266 · Cited by 902 · see citing works
The single-cell pathology landscape of breast cancer
Jackson, H. W., Fischer, J. R., Zanotelli, V. R., Ali, H. R., Mechera, R., Soysal, S. D., Moch, H., Muenst, S. E., Varga, Z., Weber, W. P., et al. (2020) · Nature · doi:10.1038/s41586-019-1876-x · Cited by 891 · see citing works
Linformer: Self-Attention with Linear Complexity
Wang, S., Li, B. Z., Khabsa, M., Fang, H., & Ma, H. (2020) · arXiv · doi:10.48550/arxiv.2006.04768 · Cited by 867 · see citing works
Multi-modal Factorized Bilinear Pooling with Co-attention Learning for Visual Question Answering
Zhou, Y., Yu, J., Jianping Fan, B., & Tao, D. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.202 · Cited by 796 · see citing works
Kim, J.-H., Jun, J., & Zhang, B.-T. (2018) · arXiv · doi:10.48550/arxiv.1805.07932 · Cited by 761 · see citing works
Chen, R. J., Lu, M. Y., Wang, J., Williamson, D. F. K., Rodig, S. J., Lindeman, N. I., & Mahmood, F. (2022) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2020.3021387 · Cited by 729 · see citing works
Self-citationSet Transformer: A Framework for Attention-based Permutation-Invariant Neural Networks
Lee, J., Lee, Y., Kim, J., Kosiorek, A. R., Choi, S., & Teh, Y. W. (2018) · arXiv · doi:10.48550/arxiv.1810.00825 · Cited by 691 · see citing works
DCAN: Deep Contour-Aware Networks for Accurate Gland Segmentation
Chen, H., Qi, X., Yu, L., & Heng, P. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.273 · Cited by 671 · see citing works
AI-based pathology predicts origins for cancers of unknown primary
Lu, M. Y., Chen, T. Y., Williamson, D. F. K., Zhao, M., Shady, M., Lipkova, J., & Mahmood, F. (2021) · Nature · doi:10.1038/s41586-021-03512-4 · Cited by 636 · see citing works
Self-citationHadamard Product for Low-rank Bilinear PoolingPreprint
Kim, J.-H., On, K.-W., Lim, W., Kim, J., Ha, J.-W., & Zhang, B. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.04325 · Cited by 604 · see citing works
Bandi, P., Geessink, O., Manson, Q., Van Dijk, M., Balkenhol, M., Hermsen, M., Ehteshami Bejnordi, B., Lee, B., Paeng, K., Zhong, A., et al. (2019) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2018.2867350 · Cited by 569 · see citing works
Self-citationDeep learning-based classification of mesothelioma improves prediction of patient outcome
Courtiol, P., Maussion, C., Moarii, M., Pronier, E., Pilcer, S., Sefta, M., Manceron, P., Toldo, S., Zaslavskiy, M., Le Stang, N., et al. (2019) · Nature Medicine · doi:10.1038/s41591-019-0583-3 · Cited by 544 · see citing works
Yao, J., Zhu, X., Jonnagaddala, J., Hawkins, N., & Huang, J. (2020) · Medical Image Analysis · doi:10.1016/j.media.2020.101789 · Cited by 522 · see citing works
Towards a Neural StatisticianPreprint
Edwards, H., & Storkey, A. (2016) · International Conference on Learning Representations · doi:10.48550/arxiv.1606.02185 · Cited by 450 · see citing works
Nguyen, D.-K., & Okatani, T. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00637 · Cited by 356 · see citing works
Deep Adversarial Training for Multi-Organ Nuclei Segmentation in Histopathology Images
Mahmood, F., Borders, D., Chen, R. J., McKay, G. N., Salimian, K. J., Baras, A. S., & Durr, N. J. (2019) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2019.2927182 · Cited by 347 · see citing works
Self-citationGeospatial immune variability illuminates differential evolution of lung adenocarcinoma
AbdulJabbar, K., Raza, S. E. A., Rosenthal, R., Jamal‐Hanjani, M., Veeriah, S., Akarca, A. U., Lund, T., Moore, D. A., Salgado, R., Bakir, M. A., et al. (2020) · Nature Medicine · doi:10.1038/s41591-020-0900-x · Cited by 287 · see citing works
WSISA: Making Survival Prediction from Whole Slide Histopathological Images
Zhu, X.-L., Yao, J., Zhu, F.-Y., & Huang, J.-Z. (2017) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2017.725 · Cited by 286 · see citing works
Diao, J. A., Wang, J. K., Chui, W. F., Mountain, V., Gullapally, S. C., Srinivasan, R., Mitchell, R. N., Glass, B., Hoffman, S., Rao, S. K., et al. (2021) · Nature Communications · doi:10.1038/s41467-021-21896-9 · Cited by 281 · see citing works
SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks
Fuchs, F. B., Worrall, D. E., Fischer, V., & Welling, M. (2020) · UvA-DARE (University of Amsterdam) · doi:10.48550/arxiv.2006.10503 · Cited by 280 · see citing works
Federated learning for computational pathology on gigapixel whole slide images
Lu, M. Y., Chen, R. J., Kong, D., Lipkova, J., Singh, R., Williamson, D. F. K., Chen, T. Y., & Mahmood, F. (2022) · Medical Image Analysis · doi:10.1016/j.media.2021.102298 · Cited by 267 · see citing works
Self-citationDeep learning-based survival prediction for multiple cancer types using histopathology images
Wulczyn, E., Steiner, D. F., Xu, Z., Sadhwani, A., Wang, H., Flament-Auvigne, I., Mermel, C. H., Chen, P.-H. C., Liu, Y., & Stumpe, M. C. (2020) · PLOS ONE · doi:10.1371/journal.pone.0233678 · Cited by 266 · see citing works
Neural Image Compression for Gigapixel Histopathology Image Analysis
Tellez, D., Litjens, G. J. S., van der Laak, J. A. W. M., & Ciompi, F. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2936841 · Cited by 246 · see citing works
Zhao, Y. S., Yang, F., Fang, Y., Liu, H., Zhou, N., Zhang, J., Sun, J., Yang, S., Menze, B. H., Fan, X., et al. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00489 · Cited by 206 · see citing works
Bias in Cross-Entropy-Based Training of Deep Survival Networks
Zadeh, S. G., & Schmid, M. C. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.2979450 · Cited by 202 · see citing works
CAMEL: A Weakly Supervised Learning Framework for Histopathology Image Segmentation
Xu, G., Song, Z., Sun, Z., Ku, C., Yang, Z., Liu, C., Wang, S., Ma, J., & Xu, W. H. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.01078 · Cited by 187 · see citing works
Shaban, M., Khurram, S. A., Fraz, M. M., Alsubaie, N. M., Masood, I., Mushtaq, S., Hassan, M., Loya, A., & Rajpoot, N. (2019) · Scientific Reports · doi:10.1038/s41598-019-49710-z · Cited by 175 · see citing works
Wong, W. H. (1986) · The Annals of Statistics · doi:10.1214/aos/1176349844 · Cited by 172 · see citing works
HistoSegNet: Semantic Segmentation of Histological Tissue Type in Whole Slide Images
Chan, L., Hosseini, M. S., Rowsell, C. H., Plataniotis, K. N., & Damaskinos, S. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.01076 · Cited by 152 · see citing works
Chen, R. J., Janković, F., Marinsek, N., Foschini, L., Kourtis, L. C., Signorini, A., Pugh, M., Shen, J., Yaari, R., Maljkovic, V., et al. (2019) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3292500.3330690 · Cited by 119 · see citing works
Self-citationFast Transformers with Clustered AttentionPreprint
Vyas, A., Katharopoulos, A., & Fleuret, F. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.04825 · Cited by 86 · see citing works
Deep Multi-instance Learning for Survival Prediction from Whole Slide Images
Yao, J., Zhu, X., & Huang, J. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-32239-7_55 · Cited by 81 · see citing works
Lu, M. Y., Chen, R. J., Wang, J., Dillon, D., & Mahmood, F. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.10825 · Cited by 80 · see citing works
Self-citationMultiple Instance Captioning: Learning Representations from Histopathology Textbooks and Articles
Gamper, J., & Rajpoot, N. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01628 · Cited by 75 · see citing works
Capturing Cellular Topology in Multi-Gigapixel Pathology Images
Lu, W., Graham, S., Bilal, M., Rajpoot, N., & Minhas, F. (2020) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00138 · Cited by 64 · see citing works
Hosseini, M. S., Chan, L., Tse, G., Tang, M. C. S., Deng, J., Norouzi, S., Rowsell, C. H., Plataniotis, K. N., & Damaskinos, S. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01202 · Cited by 62 · see citing works
Fine-grained histopathological image analysis via robust segmentation and large-scale retrieval
Zhang, X., Su, H., Yang, L., & Zhang, S. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7299174 · Cited by 57 · see citing works
No pixel-level annotations needed
van der Laak, J. A. W. M., Ciompi, F., & Litjens, G. J. S. (2019) · Nature Biomedical Engineering · doi:10.1038/s41551-019-0472-6 · Cited by 22 · see citing works
Learning Permutation Invariant Representations Using Memory Networks
Kalra, S., Adnan, M., Taylor, G., & Tizhoosh, H. R. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58526-6_40 · Cited by 21 · see citing works
Polyp segmentation and classification using predicted depth from monocular endoscopy
Mahmood, F., Yang, Z., Durr, N. J., Chen, R., Xu, W., & Borders, D. (2019) · SPIE Medical Imaging: Computer-Aided Diagnosis · doi:10.1117/12.2513117 · Cited by 20 · see citing works
Self-citationMultimodal Multitask Representation Learning for Pathology Biobank Metadata PredictionPreprint
Weng, W., Cai, Y., Lin, A. S., Tan, F. E., & Chen, P.-H. C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.07846 · Cited by 20 · see citing works
Self-citationNeedles in Haystacks: On Classifying Tiny Objects in Large ImagesPreprint
Pawlowski, N., Bhooshan, S., Ballas, N., Ciompi, F., Glocker, B., & Drozdzal, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.06037 · Cited by 12 · see citing works
Chen, R. J., Lu, M. Y., Shady, M., Lipková, J., Chen, T., Williamson, D. F. K., Joo, B., & Mahmood, F. (2021) · Clinical Cancer Research · doi:10.1158/1557-3265.adi21-po-002 · Cited by 7 · see citing works
Self-citationBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805
Flagged first, then by citations
2 of the 75 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Towards a general-purpose foundation model for computational pathology
Chen, R. J., Ding, T., Lu, M. Y., Williamson, D. F. K., Jaume, G., Song, A. H., Chen, B., Zhang, A., Shao, D., Shaban, M., et al. (2024) · Nature Medicine · doi:10.1038/s41591-024-02857-3 · Cited by 1,560 · see citing works
Self-citationTransformers in medical imaging: A survey
Shamshad, F., Khan, S., Zamir, S. W., Khan, M. H., Hayat, M., Khan, F. S., & Fu, H. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.102802 · Cited by 1,412 · see citing works
Multimodal learning with transformers: A survey
Xu, P., Zhu, X., & Clifton, D. A. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3275156 · Cited by 1,072 · see citing works
A visual-language foundation model for computational pathology
Lu, M. Y., Chen, B., Williamson, D. F. K., Chen, R. J., Liang, I., Ding, T., Jaume, G., Odintsov, I., Le, L. P., Gerber, G., et al. (2024) · Nature Medicine · doi:10.1038/s41591-024-02856-4 · Cited by 966 · see citing works
Self-citationArtificial intelligence for multimodal data integration in oncology
Lipkova, J., Chen, R. J., Chen, B., Lu, M. Y., Barbieri, M., Shao, D., Vaidya, A. J., Chen, C., Zhuang, L., Williamson, D. F. K., et al. (2022) · Cancer Cell · doi:10.1016/j.ccell.2022.09.012 · Cited by 777 · see citing works
Self-citationScaling Vision Transformers to Gigapixel Images via Hierarchical Self-Supervised Learning
Chen, R. J., Chen, C., Li, Y., Chen, T. Y., Trister, A. D., Krishnan, R. G., & Mahmood, F. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01567 · Cited by 621 · see citing works
Self-citationVision Transformers in medical computer vision—A contemplative retrospection
Parvaiz, A., Khalid, M. A., Zafar, R., Ameer, H., Ali, M. U., & Fraz, M. M. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.106126 · Cited by 288 · see citing works
Li, Y., El Habib Daho, M., Conze, P.-H., Zeghlache, R., Le Boité, H., Tadayoni, R., Cochener, B., Lamard, M., & Quellec, G. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.108635 · Cited by 274 · see citing works
Deep multimodal fusion of image and non-image data in disease diagnosis and prognosis: a review
Cui, C., Yang, H., Wang, Y., Zhao, S., Asad, Z., Coburn, L. A., Wilson, K. T., Landman, B. A., & Huo, Y. (2023) · Progress in Biomedical Engineering · doi:10.1088/2516-1091/acc2fe · Cited by 239 · see citing works
Artificial Intelligence in Oncology: Current Landscape, Challenges, and Future Directions
Lotter, W., Hassett, M. J., Schultz, N., Kehl, K. L., Van Allen, E. M., & Cerami, E. (2024) · Cancer Discovery · doi:10.1158/2159-8290.cd-23-1199 · Cited by 196 · see citing works
Multimodal data integration for oncology in the era of deep neural networks: a review
Waqas, A., Tripathi, A., Ramachandran, R. P., Stewart, P. A., & Rasool, G. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1408843 · Cited by 165 · see citing works
Jaume, G., Vaidya, A. J., Chen, R. J., Williamson, D. F. K., Liang, P. P., & Mahmood, F. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01100 · Cited by 161 · see citing works
Self-citationRecent Advances of Deep Learning for Computational Histopathology: Principles and Applications
Wu, Y., Cheng, M., Huang, S., Pei, Z., Zuo, Y., Liu, J., Yang, K., Zhu, Q., Zhang, J., Hong, H., et al. (2022) · Cancers · doi:10.3390/cancers14051199 · Cited by 145 · see citing works
Toward Explainable Artificial Intelligence for Precision Pathology
Klauschen, F., Dippel, J., Keyl, P., Jurmeister, P. S., Bockmayr, M. L., Möck, A., Buchstab, O., Alber, M., Ruff, L., Montavon, G., et al. (2023) · Annual Review of Pathology Mechanisms of Disease · doi:10.1146/annurev-pathmechdis-051222-113147 · Cited by 144 · see citing works
Waqas, A., Bui, M. M., Glassy, E. F., Naqa, I. E., Borkowski, P., Borkowski, A., & Rasool, G. (2023) · Laboratory Investigation · doi:10.1016/j.labinv.2023.100255 · Cited by 149 · see citing works
Xu, Y., & Chen, H. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01942 · Cited by 144 · see citing works
Vision Transformers for Computational Histopathology
Xu, H., Xu, Q., Cong, F., Kang, J., Han, C., Liu, Z., Madabhushi, A., & Lu, C. Y. (2023) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2023.3297604 · Cited by 146 · see citing works
Fast and scalable search of whole-slide images via self-supervised deep learning
Chen, C., Lu, M. Y., Williamson, D. F. K., Chen, T. Y., Schaumberg, A. J., & Mahmood, F. (2022) · Nature Biomedical Engineering · doi:10.1038/s41551-022-00929-8 · Cited by 130 · see citing works
Self-citationIntegrating multi-omics data with EHR for precision medicine using advanced artificial intelligence
Tong, L., Shi, W., Isgut, M., Zhong, Y., Lais, P., Gloster, L., Sun, J., Swain, A., Giuste, F., & Wang, M. D. (2024) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2023.3324264 · Cited by 130 · see citing works
Cross-Modal Translation and Alignment for Survival Analysis
Zhou, F., & Chen, H. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01964 · Cited by 129 · see citing works
Visual Language Pretrained Multiple Instance Zero-Shot Transfer for Histopathology Images
Lu, M. Y., Chen, B., Zhang, A., Williamson, D. F. K., Chen, R. J., Ding, T., Le, L. P., Chuang, Y.-S., & Mahmood, F. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01893 · Cited by 122 · see citing works
Self-citationAdversarial attacks and adversarial robustness in computational pathology
Laleh, N. G., Truhn, D., Veldhuizen, G. P., Han, T., van Treeck, M., Buelow, R. D., Langer, R., Dislich, B., Boor, P., Schulz, V., et al. (2022) · Nature Communications · doi:10.1038/s41467-022-33266-0 · Cited by 115 · see citing works
MambaMIL: Enhancing Long Sequence Modeling with Sequence Reordering in Computational Pathology
Yang, S., Wang, Y., & Chen, H. (2024) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-72083-3_28 · Cited by 114 · see citing works
Prediction of recurrence risk in endometrial cancer with multimodal deep learning
Volinsky-Fremond, S., Horeweg, N., Andani, S., Wolf, J. B., Lafarge, M. W., de Kroon, C. D., Ørtoft, G., Høgdall, E. V. S., Dijkstra, J., Jobsen, J. J., et al. (2024) · Nature Medicine · doi:10.1038/s41591-024-02993-w · Cited by 106 · see citing works
Atabansi, C. C., Nie, J., Liu, H., Song, Q., Yan, L., & Zhou, X. (2023) · BioMedical Engineering OnLine · doi:10.1186/s12938-023-01157-0 · Cited by 99 · see citing works
Teoh, J. R., Dong, J., Zuo, X., Lai, K. W., Hasikin, K. B., & Wu, X. (2024) · PeerJ Computer Science · doi:10.7717/peerj-cs.2298 · Cited by 108 · see citing works
Attention-Challenging Multiple Instance Learning for Whole Slide Image Classification
Zhang, Y., Li, H., Sun, Y., Zheng, S., Zhu, C., & Yang, L. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73668-1_8 · Cited by 90 · see citing works
Li, Z., Jiang, Y., Lu, M., Li, R., & Xia, Y. (2023) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2023.3263010 · Cited by 89 · see citing works
Self-citationArtificial intelligence applications in histopathology
Bahadir, C. D., Omar, M., Rosenthal, J., Marchionni, L., Liechty, B., Pisapia, D. J., & Sabuncu, M. R. (2024) · Nature Reviews Electrical Engineering · doi:10.1038/s44287-023-00012-7 · Cited by 84 · see citing works
Machine learning in computational histopathology: Challenges and opportunities
Cooper, M. C., Ji, Z., & Krishnan, R. G. (2023) · Genes Chromosomes and Cancer · doi:10.1002/gcc.23177 · Cited by 81 · see citing works
Challenges in AI-driven Biomedical Multimodal Data Fusion and Analysis
Liu, J., Cen, X., Yi, C., Wang, F.-A., Ding, J., Cheng, J., Wu, Q., Gai, B., Zhou, Y., He, R., et al. (2025) · Genomics, Proteomics & Bioinformatics · doi:10.1093/gpbjnl/qzaf011 · Cited by 75 · see citing works
Hou, W., Lin, C., Yu, L., Qin, J., Yu, R., & Wang, L. (2023) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2023.3253760 · Cited by 74 · see citing works
Recent advances of Transformers in medical image analysis: A comprehensive review
Xia, K., & Wang, J. (2023) · MedComm – Future Medicine · doi:10.1002/mef2.38 · Cited by 75 · see citing works
Improving diagnosis and prognosis of lung cancer using vision transformers: a scoping review
Ali, H., Mohsen, F., & Shah, Z. (2023) · BMC Medical Imaging · doi:10.1186/s12880-023-01098-z · Cited by 76 · see citing works
Al-Thelaya, K., Gilal, N. U., Alzubaidi, M. S., Majeed, F., Agus, M., Schneider, J., & Househ, M. S. (2023) · Journal of Pathology Informatics · doi:10.1016/j.jpi.2023.100335 · Cited by 63 · see citing works
Di, D., Zou, C., Feng, Y., Zhou, H., Ji, R., Dai, Q., & Gao, Y. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3209652 · Cited by 62 · see citing works
Xiong, C., Chen, H., Sung, J. J. �., & King, I. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/176 · Cited by 63 · see citing works
MuRCL: Multi-Instance Reinforcement Contrastive Learning for Whole Slide Image Classification
Zhu, Z., Yu, L., Wu, W., Yu, R., Zhang, D., & Wang, L. (2022) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2022.3227066 · Cited by 61 · see citing works
Zheng, Y., Li, J., Shi, J., Xie, F., Huai, J., Cao, M., & Jiang, Z. (2023) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2023.3264781 · Cited by 62 · see citing works
A Multimodal Knowledge-enhanced Whole-slide Pathology Foundation Model
Xu, Y., Wang, Y., Zhou, F., Ma, J., Jin, C., Yang, S., Li, J., Zhang, Z., Zhao, C., Zhou, H., et al. (2024) · arXiv · doi:10.48550/arxiv.2407.15362 · Cited by 60 · see citing works
A multimodal knowledge-enhanced whole-slide pathology foundation model
Xu, Y., Wang, Y., Zhou, F., Ma, J., Jin, C., Yang, S., Li, J., Zhang, Z., Zhao, C., Zhou, H., et al. (2025) · Nature Communications · doi:10.1038/s41467-025-66220-x · Cited by 74 · see citing works
Qu, L., Ma, Y., Luo, X., Guo, Q., Wang, M., & Song, Z. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3400876 · Cited by 58 · see citing works
Stegmüller, T., Bozorgtabar, B., Spahr, A., & Thiran, J. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00611 · Cited by 54 · see citing works
Liang, M., Chen, Q., Li, B., Wang, L., Wang, Y., Zhang, Y., Wang, R., Jiang, X. Y., & Zhang, C. (2022) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2022.107268 · Cited by 54 · see citing works
Efficient High-Resolution Deep Learning: A Survey
Bakhtiarnia, A., Zhang, Q., & Iosifidis, A. (2024) · ACM Computing Surveys · doi:10.1145/3645107 · Cited by 67 · see citing works
Harnessing multimodal data fusion to advance accurate identification of fish feeding intensity
Du, Z., Cui, M., Xu, X., Bai, Z., Han, J., Li, W., Yang, J., Liu, X., Wang, C., & Li, D. (2024) · Biosystems Engineering · doi:10.1016/j.biosystemseng.2024.08.001 · Cited by 52 · see citing works
Generative Artificial Intelligence in Anatomic Pathology
Brodsky, V., Ullah, E., Bychkov, A., Song, A. H., Walk, E. E., Louis, P., Rasool, G., Singh, R. S., Mahmood, F., Bui, M. M., et al. (2025) · Archives of Pathology & Laboratory Medicine · doi:10.5858/arpa.2024-0215-ra · Cited by 48 · see citing works
Self-citationPathology-and-Genomics Multimodal Transformer for Survival Outcome Prediction
Ding, K., Zhou, M., Metaxas, D., & Zhang, S. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-43987-2_60 · Cited by 48 · see citing works
Chen, C., Song, M., Song, W., Guo, L., & Jian, M. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3203421 · Cited by 47 · see citing works
Deep learning quantifies pathologists’ visual patterns for whole slide image diagnosis
Nan, T., Zheng, S., Qiao, S., Quan, H., Gao, X., Niu, J., Zheng, B., Guo, C., Zhang, Y., Wang, X., et al. (2025) · Nature Communications · doi:10.1038/s41467-025-60307-1 · Cited by 48 · see citing works
Deep Learning-Based Prediction of Molecular Tumor Biomarkers from H&E: A Practical Review
Couture, H. D. (2022) · Journal of Personalized Medicine · doi:10.3390/jpm12122022 · Cited by 44 · see citing works
Redlich, J.-P., Feuerhake, F., Weis, J., Schaadt, N. S., Teuber‐Hanselmann, S., Buck, C., Luttmann, S., Eberle, A., Nikolin, S., Appenzeller, A., et al. (2024) · npj Imaging · doi:10.1038/s44303-024-00020-8 · Cited by 47 · see citing works
Zheng, Y., Conrad, R. D., Green, E. J., Burks, E. J., Betke, M., Beane, J. E., & Kolachalama, V. B. (2024) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2024.3386108 · Cited by 44 · see citing works
“MCAT [6] implements a co-attention mechanism to merge WSI and genomic features, which are then processed by a : We conducted a series of ablation experiments to assess the impact of different feature extractors, node connectivity types, and the various components of the graph mixer layer (GML),…”
BackgroundMethodInfluentialApplication of Deep Learning in Histopathology Images of Breast Cancer: A Review
Zhao, Y., Zhang, J., Hu, D., Qu, H. Y., Tian, Y., & Cui, X. (2022) · Micromachines · doi:10.3390/mi13122197 · Cited by 42 · see citing works
Shao, Z., Chen, Y., Bian, H., Zhang, J., Liu, G., & Zhang, Y. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i2.25315 · Cited by 42 · see citing works
MoME: Mixture of Multimodal Experts for Cancer Survival Prediction
Xiong, C., Chen, H., Zheng, H., Wei, D., Zheng, Y., Sung, J. J. �., & King, I. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72083-3_30 · Cited by 41 · see citing works
Liu, M., Liu, Y., Cui, H., Li, C., & Ma, J. (2023) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm58861.2023.10385897 · Cited by 39 · see citing works
Bazargani, R., Fazli, L., Gleave, M. E., Goldenberg, L., Bashashati, A., & Salcudean, S. E. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103197 · Cited by 40 · see citing works
Hou, J., Zhang, R., Xie, Y., Li, C., & Qin, W. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-02257-y · Cited by 38 · see citing works
Pancancer outcome prediction via a unified weakly supervised deep learning model
Yuan, W. Q., Chen, Y. M., Zhu, B., Yang, S., Zhang, J., Mao, N., Xiang, J., Ji, Y., Ji, Y., Luo, X., et al. (2025) · Signal Transduction and Targeted Therapy · doi:10.1038/s41392-025-02374-w · Cited by 40 · see citing works
Qiu, L., Zhao, L., Hou, R., Zhao, W., Zhang, S., Lin, Z., Teng, H., & Zhao, J. (2023) · Computerized Medical Imaging and Graphics · doi:10.1016/j.compmedimag.2022.102176 · Cited by 35 · see citing works
Applications of machine and deep learning to thyroid cytology and histopathology: a review
Slabaugh, G., Beltrán, L., Rizvi, H., Deloukas, P., & Marouli, E. (2023) · Frontiers in Oncology · doi:10.3389/fonc.2023.958310 · Cited by 34 · see citing works
Cohort-Individual Cooperative Learning for Multimodal Cancer Survival Analysis
Zhou, H., Zhou, F., & Chen, H. (2024) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2024.3455931 · Cited by 33 · see citing works
Shao, Z., Wang, Y., Chen, Y., Bian, H., Liu, S., Wang, H., & Zhang, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01965 · Cited by 30 · see citing works
Li, L. K., Pan, H., Liang, Y., Shao, M., Xie, S., Lu, S., & Liao, S. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.111502 · Cited by 30 · see citing works
Ji, Z., Ge, Y., Chukwudi, C. C., Kaicheng, U., Zhang, S. M., Peng, Y., Zhu, J., Zaki, H. A., Zhang, X., Yang, S., et al. (2025) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2025.3548048 · Cited by 30 · see citing works
Mondol, R. K., Millar, E. K. A., Sowmya, A., & Meijering, E. (2024) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2024.3418341 · Cited by 29 · see citing works
Ding, S., Li, J., Wang, J., Ying, S., & Shi, J. (2024) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2024.3405535 · Cited by 30 · see citing works
Holste, G., Lin, M., Zhou, R., Wang, F., Liu, L., Yan, Q., Tassel, S. H. V., Kovacs, K. D., Chew, E. Y., Lu, Z., et al. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01207-4 · Cited by 29 · see citing works
Ma, Y., Jamdade, S., Konduri, L., & Sailem, H. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01524-2 · Cited by 29 · see citing works
Discrepancy and Gradient-Guided Multi-modal Knowledge Distillation for Pathological Glioma Grading
Xing, X., Chen, Z., Zhu, M., Hou, Y., Gao, Z., & Yuan, Y. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-16443-9_61 · Cited by 29 · see citing works
Multi-Scale Efficient Graph-Transformer for Whole Slide Image Classification
Ding, S., Li, J., Wang, J., Ying, S., & Shi, J. (2023) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2023.3317067 · Cited by 29 · see citing works
One label is all you need: Interpretable AI-enhanced histopathology for oncology
Tavolara, T. E., Su, Z., Gurcan, M. N., & Niazi, M. K. K. (2023) · Seminars in Cancer Biology · doi:10.1016/j.semcancer.2023.09.006 · Cited by 28 · see citing works
Zhi-hua, W., Yu, L., Ding, X., Liao, X., & Wang, L. (2023) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2023.3287256 · Cited by 27 · see citing works
Lin, H., Hua, J., Gong, Z., Chen, M., Qiu, B., Wu, Y., He, W., Wang, Y., Feng, Z., Liang, Y., et al. (2025) · Cancer Letters · doi:10.1016/j.canlet.2025.217557 · Cited by 27 · see citing works
Temporal Context Matters: Enhancing Single Image Prediction with Disease Progression Representations
Konwer, A., Xu, X., Bae, J., Chen, C., & Prasanna, P. (2022) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr52688.2022.01826 · Cited by 22 · see citing works
Multimodal analysis methods in predictive biomedicine
Qoku, A., Katsaouni, N., Flinner, N., Buettner, F., & Schulz, M. H. (2023) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2023.11.011 · Cited by 29 · see citing works
HONeYBEE: enabling scalable multimodal AI in oncology through foundation model-driven embeddings
Tripathi, A. G., Waqas, A., Schabath, M. B., Yılmaz, Y., & Rasool, G. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-02003-4 · Cited by 37 · see citing works
Hierarchical Transformer for Survival Prediction Using Multimodality Whole Slide Images and Genomics
Li, C., Zhu, X., Yao, J., & Huang, J. (2022) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr56361.2022.9956296 · Cited by 25 · see citing works
Huang, Q., Wu, S., Ou, Z., & Gao, Y. (2025) · Intelligent Oncology · doi:10.1016/j.intonc.2025.03.004 · Cited by 27 · see citing works
Histopathology-based protein multiplex generation using deep learning
Andani, S., Chen, B., Ficek, J., Heinke, S., Casanova, R., Hild, B. F., Sobottka, B., Bodenmiller, B., Aebersold, R. H., Ak, M., et al. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-025-01074-y · Cited by 24 · see citing works
Multi stain graph fusion for multimodal integration in pathology
Dwivedi, C., Nofallah, S., Pouryahya, M., Iyer, J. S., Leidal, K., Chung, C., Watkins, T. R., Billin, A. N., Myers, R. P., Abel, J., et al. (2022) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00200 · Cited by 24 · see citing works
Development of Prognostic Biomarkers by TMB-Guided WSI Analysis: A Two-Step Approach
Liu, X., Liu, Z., Yan, Y., Wang, K., Wang, A., Ye, X., Wang, L., Wei, W., Li, B., Sun, C., et al. (2023) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2023.3249354 · Cited by 22 · see citing works
Digital pathology and multimodal learning on oncology data
Waqas, A., Naveed, J., Shahnawaz, W., Asghar, S., Bui, M. M., & Rasool, G. (2024) · BJR|Artificial Intelligence · doi:10.1093/bjrai/ubae014 · Cited by 21 · see citing works
GTP-4o: Modality-Prompted Heterogeneous Graph Learning for Omni-Modal Biomedical Representation
Li, C., Liu, X., Wang, C., Liu, Y., Yu, W., Shao, J., & Yuan, Y. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73235-5_10 · Cited by 21 · see citing works
Zhou, Q., Zhong, W., Guo, Y., Xiao, M., Ma, H., & Huang, J. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72083-3_35 · Cited by 22 · see citing works
Benkirane, H., Vakalopoulou, M., Planchard, D., Adam, J., Olaussen, K. A., Michiels, S., & Cournède, P.-H. (2025) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1013012 · Cited by 21 · see citing works
Histo-Genomic Knowledge Association for Cancer Prognosis From Histopathology Whole Slide Images
Wang, Z., Zhang, Y., Xu, Y., Imoto, S., Chen, H., & Song, J. (2025) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2025.3526816 · Cited by 20 · see citing works
Multimodal integration strategies for clinical application in oncology
Zhang, B., Wan, Z., Luo, Y., Zhao, X., Samayoa, J. A., Zhao, W., & Wu, S. (2025) · Frontiers in Pharmacology · doi:10.3389/fphar.2025.1609079 · Cited by 23 · see citing works
Wang, W., Wang, R., Mao, L., & Min, B. (2023) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros55552.2023.10341395 · Cited by 20 · see citing works
Shao, W., Shi, H., Liu, J., Zuo, Y., Sun, L., Xia, T., Chen, W., Wan, P., Sheng, J., Zhu, Q., et al. (2024) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2024.3362852 · Cited by 20 · see citing works
Hashimoto, N., Hanada, H., Miyoshi, H., Nagaishi, M., Sato, K., Hontani, H., Ohshima, K., & Takeuchi, I. (2023) · Journal of Pathology Informatics · doi:10.1016/j.jpi.2023.100359 · Cited by 19 · see citing works
Yan, R., Zhang, X., Jiang, Z., Wang, B., Bian, X., Ren, F., & Zhou, S. K. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3611531 · Cited by 20 · see citing works
Qu, L., Yang, Z., Duan, M., Ma, Y., Wang, S., Wang, M., & Song, Z. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01962 · Cited by 19 · see citing works
Yang, Z., Wang, X., Xiang, J., Zhang, J., Yang, S., Wang, X., Yang, W., Li, Z., Han, X., & Liu, Y. (2023) · Archiv für Pathologische Anatomie und Physiologie und für Klinische Medicin · doi:10.1007/s00428-023-03502-z · Cited by 18 · see citing works
Palmal, S., Arya, N., Saha, S., & Tripathy, S. (2024) · Applied Soft Computing · doi:10.1016/j.asoc.2024.111379 · Cited by 18 · see citing works
Fu, M., Fang, M. X., Khan, R. A., Liao, B., Hu, Z., & Wu, F. (2024) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2024.102972 · Cited by 18 · see citing works
Identifying progression-specific Alzheimer’s subtypes using multimodal transformer
Machado Reyes, D., Chao, H., Hahn, J., Shen, L., Yan, P., & Initiative, F. T. A. D. N. (2024) · Journal of Personalized Medicine · doi:10.3390/jpm14040421 · Cited by 17 · see citing works
Elforaici, M. E. A., Montagnon, E., Romero, F., Le, W., Azzi, F., Trudel, D., Nguyen, B., Turcotte, S., Tang, A., & Kadoury, S. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103346 · Cited by 17 · see citing works
A survey on cell nuclei instance segmentation and classification: Leveraging context and attention
Nunes, J. D., Montezuma, D., Oliveira, D., Pereira, T., & Cardoso, J. S. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103360 · Cited by 18 · see citing works
- Authors
- Chen, R. J., Lu, M. Y., Weng, W.-H., Chen, T. Y., Williamson, D. F., Manz, T., Shady, M., & Mahmood, F.
- Affiliations
- Broad Institute (US)
- MIT Computer Science and Artificial Intelligence Laboratory (US)
- Brigham and Women's Hospital (US)
- ORCID
- Venue
- 2021 IEEE/CVF International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2021-10
- Pages
- 3995-4005
- OpenAlex
- W3203898052
- Cited by
- 398
- References
- 75
- Citations by year
- 2026: 106 · 2025: 121 · 2024: 94 · 2023: 55 · 2022: 22
- FWCI
- 19.71
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- survival prediction, whole slide images, multimodal representation learning, multiple instance learning, genomic features, tumor microenvironment, visual question answering, ordinal regression, cancer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
