Multimodal contrastive learning for spatial gene expression prediction using histology images
Open accessThe DOI matches its Crossref record; 43 of the 44 references it cites were found in a registry or an index.
Abstract
In recent years, the advent of spatial transcriptomics (ST) technology has unlocked unprecedented opportunities for delving into the complexities of gene expression patterns within intricate biological systems. Despite its transformative potential, the prohibitive cost of ST technology remains a significant barrier to its widespread adoption in large-scale studies. An alternative, more cost-effective strategy involves employing artificial intelligence to predict gene expression levels using readily accessible whole-slide images stained with Hematoxylin and Eosin (H&E). However, existing methods have yet to fully capitalize on multimodal information provided by H&E images and ST data with spatial location. In this paper, we propose mclSTExp, a multimodal contrastive learning with Transformer and Densenet-121 encoder for Spatial Transcriptomics Expression prediction. We conceptualize each spot as a “word”, integrating its intrinsic features with spatial context through the self-attention mechanism of a Transformer encoder. This integration is further enriched by incorporating image features via contrastive learning, thereby enhancing the predictive capability of our model. We conducted an extensive evaluation of highly variable genes in two breast cancer datasets and a skin squamous cell carcinoma dataset, and the results demonstrate that mclSTExp exhibits superior performance in predicting spatial gene expression. Moreover, mclSTExp has shown promise in interpreting cancer-specific overexpressed genes, elucidating immune-related genes, and identifying specialized spatial domains annotated by pathologists. Our source code is available at https://github.com/shizhiceng/mclSTExp.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures

![Figure 2 Evaluation of gene expression prediction on the HER2+ datasets by the PCC (ACGs) between the observed and predicted gene expression by STnet [21], HisToGene [15], His2ST [22], THItoGene [23], BLEEP [24], and mclSTExp.](https://figures.cochat.ai/paper-figures/eed4d9dafca54d768e758b1c355cca56/multimodal-contrastive-learning-for-spatial-gene-expression-prediction-using-figure-2.jpg)
![Figure 3 Evaluation of gene expression prediction on the cSCC datasets by the PCC (ACGs) between the observed and predicted gene expression by STnet [21], HisToGene [15], His2ST [22], THItoGene [23], BLEEP [24], and mclSTExp.](https://figures.cochat.ai/paper-figures/9e6ed288e5f6467d90caa63c3d8f94f1/multimodal-contrastive-learning-for-spatial-gene-expression-prediction-using-figure-3.jpg)
![Figure 4 Evaluation of gene expression prediction on the Alex+10x datasets by the PCC (ACGs) between the observed and predicted gene expression by STnet [21], HisToGene [15], His2ST [22], THItoGene [23], BLEEP [24], and mclSTExp.](https://figures.cochat.ai/paper-figures/e692a2c699de402aaba493ab171e099c/multimodal-contrastive-learning-for-spatial-gene-expression-prediction-using-figure-4.jpg)


Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“mclSTExp integrates image features with spatial context via a Transformer encoder, achieving superior performance in predicting highly variable genes in breast cancer and skin squamous cell carcinoma datasets (Min et al., 2024).”
From Spatial pharmacology of anti-cancer immunomodulators: a comprehensive review (2026) · cites this paper as Min et al., 2024
Experimental — accuracy not yet measuredIn the citing paper
“Several studies have explored predicting gene expression from H&E images using AI models59–62.”
From A visual–omics foundation model to bridge histopathology with spatial transcriptomics (2025) · cites this paper as [62]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 1×
“In contrast to the skip connections in ResNet [30], the dense connectivity mechanism in DenseNet-121 enhances the reusability of features, aiding the neural network in capturing image features more effectively and thereby strengthening the model’s expressive capability [26].”
ResultDensely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293
Advances in neural information processing systems 7
(1996) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(96)90463-0 · Cited by 14,281
A simple framework for contrastive learning of visual representations
Chen, T., Kornblith, S., Norouzi, M., & Hinton, G. (2020) · arXiv · doi:10.48550/arxiv.2002.05709 · Cited by 10,857 · cited 1×
“Bleep utilizes a contrastive learning [28] approach, introducing image and gene expression encoders to learn joint embedding in space.”
Spatiotemporal transcriptomic atlas of mouse organogenesis using DNA nanoball-patterned arrays
Chen, A., Liao, S., Cheng, M., Ma, K., Wu, L., Lai, Y., Qiu, X., Yang, J., Xu, J., Hao, S., et al. (2022) · Cell · doi:10.1016/j.cell.2022.04.003 · Cited by 1,784
A single-cell and spatially resolved atlas of human breast cancers
Wu, S. Z., Al-Eryani, G., Roden, D. L., Junankar, S., Harvey, K., Andersson, A., Thennavan, A., Wang, C., Torpy, J. R., Bartonicek, N., et al. (2021) · Nature Genetics · doi:10.1038/s41588-021-00911-1 · Cited by 1,775 · cited 1×
“• The higher resolution (55 μ m per spot) Alex + 10x dataset contains 9 breast cancer tissue samples, including 3 samples (2 fresh-frozen and 1 formalin-fixed-paraffin-embedded, FFPE) from 10x Genomics [30] and six samples obtained from Swarbrick’s laboratory [31].”
Self-citationExploring tissue architecture using spatial transcriptomics
Rao, A., Barkley, D., França, G. S., & Yanai, I. (2021) · Nature · doi:10.1038/s41586-021-03634-9 · Cited by 1,700 · cited 1×
“With the rapid development of spatial transcriptomics (ST) technology, we may gain a more comprehensive understanding of gene expression patterns within complex biological systems [1].”
2007 IEEE Conference on Computer Vision and Pattern Recognition
(2007) · doi:10.1109/cvpr12085.2007 · Cited by 1,688
Multimodal Analysis of Composition and Spatial Architecture in Human Squamous Cell Carcinoma
Ji, A. L., Rubin, A. J., Thrane, K., Jiang, S., Reynolds, D. L., Meyers, R. M., Guo, M. G., George, B. M., Mollbrink, A., Bergenstråhle, J., et al. (2020) · Cell · doi:10.1016/j.cell.2020.05.039 · Cited by 1,117 · cited 1×
“• The lower resolution (100 μ m per spot) cSCC dataset [29] contains 12 tissue sections obtained from four patients, with each patient contributing three sections.”
Research works printed but not linked: 2
- Xie, 2024, Spatially resolved gene expression prediction from histology images via bi-modal contrastive learning, 1
- Dosovitskiy, 2021, An image is worth 16x16 words: transformers for image recognition at scale, 1
Cited by
A visual–omics foundation model to bridge histopathology with spatial transcriptomics
Chen, W., Zhang, P., Tran, T. N., Xiao, Y., Li, S., Shah, V. V., Cheng, H., Brannan, K. W., Youker, K., Lai, L., et al. (2025) · Nature Methods · doi:10.1038/s41592-025-02707-1 · Cited by 118
Machine learning methods for histopathological image analysis: updates in 2024
Komura, D., Ochi, M., & Ishikawa, S. (2025) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2024.12.033 · Cited by 76
SpaMask: Dual masking graph autoencoder with contrastive learning for spatial transcriptomics
Min, W., Fang, D., Chen, J., & Zhang, S. (2025) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1012881 · Cited by 63
Self-citationDiffusion Generative Modeling for Spatially Resolved Gene Expression Inference from Histology ImagesPreprint
Zhu, S.-C., Zhu, Y., Tao, M., & Qiu, P. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2501.15598 · Cited by 49
“…He et al., 2020; Min et al., 2024), some making use of the spatial dependencies between image patches to enhance the prediction (Zeng et al., 2022; Zhang et al., 2024; Pang et al., 2021), and some taking both the local and global image information into consideration for better prediction (Chung…”
AI-predicted spatial transcriptomics unlocks breast cancer biomarkers from pathology
Shulman, E. D., Campagnolo, E. M., Lodha, R., Chung, Y., Stemmer, A., Cantore, T., Ru, B., Chang, T., Biswas, S., Dhruba, S. R., et al. (2026) · Cell · doi:10.1016/j.cell.2026.04.023 · Cited by 50
Self-citationFang, D., & Min, W. (2025) · Communications Biology · doi:10.1038/s42003-025-08810-5 · Cited by 39
“A comprehensive investigation of hyperparameter selection-including neighborhood size, PCA dimensionality, and loss weighting-is provided in Supplementary Text 7, and Supplementary Figs.”
InfluentialSelf-citationXue, S., Zhu, F., Chen, J., & Min, W. (2024) · Briefings in Bioinformatics · doi:10.1093/bib/bbae630 · Cited by 36
“Furthermore, spot-level gene expression prediction methods,such as BLEEP [20] and mclS-TExp [21], aim to predict spot-based spatial gene expression from histology images to eliminate the cost-prohibitive of ST field.”
Self-citationFmH2ST: foundation model-based spatial transcriptomics generation from histological images
Wang, Y., Wang, J., Xu, Y., Liu, N., Liu, B., Li, Y., & Yu, G. (2025) · Nucleic Acids Research · doi:10.1093/nar/gkaf865 · Cited by 24
“Other methods, such as HisToGene [ 16 ], Hist2ST [ 17 ], THItoGene [ 18 ], and mclSTExp [ 19 ], further leverage the spatial information.”
Zhao, J., & Min, W. (2025) · Briefings in Bioinformatics · doi:10.1093/bib/bbaf433 · Cited by 22
“Some approaches have attempted to incorporate histological image features [12, 13] into spatial clustering.”
Self-citationMasked Conditional Diffusion Model with GNN for Spatial Transcriptomics Data Imputation
Li, X. R., Min, W., Wang, S., Wang, C., & Xu, T. (2024) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm62325.2024.10822627 · Cited by 22
Self-citationHan, M., Yang, D., Cheng, J., Zhang, X., Chen, Z., Kuang, H., & Zhang, L. (2025) · Pattern Recognition · doi:10.1016/j.patcog.2025.112458 · Cited by 20
CLIBD: Bridging Vision and Genomics for Biodiversity Monitoring at Scale
Gong, Z. M., Wang, A. T., Huo, X., Haurum, J. B., Lowe, S. C., Taylor, G. W., & Chang, A. X. (2024) · VBN Forskningsportal (Aalborg Universitet) · doi:10.48550/arxiv.2405.17537 · Cited by 20
Takamatsu, M. (2025) · British Journal of Cancer · doi:10.1038/s41416-025-03206-y · Cited by 18
sCellST predicts single-cell gene expression from H& E images
Chadoutaud, L., Lerousseau, M., Herrero-Saboya, D., Ostermaier, J., Fontugne, J., Barillot, E., & Walter, T. (2026) · Nature Communications · doi:10.1038/s41467-025-67965-1 · Cited by 15
Zhao, H., Yin, X., Wang, S., Geng, Z., Yu, W., Yu, H., Ning, S., & Wang, L. (2025) · Nucleic Acids Research · doi:10.1093/nar/gkaf998 · Cited by 15
“Recently, several benchmark methods, such as SpaMask [ 38 ], SpaDAMA [ 39 ], SpaDiT [ 40 ], mclSTExp [ 41 ], SpaICL [ 42 ], and SpaViT [ 43 ], have provided powerful computational strategies for advancing spatial transcriptomics analysis on individual 2D slice.”
High-Resolution Spatial Transcriptomics from Histology Images using HisToSGE
Shi, Z., Xue, S., Zhu, F., & Min, W. (2024) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm62325.2024.10822048 · Cited by 14
“Predicting ST gene expression profiles using H&E images has become a common and cost-effective research method [11].”
Self-citationXiang, J., Hou, S., Li, Y., Quinton, R. J., Zhang, X., Eweje, F. R., Luo, X., Chen, Y. M., Li, Z., Bergstrom, C. P., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2604.03630 · Cited by 14
Exploring multi-instance learning in whole slide imaging: Current and future perspectives
Yu, J., Chen, H., Hu, L., Wu, B., Zhou, S., Zhu, J., Jiang, Y., Han, S., & Wang, Z. (2025) · Pathology - Research and Practice · doi:10.1016/j.prp.2025.156006 · Cited by 13
DANet: spatial gene expression prediction from H&E histology images through dynamic alignment
Wu, Y., Xie, J., Nie, J., Cao, J., Zeng, Y., & Wang, Z. (2025) · Briefings in Bioinformatics · doi:10.1093/bib/bbaf422 · Cited by 13
“Predicting spatial gene expression by the pathological histological image is of great significance to clinical diagnosis and scientific research [1–3].”
Masked Graph Autoencoders with Contrastive Augmentation for Spatially Resolved Transcriptomics Data
Fang, D., Zhu, F., Xie, D., & Min, W. (2024) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm62325.2024.10822738 · Cited by 13
“Furthermore, the complex graph self-supervised contrastive learning frameworks lack consideration of global semantic information and the similarity of spatial structures [13, 14].”
Self-citationDang, T. M., Li, H., Guo, Y., Ma, H., Jiang, F., Miao, Y., Zhou, Q., Gao, J., & Huang, J. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-04927-8_22 · Cited by 12
Lee, Y., Liu, X., Hao, M., Liu, T., & Regev, A. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.12.10.627865 · Cited by 12
Towards Spatial Transcriptomics-driven Pathology Foundation ModelsPreprint
Hemker, K., Song, A. H., Almagro-Pérez, C., Jaume, G., Wagner, S., Vaidya, A. J., Simidjievski, N., Jamnik, M., & Mahmood, F. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2602.14177 · Cited by 12
Spatially Gene Expression Prediction Using Dual-Scale Contrastive Learning
Qu, M., Wu, Y., Di, D., Gao, Y., Su, T., Song, Y., & Fan, L. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-05182-0_56 · Cited by 11
Shi, Z., Zhu, F., & Min, W. (2025) · Methods · doi:10.1016/j.ymeth.2024.11.016 · Cited by 10
Self-citationMulti-Slice Spatial Transcriptomics Data Integration Analysis with STG3Net
Fang, D., Zhu, F., & Min, W. (2024) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm62325.2024.10822331 · Cited by 9
Self-citationImproving cell-type composition inference in spatial transcriptomics with SpaDAMA
Huang, L., Liu, X., Zhu, F., & Min, W. (2025) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1013354 · Cited by 9
“Code availability: All code used in this paper are available at https://github.com/wenwenmin/SpaDAMA. offering invaluable insights into disease mechanisms and guiding targeted therapeutic strategies [1,2].”
Self-citationMultimodal spatial omics: From data acquisition to computational integration
Işık, E. B., Usta, Y. H., Riazi, M., Liu, H., Roach, W., Zhou, H., Nicolaou, A., Rattray, M., & Georgaka, S. (2026) · Patterns · doi:10.1016/j.patter.2026.101592 · Cited by 8
“130,131 Another approach is to utilize a multimodal contrastive loss function to map H&E and spatial proteomics (CODEX) measurements into a joint representation space for cell-type annotation.”
Liu, J., Eckstein, M., Wang, Z., Feuerhake, F., & Merhof, D. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103889 · Cited by 7
Gao, Y., Gao, Y.-L., Jiao, C.-N., Dou, X.-R., Li, F., & Liu, J. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.110338 · Cited by 7
Zha, Y., Feng, S., Gao, P., Zou, Q., & Ma, X. (2025) · Bioinformatics · doi:10.1093/bioinformatics/btaf419 · Cited by 7
“Second, morphological images is also promising for spatial transcriptomics data (Min et al. 2024”
Zhu, J., Deng, R., Guo, J., Yao, T., Xiong, J., Qu, C., Yin, M., Wang, Y., Zhao, S., Yang, H., et al. (2025) · Journal of medical imaging · doi:10.1117/1.jmi.12.6.061410 · Cited by 7
Niu, Y., Liu, J., Zhan, Y., Shi, J., Zhang, D., Reinius, M., Machado, I. P., Crispin‐Ortuzar, M., Wu, J., Li, C., et al. (2026) · Medical Image Analysis · doi:10.1016/j.media.2026.104008 · Cited by 8
Masked adversarial neural network for cell type deconvolution in spatial transcriptomics
Huang, L., Liu, X., Wang, S., & Min, W. (2024) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm62325.2024.10821782 · Cited by 6
“Spatial transcriptomics (ST) data integrate gene expression profiles with spatial information, providing crucial insights for disease research and targeted therapeutic strategies [1–3].”
BackgroundSelf-citationA comprehensive survey of computer vision methods for spatial transcriptomics
Zhu, J., Deng, R., Guo, J., Yao, T., Lu, S., Qu, C., Xiong, J., Zhu, Y., Lu, Z., Yang, Y., et al. (2026) · Briefings in Bioinformatics · doi:10.1093/bib/bbag255 · Cited by 6
“…and TME Characterization 2020 stLearn [ 12], TransformerST [ 49], HE2Gene [ 51], RedeHist [ 52], HisToSGE [ 55], MuCoST [ 56], mclSTExp [ 61], ConGcR [ 64], STMVGAE [ 65], STAIG [ 19], spaMGCN [ 72], SpaICL [ 77], stImage [ 78] Gene Set and Pathway Association/SVG Detection 2021 SpaGCN…”
InfluentialWang, H., Zhang, Y., Zhang, Y., Zhao, X., Bai, Z., Ma, X., & Zhao, C. (2025) · PLoS ONE · doi:10.1371/journal.pone.0329122 · Cited by 5
“TCGN [20], THItoGene [21], and mclSTExp [22] focus on predicting spatial gene expression based on H&E-stained histological images.”
Deep learning-based multimodal pathogenomics integration for precision cancer prognosis
Feng, X., Song, G., Zhang, Y., Guo, L., Jiang, Y., Gong, W., Feng, Y., Xu, C., Yang, Y., & He, M. (2026) · Journal of Translational Medicine · doi:10.1186/s12967-026-07682-5 · Cited by 5
“Recent advances in spatial transcriptomics technolo-gies—such as Loki [42], HistoGene [43], mcISTExp [44], BLEEP [45], and Hist2ST [46]—have enabled the mapping of molecular features to precise spatial locations within tumor tissue.”
Spatia: Multimodal Model for Prediction and Generation of Spatial Cell Phenotypes.Preprint
Kong, Z., Qiu, M., Boesen, J., Lin, X., Yun, S., Chen, T., Kellis, M., & Žitnik, M. (2025) · PubMed Central · Cited by 5
GenAR: Next-scale autoregressive generation for spatial gene expression prediction
Ouyang, J., Wang, Y., Gao, Y., Xu, Y., Yang, S., & Chen, H. (2026) · Medical Image Analysis · doi:10.1016/j.media.2026.104232 · Cited by 4
Shi, Z., Xue, S., & Min, W. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.112647 · Cited by 4
Self-citationChen, T., Xue, W., Zhang, Y., Luo, Y., Liu, C., Shen, W., Wu, S., & Wong, H. (2026) · Briefings in Bioinformatics · doi:10.1093/bib/bbag020 · Cited by 4
“This is essential for accurate downstream biological interpretation [18, 19].”
Wang, K., Shi, L., Li, X., Li, W., Wang, B., Zhou, S., Cao, B., & Zheng, P. (2026) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2026.105003 · Cited by 4
SpaCRD: Multimodal Deep Fusion of Histology and Spatial Transcriptomics for Cancer Region Detection
Xue, S., Wan, J., Zhang, L., & Min, W. (2026) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v40i14.38135 · Cited by 4
“…tasks, such as gene expression prediction (Min et al. 2024), and spatial domain identification (Hu et al. 2024; Hu, Li et al. 2021). mclSTExp (Min et al. 2024) predicts spatial gene expression from H&E-stained whole-slide images by aligning histology and transcriptomics via contrastive…”
Self-citationFang, D., Li, Y., Wang, Z., Zeng, Y., & Min, W. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2603.19766 · Cited by 4
“Complementary contrastive approaches such as BLEEP [36] and mclSTExp [25] learn a joint embedding of histology and expression by enforcing consistency across nearby spots [20].”
Self-citationMajumder, S., Kapse, S., Bhattacharya, M., Xu, X., Yurovsky, A., & Prasanna, P. (2026) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61042.2026.00777 · Cited by 3
Graph-Embedded Deep Generative Clustering for Single-Cell Multi-Omics Data Integration
Liang, C., Chen, W., Gao, L., Wang, C., Zhang, S., & Guo, F. (2026) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2026.3667525 · Cited by 3
“Advanced multi-omics single-cell integration techniques are essential to tackle these issues and to achieve a comprehensive multi-omics analysis [12], [13].”
Self-citationAdvances in predicting omics profiles from imaging data
Beachum, A. H., Xiao, X., Zhou, Y., Li, Q. Y., Xiao, G., & Xu, L. (2026) · Briefings in Bioinformatics · doi:10.1093/bib/bbag090 · Cited by 3
“Building on these histopathology foundation models, recent frameworks have incorporated contrastive learning [37–39] to improve alignment between molecular and image-derived representations before prediction.”
Spatial AI in cancer: mapping immune evasion topology through multi-modal omics and deep learning
Lang, L., Cui, Y., Wang, H., & Xiao, Y. (2026) · Frontiers in Oncology · doi:10.3389/fonc.2026.1762907 · Cited by 3
Song, Y., Fan, J., Chang, H., & Cai, W. (2026) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61042.2026.00494 · Cited by 3
“To explore image-gene relationships more effectively, bi-modal methods [17, 22, 24] co-embed WSI tiles and gene expression profiles in a shared space.”
Liu, H., Farahani, H., Li, X., Xie, Y., & Bashashati, A. (2025) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2025.3595101 · Cited by 3
“For HER2+ and cSCC datasets, previous studies [16], [17], [18], [19], [20] generally used leave-one-out cross-validation.”
Zhu, J., Deng, R., Guo, J., Yao, T., Qu, C., Xiong, J., Lu, S., Lu, Z., Zhu, Y., Lionts, M., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2512.13635 · Cited by 3
“…ST-Net [He et al., 2020], HisToGene [Pang et al., 2021], and EGN [Yang et al., 2023], which directly predict expression values from local image appearance; and retrieval-based vision–omics contrastive learning methods, such as BLEEP [Xie et al., 2024] and mlxSTExp [Min et al., 2024].”
SPATIA: Multimodal Generation and Prediction of Spatial Cell PhenotypesPreprint
Kong, Z., Qiu, M., Boesen, J., Lin, X., Yun, S., Chen, T., Kellis, M., & Žitnik, M. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2507.04704 · Cited by 3
“The central challenge is to learn unified representations that (i) capture the joint structure between image and gene modalities [5, 40], (ii) preserve spatial dependencies at the single-cell level [2, 65], and (iii) generalize across scales from local niches to whole-slide tissue context [49].”
Shi, Z., Wang, C., Wan, J., & Min, W. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2603.22821 · Cited by 3
Self-citationWang, C., & Yu, X. (2026) · Bioinformatics · doi:10.1093/bioinformatics/btag049 · Cited by 2
“Accurate clustering is crucial for downstream tasks such as tissue annotation (Rao et al. 2021), multimodal analysis (Min et al. 2024, Niu et al. 2024, Xue et al. 2024, Zhao and Min 2025), deconvolution inference (Huang et al. 2025), and disease diagnosis (Chen et al. 2023, Shi et al. 2023).”
Self-citationAI‐Driven Cancer Multi‐Omics: A Review From the Data Pipeline Perspective
Liu, S., Li, S. X., & Qian, K. (2026) · Advanced Intelligent Discovery · doi:10.1002/aidi.202500249 · Cited by 2
Hu, J., Pan, X., Yuan, Y., & Shen, H.-B. (2026) · Bioinformatics · doi:10.1093/bioinformatics/btag130 · Cited by 2
“…to the analysis of spatial transcriptomics, ranging from graph convolutional networks (scstGCN) (Xue et al. 2024) and diffusion models (SpaDiT) (Li et al. 2024) to masked autoencoders (SpaMask, SpaDAMA) (Huang et al. 2025; Min et al. 2025) and multi-view learning (mclSTexp) (Min et al. 2024).”
Yin, W., Peng, Q., Meng, F., Wan, Y., Zhang, W., & Zhou, Y. (2026) · Nature Computational Science · doi:10.1038/s43588-026-00992-0 · Cited by 2
SRLST: a unified multimodal representation learning framework for spatial transcriptomics analysis
Lan, W., Deng, X., Ling, T., He, G., Yan, X., Zheng, R., Li, M. M., Pan, S., & Pan, Y. (2026) · Bioinformatics · doi:10.1093/bioinformatics/btag524 · Cited by 2
“…robustness against noise, SpaICL (Zhao and Min 2025) adopts an image-guided curriculum learning strategy to integrate multimodal features, mclSTExp (Min et al. 2024) leverages multimodal contrastive learning for spatial gene expression prediction from histology images, SpaCross (Fang and Min 2025)…”
A Fused Transformer-based Model for Gene Expression Prediction using Histopathology Images
Junayed, M. S., Jeny, A. A., Nabavi, S., & Măndoiu, I. (2024) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm62325.2024.10821923 · Cited by 2
“On the other hand, WSIs stained with hematoxylin and eosin (H&E) present a widely accessible and cost-effective alternative that is routinely used in clinical settings [9].”
Bridging imaging and genomics: Domain knowledge guided spatial transcriptomics analysis
Zhang, W., Liu, X., Chen, T., Xu, W., Sakal, C., Nie, X., Wang, L., & Li, X. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103746 · Cited by 2
Jeong, H., Oh, J., Lee, D., Kang, J. H., & Choi, Y. (2026) · Briefings in Bioinformatics · doi:10.1093/bib/bbag037 · Cited by 2
Liu, A., Zhao, Y., Kim, W.-K., Shen, H., Ding, Z., & Deng, H. (2026) · Genome biology · doi:10.1186/s13059-026-04168-x · Cited by 2
“Our results demonstrated significant improvements over existing methods such as BLEEP [10], Hist2ST [11], HisToGene [12], mclSTExp [13], and ST-Net [14] in terms of mean correlations across top 2,000 highly variable genes (HVGs) and mean correlations among the top 50 highly expressed genes (HEGs).”
InfluentialSpatial pharmacology of anti-cancer immunomodulators: a comprehensive review
Chen, S., Tu, L., & Chen, Q. (2026) · Frontiers in Pharmacology · doi:10.3389/fphar.2026.1898418 · Cited by 1
NicheDeSig: niche-aware deconvolution and adaptive signature analysis for spatial transcriptomics
Xue, W., Zhang, J., Chen, T., Shen, W., Ma, J., Xu, Y., Wong, H., & Wu, S. (2026) · Bioinformatics · doi:10.1093/bioinformatics/btag578 · Cited by 1
“…ST analysis through masked representation learning, as in SpaMask (Min et al. 2025), cross-batch integration, as in SpaBatch (Niu et al. 2025), multimodal spatial expression prediction, as in mclSTExp (Min et al. 2024), and domain-adversarial deconvolution, as in SpaDAMA (Huang et al. 2025).”
Self-citationAre H&E-based computational models transforming molecular pathology diagnostics in cancer?
Zwaenepoel, K., Sucaet, Y., Ramadhan, A., Winne, K. D., Puppels, G. J., Jose Oramas, M., Bossi, P., Saintigny, P., Benaissa, M. H., Caponio, V. C. A., et al. (2025) · Academia oncology. · doi:10.20935/acadonco8050 · Cited by 1
Minh, L. H. N., Nguyen, T.-H., Li, T., Li, T., Huynh, H. H., Raj, M., Yang, C., Xu, M., Vinh, T., & Le, N. Q. K. (2026) · Briefings in Bioinformatics · doi:10.1093/bib/bbag365 · Cited by 1
“Contrastive frameworks such as mclSTExp [60] align image features from a DenseNet encoder with spot expression and positional embeddings from a Transformer encoder in a shared latent space, and use the learned alignment to predict spatial gene expression from H&E images alone.”
Wang, Y., Li, Y., Zhou, C., Ma, X., & Qi, J. (2026) · IEEE Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbbio.2026.3658320 · Cited by 1
“Contrastive learning-based methods [7], [12], [18] use a dual-stream network to embed image and gene data into a high-dimensional space, and then align these high-dimensional representations across modalities.”
PAIR: Reconstructing Single‐Cell Open‐Chromatin Landscapes for Transcription Factor Regulome Mapping
Su, Y., Qi, Q., Fan, Y., Wang, Y., Hao, G., Wong, K., Wang, Y., & Wang, L. (2026) · Advanced Science · doi:10.1002/advs.202524392 · Cited by 1
OrthoST: An orthogonal transformer for domain identification in spatial transcriptomics data
Wang, L., Zhang, H., Yu, K., Xie, W., Yuan, Z., & Li, W. (2026) · Information Fusion · doi:10.1016/j.inffus.2026.104400 · Cited by 1
Li, L., Dong, L., Yu, B., Li, D., Liu, H., Zhu, Y., Mei, X., Zhang, H., & Xu, D. (2026) · IEEE Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbbio.2026.3689143 · Cited by 1
“SpaMask[33]andmclSTExp[34] adoptself-supervisiontoenhancerepresentationlearning,covering graph-based learning and multi-modal image–expression alignment, respectively.”
SpaMGCL: Neighborhood-aware masked graph contrastive learning for spatial transcriptomics analysis
Zhao, J., Fang, D., & Min, W. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.132911 · Cited by 1
Self-citationDong, Y., & Su, Y. (2026) · Frontiers in Immunology · doi:10.3389/fimmu.2026.1821743 · Cited by 1
“By designing a Cross-modal Graph Contrastive Learning framework, the model can learn mapping functions that align these two graphs within a latent representation space (190, 191).”
Enhancing Super‐Resolution Spatial Transcriptomics Data by Transfer Learning
Li, X., Zhang, L., & Min, W. (2026) · Advanced Science · doi:10.1002/advs.76601 · Cited by 1
“A Bi-directional Multi-Head Attention (BMHA) mechanism [ 46 ] is employed to integrate the representations derived from both the gene encoder module and the H&E image encoder module on spatial transcriptomic data.”
Self-citationFreqFLD: Towards all-in-one facial landmark detection via frequency modulation
Shun, R., Jin, K., Hu, S., Song, B., Sun, H., Min, W., Liu, Y., & Wan, J. (2026) · Neural Networks · doi:10.1016/j.neunet.2026.109604 · Cited by 1
“Moreover, to improve cross-domain generalization, more recently, inspired by advances in low-level image restoration [13, 14, 15, 16, 17], and fine-grained visual understanding [18, 19, 20], the All-in-One paradigm has been explored to handle multiple degradations within a single framework.”
Self-citationYeom, T., Choi, D., & Lee, H. (2026) · Briefings in Bioinformatics · doi:10.1093/bib/bbag535 · Cited by 1
“We compared ST-ConMa with six gene expression prediction meth-ods: HisToGene [ 41], Hist2ST [ 14], THItoGene [ 42], BLEEP [ 43], mclST-Exp [ 44], and OmiCLIP [ 17].”
Wu, C., Zuo, M., & Xie, L. (2026) · doi:10.1109/cvprf71909.2026.00528 · Cited by 1
“BLEEP [29] and mclST-Exp [19] are methods based on contrastive learning, and they retrieve gene expression profiles from the training data.”
DeepPathway: predicting pathway expression from histopathology images
Ahsan, M., Hanley, K. P., Fergie, M., O’Leary, C., Borst, G. R., Roncaroli, F., Minhas, F., Rattray, M., Iqbal, M., & Baker, S. M. (2026) · Bioinformatics · doi:10.1093/bioinformatics/btag643 · Cited by 1
“…methods have enabled the prediction of spatial gene expression directly from histology images (He et al. 2020, Bae et al. 2021, Monjo et al. 2022, Jia et al. 2023, Xie et al. 2023, Li et al. 2024, Min et al. 2024, Pizurica et al. 2024, Xiao et al. 2024, Shi et al. 2025, Wang et al. 2025b).”
Masked adversarial neural network for cell type deconvolution in spatial transcriptomicsPreprint
Huang, L., Liu, X., Wang, S., & Min, W. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2408.05065 · Cited by 1
“Spatial transcriptomics (ST) data integrate gene expression profiles with spatial information, providing crucial insights for disease research and targeted therapeutic strategies [1–3].”
BackgroundSelf-citationSanian, M. V., Hemmat, A., Vahidi, A., Maaskola, J., Lee, J. T. H., Prete, M., Makarchuk, S., Demirci, Y., Chipampe, N.-J., Haniffa, M., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2511.14613 · Cited by 1
“Meanwhile, multi-modal contrastive formulations (e.g., mclSTExp) integrate spot context with transformer attention and image–omics contrastive learning [31].”
Zhu, J., Deng, R., Guo, J., Yao, T., Qu, C., Xiong, J., Lu, Z., Zhu, Y., Lionts, M., Yang, Y., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.14104 · Cited by 1
“We benchmark our DUET against SOTA methods, covering regression-based ST-Net [11], EGN [12] and His2ST [37]; retrieval-based BLEEP [13] and mclSTExp [19]; and foundation models Omi-CLIP [23] and UMPIRE [22].”
InfluentialBiswas, S., Patiyal, S., Chen, T. W., Stemmer, A., Dhruba, S. R., Mukherjee, S., Cantore, T., Shulman, E. D., Campagnolo, E. M., Jenkins, B. H., et al. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.05.16.725687 · Cited by 1
“…[19], DeepSpaCE [20]), attention models (HisToGene [21], CarHE[22]), capsule networks (THItoGene [23]), vision transformer-GNN hybrid model (FmH2ST [24], Hist2ST [25]), as well as contrastive learning approaches (BLEEP [26], Reg2ST [27], mclSTExp [28]), and interpretable frameworks (Stimage [29]).”
Amirabad, A. D., Zhu, J., Pati, P., Abdelmoula, W., Mansi, T., & Liao, R. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2608.14924 · Cited by 1
“…(ST-Net (He et al., 2020), EGN (Yang et al., 2023), HisToGene (Pang et al., 2021), His2ST (Zeng et al., 2022), TRIPLEX (Chung et al., 2024)) and retrieval-based methods (BLEEP (Xie et al., 2024), mclSTExp (Min et al., 2024), OpenCLIP (Ilharco et al., 2021), OmiCLIP (Chen et al., 2025)).”
Chung, Y., Ha, J. H., Song, A. H., Almagro-Pérez, C., Seo, C., Suh, W. J., Beom, J. W., Oh, K. B., Ruppin, E., Mahmood, F., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.05956 · Cited by 1
Towards Scalable Context-Aware Single-Cell Spatial Transcriptomics Prediction from Histology ImagesPreprint
Gao, Z., Gu, C., Xiang, J., Luo, X., & Heng, P.-A. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.36429 · Cited by 1
“To democratize spatial molecular profiling, numerous computational methods have been developed to directly infer ST profiles from ubiquitous H&E images [5, 6, 7, 8, 9, 10, 11, 12].”
Gustafsson, F., Boissin, C., Vallon‐Christersson, J., & Rantalainen, M. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2410.00945
“For example, one may train models to directly regress transcriptomic profiles [17, 18, 37, 45], or instead align WSI and gene-expression representations using contrastive objectives before deriving predictions [36, 66].”
A novel multi-modal retrieval framework for tracking vehicles using natural language descriptions
Zhang, C., Liu, Z., Li, K., Li, Y., Qi, X., & Ding, N. (2025) · PLoS ONE · doi:10.1371/journal.pone.0327468
Zare, H., Dehkordi, S. K., Razavian, S.-M.-J., Sajedi, S., Orr, M. E., Pater, M. S., Kolar, G. R., & Solhjoo, S. (2025) · Research Square · doi:10.21203/rs.3.rs-7541289/v1
Dual-Path Knowledge-Augmented Contrastive Alignment Network for Spatially Resolved TranscriptomicsPreprint
Zhang, W., Chu, J., Liu, X., Tong, C., & Li, X. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2511.17685
“BLEEP (Xie et al. 2023), inspired by CLIP (Radford et al. 2021), embeds images and gene profiles jointly to enable retrieval-based inference. mclST-Exp (Min et al. 2024) refines this by encoding gene expression with learnable position embeddings for better spatial integration.”
InfluentialChen, X., & Huang, J. (2025) · ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics (ACM-BCB) · doi:10.1145/3765612.3767228
“We categorize the baselines into two groups: (1) non-finetuned models,including ResNet-50 [9], CLIP [22], PLIP [11]and CONCH [15]; (2)finetunedimage–genepretrainingmodels, including BLEEP [32], MCLSTExp [17] and STimage-1K4M [5].”
Applications of large‐scale artificial intelligence models in bioinformatics
Li, M., Shang, Q., Dong, Z., You, Z. S., Zhang, L., & Xiao, M. (2025) · Quantitative Biology · doi:10.1002/qub2.70026
“For instance, mclSTExp [147] uses hematoxylin and eosin stained images and ST data with spatial locations for multimodal comparative learning.”
Ma, X., Zhan, X., Hu, Y., & Siu, S. W. I. (2026) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.113687
TSRST: A LongNet Transformer Model for Imputing Super-Resolution Spatial Transcriptomics
Xiong, Z., Cheng, R., & Zhang, J. (2025) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm66473.2025.11357199
“BLEEP [15] and mclSTExp [16] integrate histology with ST using multimodal or graph-based learning methods to uncover morphology–expression correlations.”
Self-citationFusing Pixels and Genes: Spatially-Aware Learning in Computational PathologyPreprint
Han, M., Yang, D., Qu, L., Chen, Z., Li, G., Wang, H., Wang, J., & Zhang, L. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2602.13944
“In recent years, methods such as BLEEP (Xie et al., 2024) and mclSTExp (Min et al., 2024) have explored contrastive learning frameworks to align these two modalities, followed by query-reference strategies for gene expression prediction.”
PGST: A Prototype-Guided Parameter-Efficient Network for Spatial Transcriptomics Prediction
He, Y., Hu, K., Sun, C., Wei, L., & Su, R. (2026) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2026.3666148
“We employed DenseNet121 and an image projection head with two fully connected layers as thepatchencoder,aconfigurationidenticaltomclSTExpthathas beenexperimentallyvalidatedforsuperiorperformance[31].”
Zheng, H., Xie, J., Wang, L., Xie, H., Zhi, T., Guo, Y., Zhu, B., Wang, D., & Chen, Y. (2026) · Advanced Science · doi:10.1002/advs.202521151
“With the rise of spatial transcriptomics, methods such as ST-Net [ 19 ], mclSTExp [ 20 ], and iStar [ 21 ] integrate WSIs with paired spatial data to predict gene expression at near-cellular (spot) resolution, improving spatially precise phenotype mapping and prognosis.”
Wang, C., Wang, X., Liu, D., Su, F., Sun, L., Zhao, Z., & Meng, Z. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2604.21573
“…to HAGE [24], the Pearson correlation coefficient averaged over all considered genes improves by 4.0% on cSCC and 9.8% on HER2+. Relative to mclSTExp [18], PCC(ACG) improves by 39.5% on Alex+10x, together with 9.7% and 9.0% reductions in mean squared error and mean absolute error, respectively.”
InfluentialSelf-citationGR2ST: spatial transcriptomics prediction based on graph-enhanced multimodal contrastive learning
Zhou, J., Li, S. L., Han, R., Wang, X., Wang, Y., & Li, J. W. (2026) · Bioinformatics · doi:10.1093/bioinformatics/btag209
“More recently, mclSTExp (Min et al. 2024) and Reg2ST (Wang et al. 2025) introduced multimodal contrastive learning and more flexible spot relationship modeling, respectively.”
Hierarchical MambaOut-Based Spatial Imputation Graph Network for Anatomy-Aware 3D Transcriptomics
Cui, C., Ge, Y., 韩蓓蓓, & Wang, L. (2026) · Electronics · doi:10.3390/electronics15102017
Liu, C. X., Zhu, F., & Min, W. (2026) · Bioinformatics · doi:10.1093/bioinformatics/btag342
“Although ST can generate complex gene expression profiles using many spatial spots, its high cost and low throughput limit its wide clinical application (Zhao et al. 2021, Min et al. 2024, Shi et al. 2025b).”
Self-citationMa, Y., Kang, K., Zhang, M., Liang, P., & Ma, X. (2026) · IEEE Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbbio.2026.3683521
“To fully exploit relations among cells, network-based models, such as SpaGCN [17], SpaMask [22], mclSTExp [23], SpaCross [24], SpaBatch [25], SpaICL [26], SpaCVGAE [27], 2998-4165 © 2026 IEEE.”
Metadata
- Authors
- Min, W., Shi, Z., Zhang, J., Wan, J., & Wang, C.
- Affiliations
- Yunnan University (CN)
- Zhongnan University of Economics and Law (CN)
- Shenzhen Research Institute of Big Data (CN)
- Corresponding author
- Min, Wenwen
- Venue
- Briefings in Bioinformatics
- Publisher
- Oxford University Press (OUP)
- Published
- 2024-09-23
- Volume
- 25
- Issue
- 6
- ISSN
- 1467-5463 (ISSN-L)
- 1477-4054
- PMC
- PMC11952928
- OpenAlex
- W4403853489
- Cited by
- 90
- References
- 44
- Citations by year
- 2026: 55 · 2025: 26 · 2024: 9
- FWCI
- 12.25
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Briefings in Bioinformatics (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Natural Science Foundation of China (62262069)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Single-cell and spatial transcriptomics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene expression and cancer classification
- Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
- Keywords
- spatial transcriptomics, gene expression prediction, histological images, H&E staining, multimodal contrastive learning, Transformer encoder, DenseNet121, spatial context, breast cancer, cutaneous squamous cell carcinoma, highly variable genes, immune-related genes, spatial domain
- MeSH terms
- Machine Learning, Algorithms, Breast Neoplasms, Female, Humans, Image Processing, Computer-Assisted, Gene Expression Regulation, Neoplastic, Computational Biology, Gene Expression Profiling, Transcriptome
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.