The DOI matches its Crossref record; 86 of the 86 references it cites were found in a registry or an index; 9 of the 36 sources its claims cite were checked for support.
A visual–omics foundation model to bridge histopathology with spatial transcriptomics
Open accessPublished version of a preprint
Abstract
Artificial intelligence has revolutionized computational biology. Recent developments in omics technologies, including single-cell RNA sequencing and spatial transcriptomics, provide detailed genomic data alongside tissue histology. However, current computational models focus on either omics or image analysis, lacking their integration. To address this, we developed OmiCLIP, a visual–omics foundation model linking hematoxylin and eosin images and transcriptomics using tissue patches from Visium data. We transformed transcriptomic data into ‘sentences’ by concatenating top-expressed gene symbols from each patch. We curated a dataset of 2.2 million paired tissue images and transcriptomic data across 32 organs to train OmiCLIP integrating histology and transcriptomics. Building on OmiCLIP, our Loki platform offers five key functions: tissue alignment, annotation via bulk RNA sequencing or marker genes, cell-type decomposition, image–transcriptomics retrieval and spatial transcriptomics gene expression prediction from hematoxylin and eosin-stained images. Compared with 22 state-of-the-art models on 5 simulations, and 19 public and 4 in-house experimental datasets, Loki demonstrated consistent accuracy and robustness.
Overview
CoChat Summary
TL;DR
OmiCLIP is a visual–omics foundation model trained on 2.2 million paired H&E tissue patches and spatial transcriptomics to align histology with gene expression. Built on OmiCLIP, Loki provides five multimodal analysis functions and is benchmarked against 22 state-of-the-art methods across simulations and public/in-house datasets.
Key findings
- OmiCLIP was trained on ST-bank, comprising 2,185,571 paired image–transcriptomics patches from 1,007 samples across 32 organs and 113 studies. Contrastive learning significantly improved clustering: CH scores increased for annotated cell types (P < 0.001) and across all organ types (P < 0.05).
- Loki Align aligned ST-to-ST and H&E image-to-ST data. On eight adjacent small intestine sections, median PCC ranged from 0.62 to 0.83 for ST-to-ST and 0.67 to 0.80 for image-to-ST; on two ovarian carcinosarcoma sections, median PCCs were 0.88 and 0.86 versus PASTE 0.26, GPSA 0.43 and CAST 0.71. Visium–Xenium alignment achieved a mean distance of 0.08 mm.
- Loki Annotate using marker genes achieved F1 scores of 0.59–0.96 across four benchmark datasets, versus OpenAI CLIP 0.03–0.34; combining Loki and PLIP gave the best performance, with 93% recall on CRC7K.
- Loki Decompose ranked top two with impact scores of 1.32 (ST mode) and 1.11 (image mode), outperforming methods scoring 0.87 to −1.82. Loki Retrieve improved Recall@5%/10% over OpenAI CLIP and PLIP, and Loki PredEx achieved best MSE in 28 of 39 heart samples and best PCC in 16 of 39 versus Hist2ST, HisToGene, BLEEP and mclSTExp.
Methods
- OmiCLIP uses a CoCa-based dual-encoder framework: a ViT image encoder with 224 × 224 input and a causal masking text encoder with 76-token input, producing 768-dimensional embeddings optimized by contrastive loss. It was trained for 20 epochs on one NVIDIA A100 80-GB GPU with local batch size 64.
- ST-bank was curated from public 10x Visium datasets; H&E images were cropped to spot size, and transcriptomics were converted to “sentences” of the top 50 expressed gene symbols per spot. Quality control removed duplicates, low-resolution images (<2,000 × 2,000 pixels), contaminated/low-quality cells, and housekeeping genes.
- Downstream evaluation used simulations, public datasets and in-house datasets. Metrics included CH score, PCC, Kendall’s tau, JS divergence, SSIM, F1, Recall@K and MSE; statistical comparisons used two-sided Student’s t-test and Wilcoxon rank-sum test.
Limitations
- The paper states that the major limitation is pretraining data size compared with billion-scale visual–language datasets, and that rare conditions may be underrepresented in ST-bank.
- The paper states that OmiCLIP is not generative and cannot directly generate an accurate transcriptomic profile of a query image; it retrieves similar transcriptomic profiles instead. It also notes that integrating two modalities may introduce noise or misalignment, and if one modality dominates, dual-modality gains may be minimal.
- Evident from the text: several evaluations rely on simulated data, pseudo-Visium data or similarity-based retrieval assessment because ground-truth transcriptomics were unavailable; the authors recommend fine-tuning for alignment and decomposition tasks to ensure compatibility with datasets not covered in ST-bank.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures
















Verification
Claims the paper makes
6 Valid · 5 Not valid · 14 Not checked · 121 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“Additionally, higher similarity scores were consistent with higher diagnostic values of tumors calculated by clustering-constrained-attention multiple-instance learning40 (CLAM, a SOTA WSI tumor analysis model; Fig. 3b).”
§ A visual–omics foundation model to bridge histopathology with spatial transcriptomics > Results > Loki Annotate deciphers H&E images with bulk RNA-seq data
What the sources say
- Supportedcites [40] Lu, M. Y. et al. Data-efficient and weakly supervised computational pathology on whole-slide images. Nat. Biomed. Eng. 5, 555–570 (2021). source
The citation identifies CLAM as the whole-slide tumor-analysis model used by the citing authors. The Methods (CLAM) describe its whole-slide classification approach, and the Results (Generalization to independent test cohorts) report strong diagnostic performance. The relationship between similarity scores and diagnostic values is the citing authors’ own finding, not a finding attributed to this source. (LLM, conf=0.90)
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“In three breast cancer tissues, H&E regions corresponding to tumor tissue showed high similarity with the bulk RNA-seq data from tumor biopsies, which include tumor-related markers such as COL1A1 (ref. 38) and ACTB39 (Fig. 3b and Supplementary Fig. 5).”
§ A visual–omics foundation model to bridge histopathology with spatial transcriptomics > Results > Loki Annotate deciphers H&E images with bulk RNA-seq data
What the sources say
- Partly supportedcites [38] Hayashi, M. et al. Identification of the collagen type 1 alpha 1 gene (COL1A1) as a candidate survival-related factor associated with hepatocellular carcinoma. BMC Cancer 14, 108 (2014). sourceExperimental — accuracy not yet measured
The Results (Triple-combination array; page unavailable) identify COL1A1 as associated with hepatocellular carcinoma tumor tissue, supporting its description as a tumor-related gene. The cited paper does not establish COL1A1 as a marker in breast cancer; its data concern liver tumors, not the three breast tissues or their H&E and bulk RNA-seq comparison. The citation is attached to COL1A1, not ACTB. (LLM, conf=0.84)
- Supportedcites [39] Gu, Y. et al. A pan-cancer analysis of the prognostic and immunological role of β-actin (ACTB) in human cancers. Bioengineered 12, 6166–6185 (2021). source
The citation to this paper is attached specifically to ACTB as a tumor-related marker, not to the H&E–bulk RNA-seq comparison. The Results section on ACTB expression in human cancers reports higher ACTB mRNA expression in breast tumor tissue than normal tissue in the TCGA-based analysis; the section on normal tissues also reports higher ACTB protein levels in breast cancer. (LLM, conf=0.87)
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Loki Decompose accurately predicted the distribution of VLMCs, astrocytes, neurons from L2/3, L4/5 and L6 and oligodendrocytes, aligning closely with brain anatomic ref. 49.”
§ A visual–omics foundation model to bridge histopathology with spatial transcriptomics > Results > Loki Decompose maps cell types in H&E image using scRNA-seq
What the sources say
- Supportedcites [49] Tasic, B. et al. Shared and distinct transcriptomic cell types across neocortical areas. Nature 563, 72–78 (2018). source
The citation is offered as the 'brain anatomic' cell-type reference against which the citing tool's predicted proportions are compared, and our copy of Tasic et al. (2018) is exactly such a reference: Extended Data Fig. 9 (Non-neuronal cell types) enumerates astrocytes (Aqp4+), the oligodendrocyte lineage (Sox10, Pdgfra+ OPCs, Enpp6/Opalin+ oligos) and four VLMC types (Lum/Col1a1+, Osr1-Cd74, Spp1-Col15a1, Osr1-Mc5r), while the glutamatergic taxonomy and Extended Data Fig. 11c-e assign types to L2/3, L4 (L4-IT-VISp-Rspo1), L5 (IT and PT) and L6 (L6-IT, L6-CT, L6b) in mouse neocortex, with laminar positions confirmed by RNAscope and the Allen Mouse Brain Reference Atlas diagrams. The alignment/accuracy statement itself is the citing paper's own result and is not judged here; the only caveat is that Tasic et al. sample two areas (VISp and ALM) rather than the whole cortex. (LLM, conf=0.82)
plainstates an associationSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“As an application of Tangram with OmiCLIP embeddings instead of gene expression data, Loki Decompose applied Tangram’s nonconvex optimization algorithm47 to deconvolute the OmiCLIP embeddings of an H&E image patch or the embeddings of a Visium spot’s transcriptomic profile rather than raw gene expression data, providing the cell-type composition of an image patch or a Visium spot (Fig. 5a).”
§ A visual–omics foundation model to bridge histopathology with spatial transcriptomics > Results > Loki Decompose maps cell types in H&E image using scRNA-seq
What the sources say
- Could not verifycites [47] Jain, P. & Kar, P. Non-convex optimization for machine learning. Found. Trends Mach. Learn. 10, 142–363 (2017). sourceExperimental — accuracy not yet measured
Our copy of Jain & Kar (2017) contains no mention of Tangram, Loki Decompose, OmiCLIP, Visium, H&E imaging or cell-type deconvolution; searches for those terms returned only the monograph's generic material (§1.1 'Non-convex Optimization', §6 'Stochastic Optimization Techniques', §6.1–6.7 orthogonal tensor decomposition, alternating minimization) and an unrelated §7 gene-expression-analysis example about sparse regression on microarray data. So the monograph does establish what a non-convex optimization problem/algorithm is in general, but it never states or defines the specific algorithm the citing sentence attributes to it ('Tangram's nonconvex optimization algorithm') nor its application to embedding-based deconvolution. Nothing in our copy contradicts the claim; it simply does not address it, so the attribution cannot be checked against this source. (LLM, conf=0.70)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“results demonstrated that OmiCLIP is robust to variations in image quality.”
§ A visual–omics foundation model to bridge histopathology with spatial transcriptomics > Results > Loki platform powered by contrastive-aligned visual–omics
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
54 more claims
Create a free account to see all 146 claims and to verify the 27 references still open.
- It may state two separate claims as one. 61
- Probably not a substantial claim. 39
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“For example, the OmiCLIP model links H&E images with transcriptomics data, enabling the prediction of gene expression from histology (Chen et al., 2025).”
From Spatial pharmacology of anti-cancer immunomodulators: a comprehensive review (2026) · cites this paper as Chen et al., 2025
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ A visual–omics foundation model to bridge histopathology with spatial transcriptomics > Methods > Loki Decompose
“The mapping matrix M of dimensions spots-by-cells quantifies the likelihood that a given single cell is located within a particular spot. The scRNA-seq data matrix S is structured as cells-by-embeddings, while the ST data or H&E image matrix G is formatted as spots-by-embeddings.”
§ A visual–omics foundation model to bridge histopathology with spatial transcriptomics > Methods > Loki Decompose
“The loss function aims to minimize the cosine distance between the projected single-cell embeddings MTS and the embeddings of ST data or H&E images G”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ A visual–omics foundation model to bridge histopathology with spatial transcriptomics > Methods > Loki Annotate
“The OmiCLIP image encoder encoded images resized to 20 × 20 pixels, consistent with its pretraining.”
§ A visual–omics foundation model to bridge histopathology with spatial transcriptomics > Methods > OmiCLIP model training
“The image encoder was based on a standard vision transformer (ViT)82 with an input image size of 224 × 224 pixels.”
Citations
Cites
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472 · cited 1×
“The image encoder was based on a standard vision transformer (ViT) 82 with an input image size of 224 × 224 pixels.”
Comprehensive Integration of Single-Cell Data
Stuart, T., Butler, A., Hoffman, P., Hafemeister, C., Papalexi, E., Mauck, W. M., Hao, Y., Stoeckius, M., Smibert, P., & Satija, R. (2019) · Cell · doi:10.1016/j.cell.2019.05.031 · Cited by 15,453
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293
SCANPY: large-scale single-cell gene expression data analysis
Wolf, F. A., Angerer, P., & Theis, F. J. (2018) · Genome Biology · doi:10.1186/s13059-017-1382-0 · Cited by 8,881 · cited 2×
“Data pre-processing involved removing duplicates and excluding low-resolution H&E images ( 2,000 × 2,000 pixels), and normalizing raw count matrices following standard protocols using Seurat 70 and Scanpy 71 .”
Dictionary learning for integrative, multimodal and scalable single-cell analysis
Hao, Y., Stuart, T., Kowalski, M. H., Choudhary, S., Hoffman, P., Hartman, A., Srivastava, A., Molla, G., Madad, S., Fernandez-Granda, C., et al. (2024) · Nature Biotechnology · doi:10.1038/s41587-023-01767-y · Cited by 5,631 · cited 1×
“Loki accurately predicted highly variable gene expression, as demonstrated by the spatial distribution of the predicted gene expression (Extended Data Fig.”
A dendrite method for cluster analysis
Caliński, T., & Harabasz, J. (1974) · Communications in Statistics · doi:10.1080/03610927408827101 · Cited by 4,480 · cited 1×
“Therefore, we computed the Calinski–Harabasz (CH) score 29 , a widely used clustering validation metric, which balances the dispersion between clusters with the cohesion within clusters (Methods).”
Point Set Registration: Coherent Point Drift
Myronenko, A., & Song, X. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.46 · Cited by 2,762
Data-efficient and weakly supervised computational pathology on whole-slide images
Lu, M. Y., Williamson, D. F. K., Chen, T. Y., Chen, R. J., Barbieri, M., & Mahmood, F. (2021) · Nature Biomedical Engineering · doi:10.1038/s41551-020-00682-w · Cited by 2,282 · cited 2×
“Additionally, higher similarity scores were consistent with higher diagnostic values of tumors calculated by clustering-constrained-attention multiple-instance learning 40 (CLAM, a SOTA WSI tumor analysis model; Fig.”
Efficient Non-Maximum Suppression
Neubeck, A., & Van Gool, L. (2006) · 18th International Conference on Pattern Recognition (ICPR'06) · doi:10.1109/icpr.2006.479 · Cited by 2,118 · cited 2×
“In summary, Loki Annotate effectively anno-tates H&E images by using tissue-type-specific bulk RNA-seq data as a reference.”
McCarthy, D. J., Campbell, K. R., Lun, A. T. L., & Wills, Q. F. (2017) · Bioinformatics · doi:10.1093/bioinformatics/btw777 · Cited by 1,852
Research works printed but not linked: 1
- Xie, R. et al. Spatially resolved gene expression prediction from histology images via bi-modal contrastive learning. Advances in Neural Information Processing Systems 36 (2024).
Cited by
Liu, Y., Dai, Y., & Wang, L. (2025) · Cancer Cell · doi:10.1016/j.ccell.2025.12.009 · Cited by 193
“In the histology image-omics integration category, models such as LOKI, 133 which uses contrastive language-image pretraining, align H&E histology with spatial transcriptomics to enable zero-shot tasks, including spatial alignment, cell-type decomposition, clustering, and cross-modal retrieval.”
Li, Z., Li, Y., Xiang, J., Wang, X., Yang, S., Zhang, X., Eweje, F. R., Chen, Y. M., Luo, X., Li, Y., et al. (2026) · Nature Medicine · doi:10.1038/s41591-025-04060-4 · Cited by 95
Spatial architecture of development and disease
Lázár, E., & Lundeberg, J. (2025) · Nature Reviews Genetics · doi:10.1038/s41576-025-00892-5 · Cited by 55
“More recently, models such as RNAPath 183 , Starfysh 184 , OmiCLIP 185 and especially ENLIGHT-DeepPT 186 , have embraced complex multimodal learning strategies, integrating spatial priors, transformer-based fusion, pathway knowledge and patient-level clinical outcomes.”
Emerging AI approaches for cancer spatial omics
Noorbakhsh, J., pour, A. F., & Chuang, J. H. (2025) · GigaScience · doi:10.1093/gigascience/giaf128 · Cited by 21
Single‐Cell and Spatial Omics: Methods and Applications
Cen, X., Huang, X., Deng, E., Gong, X., Tan, N., Ye, J. T., Wang, Y., Eils, R., Luo, Q., Li, Y., et al. (2026) · MedComm · doi:10.1002/mco2.70713 · Cited by 18
Metadata
- Authors
- Chen, W., Zhang, P., Tran, T. N., Xiao, Y., Li, S., Shah, V. V., Cheng, H., Brannan, K. W., Youker, K., Lai, L., Fang, L., Yang, Y., Le, N.-T., Abe, J.-I., Chen, S.-H., Ma, Q., Chen, K., Song, Q., Cooke, J. P., & Wang, G.
- Affiliations
- Houston Methodist (US)
- The Ohio State University (US)
- University of Florida (US)
- The University of Texas MD Anderson Cancer Center (US)
- Houston Methodist Research Institute (US)
- Corresponding author
- Wang, Guangyu
- ORCID
- Venue
- Nature Methods
- Published
- 2025-05-29
- Volume
- 22
- Issue
- 7
- Pages
- 1568-1582
- ISSN
- 1548-7091 (ISSN-L)
- 1548-7105
- PMC
- PMC12240810
- OpenAlex
- W4410866390
- Semantic Scholar
- e31363f03f452a7f272a10fe8c178ee6dfbae576
- Cited by
- 118
- References
- 84
- Citations by year
- 2026: 86 · 2025: 32
- FWCI
- 35.47
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Nature Methods (publishedVersion, cc-by)
- Research Square (acceptedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- PubMed Central (submittedVersion, cc-by)
- Digital Commons-TMC (Texas Medical Center) (submittedVersion)
- Funders
- U.S. Department of Health & Human Services | NIH | National Institute of General Medical Sciences (R35GM150460)
- U.S. Department of Health & Human Services | NIH | National Cancer Institute (R01 CA284315)
- Cancer Prevention and Research Institute of Texas (RR220017)
- U.S. Department of Health & Human Services | NIH | National Institute of Neurological Disorders and Stroke (K22 NS112678)
- U.S. Department of Health & Human Services | NIH | National Heart, Lung, and Blood Institute (R01HL169204-01A1)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Single-cell and spatial transcriptomics
- Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Biophysics › Cell Image Analysis Techniques
- Keywords
- histopathology, spatial transcriptomics, spatial gene expression prediction, tissue annotation, multimodal data integration
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-06, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-06 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: A visual–omics foundation model to bridge histopathology with spatial transcriptomics. https://app.cochat.ai/papers/10.1038/s41592-025-02707-1
- The paper
- Chen, W., Zhang, P., Tran, T. N., Xiao, Y., Li, S., Shah, V. V., Cheng, H., Brannan, K. W., Youker, K., Lai, L., Fang, L., Yang, Y., Le, N.-T., Abe, J.-I., Chen, S.-H., Ma, Q., Chen, K., Song, Q., Cooke, J. P., & Wang, G. (2025). A visual–omics foundation model to bridge histopathology with spatial transcriptomics. Nature Methods. https://doi.org/10.1038/s41592-025-02707-1
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.