Inferring super-resolution tissue architecture by integrating spatial transcriptomics with histology
Open accessAbstract
Spatial transcriptomics (ST) has demonstrated enormous potential for generating intricate molecular maps of cells within tissues. Here we present iStar, a method based on hierarchical image feature extraction that integrates ST data and high-resolution histology images to predict spatial gene expression with super-resolution. Our method enhances gene expression resolution to near-single-cell levels in ST and enables gene expression prediction in tissue sections where only histology images are available.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Some deep learning approaches have focused on detecting molecular alterations in image data, while most existing studies have centered on inferring spatial transcriptome or bulk transcriptome information from histopathology images.
“While some deep learning approaches have focused on detecting molecular alterations in image data [132], most existing studies have centered on inferring spatial transcriptome or bulk transcriptome information from histopathology images [133–136].”
From Challenges in AI-driven Biomedical Multimodal Data Fusion and Analysis (2025) · cites this paper as [134]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892 · see citing works
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534 · see citing works
Emerging Properties in Self-Supervised Vision Transformers
Caron, M., Touvron, H., Misra, I., Jegou, H., Mairal, J., Bojanowski, P., & Joulin, A. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00951 · Cited by 5,638 · see citing works
Aran, D., Looney, A. P., Liu, L., Wu, E., Fong, V., Hsu, A., Chak, S., Naikawadi, R. P., Wolters, P. J., Abate, A. R., et al. (2019) · Nature Immunology · doi:10.1038/s41590-018-0276-y · Cited by 4,984 · see citing works
Visualization and analysis of gene expression in tissue sections by spatial transcriptomics
Ståhl, P. L., Salmén, F., Vickovic, S., Lundmark, A., Navarro, J. F., Magnusson, J., Giacomello, S., Asp, M., Westholm, J. O., Huss, M., et al. (2016) · Science · doi:10.1126/science.aaf2403 · Cited by 3,834 · see citing works
B cells and tertiary lymphoid structures promote immunotherapy response
Helmink, B. A., Reddy, S. M., Gao, J., Zhang, S., Basar, R., Thakur, R., Yizhak, K., Sade-Feldman, M., Blando, J., Han, G., et al. (2020) · Nature · doi:10.1038/s41586-019-1922-8 · Cited by 2,731 · cited 1× · see citing works
“The presence of TLSs has been shown to be associated with positive clinical outcomes and responses to immunotherapy 36–38 .”
Self-citationWhat is principal component analysis?
Ringnér, M. (2008) · Nature Biotechnology · doi:10.1038/nbt0308-303 · Cited by 2,407 · see citing works
B cells are associated with survival and immunotherapy response in sarcoma
Petitprez, F., de Reyniès, A., Keung, E. Z., Chen, T. W.-W., Sun, C.-M., Calderaro, J., Jeng, Y.-M., Hsiao, L.-P., Lacroix, L., Bougoüin, A., et al. (2020) · Nature · doi:10.1038/s41586-019-1906-8 · Cited by 1,942 · see citing works
Tertiary lymphoid structures in the era of cancer immunotherapy
Sautès-Fridman, C., Petitprez, F., Calderaro, J., & Fridman, W. H. (2019) · Nature reviews. Cancer · doi:10.1038/s41568-019-0144-6 · Cited by 1,884 · cited 1× · see citing works
“Next, we show that iStar can be utilized to detect multicellular structures, such as tertiary lymphoid structures (TLSs), which are clusters of highly organized immune cells formed in non-lymphoid tissues, often found at sites of inflammation, including a variety of solid tumors 34,35 .”
Spatiotemporal transcriptomic atlas of mouse organogenesis using DNA nanoball-patterned arrays
Chen, A., Liao, S., Cheng, M., Ma, K., Wu, L., Lai, Y., Qiu, X., Yang, J., Xu, J., Hao, S., et al. (2022) · Cell · doi:10.1016/j.cell.2022.04.003 · Cited by 1,784 · see citing works
Self-citationThree-dimensional intact-tissue sequencing of single-cell transcriptional states
Wang, X., Allen, W. E., Wright, M. A., Sylwestrak, E. L., Samusik, N., Vesuna, S., Evans, K., Liu, C., Ramakrishnan, C., Liu, J., et al. (2018) · Science · doi:10.1126/science.aat5691 · Cited by 1,620 · see citing works
Han, K., Xiao, A., Wu, E., Guo, J., Xu, C., & Wang, Y. (2021) · arXiv · doi:10.48550/arxiv.2103.00112 · Cited by 1,394 · cited 3× · see citing works
“23 and 24).”
Highly sensitive spatial transcriptomics at near-cellular resolution with Slide-seqV2
Stickels, R. R., Murray, E., Kumar, P., Li, J., Marshall, J. L., Di Bella, D. J., Arlotta, P., Macosko, E. Z., & Chen, F. (2021) · Nature Biotechnology · doi:10.1038/s41587-020-0739-1 · Cited by 1,295 · see citing works
Hu, J., Li, X., Coleman, K., Schroeder, A., Ma, N., Irwin, D. J., Lee, E. B., Shinohara, R. T., & Li, M. (2021) · Nature Methods · doi:10.1038/s41592-021-01255-8 · Cited by 1,113 · see citing works
Self-citationSpatialDE: identification of spatially variable genes
Svensson, V., Teichmann, S. A., & Stegle, O. (2018) · Nature Methods · doi:10.1038/nmeth.4636 · Cited by 695 · see citing works
Spatially resolved transcriptomics and beyond
Crosetto, N., Bienko, M., & van Oudenaarden, A. (2015) · Nature Reviews Genetics · doi:10.1038/nrg3832 · Cited by 509 · see citing works
Flagged first, then by citations
Research works printed but not linked: 40
- [1]Burgess DJ Spatial transcriptomics coming of age. Nat. Rev. Genet 20, 317 (2019). 30980030
- [2]Asp M, Bergenstrahle J. & Lundeberg J. Spatially Resolved Transcriptomes-Next Generation Tools for Tissue Exploration. Bioessays 42, e1900221 (2020). 32363691
- [4]Moor AE & Itzkovitz S. Spatial transcriptomics: paving the way for tissue-level systems biology. Curr. Opin. Biotechnol 46, 126– 133 (2017). 28346891
- [6]Sun S, Zhu J. & Zhou X. Statistical analysis of spatial expression patterns for spatially resolved transcriptomic studies. Nat. Methods 17, 193– 200 (2020). 31988518 PMC7233129
- [8]Dries R. Giotto: a toolbox for integrative analysis and visualization of spatial expression data. Genome Biol. 22, 78 (2021). 33685491 PMC7938609
- [9]Pham D. stLearn: integrating spatial location, tissue morphology and gene expression to find cell types, cell-cell interactions and spatial trajectories within undissociated tissues. Preprint at bioRxiv, 10.1101/2020.05.31.125658 (2020). · DOI
- [10]Asp M. A Spatiotemporal Organ-Wide Gene Expression and Cell Atlas of the Developing Human Heart. Cell 179, 1647– 1660 e1619 (2019). 31835037
- [12]Lubeck E, Coskun AF, Zhiyentayev T, Ahmad M. & Cai L. Single-cell in situ RNA profiling by sequential hybridization. Nat. Methods 11, 360– 361 (2014). 24681720 PMC4085791
- [13]Shah S, Lubeck E, Zhou W. & Cai L. In Situ Transcription Profiling of Single Cells Reveals Spatial Organization of Cells in the Mouse Hippocampus. Neuron 92, 342– 357 (2016). 27764670 PMC5087994
- [14]Eng CL Transcriptome-scale super-resolved imaging in tissues by RNA seqFISH. Nature 568, 235– 239 (2019). 30911168 PMC6544023
- [15]Moffitt JR Molecular, spatial, and functional single-cell profiling of the hypothalamic preoptic region. Science 362, eaau5324 (2018). PMC6482113 30385464
- [16]Chen KH, Boettiger AN, Moffitt JR, Wang S. & Zhuang X. RNA imaging. Spatially resolved, highly multiplexed RNA profiling in single cells. Science 348, aaa6090 (2015). PMC4662681 25858977
- [19]Badea L. & Stanescu E. Identifying transcriptomic correlates of histology using deep learning. PLoS One 15, e0242858 (2020). PMC7688140 33237966
- [20]Ash JT, Darnell G, Munro D. & Engelhardt BE Joint analysis of expression levels and histological images identifies genes associated with tissue morphology. Nat. Commun 12, 1609 (2021). 33707455 PMC7952575
- [21]Schmauch B. A deep learning model to predict RNA-Seq expression of tumours from whole slide images. Nat. Commun 11, 3877 (2020). 32747659 PMC7400514
- [22]Chen RJ Scaling vision transformers to gigapixel images via hierarchical self-supervised learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 16144– 16155 (2022).
- [26]Bao H, Dong L, Piao S. & Wei F. Beit: Bert pre-training of image transformers. Preprint at arXiv, 10.48550/arXiv.2106.08254 (2021). · DOI · Paper page
- [27]Janesick A. High resolution mapping of the breast cancer tumor microenvironment using integrated single cell, spatial and in situ analysis of FFPE tissue. Preprint at bioRxiv, 10.1101/2022.10.06.510405 (2022). PMC10730913 38114474 · DOI
- [28]Bergenstråhle L. Super-resolved spatial transcriptomics by deep data fusion. Nat. biotechnol 40, 476– 479 (2022). 34845373
- [30]Hamerly G. & Elkan C. Learning the k in k-means. Advances in neural information processing systems 16 (2003).
- [31]Andersson A. Spatial deconvolution of HER2-positive breast cancer delineates tumor-associated cell type interactions. Nat. Commun 12, 6012 (2021). 34650042 PMC8516894
- [33]Wu SZ A single-cell and spatially resolved atlas of human breast cancers. Nat. genetics 53, 1334– 1347 (2021). 34493872 PMC9044823
- [35]Fridman WH B cells and tertiary lymphoid structures as determinants of tumour immune contexture and clinical outcome. Nat. Rev. Clin. Oncol 19, 441– 457 (2022). 35365796
- [38]Cabrita R. Tertiary lymphoid structures improve immunotherapy and survival in melanoma. Nature 577, 561– 565 (2020). 31942071
- [39]Lin JR Multiplexed 3D atlas of state transitions and immune interaction in colorectal cancer. Cell 186, 363– 381 e319 (2023). 36669472 PMC10019067
- [40]Steiner A. How to train your vit? data, augmentation, and regularization in vision transformers. Preprint at arXiv, 10.48550/arXiv.2106.10270 (2021). · DOI · Paper page
- [41]Hölscher DL Next-Generation Morphometry for pathomics-data mining in histopathology. Nat. Commun 14, 470 (2023). 36709324 PMC9884209
- [42]Rappez L. SpaceM reveals metabolic states of single cells. Nat. Methods 18, 799– 805 (2021). 34226721 PMC7611214
- [43]Xu B, Wang N, Chen T. & Li M. Empirical evaluation of rectified activations in convolutional network. Preprint arXiv, 10.48550/arXiv.1505.00853 (2015). · DOI · Paper page
- [44]Clevert D-A, Unterthiner T. & Hochreiter S. Fast and accurate deep network learning by exponential linear units (ELUs). Preprint at arXiv, 10.48550/arXiv.1511.07289 (2015). · DOI · Paper page
- [46]McInnes L, Healy J. & Melville J. Umap: Uniform manifold approximation and projection for dimension reduction. Preprint at arXiv, 10.48550/arXiv.1802.03426 (2018). · DOI · Paper page
- [47]Schurch CM Coordinated Cellular Neighborhoods Orchestrate Antitumoral Immunity at the Colorectal Cancer Invasive Front. Cell 182, 1341– 1359 e1319 (2020). 32763154 PMC7479520
- [49]Hu J. Iterative transfer learning with neural network for clustering and cell type classification in single-cell RNA-seq analysis. Nat machine intelligence 2, 607– 618 (2020). PMC8009055 33817554
- [50]Lu Y. The level weighted structural similarity loss: A step away from MSE. Proceedings of the AAAI Conference on Artificial Intelligence 33, 9989– 9990 (2019).
- [51]Lai W-S, Huang J-B, Ahuja N. & Yang M-H Deep laplacian pyramid networks for fast and accurate super-resolution. Proceedings of the IEEE conference on computer vision and pattern recognition, 624– 632 (2017).
- [52]Dahl R, Norouzi M. & Shlens J. Pixel recursive super resolution. Proceedings of the IEEE international conference on computer vision, 5439– 5448 (2017).
- [53]Masutani EM, Bahrami N. & Hsiao A. Deep learning single-frame and multiframe super-resolution for cardiac MRI. Radiology 295, 552– 561 (2020). 32286192 PMC7263289
- [54]Anwar S, Khan S. & Barnes N. A deep journey into super-resolution: A survey. ACM Computing Surveys (CSUR) 53, 1– 34 (2020).
- [55]Wang Z, Chen J. & Hoi SC Deep learning for image super-resolution: A survey. IEEE transactions on pattern analysis and machine intelligence 43, 3365– 3387 (2020). 32217470
- [56]Ma Y. & Zhou X. Spatially informed cell-type deconvolution for spatial transcriptomics. Nat. Biotechnol 40, 1349– 1359 (2022). 35501392 PMC9464662
Cited by
Sorted by citations
Towards a general-purpose foundation model for computational pathology
Chen, R. J., Ding, T., Lu, M. Y., Williamson, D. F. K., Jaume, G., Song, A. H., Chen, B., Zhang, A., Shao, D., Shaban, M., et al. (2024) · Nature Medicine · doi:10.1038/s41591-024-02857-3 · Cited by 1,560 · see citing works
Resolving tissue complexity by multimodal spatial omics modeling with MISO
Coleman, K., Schroeder, A., Loth, M., Zhang, D., Park, J. H., Sung, J.-Y., Blank, N., Cowan, A. J., Qian, X., Chen, J., et al. (2025) · Nature Methods · doi:10.1038/s41592-024-02574-2 · Cited by 87 · see citing works
Self-citationMultimodal contrastive learning for spatial gene expression prediction using histology images
Min, W., Shi, Z., Zhang, J., Wan, J., & Wang, C. (2024) · Briefings in Bioinformatics · doi:10.1093/bib/bbae551 · Cited by 90 · see citing works
Machine learning methods for histopathological image analysis: updates in 2024
Komura, D., Ochi, M., & Ishikawa, S. (2025) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2024.12.033 · Cited by 76 · see citing works
Challenges in AI-driven Biomedical Multimodal Data Fusion and Analysis
Liu, J., Cen, X., Yi, C., Wang, F.-A., Ding, J., Cheng, J., Wu, Q., Gai, B., Zhou, Y., He, R., et al. (2025) · Genomics, Proteomics & Bioinformatics · doi:10.1093/gpbjnl/qzaf011 · Cited by 75 · see citing works
Generative Artificial Intelligence in Anatomic Pathology
Brodsky, V., Ullah, E., Bychkov, A., Song, A. H., Walk, E. E., Louis, P., Rasool, G., Singh, R. S., Mahmood, F., Bui, M. M., et al. (2025) · Archives of Pathology & Laboratory Medicine · doi:10.5858/arpa.2024-0215-ra · Cited by 48 · see citing works
Advancing biological understanding of cellular senescence with computational multiomics
Li, S., Agudelo Garcia, P. A., Aliferis, C., Becich, M. J., Calyeca, J., Cosgrove, B. D., Elisseeff, J., Farzad, N., Fertig, E. J., Glass, C., et al. (2025) · Nature Genetics · doi:10.1038/s41588-025-02314-y · Cited by 40 · see citing works
- Authors
- Zhang, D., Schroeder, A., Yan, H., Yang, H., Hu, J., Lee, M. Y. Y., Cho, K. S., Susztak, K., Xu, G. X., Feldman, M. D., Lee, E. B., Furth, E. E., Wang, L., & Li, M.
- Affiliations
- University of Pennsylvania (US)
- Emory University (US)
- The University of Texas MD Anderson Cancer Center (US)
- Indiana University Indianapolis (US)
- Corresponding author
- Zhang, Daiwei
- Li, Mingyao
- ORCID
- Venue
- Nature Biotechnology
- Published
- 2024-01-02
- Volume
- 42
- Issue
- 9
- Pages
- 1372-1377
- ISSN
- 0733-222X (ISSN-L)
- 1087-0156
- 1546-1696
- PMC
- PMC11260191
- OpenAlex
- W4390500930
- Cited by
- 219
- References
- 16
- Citations by year
- 2026: 93 · 2025: 73 · 2024: 53
- FWCI
- 29.81
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Licence
- Locations
- Nature Biotechnology (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- IUScholarWorks (Indiana University) (submittedVersion)
- Digital Commons-TMC (Texas Medical Center) (submittedVersion)
- Funders
- U.S. Department of Health & Human Services | NIH | National Institute of General Medical Sciences (R01GM125301)
- U.S. Department of Health & Human Services | NIH | National Heart, Lung, and Blood Institute (R01HL150359)
- U.S. Department of Health & Human Services | NIH | National Eye Institute (R01EY030192)
- U.S. Department of Health & Human Services | NIH | National Institute on Aging (P01AG066597)
- U.S. Department of Health & Human Services | NIH | National Human Genome Research Institute (R01HG013158)
- U.S. Department of Health & Human Services | NIH | National Cancer Institute (R01CA266280, U01CA264583)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Single-cell and spatial transcriptomics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Biophysics › Cell Image Analysis Techniques
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene expression and cancer classification
- Keywords
- spatial transcriptomics, histology, tissue structure, gene expression prediction, ISTAR, single-cell resolution, tissue sections
- MeSH terms
- Algorithms, Animals, Humans, Image Processing, Computer-Assisted, Gene Expression Profiling, Single-Cell Analysis, Transcriptome
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
