Generation of synthetic whole-slide image tiles of tumours from RNA-sequencing data via cascaded diffusion models
Open accessThe DOI matches its Crossref record; 75 of the 75 references it cites were found in a registry or an index.
Abstract
Training machine-learning models with synthetically generated data can alleviate the problem of data scarcity when acquiring diverse and sufficiently large datasets is costly and challenging. Here we show that cascaded diffusion models can be used to synthesize realistic whole-slide image tiles from latent representations of RNA-sequencing data from human tumours. Alterations in gene expression affected the composition of cell types in the generated synthetic image tiles, which accurately preserved the distribution of cell types and maintained the cell fraction observed in bulk RNA-sequencing data, as we show for lung adenocarcinoma, kidney renal papillary cell carcinoma, cervical squamous cell carcinoma, colon adenocarcinoma and glioblastoma. Machine-learning models pretrained with the generated synthetic data performed better than models trained from scratch. Synthetic data may accelerate the development of machine-learning models in scarce-data settings and allow for the imputation of missing data modalities.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Additionally, we demonstrate the utility of pretraining with synthetic fine-grained morphological descriptions, suggesting the scaling potential of TITAN pretraining with synthetic data89–91.”
From A multimodal whole-slide foundation model for pathology (2025) · cites this paper as [91]
In the citing paper
“Carrillo-Perez et al. developed a cascaded-diffusion model to synthesize WSIs from RNA-seq data, which accurately preserved the distribution of cell types in WSIs [157].”
From Challenges in AI-driven Biomedical Multimodal Data Fusion and Analysis (2025) · cites this paper as [157]
In the citing paper
“Generative AI models, such as large language models or vision-language models capable of synthesizing images, additionally allow for tailored data augmentation by creating new examples that meet a set of targeted criteria.71, 72, 73”
From Bias in artificial intelligence for medical imaging: fundamentals, detection, avoidance, mitigation, challenges, ethics, and prospects (2024) · cites this paper as [71]
Citations
Cites
Sung, H., Ferlay, J., Siegel, R. L., Laversanne, M., Soerjomataram, I., Jemal, A., & Bray, F. (2021) · CA: A Cancer Journal for Clinicians · doi:10.3322/caac.21660 · Cited by 108,699 · cited 1×
“Cancer is one of the leading causes of death worldwide, just behind cardiovascular diseases 1 .”
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
A Threshold Selection Method from Gray-Level Histograms
Otsu, N. (1979) · IEEE Transactions on Systems, Man, and Cybernetics · doi:10.1109/tsmc.1979.4310076 · Cited by 43,398 · cited 1×
“First, a mask of the tissue in the higher resolution of the SVS file was obtained using the Otsu threshold method 60 .”
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293 · cited 1×
“Similarly, Dall-E 2 authors obtained a CLIP image-text embedding 35 to condition the diffusion model during the generation step.”
Robust enumeration of cell subsets from tissue expression profiles
Newman, A. M., Liu, C. L., Green, M. R., Gentles, A. J., Feng, W., Xu, Y., Hoang, C. D., Diehn, M., & Alizadeh, A. A. (2015) · Nature Methods · doi:10.1038/nmeth.3337 · Cited by 13,224 · cited 3×
“The results demonstrated a consistent pattern: synthetic tiles generated using profiles featuring higher fractions, as predicted by CIBERSORTx, exhibited a noticeably augmented proportion of the corresponding cell types in terms of proliferation.”
Nishimura, R., Osako, T., Nishiyama, Y., Tashima, R., Nakano, M., Fujisue, M., Toyozumi, Y., & Arima, N. (2013) · Oncology · doi:10.1159/000353099 · Cited by 13,166
NCBI GEO: archive for functional genomics data sets—update
Barrett, T., Wilhite, S. E., Ledoux, P., Evangelista, C., Kim, I. F., Tomashevsky, M., Marshall, K. A., Phillippy, K. H., Sherman, P. M., Holko, M., et al. (2012) · Nucleic Acids Research · doi:10.1093/nar/gks1193 · Cited by 10,418 · cited 1×
“For instance, there are thousands of gene-expression series in the Gene Expression Omnibus (GEO) platform 22 for which the Article https://doi.org/10.1038/s41551-024-01193-8 models 34 .”
InfluentialNear-optimal probabilistic RNA-seq quantification
Bray, N. L., Pimentel, H., Melsted, P., & Pachter, L. (2016) · Nature Biotechnology · doi:10.1038/nbt.3519 · Cited by 9,723 · cited 1×
“The raw sequencing reads were aligned to the human transcriptome and quantified using the Kallisto–Sleuth algorithm 57 .”
The cancer genome atlas pan-cancer analysis project
The Cancer Genome Atlas Research Network, Weinstein, J. N., Collisson, E. A., Mills, G. B., Shaw, K. R. M., Ozenberger, B. A., Ellrott, K., Shmulevich, I., Sander, C., & Stuart, J. M. (2013) · Nature Genetics · doi:10.1038/ng.2764 · Cited by 9,092 · cited 10×
“Similarly, synthetic tiles generated using RNA-seq data sourced from patients exhibiting higher fractions of haematopoietic cells demonstrated an elevated presence of lymphocytes, except for TCGA-KIRP (Supplementary Table 1).”
InfluentialDenoising Diffusion Probabilistic Models
Ho, J., Jain, A., & Abbeel, P. (2020) · arXiv · doi:10.48550/arxiv.2006.11239 · Cited by 8,712 · cited 1×
“This can be achieved by applying the denoising function f to the samples to obtain x 0 and then making the transition to x t − n by using the predicted ˆ x 0 (ref.”
Research works printed but not linked: 7
- Lee, C. & van der Schaar, M. A variational information bottleneck approach to multi-omics data integration. In International Conference on Artificial Intelligence and Statistics 1513–1521 (PMLR, 2021).
- Quiros, A. C., Murray-Smith, R. & Yuan, K. PathologyGAN: learning deep representations of cancer tissue. In Proceedings of the Third Conference on Medical Imaging with Deep Learning 121, 669–695 (PMLR, 2020).
- Quiros, A. C., Murray-Smith, R. & Yuan, K. Learning a low dimensional manifold of real cancer tissue with PathologyGAN. Preprint at https://arxiv.org/abs/1907.02644v5 (2020). · arXiv
- Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V. & Courville, A. C. Improved training of wasserstein gans. In Advances in Neural Information Processing Systems 30 (eds Guyon, I. et al.) 5769–5779 (Curran Associates, 2017).
- Zhao, S., Song, J. & Ermon, S. Infovae: balancing learning and inference in variational autoencoders. Proc. AAAI Conf. Artif. Intell. 33, 5885–5892 (2019).
- Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B. & Hochreiter, S. Gans trained by a two time-scale update rule converge to a local nash equilibrium. In Advances in Neural Information Processing Systems 30 (eds Guyon, I. et al.) 6629–6640 (Curran Associates, 2017).
- Lehrer, M. et al. in Advances in Biology and Treatment of Glioblastoma (ed. Somasundaram, K.) 143–159 (Springer, 2017).
Cited by
Koçak, B., Ponsiglione, A., Stanzione, A., Bluethgen, C., Santinha, J., Ugga, L., Huisman, M., Klontzas, M. E., Cannella, R., & Cuocolo, R. (2024) · Diagnostic and Interventional Radiology · doi:10.4274/dir.2024.242854 · Cited by 168
Wang, J., Wang, K., Yu, Y., Lu, Y., Xiao, W., Sun, Z., Liu, F., Zou, Z., Gao, Y., Yang, L., et al. (2024) · Nature Medicine · doi:10.1038/s41591-024-03359-y · Cited by 231
“…AI can synthesize high-quality chest X-ray images 5–7 , morphologically preserving three-dimensional (3D) images of the brain 8 and two-dimensional (2D) images of histopathology 9,10 and dermatology 10,11 , and consequently enhancing medical image understanding and improving downstream analysis.”
A multimodal whole-slide foundation model for pathology
Ding, T., Wagner, S. J., Song, A. H., Chen, R. J., Lu, M. Y., Zhang, A., Vaidya, A. J., Jaume, G., Shaban, M., Kim, A., et al. (2025) · Nature Medicine · doi:10.1038/s41591-025-03982-3 · Cited by 171
Challenges in AI-driven Biomedical Multimodal Data Fusion and Analysis
Liu, J., Cen, X., Yi, C., Wang, F.-A., Ding, J., Cheng, J., Wu, Q., Gai, B., Zhou, Y., He, R., et al. (2025) · Genomics, Proteomics & Bioinformatics · doi:10.1093/gpbjnl/qzaf011 · Cited by 75
“Carrillo-Perez et al. developed a cascaded-diffusion model to synthesize WSIs from RNA-seq data, which accurately preserved the distribution of cell types in WSIs [157].”
Generative Artificial Intelligence in Anatomic Pathology
Brodsky, V., Ullah, E., Bychkov, A., Song, A. H., Walk, E. E., Louis, P., Rasool, G., Singh, R. S., Mahmood, F., Bui, M. M., et al. (2025) · Archives of Pathology & Laboratory Medicine · doi:10.5858/arpa.2024-0215-ra · Cited by 48
Metadata
- Authors
- Carrillo-Perez, F., Pizurica, M., Zheng, Y., Nandi, T. N., Madduri, R., Shen, J., & Gevaert, O.
- Affiliations
- Ghent University (BE)
- Stanford Medicine (US)
- Argonne National Laboratory (US)
- Corresponding author
- Gevaert, Olivier
- Venue
- Nature Biomedical Engineering
- Published
- 2024-03-21
- Volume
- 9
- Issue
- 3
- Pages
- 320-332
- ISSN
- 2157-846X (ISSN-L)
- OpenAlex
- W4393055385
- Semantic Scholar
- 9eedc2a6a4bc551eca06dc1da9fc518a37f05773
- Cited by
- 37
- References
- 69
- Citations by year
- 2026: 13 · 2025: 19 · 2024: 5
- FWCI
- 5.46
- Type
- journal article
- Language
- en
- Open access
- green
- biblio.ugent.be
- Licence
- other-oa
- www.springernature.com/gp/researchers/text-and-data-mining (tdm)
- www.springernature.com/gp/researchers/text-and-data-mining (vor)
- Locations
- Nature Biomedical Engineering (publishedVersion)
- PubMed (publishedVersion)
- Ghent University Academic Bibliography (Ghent University) (submittedVersion, other-oa)
- Funders
- National Cancer Institute (R01CA260271)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Single-cell and spatial transcriptomics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene expression and cancer classification
- Keywords
- synthetic data generation, RNA sequencing, tumours, cell type composition, gene expression changes, data scarcity, lung adenocarcinoma, papillary renal cell carcinoma, cervical squamous cell carcinoma, colon adenocarcinoma, glioblastoma, missing data imputation, pre-trained models
- MeSH terms
- Machine Learning, Algorithms, Humans, Image Processing, Computer-Assisted, Neoplasms, Sequence Analysis, RNA
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Generation of synthetic whole-slide image tiles of tumours from RNA-sequencing data via cascaded diffusion models. https://app.cochat.ai/papers/10.1038/s41551-024-01193-8
- The paper
- Carrillo-Perez, F., Pizurica, M., Zheng, Y., Nandi, T. N., Madduri, R., Shen, J., & Gevaert, O. (2024). Generation of synthetic whole-slide image tiles of tumours from RNA-sequencing data via cascaded diffusion models. Nature Biomedical Engineering. https://doi.org/10.1038/s41551-024-01193-8
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.