A deep generative model of 3D single-cell organization
Open accessPublished version of a preprint
The DOI matches its Crossref record; 37 of the 38 references it cites were found in a registry or an index.
Abstract
We introduce a framework for end-to-end integrative modeling of 3D single-cell multi-channel fluorescent image data of diverse subcellular structures. We employ stacked conditional β -variational autoencoders to first learn a latent representation of cell morphology, and then learn a latent representation of subcellular structure localization which is conditioned on the learned cell morphology. Our model is flexible and can be trained on images of arbitrary subcellular structures and at varying degrees of sparsity and reconstruction fidelity. We train our full model on 3D cell image data and explore design trade-offs in the 2D setting. Once trained, our model can be used to predict plausible locations of structures in cells where these structures were not imaged. The trained model can also be used to quantify the variation in the location of subcellular structures by generating plausible instantiations of each structure in arbitrary cell geometries. We apply our trained model to a small drug perturbation screen to demonstrate its applicability to new data. We show how the latent representations of drugged cells differ from unperturbed cells as expected by on-target effects of the drugs.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures






![Fig 7 Residual block used in this model. Each layer of our model is a modified residual layer. In the encoder, the layer input, x, is passed through a 4x convolution kernel with a stride of 2, then a 3x convolution kernel with a stride of 1 or a 1x convolution kernel with a subsequent avg-pooling step. These results are summed along the channel dimensions, added, and passed to the next layer. With the decoder, 4x convolution is replaced with transposed convolution, and pooling replaced with linear upsampling. In the case of the conditional model (components with dotted lines) MT, the reference input xr is linearly interpolated to be the same size as the output, and passed through a 1x kernel. The target label is passed through a 1x kernel, and added to each pixel of the output. Spectral weight normalization [36] is utilized at every convolutional or fully-connected operation. In the case of the 3D model the convolutions are three-dimensional, and the 2D model uses two-dimensional convolutions.](https://figures.cochat.ai/paper-figures/947d61dcd6614b7492ee5a8bf8b6d6e6/a-deep-generative-model-of-3d-single-cell-organization-figure-7.jpg)
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Other systematic image-based approaches have catalogued the location of human proteins in several cell types and used the locations of proteins and structures within cells to identify differences in intracellular spatial patterns among cells in distinct states6–12,19,29.”
From Integrated intracellular organization and its variations in human iPS cells (2023) · cites this paper as [29]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
CellProfiler: image analysis software for identifying and quantifying cell phenotypes
Carpenter, A. E., Jones, T. R., Lamprecht, M. R., Clarke, C., Kang, I. H., Friman, O., Guertin, D. A., Chang, J. H., Lindquist, R. A., Moffat, J., et al. (2006) · Genome Biology · doi:10.1186/gb-2006-7-10-r100 · Cited by 5,157
Progressive Growing of GANs for Improved Quality, Stability, and Variation
Karras, T., Aila, T., Laine, S., & Lehtinen, J. (2017) · arXiv · doi:10.48550/arxiv.1710.10196 · Cited by 4,727
High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs
Wang, T.-C., Liu, M.-Y., Zhu, J.-Y., Tao, A., Kautz, J., & Catanzaro, B. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00917 · Cited by 4,332
Spectral Normalization for Generative Adversarial Networks
Miyato, T., Kataoka, T., Koyama, M., & Yoshida, Y. (2018) · arXiv · doi:10.48550/arxiv.1802.05957 · Cited by 3,438
(2017) · International Conference on Learning Representations · Cited by 3,248
Deep generative modeling for single-cell transcriptomics
Lopez, R., Regier, J., Cole, M. B., Jordan, M. I., & Yosef, N. (2018) · Nature Methods · doi:10.1038/s41592-018-0229-2 · Cited by 2,530
A draft map of the human proteome
Kim, M.-S., Pinto, S. M., Getnet, D., Nirujogi, R. S., Manda, S. S., Chaerkady, R., Madugundu, A. K., Kelkar, D. S., Isserlin, R., Jain, S., et al. (2014) · Nature · doi:10.1038/nature13302 · Cited by 2,198
NIPS 2016 Tutorial: Generative Adversarial Networks
Goodfellow, I. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1701.00160 · Cited by 1,229
Research works printed but not linked: 3
- Kingma DP, Welling M. Auto-encoding variational bayes. arXiv preprint arXiv:13126114. 2013;.
- Alemi AA, Poole B, Fischer I, Dillon JV, Saurous RA, Murphy K. Fixing a broken ELBO. arXiv preprint arXiv:171100464. 2017;.
- GR Johnson, 2017, Building a 3D Integrated Cell, bioRxiv
Cited by
Generative artificial intelligence empowers digital twins in drug discovery and clinical trials
Bordukova, M., Makarov, N., Rodriguez-Esteban, R., Schmich, F., & Menden, M. P. (2024) · Expert Opinion on Drug Discovery · doi:10.1080/17460441.2023.2273839 · Cited by 235
“For instance, state-of-the-art cell DTs focus on predicting changes of single cellular characteristics, e.g. transcriptome [6–8] or cell morphology [9,10], while a multimodal DT could jointly cover numerous cell characteristics.”
BackgroundGenerative-AI-Driven Human Digital Twin in IoT Healthcare: A Comprehensive Survey
Chen, J., Shi, Y., Yi, C., Du, H., Kang, J., & Niyato, D. T. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3421918 · Cited by 198
Self-supervised deep learning encodes high-resolution features of protein subcellular localization
Kobayashi, H., Cheveralls, K. C., Leonetti, M. D., & Royer, L. A. (2022) · Nature Methods · doi:10.1038/s41592-022-01541-z · Cited by 133
Integrated intracellular organization and its variations in human iPS cells
Viana, M. P., Chen, J., Knijnenburg, T. A., Vasan, R., Yan, C., Arakaki, J. E., Bailey, M., Berry, B., Borensztejn, A., Brown, E. M., et al. (2023) · Nature · doi:10.1038/s41586-022-05563-7 · Cited by 132
“spatial patterns among cells in distinct states 6–12, 19, 29 .”
Self-citationSynthetic data in biomedicine via generative artificial intelligence
van Breugel, B., Liu, T., Oglić, D., & van der Schaar, M. (2024) · Nature Reviews Bioengineering · doi:10.1038/s44222-024-00245-7 · Cited by 149
Establishing a conceptual framework for holistic cell states and state transitions
Rafelski, S. M., & Theriot, J. A. (2024) · Cell · doi:10.1016/j.cell.2024.04.035 · Cited by 91
Self-citationBuilding the next generation of virtual cells to understand cellular biology
Johnson, G. T., Agmon, E., Akamatsu, M., Lundberg, E. K., Lyons, B. M., Ouyang, W., Quintero, O. A., Riel-Mehan, M. M., Rafelski, S. M., & Horwitz, R. (2023) · Biophysical Journal · doi:10.1016/j.bpj.2023.04.006 · Cited by 41
Self-citationGregor, B. W., Coston, M. E., Adams, E. M., Arakaki, J., Borensztejn, A., Do, T. P., Fuqua, M. A., Haupt, A., Hendershott, M. C., Leung, W. W., et al. (2023) · Nature Protocols · doi:10.1038/s41596-023-00912-w · Cited by 25
Self-citationProgress and new challenges in image-based profiling
Serrano, E., Peters, J., Wagner, J., Graham, R. E., Chen, Z., Feng, B. Y., Miranda, G., Kalinin, A. A., Vulliard, L., Tomkinson, J., et al. (2026) · Molecular Systems Biology · doi:10.1038/s44320-026-00197-7 · Cited by 26
Generative models of morphogenesis in developmental biology
Stillman, N., & Mayor, R. (2023) · Seminars in Cell and Developmental Biology · doi:10.1016/j.semcdb.2023.02.001 · Cited by 21
Improving and evaluating deep learning models of cellular organization
Sun, H., Fu, X., Abraham, S., Shen, J., & Murphy, R. F. (2022) · Bioinformatics · doi:10.1093/bioinformatics/btac688 · Cited by 17
(2025) · doi:10.1091/mbc.e24-10-0486 · Cited by 11
AI-powered microscopy image analysis for parasitology: integrating human expertise
Feng, R., Li, S., & Zhang, Y. (2024) · Trends in Parasitology · doi:10.1016/j.pt.2024.05.005 · Cited by 12
Virtual staining for pixel-wise and quantitative analysis of single cell images
Yılmaz, A., Aydin, T., & Varol, R. (2023) · Scientific Reports · doi:10.1038/s41598-023-45150-y · Cited by 10
Partially shared multi-modal embedding learns holistic representation of cell state
Zhang, X., Shivashankar, G. V., & Uhler, C. (2026) · Nature Computational Science · doi:10.1038/s43588-025-00948-w · Cited by 14
Interpretable representation learning for 3D multi-piece intracellular structures using point clouds
Vasan, R., Ferrante, A. J., Borensztejn, A., Frick, C. L., Garrison, P., Gaudreault, N., Mogre, S. S., Mohammed, F. S., Morris, B., Pires, G. G., et al. (2025) · Nature Methods · doi:10.1038/s41592-025-02729-9 · Cited by 7
“For instance, one could incorporate reference information about other intracellular structures for answering questions about intracellular structure colocalization 35,36 .”
Self-citationWei, Z., Liu, X., Yan, R., Sun, G., Yu, W., Liu, Q., & Guo, Q. (2022) · Frontiers in Genetics · doi:10.3389/fgene.2022.1002327 · Cited by 7
“) is defined as follows (Donovan-Maiye et al., 2018): MHSA(α, β, μ) Cat(h1, h2,//, hNh)W, (12)”
Optimizing Generative Adversarial Network (GAN) Models for Non-Pneumatic Tire Design
Seong, J., Ji, S.-M., Choi, D.-H., Lee, S., & Lee, S. (2023) · Applied Sciences · doi:10.3390/app131910664 · Cited by 8
Mirvis, M., Weingard, B., Goodman, S. N., & Marshall, W. F. (2026) · BMC Biology · doi:10.1186/s12915-026-02556-0 · Cited by 7
Wei, Z., Liu, W., Yu, W., Liu, X., Yan, R., Liu, Q., & Guo, Q. (2022) · International Journal of Molecular Sciences · doi:10.3390/ijms231810827 · Cited by 7
Labitigan, R. L. D., Sanborn, A. L., Hao, C. V., Chan, C. K., Belliveau, N. M., Brown, E. M., Mehrotra, M., & Theriot, J. A. (2022) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2022.12.27.522042 · Cited by 7
Self-citationLabitigan, R. L. D., Sanborn, A. L., Hao, C. V., Chan, C. K., Belliveau, N. M., Brown, E. M., Mehrotra, M., & Theriot, J. A. (2024) · eLife · doi:10.7554/elife.94964.1 · Cited by 9
Self-citationBasal body organization and cell geometry during the cell cycle in Tetrahymena thermophila
Sun, H., Soh, A. W. J., Mitchell, L. E., Pearson, C. G., & Murphy, R. F. (2023) · Molecular Biology of the Cell · doi:10.1091/mbc.e22-11-0508 · Cited by 4
Facilitating cell segmentation with the projection-enhancement network
Eddy, C., Naylor, A., Cunningham, C. T., & Sun, B. (2023) · Physical Biology · doi:10.1088/1478-3975/acfe53 · Cited by 5
Labitigan, R. L. D., Sanborn, A. L., Hao, C. V., Chan, C. K., Belliveau, N. M., Brown, E. M., Mehrotra, M., & Theriot, J. A. (2024) · eLife · doi:10.7554/elife.94964 · Cited by 10
Self-citationZeng, G.-H., Zhu, X.-Z., Yang, H.-R., Liang, Y.-J., Zhai, Y.-J., & Xu, Y.-Y. (2025) · Bioinformatics · doi:10.1093/bioinformatics/btaf331 · Cited by 3
Generative machine learning unlocks the first proteome-wide image of human cellsPreprint
Sun, H., Kahnert, K., Hansen, J. N., Leineweber, W. D., Li, M., Feng, W., Navarro, F. B., Axelsson, U., Ouyang, W., & Lundberg, E. K. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.03.31.715748 · Cited by 3
Li, Y., Zeng, G.-H., Liang, Y.-J., Yang, H.-R., Zhu, X., Zhai, Y.-J., Duan, L., & Xu, Y. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.108913 · Cited by 1
Using Dimensionality Reduction to Visualize Phenotypic Changes in High-Throughput Microscopy
Lu, A. X., & Moses, A. M. (2024) · Methods in molecular biology · doi:10.1007/978-1-0716-3834-7_15 · Cited by 2
RETRACTED ARTICLE: Exploring subcellular location anomalies: a novel quantum bioimaging paradigm
Kumar, K., Abbas, S. H., Gupta, M., Waris, S. F., Bordoloi, D., & Kirubakaran, M. K. (2024) · Optical and Quantum Electronics · doi:10.1007/s11082-023-06272-9 · Cited by 1
RetractedLitsios, A., Lo, D. S., Fischbach, A., Friesen, H., Daiejavad, A., Sahin, N., Masinas, M. P. D., Nguyen, T., Pons, C., Hou, J., et al. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.08.25.746993 · Cited by 1
Seong, J., Ji, S.-M., Choi, D.-H., & Lee, S. (2023) · ACM Conference on Research in Adaptive and Convergent Systems (RACS) · doi:10.1145/3599957.3606243
Morphogenesis of Images using Neural Cellular Automata
Tumdi, S., Patel, D., Dholakia, D., Patel, A., & Saval, P. (2024) · International Conference on Computing, Communication and Networking Technologies (ICCCNT) · doi:10.1109/icccnt61001.2024.10725715
Potential of Interactive VR and Digital Twin for Drug Development and Clinical Trials
Malviya, R., & Rajput, S. (2025) · Wiley eBooks · doi:10.1002/9781394315680.ch8
Micellangelo: a Generative Model of Cell-Topography InteractionsPreprint
Konshin, N., Minartz, K., de Boer, J., & Menkovski, V. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.11.24.690184
DEEPCPG: A Novel Deep Learning Framework for Single-Cell DNA Methylation State Prediction
Ganji, S. (2025) · International Conference on Electronic Circuits and Signaling Technologies (ICECST) · doi:10.1109/icecst66106.2025.11307553
Integration and harmonization of cell shape images for generative modelingPreprint
Khang, A., Young, M. W., Batan, D., & Anseth, K. S. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.02.07.704589
Wang, Y.-Y., Li, Y., He, Y., Xiong, Y., Li, Y., & Xu, Y.-Y. (2026) · Displays · doi:10.1016/j.displa.2026.103426
Learning Continuous Morphological Trajectories via Latent Principal CurvesPreprint
Magana, S., Zhao, W., & Duc, K. D. (2026) · bioRxiv · doi:10.64898/2026.03.16.712066
The development and application of deep learning for stem cell research
Li, Z., Zhang, W., Wu, Y., Ye, Y., Chen, Y., Wu, J., & Chen, D. (2026) · Journal of genetics and genomics/Journal of Genetics and Genomics · doi:10.1016/j.jgg.2026.05.008
Algebraic Representation of Mitochondrial Dynamics
Mostov, R., Lewis, G. R., Sturm, G., & Marshall, W. F. (2026) · Bulletin of Mathematical Biology · doi:10.1007/s11538-026-01738-9
Recent advances in deep learning for biological microscopy image analysis beyond segmentation
Cao, K., Gao, C., Zhang, X., Pan, X., Chen, J., & Wang, H. (2026) · BioTechniques · doi:10.1080/07366205.2026.2730974
Huang, Y., Iino, Y., & Toyoshima, Y. (2026) · IEEE Access · doi:10.1109/access.2026.3738430
Geometry‐aware modeling with cellPACK for evaluating spatial trends in intracellular organization
Mogre, S. S., Li, R., Scibisz, A., Thảo, Đ. T., Dixon, J. C., Viana, M. P., Nazockdast, E., Rafelski, S. M., Theriot, J. A., Riel-Mehan, M. M., et al. (2026) · Protein Science · doi:10.1002/pro.70832
Self-citation
Metadata
- Authors
- Donovan-Maiye, R. M., Brown, J. M., Chan, C. K., Ding, L., Yan, C., Gaudreault, N., Theriot, J. A., Maleckar, M. M., Knijnenburg, T. A., & Johnson, G. R.
- Affiliations
- Allen Institute for Cell Science (US)
- Howard Hughes Medical Institute (US)
- Corresponding author
- Knijnenburg, Theo A.
- ORCID
- Venue
- PLOS Computational Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2022-01-18
- Volume
- 18
- Issue
- 1
- Pages
- e1009155
- ISSN
- 1553-734X (ISSN-L)
- 1553-7358
- PMC
- PMC8797242
- OpenAlex
- W4205199447
- Cited by
- 44
- References
- 38
- Citations by year
- 2026: 11 · 2025: 8 · 2024: 10 · 2023: 10 · 2022: 5
- FWCI
- 3.65
- Type
- journal article
- Language
- en
- Update notices
- new_version (2022-01-28)
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS Computational Biology (publishedVersion, cc-by)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Allen Institute
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Biophysics › Cell Image Analysis Techniques
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Single-cell and spatial transcriptomics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Biophysics › Advanced Fluorescence Microscopy Techniques
- Keywords
- conditional variational autoencoder, cell morphology, latent representations
- MeSH terms
- Cell Nucleus, Cells, Cultured, Humans, Microscopy, Fluorescence, Models, Biological, Computational Biology, Imaging, Three-Dimensional, Intracellular Space, Cell Shape, Induced Pluripotent Stem Cells, Single-Cell Analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.