The DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
CLAY: A Controllable Large-scale Generative Model for Creating High-quality 3D Assets
Open accessAbstract
In the realm of digital creativity, our potential to craft intricate 3D worlds from imagination is often hampered by the limitations of existing digital tools, which demand extensive expertise and efforts. To narrow this disparity, we introduce CLAY, a 3D geometry and material generator designed to effortlessly transform human imagination into intricate 3D digital structures. CLAY supports classic text or image inputs as well as 3D-aware controls from diverse primitives (multi-view images, voxels, bounding boxes, point clouds, implicit representations, etc). At its core is a large-scale generative model composed of a multi-resolution Variational Autoencoder (VAE) and a minimalistic latent Diffusion Transformer (DiT), to extract rich 3D priors directly from a diverse range of 3D geometries. Specifically, it adopts neural fields to represent continuous and complete surfaces and uses a geometry generative module with pure transformer blocks in latent space. We present a progressive training scheme to train CLAY on an ultra large 3D model dataset obtained through a carefully designed processing pipeline, resulting in a 3D native geometry generator with 1.5 billion parameters. For appearance generation, CLAY sets out to produce physically-based rendering (PBR) textures by employing a multi-view material diffusion model that can generate 2K resolution textures with diffuse, roughness, and metallic modalities. We demonstrate using CLAY for a range of controllable 3D asset creations, from sketchy conceptual designs to production ready assets with intricate details. Even first time users can easily use CLAY to bring their vivid 3D imaginations to life, unleashing unlimited creativity.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
High-Resolution Image Synthesis with Latent Diffusion Models
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01042 · Cited by 20,302
NeRF: representing scenes as neural radiance fields for view synthesis
Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, J. T., Ramamoorthi, R., & Ng, R. (2022) · Communications of the ACM · doi:10.1145/3503250 · Cited by 6,852
3D Gaussian Splatting for Real-Time Radiance Field Rendering
Kerbl, B., Kopanas, G., Leimkuehler, T., & Drettakis, G. (2023) · ACM Transactions on Graphics · doi:10.1145/3592433 · Cited by 6,025
Adding Conditional Control to Text-to-Image Diffusion Models
Zhang, L., Rao, A., & Agrawala, M. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00355 · Cited by 4,547
Self-citationHigh-Resolution Image Synthesis with Latent Diffusion Models
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2021) · arXiv · doi:10.48550/arxiv.2112.10752 · Cited by 737
Objaverse: A Universe of Annotated 3D Objects
Deitke, M., Schwenk, D., Salvador, J., Weihs, L., Michel, O., VanderBilt, E., Schmidt, L., Ehsanit, K., Kembhavi, A., & Farhadi, A. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01263 · Cited by 725
SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis
Podell, D., English, Z., Lacey, K., Blattmann, A., Dockhorn, T., Müller, J., Penna, J., & Rombach, R. (2023) · arXiv · doi:10.48550/arxiv.2307.01952 · Cited by 455
Latent-NeRF for Shape-Guided Generation of 3D Shapes and Textures
Metzer, G., Richardson, E., Patashnik, O., Giryes, R., & Cohen-Or, D. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01218 · Cited by 336 · cited 1×
“…al. 2023b; Zhu et al. 2024], ranging from DMTet [Shen et al. 2021] to the most recent 3D Gaussian splatting [Kerbl et al. 2023], Various modifications managed to elevate the performance [Chen et al. 2023a; Li et al. 2024; Metzer et al. 2023; Seo et al. 2024; Wang et al. 2023; Zhang et al. 2023a].”
ULIP-2: Towards Scalable Multimodal Pre-Training for 3D Understanding
Xue, L., Yu, N., Zhang, S., Panagopoulou, A., Li, J., Martín-Martín, R., Wu, J., Xiong, C., Xu, R., Niebles, J. C., et al. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.02558 · Cited by 144
Cited by
Structured 3D Latents for Scalable and Versatile 3D Generation
Xiang, J., Lv, Z., Xu, S., Deng, Y., Wang, R., Zhang, B., Chen, D., Tong, X., & Yang, J. (2025) · 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52734.2025.02000 · Cited by 278
One Model to Rig Them All: Diverse Skeleton Rigging with UniRig
Zhang, J.-P., Pu, C.-F., Guo, M.-H., Cao, Y.-P., & Hu, S.-M. (2025) · ACM Transactions on Graphics · doi:10.1145/3730930 · Cited by 12
“The rapid advancements in AI-driven 3D content creation [Holden et al. 2017; Peng et al. 2024; Poole et al. 2022; Siddiqui et al. 2024; Yu et al. 2024; Zhang et al. 2024b] are revolutionizing computer graphics, enabling the generation of complex 3D models at an unprecedented scale and speed.”
ArtiLatent: Realistic Articulated 3D Object Generation via Structured Latents
Chen, H., Lan, Y., Chen, Y., & Pan, X. (2025) · Proceedings of the SIGGRAPH Asia 2025 Conference Papers · doi:10.1145/3757377.3763922 · Cited by 4
“[Chen et al. 2024, 2025; Lan et al. 2024a,b; Li et al. 2025; Xiang et al. 2024; Zhang et al. 2023, 2024; Zhao et al. 2025].”
Skin Tokens: A Learned Compact Representation for Unified Autoregressive Rigging
Zhang, J.-P., Pu, C.-F., Guo, M.-H., Cao, Y.-P., & Hu, S.-M. (2026) · arXiv · doi:10.48550/arxiv.2602.04805
“The rapid advancement of generative models has enabled the creation of 3D assets at an unprecedented scale [Jiang 2024; Li et al. 2025; Xiang et al. 2024; Zhang et al. 2024].”
Rigel3D: Rig-aware Latents for Animation-Ready 3D Asset Generation
Chatzis, N., Loizou, M., & Kalogerakis, E. (2026) · arXiv · doi:10.48550/arxiv.2605.13129
Metadata
- Authors
- Zhang, L., Wang, Z., Zhang, Q., Qiu, Q., Pang, A., Jiang, H., Yang, W., Xu, L., & Yu, J.
- Venue
- ACM Transactions on Graphics
- Published
- 2024-07-19
- Volume
- 43
- Issue
- 4
- Pages
- 1-20
- ISSN
- 0730-0301 (ISSN-L)
- 1557-7368
- OpenAlex
- W4400822286
- Cited by
- 160
- References
- 9
- FWCI
- 32.22
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: CLAY: A Controllable Large-scale Generative Model for Creating High-quality 3D Assets. https://app.cochat.ai/papers/10.1145/3658146
- The paper
- Zhang, L., Wang, Z., Zhang, Q., Qiu, Q., Pang, A., Jiang, H., Yang, W., Xu, L., & Yu, J. (2024). CLAY: A Controllable Large-scale Generative Model for Creating High-quality 3D Assets. ACM Transactions on Graphics. https://doi.org/10.1145/3658146
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.