The DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
3DShape2VecSet: A 3D Shape Representation for Neural Fields and Generative Diffusion Models
Open accessAbstract
We introduce 3DShape2VecSet, a novel shape representation for neural fields designed for generative diffusion models. Our shape representation can encode 3D shapes given as surface models or point clouds, and represents them as neural fields. The concept of neural fields has previously been combined with a global latent vector, a regular grid of latent vectors, or an irregular grid of latent vectors. Our new representation encodes neural fields on top of a set of vectors. We draw from multiple concepts, such as the radial basis function representation, and the cross attention and self-attention function, to design a learnable representation that is especially suitable for processing with transformers. Our results show improved performance in 3D shape encoding and 3D shape generative modeling tasks. We demonstrate a wide variety of generative applications: unconditioned generation, category-conditioned generation, text-conditioned generation, point-cloud completion, and image-conditioned generation. Code: https://1zb.github.io/3DShape2VecSet/.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
High-Resolution Image Synthesis with Latent Diffusion Models
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01042 · Cited by 20,302
End-to-End Object Detection with Transformers
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58452-8_13 · Cited by 18,285
End-to-End Object Detection with Transformers
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. (2020) · arXiv · doi:10.48550/arxiv.2005.12872 · Cited by 18,253
Denoising Diffusion Probabilistic Models
Ho, J., Jain, A., & Abbeel, P. (2020) · arXiv · doi:10.48550/arxiv.2006.11239 · Cited by 8,712
Taming Transformers for High-Resolution Image Synthesis
Esser, P., Rombach, R., & Ommer, B. (2020) · arXiv · doi:10.48550/arxiv.2012.09841 · Cited by 2,514
Guo, M.-H., Cai, J.-X., Liu, Z.-N., Mu, T.-J., Martin, R. R., & Hu, S.-M. (2021) · Computational Visual Media · doi:10.1007/s41095-021-0229-5 · Cited by 2,100
Reconstruction and representation of 3D objects with radial basis functions
Carr, J. C., Beatson, R. K., Cherrie, J. B., Mitchell, T. J., Fright, W. R., McCallum, B. C., & Evans, T. R. (2001) · Proceedings of the 28th annual conference on Computer graphics and interactive techniques · doi:10.1145/383259.383266 · Cited by 1,852 · cited 1×
“In this way, we can represent the continuous function O(·) as a set of 𝑀 points including their corresponding weights, (cid:8) (cid:9) However, in order to retain the details of a 3d shape, we often need a very large number of points ( e.g ., 𝑀 = 80 , 000 in [Carr et al. 2001]).”
BackgroundLearning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling
Wu, J., Zhang, C., Xue, T., Freeman, W. T., & Tenenbaum, J. B. (2016) · arXiv · doi:10.48550/arxiv.1610.07584 · Cited by 1,393 · cited 3×
“3D Models Representation 3D-GAN [Wu et al. 2016] GAN Voxels l-GAN [Achlioptas et al. 2018] GAN Point Clouds IM-GAN [Chen and Zhang 2019] GAN Fields PointFlow [Yang et al. 2019] NF Point Clouds GenVoxelNet [Xie et al. 2020] EBM Voxels PointGrow [Sun et al. 2020] AR Point Clouds PolyGen [Nash et al.…”
Background
Cited by
Wonder3D: Single Image to 3D Using Cross-Domain Diffusion
Long, X., Guo, Y.-C., Lin, C., Liu, Y., Dou, Z., Liu, L., Ma, Y., Zhang, S.-H., Habermann, M., Theobalt, C., et al. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.00951 · Cited by 377
Structured 3D Latents for Scalable and Versatile 3D Generation
Xiang, J., Lv, Z., Xu, S., Deng, Y., Wang, R., Zhang, B., Chen, D., Tong, X., & Yang, J. (2025) · 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52734.2025.02000 · Cited by 278
Self-citationA Survey on Deep Generative 3D-aware Image Synthesis
Xia, W., & Xue, J.-H. (2024) · ACM Computing Surveys · doi:10.1145/3626193 · Cited by 20
GEM3D: GEnerative Medial Abstractions for 3D Shape Synthesis
Petrov, D., Goyal, P., Thamizharasan, V., Kim, V., Gadelha, M., Averkiou, M., Chaudhuri, S., & Kalogerakis, E. (2024) · Special Interest Group on Computer Graphics and Interactive Techniques Conference Conference Papers · doi:10.1145/3641519.3657415 · Cited by 12
Zhao, Y., Zhao, Z., Li, J., Dong, S., & Gao, S. (2024) · 2024 International Conference on 3D Vision (3DV) · doi:10.1109/3dv62453.2024.00132 · Cited by 10
“In light of the recent successes in high-quality 3D shape generation [23, 58, 60, 62, 63], we propose to generate both scene layout and 3D furniture shapes, simultaneously.”
BackgroundOne Model to Rig Them All: Diverse Skeleton Rigging with UniRig
Zhang, J.-P., Pu, C.-F., Guo, M.-H., Cao, Y.-P., & Hu, S.-M. (2025) · ACM Transactions on Graphics · doi:10.1145/3730930 · Cited by 12
ArtiLatent: Realistic Articulated 3D Object Generation via Structured Latents
Chen, H., Lan, Y., Chen, Y., & Pan, X. (2025) · Proceedings of the SIGGRAPH Asia 2025 Conference Papers · doi:10.1145/3757377.3763922 · Cited by 4
“[Chen et al. 2024, 2025; Lan et al. 2024a,b; Li et al. 2025; Xiang et al. 2024; Zhang et al. 2023, 2024; Zhao et al. 2025].”
Skin Tokens: A Learned Compact Representation for Unified Autoregressive Rigging
Zhang, J.-P., Pu, C.-F., Guo, M.-H., Cao, Y.-P., & Hu, S.-M. (2026) · arXiv · doi:10.48550/arxiv.2602.04805
“We utilize the 3DShape2Vecset back-bone [Zhang et al. 2023] (110M parameters) with an asymmetric design (2 encoder layers, 10 decoder layers) to accelerate subsequent training, and PMPE encoding [Bhat et al. 2025].”
ViP-Rig: Visual-Prompted Controllable Rigging
Qin, Z., Sun, M., Mao, Y., Xu, J., Guo, J., Hu, C., Zhao, W., Jiang, J., & Liu, X. (2026) · arXiv · doi:10.48550/arxiv.2607.27982
“…structural cues and compress the dense features into K fixed-length conditioning tokens: Because the autoregressive backbone cannot directly process the mesh, we follow the sampling and encoding pipeline of 3DShape2VecSet (Zhang et al. 2023) to convert the input mesh M into geometry features F G .”
Rigel3D: Rig-aware Latents for Animation-Ready 3D Asset Generation
Chatzis, N., Loizou, M., & Kalogerakis, E. (2026) · arXiv · doi:10.48550/arxiv.2605.13129
Metadata
- Authors
- Zhang, B., Tang, J., Nießner, M., & Wonka, P.
- Venue
- ACM Transactions on Graphics
- Published
- 2023-07-26
- Volume
- 42
- Issue
- 4
- Pages
- 1-16
- ISSN
- 0730-0301 (ISSN-L)
- 1557-7368
- OpenAlex
- W4385275714
- Cited by
- 178
- References
- 13
- FWCI
- 25.07
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.