Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Cited by 220

The DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.

CLIP-Forge: Towards Zero-Shot Text-to-Shape Generation

Open access
Sanghi, Aditya, Chu, Hang, Lambourne, Joseph G., Wang, Ye, Cheng, Chin-Yi, Fumero, Marco, Malekshan, Kamal Rahimi · 2022 · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Abstract

Generating shapes using natural language can enable new ways of imagining and creating the things around us. While significant recent progress has been made in text-to-image generation, text-to-shape generation remains a challenging problem due to the unavailability of paired text and shape data at a large scale. We present a simple yet effective method for zero-shot text-to-shape gener-ation that circumvents such data scarcity. Our proposed method, named CLIP-Forge, is based on a two-stage training process, which only depends on an unlabelled shape dataset and a pre-trained image-text network such as CLIP. Our method has the benefits of avoiding expensive inference time optimization, as well as the ability to generate multiple shapes for a given text. We not only demonstrate promising zero-shot generalization of the CLIP-Forge model qualitatively and quantitatively, but also provide extensive compar-ative evaluations to better understand its behavior.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/cvpr52688.2022.01805

Cites

12 of 12 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 12 references

Research works printed but not linked: 49

  1. van den oord, 2017, Neural discrete representation learning, ArXiv Preprint
  2. nash, 2020, Polygen: An autoregressive generative model of 3d meshes, ICML
  3. chun-liang, 2018, Point cloud gan, ar Xiv preprint
  4. lampert, 2009, Learning to Detect Unseen Object Classes by Betweenclass Attribute Transfer
  5. kumar, 2020, Videoflow A conditional flow-based model for stochastic video generation
  6. kingma, 2018, Glow Generative flow with invertible 1×1 convolutions
  7. mishra, 2019, Machine Learning in the AWS Cloud Add Intelligence to Applications with Amazon SageMaker and Amazon Rekognition
  8. 10.1109/CVPR.2019.00459 · DOI · Paper page
  9. liu, 2019, An acceleration framework for high resolution image synthesis, ar Xiv preprint
  10. lin, 2018, Generating animated videos of human activities from natural language descriptions
  11. kim, 2019, Flowavenet A generative flow for raw audio
  12. karpathy, 2014, Deep frag-ment embeddings for bidirectional image sentence mapping, ArXiv Preprint
  13. achlioptas, 2018, Learning representations and generative models for 3d point clouds, International Conference on Machine Learning, 40
  14. achlioptas, 0, Referit3d: N eurallisteners for fine-grained 3d object identification in real-world scenes, European Conference on Computer Vision, 422
  15. pin-hao, 2021, Text-guided graph neural networks for re-ferring 3d instance segmentation, Proceedings of the AAAI Conference on Artificial Intelligence, 35, 1610, 10.1609/aaai.v35i2.16253 · DOI
  16. ioffe, 2015, Batch normalization: Accelerating deep network training by reducing internal co-variate shift, International Conference on Machine Learning, 448
  17. huq, 2020, Holistic static and animated 3d scene generation from diverse text descriptions, CoRR abs/2010 01549
  18. armand, 0, Learning visual features from large weakly supervised data, European Conference on Computer Vision, 67
  19. rezende, 2015, Variational inference with normalizing flows, International Conference on Ma-chine Learning, 1530
  20. rothfuss, 2019, Noise regularization for conditional density estimation, ArXiv Preprint
  21. yang, 2018, Foldingnet: Point cloud auto-encoder via deep grid deformation, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 206
  22. wang, 2018, Pixel2mesh: Generating 3d mesh models from single rgb images, ECCV
  23. tang, 2021, Part2word Learning joint embedding of point clouds and text by matching parts to words
  24. 10.1162/tacl_a_00177 · DOI · Paper page
  25. shen, 2021, How much can clip benefit vision-and-language tasks?, ArXiv Preprint
  26. sariyildiz, 0, Learning visual representations with caption annotations, Computer Vision-ECCV 2020 16th European Conference Glasgow UK August 23–28 2020 Proceedings Part VIII 16, 153
  27. 10.1109/CVPR46437.2021.01101 · DOI · Paper page
  28. pakhomov, 2021, Segmentation in style: Unsuper-vised semantic image segmentation with stylegan and clip, ArXiv Preprint
  29. dinh, 2016, Density estimation using real nvp, ar Xiv preprint
  30. 10.1109/CVPR46437.2021.01268 · DOI · Paper page
  31. fang, 2021, Clip2video: Mastering video-text retrieval via image clip, ArXiv Preprint
  32. 10.7551/mitpress/7287.001.0001 · DOI · Paper page
  33. frans, 2021, Clipdraw Exploring text-to-drawing synthesis through language-image encoders
  34. frome, 2013, DeViSE A deep visual-semantic embedding model [J]
  35. 10.1609/aaai.v33i01.3301126 · DOI
  36. chang, 2015, ShapeNet An Information-Rich 3D Model Repository
  37. achlioptas, 2018, Learning to refer to 3D objects with natural language
  38. 10.1007/978-3-030-20893-6_7 · DOI · Paper page
  39. chen, 0, Scanrefer: 3d object localization in rgb-d scans using natu-ral language, Computer Vision-ECCV 2020 16th Euro-pean Conference Glasgow UK August 23–28 2020 Pro-ceedings Part XX 16, 202
  40. 10.1109/CVPR42600.2020.00700 · DOI · Paper page
  41. 10.1109/CVPR.2019.00609 · DOI · Paper page
  42. choy, 0, 3d-r2n2: A unified approach for single and multi-view 3d object reconstruction, European Conference on Computer Vision, 628
  43. pumarola, 2020, C-flow Conditional generative flow models for images and 3d point clouds
  44. peng, 0, Convolutional occupancy networks, Computer Vision-ECCV 2020 16th European Conference Glasgow UK August 23–28 2020 Proceedings Part III 16, 523
  45. qi, 2017, Pointnet: Deep learning on point sets for 3d classification and segmentation, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 652
  46. papamakarios, 2017, Masked autoregressive flow for density estimation, ar Xiv preprint
  47. palatucci, 2009, Zero-shot learning with semantic out-put codes, Advances in Neural Infor-Mation Processing Systems, 22
  48. 10.1109/ICCV48922.2021.00209 · DOI · Paper page
  49. 10.1109/CVPR.2019.00025 · DOI · Paper page

Cited by

See all 220

Metadata

Authors
Sanghi, A., Chu, H., Lambourne, J. G., Wang, Y., Cheng, C.-Y., Fumero, M., & Malekshan, K. R.
Venue
2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Publisher
IEEE
Published
2022-06
Pages
18582-18592
ISSN
1063-6919 (ISSN-L)
OpenAlex
W3201927751
Cited by
220
References
12
FWCI
19.41
Type
conference paper
Language
en
Open access
green

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: CLIP-Forge: Towards Zero-Shot Text-to-Shape Generation. https://app.cochat.ai/papers/10.1109/cvpr52688.2022.01805
The paper
Sanghi, A., Chu, H., Lambourne, J. G., Wang, Y., Cheng, C.-Y., Fumero, M., & Malekshan, K. R. (2022). CLIP-Forge: Towards Zero-Shot Text-to-Shape Generation. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). https://doi.org/10.1109/cvpr52688.2022.01805

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.