CoChat
Sign Up
PreprintCited by 93D Shape Modeling and AnalysisGenerative Adversarial Networks and Image SynthesisAdvanced Neural Network Applications

The DOI matches its DataCite record.

Unsupervised 3D Reconstruction from a Single Image via Adversarial Learning

Open access
Wang, Lingjing, Fang, Yi·2017·arXiv
DOI 10.48550/arxiv.1711.09312arXiv 1711.09312
Open this paper in CoChat — free accountOpen PDF

Abstract

Recent advancements in deep learning opened new opportunities for learning a high-quality 3D model from a single 2D image given sufficient training on large-scale data sets. However, the significant imbalance between available amount of images and 3D models, and the limited availability of labeled 2D image data (i.e. manually annotated pairs between images and their corresponding 3D models), severely impacts the training of most supervised deep learning methods in practice. In this paper, driven by a novel design of adversarial networks, we have developed an unsupervised learning paradigm to reconstruct 3D models from a single 2D image, which is free of manually annotated pairwise input image and its associated 3D model. Particularly, the paradigm begins with training an adaption network via autoencoder with adversarial loss, which embeds unpaired 2D synthesized image domain with real world image domain to a shared latent vector space. Then, we jointly train a 3D deconvolutional network to transform the latent vector space to the 3D object space together with the embedding process. Our experiments verify our network's robust and superior performance in handling 3D volumetric object generation from a single 2D image.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.1711.09312

Cites

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

No references could be resolved yet.

Research works printed but not linked: 31

  1. [2][2] Jorge and Ruiz-Ascencio Fuentes-Pacheco Jos´ e and Rend´ on-Mancha, Visual simultaneous localization and mapping: a survey , Artificial Intelligence Review 43 (2015), no. 1, 55-81. -
  2. [3][3] Fred and Lazebnik Rothganger Svetlana and Schmid, Segmenting, modeling, and matching video clips containing multiple moving objects , IEEE transactions on pattern analysis and machine intelligence 29 (2007), no. 3. -
  3. [4][4] Changchang Wu, Towards linear-time incremental structure from motion , 3DTV-Conference, 2013 International Conference on, 2013, pp. 127-134. -
  4. [6][6] Florian and Chaouch Chabot Mohamed and Rabarisoa, Accurate 3D car pose estimation , Image Processing (ICIP), 2016 IEEE International Conference on, 2016, pp. 3807-3811. -
  5. [7][7] Jason and Gupta Rock Tanmay and Thorsen, Completing 3D object shape from one depth image , Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 2484-2493. -
  6. [8][8] Christopher and Stark Bongsoo Choy Michael and Corbett-Davies, Enriching object detection with 2d-3d registration and continuous viewpoint estimation , Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 2512-2520. -
  7. [9][9] Masatoshi and Kanade Okutomi Takeo, A multiple-baseline stereo , IEEE Transactions on pattern analysis and machine intelligence 15 (1993), no. 4, 353-363. -
  8. [10][10] Michael and Snavely Goesele Noah and Curless, Multi-view stereo for community photo collections , Computer Vision, 2007. ICCV 2007. IEEE 11th International Conference on, 2007, pp. 1-8. -
  9. [11][11] Yasutaka and Ponce Furukawa Jean, Carved visual hulls for image-based modeling , Computer Vision-ECCV 2006 (2006), 564-577. -
  10. [12][12] Zhirong and Song Wu Shuran and Khosla, 3d shapenets: A deep representation for volumetric shapes , Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 1912-1920. -
  11. [13][13] Hang and Maji Su Subhransu and Kalogerakis, Multi-view convolutional neural networks for 3d shape recognition , Proceedings of the IEEE international conference on computer vision, 2015, pp. 945-953. -
  12. [14][14] Abhishek and Grau Sharma Oliver and Fritz, Vconv-dae: Deep volumetric shape learning without object labels , Computer Vision-ECCV 2016 Workshops, 2016, pp. 236-250. -
  13. [15][15] Daniel and Scherer Maturana Sebastian, Voxnet: A 3d convolutional neural network for real-time object recognition , Intelligent Robots and Systems (IROS), 2015 IEEE/RSJ International Conference on, 2015, pp. 922-928. -
  14. [16][16] Song and Bai Bai Xiang and Zhou, Gift: A real-time and scalable 3d shape search engine , Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 5023-5032. -
  15. [17][17] Rohit and Fouhey Girdhar David F and Rodriguez, Learning a predictable and generative vector representation for objects , European Conference on Computer Vision, 2016, pp. 484-499. -
  16. [18][18] Jiajun and Zhang Wu Chengkai and Xue, Learning a probabilistic latent space of object shapes via 3d generative-adversarial modeling , Advances in Neural Information Processing Systems, 2016, pp. 82-90. -
  17. [19][19] Christopher B and Xu Choy Danfei and Gwak, 3d-r2n2: A unified approach for single and multi-view 3d object reconstruction , European Conference on Computer Vision, 2016, pp. 628-644. -
  18. [20][20] Haoqiang and Su Fan Hao and Guibas, Apoint set generation network for 3d object reconstruction from a single image , arXiv preprint arXiv:1612.00603 (2016). - · arXiv
  19. [21][21] Fisher and Seff Yu Ari and Zhang, Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop , arXiv preprint arXiv:1506.03365 (2015). - · arXiv
  20. [22][22] Ziwei and Luo Liu Ping and Wang, Deep learning face attributes in the wild , Proceedings of the IEEE International Conference on Computer Vision, 2015, pp. 3730-3738. -
  21. [23][23] Alec and Metz Radford Luke and Chintala, Unsupervised representation learning with deep convolutional generative adversarial networks , arXiv preprint arXiv:1511.06434 (2015). - · arXiv
  22. [24][24] Martin and Chintala Arjovsky Soumith and Bottou, Wasserstein gan , arXiv preprint arXiv:1701.07875 (2017). - · arXiv
  23. [25][25] David and Schumm Berthelot Tom and Metz, BEGAN: Boundary Equilibrium Generative Adversarial Networks , arXiv preprint arXiv:1703.10717 (2017). - · arXiv
  24. [26][26] Jun-Yan and Park Zhu Taesung and Isola, Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks , arXiv preprint arXiv:1703.10593 (2017). - · arXiv
  25. [27][27] Guillaume and Zeghidour Lample Neil and Usunier, Fader Networks: Manipulating Images by Sliding Attributes , arXiv preprint arXiv:1706.00409 (2017). - · arXiv
  26. [28][28] Sergey and Szegedy Ioffe Christian, Batch normalization: Accelerating deep network training by reducing internal covariate shift , International Conference on Machine Learning, 2015, pp. 448-456. -
  27. [29][29] Bing and Wang Xu Naiyan and Chen, Empirical evaluation of rectified activations in convolutional network , arXiv preprint arXiv:1505.00853 (2015). - · arXiv
  28. [30][30] Angel X and Funkhouser Chang Thomas and Guibas, Shapenet: An information-rich 3d model repository , arXiv preprint arXiv:1512.03012 (2015). - · arXiv
  29. [31][31] Yu and Mottaghi Xiang Roozbeh and Savarese, Beyond pascal: A benchmark for 3d object detection in the wild , Applications of Computer Vision (WACV), 2014 IEEE Winter Conference on, 2014, pp. 75-82. -
  30. [33][33] Joseph J and Zitnick Lim C Lawrence and Doll´ ar, Sketch tokens: A learned mid-level representation for contour and object detection , Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2013, pp. 3158-3165. -
  31. [34][34] Abhishek and Tulsiani Kar Shubham and Carreira, Category-specific object reconstruction from a single image , Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 1966-1974.

Cited by

Sorted by citations

See all 9

Metadata

Authors
Wang, L., & Fang, Y.
ORCID
Venue
arXiv
Publisher
arXiv
Published
2017
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.1711.09312
arXiv
1711.09312
OpenAlex
W2768377388
Cited by
9
References
0
Citations by year
2025: 1 · 2022: 1 · 2021: 2 · 2020: 3 · 2019: 1 · 2018: 1
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › 3D Shape Modeling and Analysis
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
Keywords
single-image 3D reconstruction, adversarial learning, autoencoder, adversarial loss, latent space embedding, domain adaptation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review