The DOI matches its Crossref record; 55 of the 55 references it cites were found in a registry or an index.
What Do Single-View 3D Reconstruction Networks Learn?
Abstract
Convolutional networks for single-view object reconstruction have shown impressive performance and have become a popular subject of research. All existing techniques are united by the idea of having an encoder-decoder network that performs non-trivial reasoning about the 3D structure of the output space. In this work, we set up two alternative approaches that perform image classification and retrieval respectively. These simple baselines yield better results than state-of-the-art methods, both qualitatively and quantitatively. We show that encoder-decoder methods are statistically indistinguishable from these baselines, thus indicating that the current state of the art in single-view object reconstruction does not actually perform reconstruction but image classification. We identify aspects of popular experimental procedures that elicit this behavior and discuss ways to improve the current state of research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
(1967) · Cited by 23,719
Marching cubes: A high resolution 3D surface construction algorithm
Lorensen, W. E., & Cline, H. E. (1987) · Proceedings of the 14th annual conference on Computer graphics and interactive techniques · doi:10.1145/37401.37422 · Cited by 10,104
Marching cubes: A high resolution 3D surface construction algorithm
Lorensen, W. E., & Cline, H. E. (1987) · ACM SIGGRAPH Computer Graphics · doi:10.1145/37402.37422 · Cited by 8,647
Multidimensional Scaling by Optimizing Goodness of Fit to a Nonmetric Hypothesis
Kruskal, J. B. (1964) · Psychometrika · doi:10.1007/bf02289565 · Cited by 7,317
The Kolmogorov-Smirnov Test for Goodness of Fit
Massey, F. J. (1951) · Journal of the American Statistical Association · doi:10.1080/01621459.1951.10500769 · Cited by 7,210
Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
Eigen, D., Puhrsch, C., & Fergus, R. (2014) · arXiv · doi:10.48550/arxiv.1406.2283 · Cited by 3,442
A Nonlinear Mapping for Data Structure Analysis
Sammon, J. W. (1969) · IEEE Transactions on Computers · doi:10.1109/t-c.1969.222678 · Cited by 3,380
Unsupervised Monocular Depth Estimation with Left-Right Consistency
Godard, C., Aodha, O. M., & Brostow, G. J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.699 · Cited by 3,270
A Point Set Generation Network for 3D Object Reconstruction from a Single Image
Fan, H., Su, H., & Guibas, L. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.264 · Cited by 2,402
ShapeNet: An Information-Rich 3D Model Repository
Chang, A. X., Funkhouser, T., Guibas, L., Hanrahan, P., Huang, Q., Li, Z., Savarese, S., Savva, M., Song, S., Su, H., et al. (2015) · arXiv · doi:10.48550/arxiv.1512.03012 · Cited by 2,330
Self-citationZhang, R., Tsai, P.-S., Cryer, J. E., & Shah, M. (1999) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.784284 · Cited by 1,731
Tanks and temples: benchmarking large-scale scene reconstruction
Knapitsch, A., Park, J., Zhou, Q.-Y., & Koltun, V. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073599 · Cited by 1,707
Self-citation3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction
Choy, C. B., Xu, D., Gwak, J., Chen, K., & Savarese, S. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46484-8_38 · Cited by 1,632
ShapeNet: An Information-Rich 3D Model Repository
Chang, A. X., Funkhouser, T. A., Guibas, L. J., Hanrahan, P., Huang, Q., Li, Z., Savarese, S., Savva, M., Song, S., Su, H., et al. (2023) · doi:10.5281/zenodo.10089705 · Cited by 1,623
Self-citationOn Visual Similarity Based 3D Model Retrieval
Chen, D., Tian, X., Shen, Y., & Ouhyoung, M. (2003) · Computer Graphics Forum · doi:10.1111/1467-8659.00669 · Cited by 1,398
Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling
Wu, J., Zhang, C., Xue, T., Freeman, W. T., & Tenenbaum, J. B. (2016) · arXiv · doi:10.48550/arxiv.1610.07584 · Cited by 1,393 · cited 1×
“A cluster of approaches used voxel-based representations of 3D shapes and generated them with 3D up-convolutions from a latent representation [4, 10, 53].”
MethodPixel2Mesh: Generating 3D Mesh Models from Single RGB Images
Wang, N., Zhang, Y., Li, Z., Fu, Y., Liu, W., & Jiang, Y.-G. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01252-6_4 · Cited by 1,306
Self-citationMegaDepth: Learning Single-View Depth Prediction from Internet Photos
Li, Z., & Snavely, N. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00218 · Cited by 1,135
Self-citationOpen3D: A Modern Library for 3D Data Processing
Zhou, Q.-Y., Park, J., & Koltun, V. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.09847 · Cited by 1,124
Self-citation(2005) · Cited by 981
Kato, H., Ushiku, Y., & Harada, T. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00411 · Cited by 954
Octree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputs
Tatarchenko, M., Dosovitskiy, A., & Brox, T. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.230 · Cited by 742
Self-citationLearning a Predictable and Generative Vector Representation for Objects
Girdhar, R., Fouhey, D. F., Rodriguez, M., & Gupta, A. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46466-4_29 · Cited by 646
Hoiem, D., Efros, A. A., & Hebert, M. (2005) · ACM Transactions on Graphics · doi:10.1145/1073204.1073232 · Cited by 616
SHAPE FROM SHADING: A METHOD FOR OBTAINING THE SHAPE OF A SMOOTH OPAQUE OBJECT FROM ONE VIEW
Horn, B. K. P. (1970) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 534
Learning Category-Specific Mesh Reconstruction from Image Collections
Kanazawa, A., Tulsiani, S., Efros, A. A., & Malik, J. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01267-0_23 · Cited by 530
Multi-view Supervision for Single-View Reconstruction via Differentiable Ray Consistency
Tulsiani, S., Zhou, T., Efros, A. A., & Malik, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.30 · Cited by 519
Pix3D: Dataset and Methods for Single-Image 3D Shape Modeling
Sun, X., Wu, J., Zhang, X., Zhang, Z., Zhang, C., Xue, T., Tenenbaum, J. B., & Freeman, W. T. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00314 · Cited by 455
Depth Transfer: Depth Extraction from Video Using Non-Parametric Sampling
Karsch, K., Liu, C., & Kang, S. B. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2014.2316835 · Cited by 453
Learning Efficient Point Cloud Generation for Dense 3D Object Reconstruction
Lin, C.-H., Kong, C., & Lucey, S. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12278 · Cited by 385
AtlasNet: A Papier-Mâché Approach to Learning 3D Surface GenerationPreprint
Groueix, T., Fisher, M., Kim, V. G., Russell, B. C., & Aubry, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.05384 · Cited by 342
Multi-view 3D Models from Single Images with a Convolutional Network
Tatarchenko, M., Dosovitskiy, A., & Brox, T. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46478-7_20 · Cited by 337
Self-citationLearning Shape Abstractions by Assembling Volumetric Primitives
Tulsiani, S., Su, H., Guibas, L. J., Efros, A. A., & Malik, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.160 · Cited by 321
Hierarchical Surface Prediction for 3D Object Reconstruction
Hane, C., Tulsiani, S., & Malik, J. (2017) · 2017 International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2017.00054 · Cited by 307
A geometric approach to shape from defocus
Favaro, P., & Soatto, S. (2005) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2005.43 · Cited by 276
Perspective Transformer Nets: Learning Single-View 3D Object Reconstruction without 3D Supervision
Yan, X., Yang, J., Yumer, E., Guo, Y., & Lee, H. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.00814 · Cited by 267
MarrNet: 3D Shape Reconstruction via 2.5D Sketches
Wu, J., Wang, Y., Xue, T., Sun, X., Freeman, W. T., & Tenenbaum, J. B. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.03129 · Cited by 244
Learning 3-D Scene Structure from a Single Still Image
Saxena, A., Sun, M., & Ng, A. Y. (2007) · 2007 IEEE 11th International Conference on Computer Vision · doi:10.1109/iccv.2007.4408828 · Cited by 223
Monocular Relative Depth Perception with Web Stereo Data Supervision
Xian, K., Shen, C., Cao, Z., Lu, H., Xiao, Y., Li, R., & Luo, Z. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00040 · Cited by 215
3D Shape Reconstruction from Sketches via Multi-view Convolutional Networks
Lun, Z., Gadelha, M., Kalogerakis, E., Maji, S., & Wang, R. (2017) · 2017 International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2017.00018 · Cited by 189
Soltani, A. A., Huang, H., Wu, J., Kulkarni, T. D., & Tenenbaum, J. B. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.269 · Cited by 186
OctNetFusion: Learning Depth Fusion from Data
Riegler, G., Ulusoy, A. O., Bischof, H., & Geiger, A. (2017) · 2017 International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2017.00017 · Cited by 177
Joint embeddings of shapes and images via CNN image purification
Li, Y., Su, H., Qi, C. R., Fish, N., Cohen-Or, D., & Guibas, L. J. (2015) · ACM Transactions on Graphics · doi:10.1145/2816795.2818071 · Cited by 174
Matryoshka Networks: Predicting 3D Geometry via Nested Shape Layers
Roth, S., & Richter, S. R. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00207 · Cited by 130
Self-citationIm2Struct: Recovering 3D Shape Structure from a Single RGB Image
Niu, C., Li, J., & Xu, K. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00475 · Cited by 125
Shin, D., Fowlkes, C. C., & Hoiem, D. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00323 · Cited by 115
Rethinking Reprojection: Closing the Loop for Pose-Aware Shape Reconstruction from a Single Image
Zhu, R., Galoogahi, H. K., Wang, C., & Lucey, S. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.16 · Cited by 97
Large-Scale 3D Shape Reconstruction and Segmentation from ShapeNet Core55Preprint
Yi, L., Shao, L., Savva, M., Huang, H., Zhou, Y., Wang, Q.-R., Graham, B., Engelcke, M., Klokov, R., Lempitsky, V., et al. (2017) · arXiv.org · doi:10.48550/arxiv.1710.06104 · Cited by 69
Learning Category-Specific Deformable 3D Models for Object Reconstruction
Tulsiani, S., Kar, A., Carreira, J., & Malik, J. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2574713 · Cited by 66
Fast and globally optimal single view reconstruction of curved objects
Oswald, M. R., Töppe, E., & Cremers, D. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6247718 · Cited by 37
Scaling CNNs for High Resolution Volumetric Reconstruction from a Single Image
Johnston, A., Garg, R., Carneiro, G., & Reid, I. (2017) · 2017 IEEE International Conference on Computer Vision Workshops (ICCVW) · doi:10.1109/iccvw.2017.114 · Cited by 34
Compact Model Representation for 3D Reconstruction
Pontes, J. K., Kong, C., Eriksson, A. P., Fookes, C., Sridharan, S., & Lucey, S. (2017) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2017.00020 · Cited by 25
Cited by
Convolutional Occupancy Networks
Peng, S., Niemeyer, M., Mescheder, L., Pollefeys, M., & Geiger, A. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58580-8_31 · Cited by 801
PoinTr: Diverse Point Cloud Completion with Geometry-Aware Transformers
Yu, X., Rao, Y., Wang, Z., Liu, Z., Lu, J., & Zhou, J. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01227 · Cited by 544
Image-Based 3D Object Reconstruction: State-of-the-Art and Trends in the Deep Learning Era
Han, X.-F., Laga, H., & Bennamoun, M. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2954885 · Cited by 431
Probing the 3D Awareness of Visual Foundation Models
Banani, M. E., Raj, A., Maninis, K.-K., Kar, A., Li, Y., Rubinstein, M., Sun, D., Guibas, L., Johnson, J., & Jampani, V. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.02059 · Cited by 52
Metadata
- Authors
- Tatarchenko, M., Richter, S. R., Ranftl, R., Li, Z., Koltun, V., & Brox, T.
- Venue
- 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2019-06
- Pages
- 3400-3409
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2944579304
- Cited by
- 424
- References
- 55
- FWCI
- 62.09
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.