The DOI matches its Crossref record; 64 of the 64 references it cites were found in a registry or an index.
Pix3D: Dataset and Methods for Single-Image 3D Shape Modeling
Abstract
We study 3D shape modeling from a single image and make contributions to it in three aspects. First, we present Pix3D, a large-scale benchmark of diverse image-shape pairs with pixel-level 2D-3D alignment. Pix3D has wide applications in shape-related tasks including reconstruction, retrieval, viewpoint estimation, etc. Building such a large-scale dataset, however, is highly challenging; existing datasets either contain only synthetic data, or lack precise alignment between 2D images and 3D shapes, or only have a small number of images. Second, we calibrate the evaluation criteria for 3D shape reconstruction through behavioral studies, and use them to objectively and systematically benchmark cutting-edge reconstruction algorithms on Pix3D. Third, we design a novel model that simultaneously performs 3D reconstruction and pose estimation; our multi-task learning approach achieves state-of-the-art performance on both tasks.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Self-citationAn Algorithm for Least-Squares Estimation of Nonlinear Parameters
Marquardt, D. W. (1963) · Journal of the Society for Industrial and Applied Mathematics · doi:10.1137/0111030 · Cited by 30,728
Are we ready for autonomous driving? The KITTI vision benchmark suite
Geiger, A., Lenz, P., & Urtasun, R. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248074 · Cited by 14,897
A method for the solution of certain non-linear problems in least squares
Levenberg, K. (1944) · Quarterly of Applied Mathematics · doi:10.1090/qam/10666 · Cited by 12,359
Indoor segmentation and support inference from rgbd images
Silberman, N., Hoiem, D., Kohli, P., & Fergus, R. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33715-4_54 · Cited by 5,760
The Earth Mover's Distance as a Metric for Image Retrieval
Rubner, Y., Tomasi, C., & Guibas, L. J. (2000) · International Journal of Computer Vision · doi:10.1023/a:1026543900054 · Cited by 4,568
ScanNet: Richly-Annotated 3D Reconstructions of Indoor Scenes
Dai, A., Chang, A. X., Savva, M., Halber, M., Funkhouser, T., & Niessner, M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.261 · Cited by 4,368
EPnP: An Accurate O(n) Solution to the PnP Problem
Lepetit, V., Moreno-Noguer, F., & Fua, P. (2009) · International Journal of Computer Vision · doi:10.1007/s11263-008-0152-6 · Cited by 3,298
SUN database: Large-scale scene recognition from abbey to zoo
Xiao, J., Hays, J., Ehinger, K. A., Oliva, A., & Torralba, A. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5539970 · Cited by 3,283
Product Quantization for Nearest Neighbor Search
Jégou, H., Douze, M., & Schmid, C. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.57 · Cited by 2,956 · cited 1×
“For each test image, we retrieve its K nearest neighbors from the test set, and use Recall@K [28] to compute how many retrieved images are actually depicting the same shape.”
MethodA Point Set Generation Network for 3D Object Reconstruction from a Single Image
Fan, H., Su, H., & Guibas, L. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.264 · Cited by 2,402
ShapeNet: An Information-Rich 3D Model Repository
Chang, A. X., Funkhouser, T., Guibas, L., Hanrahan, P., Huang, Q., Li, Z., Savarese, S., Savva, M., Song, S., Su, H., et al. (2015) · arXiv · doi:10.48550/arxiv.1512.03012 · Cited by 2,330
KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera
Izadi, S., Kim, D., Hilliges, O., Molyneaux, D., Newcombe, R., Kohli, P., Shotton, J., Hodges, S., Freeman, D., Davison, A., et al. (2011) · Proceedings of the 24th annual ACM symposium on User interface software and technology · doi:10.1145/2047196.2047270 · Cited by 2,110
SUN RGB-D: A RGB-D scene understanding benchmark suite
Song, S., Lichtenberg, S. P., & Xiao, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298655 · Cited by 1,943
Tanks and temples: benchmarking large-scale scene reconstruction
Knapitsch, A., Park, J., Zhou, Q.-Y., & Koltun, V. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073599 · Cited by 1,707
3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction
Choy, C. B., Xu, D., Gwak, J., Chen, K., & Savarese, S. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46484-8_38 · Cited by 1,632
ShapeNet: An Information-Rich 3D Model Repository
Chang, A. X., Funkhouser, T. A., Guibas, L. J., Hanrahan, P., Huang, Q., Li, Z., Savarese, S., Savva, M., Song, S., Su, H., et al. (2023) · doi:10.5281/zenodo.10089705 · Cited by 1,623
OctNet: Learning Deep 3D Representations at High Resolutions
Riegler, G., Ulusoy, A. O., & Geiger, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.701 · Cited by 1,589
Shilane, P., Min, P., Kazhdan, M., & Funkhouser, T. (2004) · Proceedings Shape Modeling Applications, 2004. · doi:10.1109/smi.2004.1314504 · Cited by 1,419
Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling
Wu, J., Zhang, C., Xue, T., Freeman, W. T., & Tenenbaum, J. B. (2016) · arXiv · doi:10.48550/arxiv.1610.07584 · Cited by 1,393
Self-citationA large-scale hierarchical multi-view RGB-D object dataset
Lai, K., Bo, L., Ren, X., & Fox, D. (2011) · 2011 IEEE International Conference on Robotics and Automation · doi:10.1109/icra.2011.5980382 · Cited by 1,316
Semantic Scene Completion from a Single Depth Image
Song, S., Yu, F., Zeng, A., Chang, A. X., Savva, M., & Funkhouser, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.28 · Cited by 1,209
Analyzing appearance and contour based methods for object categorization
Leibe, B., & Schiele, B. (2003) · 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings. · doi:10.1109/cvpr.2003.1211497 · Cited by 787
Octree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputs
Tatarchenko, M., Dosovitskiy, A., & Brox, T. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.230 · Cited by 742
Numerical Geometry of Non-Rigid Shapes
Bronstein, A. M., Bronstein, M. M., & Kimmel, R. (2009) · Monographs in Computer Science · doi:10.1007/978-0-387-73301-2 · Cited by 734
Parametric correspondence and chamfer matching: two new techniques for image matching
Barrow, H. G., Tenenbaum, J. M., Bolles, R. C., & Wolf, H. C. (1977) · Defense Technical Information Center (DTIC) · Cited by 732
Self-citationBeyond PASCAL: A benchmark for 3D object detection in the wild
Xiang, Y., Mottaghi, R., & Savarese, S. (2014) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2014.6836101 · Cited by 727
FAUST: Dataset and Evaluation for 3D Mesh Registration
Bogo, F., Romero, J., Loper, M., & Black, M. J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.491 · Cited by 689
Render for CNN: Viewpoint Estimation in Images Using CNNs Trained with Rendered 3D Model Views
Su, H., Qi, C. R., Li, Y., & Guibas, L. J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.308 · Cited by 680
Learning a Predictable and Generative Vector Representation for Objects
Girdhar, R., Fouhey, D. F., Rodriguez, M., & Gupta, A. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46466-4_29 · Cited by 646
Benchmarking in Manipulation Research: Using the Yale-CMU-Berkeley Object and Model Set
Calli, B., Walsman, A., Singh, A., Srinivasa, S., Abbeel, P., & Dollar, A. M. (2015) · IEEE Robotics & Automation Magazine · doi:10.1109/mra.2015.2448951 · Cited by 602
Efficient Implementation of Marching Cubes' Cases with Topological Guarantees
Lewiner, T., Lopes, H., Vieira, A. W., & Tavares, G. (2003) · Journal of Graphics Tools · doi:10.1080/10867651.2003.10487582 · Cited by 520
Multi-view Supervision for Single-View Reconstruction via Differentiable Ray Consistency
Tulsiani, S., Zhou, T., Efros, A. A., & Malik, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.30 · Cited by 519
T-LESS: An RGB-D Dataset for 6D Pose Estimation of Texture-Less Objects
Hodan, T., Haluza, P., Obdrzalek, S., Matas, J., Lourakis, M., & Zabulis, X. (2017) · 2017 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv.2017.103 · Cited by 514
Seeing 3D Chairs: Exemplar Part-Based 2D-3D Alignment Using a Large Dataset of CAD Models
Aubry, M., Maturana, D., Efros, A. A., Russell, B. C., & Sivic, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.487 · Cited by 473
3D generic object categorization, localization and pose estimation
Savarese, S., & Fei-Fei, L. (2007) · 2007 IEEE 11th International Conference on Computer Vision · doi:10.1109/iccv.2007.4408987 · Cited by 384
Tulsiani, S., & Malik, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298758 · Cited by 346
AtlasNet: A Papier-Mâché Approach to Learning 3D Surface GenerationPreprint
Groueix, T., Fisher, M., Kim, V. G., Russell, B. C., & Aubry, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.05384 · Cited by 342
Multi-view 3D Models from Single Images with a Convolutional Network
Tatarchenko, M., Dosovitskiy, A., & Brox, T. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46478-7_20 · Cited by 337
BigBIRD: A large-scale 3D database of object instances
Singh, A., Sha, J., Narayan, K. S., Achim, T., & Abbeel, P. (2014) · 2014 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2014.6906903 · Cited by 311
Hierarchical Surface Prediction for 3D Object Reconstruction
Hane, C., Tulsiani, S., & Malik, J. (2017) · 2017 International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2017.00054 · Cited by 307
SceneNet RGB-D: Can 5M Synthetic Images Beat Generic ImageNet Pre-training on Indoor Segmentation?
McCormac, J., Handa, A., Leutenegger, S., & Davison, A. J. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.292 · Cited by 291
ObjectNet3D: A Large Scale Database for 3D Object Recognition
Xiang, Y., Kim, W., Chen, W., Ji, J., Choy, C., Su, H., Mottaghi, R., Guibas, L., & Savarese, S. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46484-8_10 · Cited by 282
Perspective Transformer Nets: Learning Single-View 3D Object Reconstruction without 3D Supervision
Yan, X., Yang, J., Yumer, E., Guo, Y., & Lee, H. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.00814 · Cited by 267
Category-specific object reconstruction from a single image
Kar, A., Tulsiani, S., Carreira, J., & Malik, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298807 · Cited by 255
A category-level 3-D object dataset: Putting the Kinect to work
Janoch, A., Karayev, S., Jia, Y., Barron, J. T., Fritz, M., Saenko, K., & Darrell, T. (2011) · 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops) · doi:10.1109/iccvw.2011.6130382 · Cited by 250
Parsing IKEA Objects: Fine Pose Estimation
Lim, J. J., Pirsiavash, H., & Torralba, A. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.372 · Cited by 249
MarrNet: 3D Shape Reconstruction via 2.5D Sketches
Wu, J., Wang, Y., Xue, T., Sun, X., Freeman, W. T., & Tenenbaum, J. B. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.03129 · Cited by 244
Self-citationSingle Image 3D Interpreter Network
Tian, Y., Wu, J., Xue, T., Lim, J. J., Tenenbaum, J. B., Torralba, A. B., & Freeman, W. T. (2016) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 241
Self-citationPose estimation for category specific multiview object localization
Ozuysal, M., Lepetit, V., & Fua, P. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206633 · Cited by 228
Learning to Generate Chairs, Tables and Cars with Convolutional Networks
Dosovitskiy, A., Springenberg, J., Tatarchenko, M., & Brox, T. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2567384 · Cited by 226
Aligning 3D models to RGB-D images of cluttered scenes
Gupta, S., Arbelaez, P., Girshick, R., & Malik, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299105 · Cited by 199
Soltani, A. A., Huang, H., Wu, J., Kulkarni, T. D., & Tenenbaum, J. B. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.269 · Cited by 186
Self-citationA Large Dataset of Object ScansPreprint
Choi, S., Zhou, Q.-Y., Miller, S., & Koltun, V. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.02481 · Cited by 173
RGBD Datasets: Past, Present and Future
Firman, M. (2016) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2016.88 · Cited by 150
3D Object Detection and Viewpoint Estimation with a Deformable 3D Cuboid Model
Fidler, S., Dickinson, S., & Urtasun, R. (2012) · doi:10.13140/2.1.2839.7440 · Cited by 149
Robust 3D Shape Correspondence in the Spectral Domain
Jain, V., & Zhang, H. (2006) · IEEE International Conference on Shape Modeling and Applications 2006 (SMI'06) · doi:10.1109/smi.2006.31 · Cited by 148
Weakly Supervised 3D Reconstruction with Adversarial Constraint
Gwak, J., Choy, C., Chandraker, M., Garg, A., & Savarese, S. (2017) · International Conference on 3D Vision · doi:10.1109/3dv.2017.00038 · Cited by 124
Simple and Efficient Mesh Editing with Consistent Local Frames
Paries, N., Degener, P., & Klein, R. (2007) · doi:10.1109/pg.2007.43 · Cited by 113
Context-based search for 3D models
Fisher, M., & Hanrahan, P. (2010) · ACM SIGGRAPH Conference and Exhibition on Computer Graphics and Interactive Techniques in Asia (SIGGRAPH Asia) · doi:10.1145/1866158.1866204 · Cited by 85
Building a database of 3D scenes from user annotations
Russell, B. C., & Torralba, A. B. (2009) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2009.5206643 · Cited by 76
Large-Scale 3D Shape Reconstruction and Segmentation from ShapeNet Core55Preprint
Yi, L., Shao, L., Savva, M., Huang, H., Zhou, Y., Wang, Q.-R., Graham, B., Engelcke, M., Klokov, R., Lempitsky, V., et al. (2017) · arXiv.org · doi:10.48550/arxiv.1710.06104 · Cited by 69
Context-based search for 3D models
Fisher, M., & Hanrahan, P. (2010) · ACM Transactions on Graphics · doi:10.1145/1882261.1866204 · Cited by 67
Efficient Globally Optimal 2D-to-3D Deformable Shape Matching
Lähner, Z., Rodolà, E., Schmidt, F. R., et al. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.240 · Cited by 33
Cited by
Occupancy Networks: Learning 3D Reconstruction in Function Space
Mescheder, L., Oechsle, M., Niemeyer, M., Nowozin, S., & Geiger, A. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00459 · Cited by 2,708
Objaverse: A Universe of Annotated 3D Objects
Deitke, M., Schwenk, D., Salvador, J., Weihs, L., Michel, O., VanderBilt, E., Schmidt, L., Ehsanit, K., Kembhavi, A., & Farhadi, A. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01263 · Cited by 725
Gkioxari, G., Johnson, J., & Malik, J. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00988 · Cited by 441
Image-Based 3D Object Reconstruction: State-of-the-Art and Trends in the Deep Learning Era
Han, X.-F., Laga, H., & Bennamoun, M. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2954885 · Cited by 431
What Do Single-View 3D Reconstruction Networks Learn?
Tatarchenko, M., Richter, S. R., Ranftl, R., Li, Z., Koltun, V., & Brox, T. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00352 · Cited by 424
Metadata
- Authors
- Sun, X., Wu, J., Zhang, X., Zhang, Z., Zhang, C., Xue, T., Tenenbaum, J. B., & Freeman, W. T.
- Venue
- 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
- Publisher
- IEEE
- Published
- 2018-06
- Pages
- 2974-2983
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2962988048
- Cited by
- 455
- References
- 64
- FWCI
- 63.57
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.