The DOI matches its Crossref record; 65 of the 65 references it cites were found in a registry or an index.
Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstruction
Open accessAbstract
Traditional approaches for learning 3D object categories have been predominantly trained and evaluated on synthetic datasets due to the unavailability of real 3D-annotated category-centric data. Our main goal is to facilitate advances in this field by collecting real-world data in a magnitude similar to the existing synthetic counterparts. The principal contribution of this work is thus a large-scale dataset, called Common Objects in 3D, with real multi-view images of object categories annotated with camera poses and ground truth 3D point clouds. The dataset contains a total of 1.5 million frames from nearly 19,000 videos capturing objects from 50 MS-COCO categories and, as such, it is significantly larger than alternatives both in terms of the number of categories and objects.We exploit this new dataset to conduct one of the first large-scale "in-the-wild" evaluations of several new-view-synthesis and category-centric 3D reconstruction methods. Finally, we contribute NerFormer - a novel neural rendering method that leverages the powerful Transformer to reconstruct an object given a small number of its views.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Exploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
Cortes, C., & Vapnik, V. (1995) · Machine Learning · doi:10.1007/bf00994018 · Cited by 59,421
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
Zhang, R., Isola, P., Efros, A. A., Shechtman, E., & Wang, O. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00068 · Cited by 14,623
Structure-from-Motion Revisited
Schonberger, J. L., & Frahm, J.-M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.445 · Cited by 6,433
NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis
Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, J. T., Ramamoorthi, R., & Ng, R. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58452-8_24 · Cited by 5,351
DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation
Park, J. J., Florence, P., Straub, J., Newcombe, R., & Lovegrove, S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00025 · Cited by 3,422
Generalised Dice Overlap as a Deep Learning Loss Function for Highly Unbalanced Segmentations
Sudre, C. H., Li, W., Vercauteren, T., Ourselin, S., & Jorge Cardoso, M. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-67558-9_28 · Cited by 2,829
Occupancy Networks: Learning 3D Reconstruction in Function Space
Mescheder, L., Oechsle, M., Niemeyer, M., Nowozin, S., & Geiger, A. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00459 · Cited by 2,708
A Point Set Generation Network for 3D Object Reconstruction from a Single Image
Fan, H., Su, H., & Guibas, L. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.264 · Cited by 2,402
ShapeNet: An Information-Rich 3D Model Repository
Chang, A. X., Funkhouser, T., Guibas, L., Hanrahan, P., Huang, Q., Li, Z., Savarese, S., Savva, M., Song, S., Su, H., et al. (2015) · arXiv · doi:10.48550/arxiv.1512.03012 · Cited by 2,330
Pixelwise View Selection for Unstructured Multi-View Stereo
Schönberger, J. L., Zheng, E., Frahm, J.-M., & Pollefeys, M. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46487-9_31 · Cited by 2,315
3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction
Choy, C. B., Xu, D., Gwak, J., Chen, K., & Savarese, S. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46484-8_38 · Cited by 1,632
ShapeNet: An Information-Rich 3D Model Repository
Chang, A. X., Funkhouser, T. A., Guibas, L. J., Hanrahan, P., Huang, Q., Li, Z., Savarese, S., Savva, M., Song, S., Su, H., et al. (2023) · doi:10.5281/zenodo.10089705 · Cited by 1,623
Improving generalization with active learning
Cohn, D., Atlas, L., & Ladner, R. (1994) · Machine Learning · doi:10.1007/bf00993277 · Cited by 1,591
pixelNeRF: Neural Radiance Fields from One or Few Images
Yu, A., Ye, V., Tancik, M., & Kanazawa, A. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00455 · Cited by 1,560
Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling
Wu, J., Zhang, C., Xue, T., Freeman, W. T., & Tenenbaum, J. B. (2016) · arXiv · doi:10.48550/arxiv.1610.07584 · Cited by 1,393 · cited 1×
“Early methods [18, 70, 58, 61, 40] predicted a global scene encoding z global = Φ CNN ( I src ) with a deep convolutional network Φ CNN that solely analyzed the colors of source image pixels.”
MethodSupport vector machine active learning for image retrieval
Tong, S., & Chang, E. (2001) · Proceedings of the ninth ACM international conference on Multimedia · doi:10.1145/500141.500159 · Cited by 1,328
Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images
Wang, N., Zhang, Y., Li, Z., Fu, Y., Liu, W., & Jiang, Y.-G. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01252-6_4 · Cited by 1,306
PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization
Saito, S., Huang, Z., Natsume, R., Morishima, S., Li, H., & Kanazawa, A. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00239 · Cited by 1,228
PointRend: Image Segmentation As Rendering
Kirillov, A., Wu, Y., He, K., & Girshick, R. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00982 · Cited by 1,135
Kato, H., Ushiku, Y., & Harada, T. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00411 · Cited by 954
Unstructured lumigraph rendering
Buehler, C., Bosse, M., McMillan, L., Gortler, S., & Cohen, M. (2001) · Proceedings of the 28th annual conference on Computer graphics and interactive techniques · doi:10.1145/383259.383309 · Cited by 881
Differentiable Volumetric Rendering: Learning Implicit 3D Representations Without 3D Supervision
Niemeyer, M., Mescheder, L., Oechsle, M., & Geiger, A. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00356 · Cited by 834
Beyond PASCAL: A benchmark for 3D object detection in the wild
Xiang, Y., Mottaghi, R., & Savarese, S. (2014) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2014.6836101 · Cited by 727
IBRNet: Learning Multi-View Image-Based Rendering
Wang, Q., Wang, Z., Genova, K., Srinivasan, P., Zhou, H., Barron, J. T., Martin-Brualla, R., Snavely, N., & Funkhouser, T. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00466 · Cited by 725
Neural volumes: learning dynamic renderable volumes from images
Lombardi, S., Simon, T., Saragih, J., Schwartz, G., Lehrmann, A., & Sheikh, Y. (2019) · ACM Transactions on Graphics · doi:10.1145/3306346.3323020 · Cited by 693
PointFlow: 3D Point Cloud Generation With Continuous Normalizing Flows
Yang, G., Huang, X., Hao, Z., Liu, M.-Y., Belongie, S., & Hariharan, B. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00464 · Cited by 656 · cited 1×
“3D reconstruction A vast amount of methods studied fully supervised 3D reconstruction from 2D images making use of several different representations: voxel grids [10, 18], meshes [19, 67], point clouds [14, 73], signed distance fields, [49, 2] or continuous occupancy fields [42, 8, 17, 16].”
BackgroundLearning a Predictable and Generative Vector Representation for Objects
Girdhar, R., Fouhey, D. F., Rodriguez, M., & Gupta, A. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46466-4_29 · Cited by 646
Soft Rasterizer: A Differentiable Renderer for Image-Based 3D Reasoning
Liu, S., Chen, W., Li, T., & Li, H. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00780 · Cited by 641
Deep blending for free-viewpoint image-based rendering
Hedman, P., Philip, J., Price, T., Frahm, J.-M., Drettakis, G., & Brostow, G. (2018) · ACM Transactions on Graphics · doi:10.1145/3272127.3275084 · Cited by 617
Learning Category-Specific Mesh Reconstruction from Image Collections
Kanazawa, A., Tulsiani, S., Efros, A. A., & Malik, J. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01267-0_23 · Cited by 530
Multi-view Supervision for Single-View Reconstruction via Differentiable Ray Consistency
Tulsiani, S., Zhou, T., Efros, A. A., & Malik, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.30 · Cited by 519
Support vector machine active learning for image retrieval
Tong, S., & Chang, E. (2001) · Proceedings of the ninth ACM international conference on Multimedia - MULTIMEDIA '01 · doi:10.1145/500156.500159 · Cited by 484
Gkioxari, G., Johnson, J., & Malik, J. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00988 · Cited by 441
SAL: Sign Agnostic Learning of Shapes From Raw Data
Atzmon, M., & Lipman, Y. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00264 · Cited by 433
SynSin: End-to-End View Synthesis From a Single Image
Wiles, O., Gkioxari, G., Szeliski, R., & Johnson, J. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00749 · Cited by 418
Accelerating 3D deep learning with PyTorch3D
Johnson, J., Ravi, N., Reizenstein, J., Novotny, D., Tulsiani, S., Lassner, C., & Branson, S. (2020) · SIGGRAPH Asia 2020 Courses · doi:10.1145/3415263.3419160 · Cited by 413
Self-citationLocal Deep Implicit Functions for 3D Shape
Genova, K., Cole, F., Sud, A., Sarna, A., & Funkhouser, T. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00491 · Cited by 370
Panoptic Studio: A Massively Multiview System for Social Interaction Capture
Joo, H., Simon, T., Li, X., Liu, H., Tan, L., Gui, L., Banerjee, S., Godisart, T., Nabbe, B., Matthews, I., et al. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2782743 · Cited by 344
HoloGAN: Unsupervised Learning of 3D Representations From Natural Images
Nguyen-Phuoc, T., Li, C., Theis, L., Richardt, C., & Yang, Y. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00768 · Cited by 340
Multi-view 3D Models from Single Images with a Convolutional Network
Tatarchenko, M., Dosovitskiy, A., & Brox, T. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46478-7_20 · Cited by 337
Learning Shape Templates With Structured Implicit Functions
Genova, K., Cole, F., Vlasic, D., Sarna, A., Freeman, W., & Funkhouser, T. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00725 · Cited by 323
GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis
Schwarz, K., Liao, Y., Niemeyer, M., & Geiger, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.02442 · Cited by 314
Objectron: A Large Scale Dataset of Object-Centric Videos in the Wild with Pose Annotations
Ahmadyan, A., Zhang, L., Ablavatski, A., Wei, J., & Grundmann, M. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00773 · Cited by 262
3D Shape Induction from 2D Views of Multiple Objects
Gadelha, M., Maji, S., & Wang, R. (2017) · 2017 International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2017.00053 · Cited by 260
A Large Dataset of Object ScansPreprint
Choi, S., Zhou, Q.-Y., Miller, S., & Koltun, V. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.02481 · Cited by 254
Parsing IKEA Objects: Fine Pose Estimation
Lim, J. J., Pirsiavash, H., & Torralba, A. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.372 · Cited by 249
Occupancy Flow: 4D Reconstruction by Learning Particle Dynamics
Niemeyer, M., Mescheder, L., Oechsle, M., & Geiger, A. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00548 · Cited by 246
Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderer
Chen, W., Gao, J., Ling, H., Smith, E. J., Lehtinen, J., Jacobson, A., & Fidler, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.01210 · Cited by 238
Unsupervised Learning of Probably Symmetric Deformable 3D Objects From Images in the Wild
Wu, S., Rupprecht, C., & Vedaldi, A. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00008 · Cited by 233
Learning a Multi-View Stereo Machine
Kar, A., Häne, C., & Malik, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.05375 · Cited by 209
Multi-view Consistency as Supervisory Signal for Learning Shape and Pose Prediction
Tulsiani, S., Efros, A. A., & Malik, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00306 · Cited by 202
Self-supervised Single-View 3D Reconstruction via Semantic Consistency
Li, X., Liu, S., Kim, K., Mello, S. D., Jampani, V., Yang, M., & Kautz, J. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58568-6_40 · Cited by 197
Optimizing the Latent Space of Generative Networks
Bojanowski, P., Joulin, A., López-Paz, D., & Szlam, A. D. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.05776 · Cited by 185
HoloGAN: Unsupervised Learning of 3D Representations From Natural Images
Nguyen-Phuoc, T., Li, C., Theis, L., Richardt, C., & Yang, Y.-L. (2019) · 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW) · doi:10.1109/iccvw.2019.00255 · Cited by 178
Shape and Viewpoint Without Keypoints
Goel, S., Kanazawa, A., & Malik, J. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58555-6_6 · Cited by 138
Canonical Surface Mapping via Geometric Cycle Consistency
Kulkarni, N. V., Tulsiani, S., & Gupta, A. K. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00229 · Cited by 117
Articulation-Aware Canonical Surface Mapping
Kulkarni, N. V., Gupta, A. K., Fouhey, D. F., & Tulsiani, S. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00053 · Cited by 101
GRF: Learning a General Radiance Field for 3D Scene Representation and Rendering
Trevithick, A., & Yang, B. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · Cited by 82
Learning Category-Specific Deformable 3D Models for Object Reconstruction
Tulsiani, S., Kar, A., Carreira, J., & Malik, J. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2574713 · Cited by 77
Capturing the Geometry of Object Categories from Video Supervision
Novotný, D., Larlus, D., & Vedaldi, A. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2871117 · Cited by 22
Cited by
Zero-1-to-3: Zero-shot One Image to 3D Object
Liu, R., Wu, R., Van Hoorick, B., Tokmakov, P., Zakharov, S., & Vondrick, C. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00853 · Cited by 841
Neural Fields in Visual Computing and Beyond
Xie, Y., Takikawa, T., Saito, S., Litany, O., Yan, S., Khan, N., Tombari, F., Tompkin, J., sitzmann, V., & Sridhar, S. (2022) · Computer Graphics Forum · doi:10.1111/cgf.14505 · Cited by 484
PixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstruction
Charatan, D., Li, S. L., Tagliasacchi, A., & Sitzmann, V. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01840 · Cited by 420
“Preserving end-to-end locality and shift equivariance between encoder and scene representation via pixel-aligned features [14, 23, 39, 52, 58] or via transformers [35, 54] has enabled generalization to unbounded scenes.”
BackgroundGoogle Scanned Objects: A High-Quality Dataset of 3D Scanned Household Items
Downs, L., Francis, A., Koenig, N., Kinman, B., Hickman, R., Reymann, K., McHugh, T. B., & Vanhoucke, V. (2022) · 2022 International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra46639.2022.9811809 · Cited by 377
LERF: Language Embedded Radiance Fields
Kerr, J., Kim, C. M., Goldberg, K., Kanazawa, A., & Tancik, M. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01807 · Cited by 370
Metadata
- Authors
- Reizenstein, J., Shapovalov, R., Henzler, P., Sbordone, L., Labatut, P., & Novotny, D.
- Venue
- 2021 IEEE/CVF International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2021-10
- Pages
- 10881-10891
- OpenAlex
- W3196466825
- Cited by
- 352
- References
- 65
- FWCI
- 38.38
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.