The DOI matches its Crossref record; 69 of the 69 references it cites were found in a registry or an index.
Occupancy Networks: Learning 3D Reconstruction in Function Space
Open accessAbstract
With the advent of deep neural networks, learning-based approaches for 3D reconstruction have gained popularity. However, unlike for images, in 3D there is no canonical representation which is both computationally and memory efficient yet allows for representing high-resolution geometry of arbitrary topology. Many of the state-of-the-art learning-based 3D reconstruction approaches can hence only represent very coarse 3D geometry or are limited to a restricted domain. In this paper, we propose Occupancy Networks, a new representation for learning-based 3D reconstruction methods. Occupancy networks implicitly represent the 3D surface as the continuous decision boundary of a deep neural network classifier. In contrast to existing approaches, our representation encodes a description of the 3D output at infinite resolution without excessive memory footprint. We validate that our representation can efficiently encode 3D structure and can be inferred from various kinds of input. Our experiments demonstrate competitive results, both qualitatively and quantitatively, for the challenging tasks of 3D reconstruction from single images, noisy point clouds and coarse discrete voxel grids. We believe that occupancy networks will become a useful tool in a wide variety of learning-based 3D tasks.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Auto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067
GAN(Generative Adversarial Nets)
(2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,270
Fronts propagating with curvature-dependent speed: Algorithms based on Hamilton-Jacobi formulations
Osher, S., & Sethian, J. A. (1988) · Journal of Computational Physics · doi:10.1016/0021-9991(88)90002-2 · Cited by 14,062
Marching cubes: A high resolution 3D surface construction algorithm
Lorensen, W. E., & Cline, H. E. (1987) · Proceedings of the 14th annual conference on Computer graphics and interactive techniques · doi:10.1145/37401.37422 · Cited by 10,104
Vision meets robotics: The KITTI dataset
Geiger, A., Lenz, P., Stiller, C., & Urtasun, R. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913491297 · Cited by 9,913
Self-citationPointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
Charles, R. Q., Su, H., Kaichun, M., & Guibas, L. J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.16 · Cited by 9,237
3D ShapeNets: A deep representation for volumetric shapes
Wu, Z., Song, S., Khosla, A., Yu, F., Zhang, L., Tang, X., & Xiao, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298801 · Cited by 4,550
High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs
Wang, T.-C., Liu, M.-Y., Zhu, J.-Y., Tao, A., Kautz, J., & Catanzaro, B. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00917 · Cited by 4,332
Geometric Deep Learning: Going beyond Euclidean data
Bronstein, M. M., Bruna, J., LeCun, Y., Szlam, A., & Vandergheynst, P. (2017) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2693418 · Cited by 3,661
VoxNet: A 3D Convolutional Neural Network for real-time object recognition
Maturana, D., & Scherer, S. (2015) · 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2015.7353481 · Cited by 3,458
DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation
Park, J. J., Florence, P., Straub, J., Newcombe, R., & Lovegrove, S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00025 · Cited by 3,422
A volumetric method for building complex models from range images
Curless, B., & Levoy, M. (1996) · Proceedings of the 23rd annual conference on Computer graphics and interactive techniques · doi:10.1145/237170.237269 · Cited by 3,124
A Point Set Generation Network for 3D Object Reconstruction from a Single Image
Fan, H., Su, H., & Guibas, L. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.264 · Cited by 2,402
Screened poisson surface reconstruction
Kazhdan, M., & Hoppe, H. (2013) · ACM Transactions on Graphics · doi:10.1145/2487228.2487237 · Cited by 2,231
Poisson Surface Reconstruction
Kazhdan, M. M., Bolitho, M., & Hoppe, H. (2006) · doi:10.2312/sgp/sgp06/061-070 · Cited by 2,125
End-to-End Recovery of Human Shape and Pose
Kanazawa, A., Black, M. J., Jacobs, D. W., & Malik, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00744 · Cited by 1,870
3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction
Choy, C. B., Xu, D., Gwak, J., Chen, K., & Savarese, S. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46484-8_38 · Cited by 1,632
Deep Metric Learning via Lifted Structured Feature Embedding
Song, H. O., Xiang, Y., Jegelka, S., & Savarese, S. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.434 · Cited by 1,630
ShapeNet: An Information-Rich 3D Model Repository
Chang, A. X., Funkhouser, T. A., Guibas, L. J., Hanrahan, P., Huang, Q., Li, Z., Savarese, S., Savva, M., Song, S., Su, H., et al. (2023) · doi:10.5281/zenodo.10089705 · Cited by 1,623
OctNet: Learning Deep 3D Representations at High Resolutions
Riegler, G., Ulusoy, A. O., & Geiger, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.701 · Cited by 1,589
Self-citationVolumetric and Multi-view CNNs for Object Classification on 3D Data
Qi, C. R., Su, H., NieBner, M., Dai, A., Yan, M., & Guibas, L. J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.609 · Cited by 1,504
The ball-pivoting algorithm for surface reconstruction
Bernardini, F., Mittleman, J., Rushmeier, H., Silva, C., & Taubin, G. (1999) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/2945.817351 · Cited by 1,495
Learning Implicit Fields for Generative Shape Modeling
Chen, Z., & Zhang, H. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00609 · Cited by 1,443
Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling
Wu, J., Zhang, C., Xue, T., Freeman, W. T., & Tenenbaum, J. B. (2016) · arXiv · doi:10.48550/arxiv.1610.07584 · Cited by 1,393 · cited 4×
“Voxel Representations: Due to their simplicity, voxels are the most commonly used representation for discriminative [45,55,63] and generative [9,23,58,64,75,77] 3D tasks.”
BackgroundInfluentialPixel2Mesh: Generating 3D Mesh Models from Single RGB Images
Wang, N., Zhang, Y., Li, Z., Fu, Y., Liu, W., & Jiang, Y.-G. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01252-6_4 · Cited by 1,306
Geometric modeling using octree encoding
Meagher, D. (1982) · Computer Graphics and Image Processing · doi:10.1016/0146-664x(82)90104-6 · Cited by 1,266
Cremers, D., Rousson, M., & Deriche, R. (2007) · International Journal of Computer Vision · doi:10.1007/s11263-006-8711-1 · Cited by 1,006
Which Training Methods for GANs do actually Converge?
Mescheder, L., Geiger, A., & Nowozin, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.04406 · Cited by 768
Self-citationOctree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputs
Tatarchenko, M., Dosovitskiy, A., & Brox, T. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.230 · Cited by 742
Deep Sliding Shapes for Amodal 3D Object Detection in RGB-D Images
Song, S., & Xiao, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.94 · Cited by 681
Learning a Predictable and Generative Vector Representation for Objects
Girdhar, R., Fouhey, D. F., Rodriguez, M., & Gupta, A. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46466-4_29 · Cited by 646
Shape Completion Using 3D-Encoder-Predictor CNNs and Shape Synthesis
Dai, A., Qi, C. R., & NieBner, M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.693 · Cited by 633
Adversarially Learned Inference
Dumoulin, V., Belghazi, M. I., Poole, B., Mastropietro, O., Lamb, A., Arjovsky, M., & Courville, A. C. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.00704 · Cited by 579
Generating 3D Faces Using Convolutional Mesh Autoencoders
Ranjan, A., Bolkart, T., Sanyal, S., & Black, M. J. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01219-9_43 · Cited by 560
Learning Category-Specific Mesh Reconstruction from Image Collections
Kanazawa, A., Tulsiani, S., Efros, A. A., & Malik, J. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01267-0_23 · Cited by 530
Multi-view Supervision for Single-View Reconstruction via Differentiable Ray Consistency
Tulsiani, S., Zhou, T., Efros, A. A., & Malik, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.30 · Cited by 519
Rapid Octree Construction from Image Sequences
Szeliski, R. (1993) · CVGIP: Image Understanding · doi:10.1006/ciun.1993.1029 · Cited by 475
Learning Representations and Generative Models for 3D Point Clouds
Achlioptas, P., Diamanti, O., Mitliagkas, I., & Guibas, L. J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.02392 · Cited by 470
Pix3D: Dataset and Methods for Single-Image 3D Shape Modeling
Sun, X., Wu, J., Zhang, X., Zhang, Z., Zhang, C., Xue, T., Tenenbaum, J. B., & Freeman, W. T. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00314 · Cited by 455
Generative and Discriminative Voxel Modeling with Convolutional Neural Networks
Brock, A., Lim, T., Ritchie, J. M., & Weston, N. J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1608.04236 · Cited by 408
Simplifying surfaces with color and texture using quadric error metrics
Garland, M., & Heckbert, P. S. (1998) · doi:10.5555/288216.288280 · Cited by 403
Modulating early visual processing by language
de Vries, H., Strub, F., Mary, J., Larochelle, H., Pietquin, O., & Courville, A. C. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.00683 · Cited by 369
SurfaceNet: An End-to-End 3D Neural Network for Multiview Stereopsis
Ji, M., Gall, J., Zheng, H., Liu, Y., & Fang, L. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.253 · Cited by 361
Hierarchical Surface Prediction for 3D Object Reconstruction
Hane, C., Tulsiani, S., & Malik, J. (2017) · 2017 International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2017.00054 · Cited by 307
3D Shape Induction from 2D Views of Multiple Objects
Gadelha, M., Maji, S., & Wang, R. (2017) · 2017 International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2017.00053 · Cited by 260
Deep Marching Cubes: Learning Explicit Surface Representations
Liao, Y., Donne, S., & Geiger, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00308 · Cited by 255
Self-citationMarrNet: 3D Shape Reconstruction via 2.5D Sketches
Wu, J., Wang, Y., Xue, T., Sun, X., Freeman, W. T., & Tenenbaum, J. B. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.03129 · Cited by 244
Improving generalization performance using double backpropagation
Drucker, H., & Le Cun, Y. (1992) · IEEE Transactions on Neural Networks · doi:10.1109/72.165600 · Cited by 239
Learning 3D Shape Completion from Laser Scan Data with Weak Supervision
Stutz, D., & Geiger, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00209 · Cited by 229
Self-citationSSD: Smooth Signed Distance Surface Reconstruction
Calakli, F., & Taubin, G. (2011) · Computer Graphics Forum · doi:10.1111/j.1467-8659.2011.02058.x · Cited by 220
3D Mesh Labeling via Deep Convolutional Neural Networks
Guo, K., Zou, D., & Chen, X. (2015) · ACM Transactions on Graphics · doi:10.1145/2835487 · Cited by 217
Learning a Multi-View Stereo Machine
Kar, A., Häne, C., & Malik, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.05375 · Cited by 209
Learning Shape Priors for Single-View 3D Completion And Reconstruction
Wu, J., Zhang, C., Zhang, X., Zhang, Z., Freeman, W. T., & Tenenbaum, J. B. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01252-6_40 · Cited by 175
Learning to Reconstruct Shapes from Unseen ClassesPreprint
Zhang, X., Zhang, Z., Zhang, C.-K., Tenenbaum, J., Freeman, B., & Wu, J. (2018) · Neural Information Processing Systems · doi:10.48550/arxiv.1812.11166 · Cited by 170
A finite element method for the simulation of a Rayleigh-Taylor instability
Dervieux, A., & Thomasset, F. (1980) · Lecture Notes in Mathematics · doi:10.1007/bfb0086904 · Cited by 163
Structure and motion from silhouettes
Wong, K. K., & Cipolla, R. (2002) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2001.937627 · Cited by 119
Modelling Dynamic Scenes by Registering Multi-View Image Sequences
Pons, J.-P., Keriven, R., & Faugeras, O. D. (2005) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2005.227 · Cited by 111
Yezzi, A. J., & Soatto, S. (2003) · International Journal of Computer Vision · doi:10.1023/a:1023079624234 · Cited by 107
Deep Level Sets: Implicit Surface Representations for 3D Shape InferencePreprint
Michalkiewicz, M., Pontes, J. K., Jack, D., Baktashmotlagh, M., & Eriksson, A. P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.06802 · Cited by 107
RayNet: Learning Volumetric 3D Reconstruction with Ray Potentials
Paschalidou, D., Ulusoy, A. O., Schmitt, C., Gool, L. V., & Geiger, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00410 · Cited by 102
Self-citationAtlasNet: Multi-atlas Non-linear Deep Networks for Medical Image Segmentation
Vakalopoulou, M., Chassagnon, G., Bus, N., Marini, R., Zacharaki, E. I., Revel, M., & Nikos Paragios, A. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00937-3_75 · Cited by 85
Space-time isosurface evolution for temporally coherent 3D reconstruction
Goldlüecke, B., & Magnor, M. A. (2004) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2004.1315053 · Cited by 75
3D shape segmentation via shape fully convolutional networks
Wang, P., Gan, Y., Shui, P., Yu, F., Zhang, Y., Chen, S., & Sun, Z. (2017) · Computers & Graphics · doi:10.1016/j.cag.2017.07.030 · Cited by 67
Augmented Reality meets Deep Learning
Alhaija, H. A., Mustikovela, S. K., Mescheder, L., Geiger, A., & Rother, C. (2017) · British Machine Vision Conference (BMVC) · doi:10.5244/c.31.81 · Cited by 57
Self-citationTowards Probabilistic Volumetric Reconstruction Using Ray Potentials
Ulusoy, A. O., Geiger, A., & Black, M. J. (2015) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2015.9 · Cited by 56
Self-citationFrom Point Clouds to Mesh Using Regression
Ladický, Ľ., Saurer, O., Jeong, S., Maninchedda, F., & Pollefeys, M. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.420 · Cited by 47
Augmented Reality Meets Deep Learning for Car Instance Segmentation in Urban Scenes
Alhaija, H., Mustikovela, S., Mescheder, L., Geiger, A., & Rother, C. (2017) · MPG.PuRe (Max Planck Society) · Cited by 40
Self-citationOctNetFusion: Learning Depth Fusion from DataPreprint
Riegler, G., Ulusoy, A. O., Bischof, H., & Geiger, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.01047 · Cited by 15
Self-citation
Cited by
NeRF: representing scenes as neural radiance fields for view synthesis
Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, J. T., Ramamoorthi, R., & Ng, R. (2022) · Communications of the ACM · doi:10.1145/3503250 · Cited by 6,852
NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis
Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, J. T., Ramamoorthi, R., & Ng, R. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58452-8_24 · Cited by 5,351
Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains
Tancik, M., Srinivasan, P. P., Mildenhall, B., Fridovich-Keil, S., Raghavan, N., Singhal, U., Ramamoorthi, R., Barron, J. T., & Ng, R. (2020) · arXiv · doi:10.48550/arxiv.2006.10739 · Cited by 1,459
PlenOctrees for Real-time Rendering of Neural Radiance Fields
Yu, A., Li, R., Tancik, M., Li, H., Ng, R., & Kanazawa, A. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00570 · Cited by 854
Differentiable Volumetric Rendering: Learning Implicit 3D Representations Without 3D Supervision
Niemeyer, M., Mescheder, L., Oechsle, M., & Geiger, A. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00356 · Cited by 834
Self-citation
Metadata
- Authors
- Mescheder, L., Oechsle, M., Niemeyer, M., Nowozin, S., & Geiger, A.
- Venue
- 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2019-06
- Pages
- 4455-4465
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2904383797
- Cited by
- 2,708
- References
- 69
- FWCI
- 301.55
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.