The DOI matches its Crossref record; 61 of the 61 references it cites were found in a registry or an index.
DeepVoxels: Learning Persistent 3D Feature Embeddings
Open accessAbstract
In this work, we address the lack of 3D understanding of generative neural networks by introducing a persistent 3D feature embedding for view synthesis. To this end, we propose DeepVoxels, a learned representation that encodes the view-dependent appearance of a 3D scene without having to explicitly model its geometry. At its core, our approach is based on a Cartesian 3D grid of persistent embedded features that learn to make use of the underlying 3D scene structure. Our approach combines insights from 3D geometric computer vision with recent advances in learning image-to-image mappings based on adversarial loss functions. DeepVoxels is supervised, without requiring a 3D reconstruction of the scene, using a 2D re-rendering loss and enforces perspective and multi-view geometry in a principled manner. We apply our persistent 3D scene representation to the problem of novel view synthesis demonstrating high-quality results for a variety of challenging scenes.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545
Image-to-Image Translation with Conditional Adversarial Networks
Isola, P., Zhu, J.-Y., Zhou, T., & Efros, A. A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.632 · Cited by 22,930
Multiple View Geometry in Computer Vision
Hartley, R., & Zisserman, A. (2004) · doi:10.1017/cbo9780511811685 · Cited by 21,070
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
Auto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067
GAN(Generative Adversarial Nets)
(2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,270
Unsupervised representation learning with deep convolutional generative adversarial networks
Radford, A., Metz, L., & Chintala, S. (2015) · arXiv · doi:10.48550/arxiv.1511.06434 · Cited by 12,235
(2017) · Cited by 11,625
PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
Charles, R. Q., Su, H., Kaichun, M., & Guibas, L. J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.16 · Cited by 9,237
Conditional Generative Adversarial Nets
Mirza, M., & Osindero, S. (2014) · arXiv · doi:10.48550/arxiv.1411.1784 · Cited by 8,863
Structure-from-Motion Revisited
Schonberger, J. L., & Frahm, J.-M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.445 · Cited by 6,433
High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs
Wang, T.-C., Liu, M.-Y., Zhu, J.-Y., Tao, A., Kautz, J., & Catanzaro, B. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00917 · Cited by 4,332
Bundle Adjustment — A Modern Synthesis
Triggs, B., McLauchlan, P. F., Hartley, R. I., & Fitzgibbon, A. W. (2000) · Lecture Notes in Computer Science · doi:10.1007/3-540-44480-7_21 · Cited by 3,988
Computer vision: algorithms and applications
(2011) · Choice Reviews Online · doi:10.5860/choice.48-5140 · Cited by 3,911
VoxNet: A 3D Convolutional Neural Network for real-time object recognition
Maturana, D., & Scherer, S. (2015) · 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2015.7353481 · Cited by 3,458
Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
Eigen, D., Puhrsch, C., & Fergus, R. (2014) · arXiv · doi:10.48550/arxiv.1406.2283 · Cited by 3,442
Accurate, Dense, and Robust Multiview Stereopsis
Furukawa, Y., & Ponce, J. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.161 · Cited by 3,019
Photo tourism: exploring photo collections in 3D
Snavely, N., Seitz, S. M., & Szeliski, R. (2006) · ACM Transactions on Graphics · doi:10.1145/1141911.1141964 · Cited by 2,866
PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
Qi, C. R., Su, H., Mo, K., & Guibas, L. J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.00593 · Cited by 2,817
Pixelwise View Selection for Unstructured Multi-View Stereo
Schönberger, J. L., Zheng, E., Frahm, J.-M., & Pollefeys, M. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46487-9_31 · Cited by 2,315
Poisson Surface Reconstruction
Kazhdan, M. M., Bolitho, M., & Hoppe, H. (2006) · doi:10.2312/sgp/sgp06/061-070 · Cited by 2,125
Conditional Image Generation with PixelCNN Decoders
van den Oord, A., Kalchbrenner, N., Vinyals, O., Espeholt, L., Graves, A., & Kavukcuoglu, K. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.05328 · Cited by 1,833
3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction
Choy, C. B., Xu, D., Gwak, J., Chen, K., & Savarese, S. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46484-8_38 · Cited by 1,632
OctNet: Learning Deep 3D Representations at High Resolutions
Riegler, G., Ulusoy, A. O., & Geiger, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.701 · Cited by 1,589
Volumetric and Multi-view CNNs for Object Classification on 3D Data
Qi, C. R., Su, H., NieBner, M., Dai, A., Yan, M., & Guibas, L. J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.609 · Cited by 1,504
View interpolation for image synthesis
Chen, S. E., & Williams, L. (1993) · Proceedings of the 20th annual conference on Computer graphics and interactive techniques · doi:10.1145/166117.166153 · Cited by 1,097
Agarwal, S., Furukawa, Y., Snavely, N., Simon, I., Curless, B., Seitz, S. M., & Szeliski, R. (2011) · Communications of the ACM · doi:10.1145/2001269.2001293 · Cited by 1,072
Agarwal, S., Snavely, N., Simon, I., Seitz, S. M., & Szeliski, R. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459148 · Cited by 1,067
An Intriguing Failing of Convolutional Neural Networks and the CoordConv Solution
Liu, R., Lehman, J., Molino, P., Such, F. P., Frank, E., Sergeev, A., & Yosinski, J. (2018) · arXiv · doi:10.48550/arxiv.1807.03247 · Cited by 784
Learning-based view synthesis for light field cameras
Kalantari, N. K., Wang, T.-C., & Ramamoorthi, R. (2016) · ACM Transactions on Graphics · doi:10.1145/2980179.2980251 · Cited by 726
Photo tourism: exploring photo collections in 3D
Snavely, N., Seitz, S. M., & Szeliski, R. (2006) · ACM SIGGRAPH 2006 Papers on - SIGGRAPH '06 · doi:10.1145/1179352.1141964 · Cited by 696
Deep Convolutional Inverse Graphics Network
Kulkarni, T. D., Whitney, W., Kohli, P., & Tenenbaum, J. B. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1503.03167 · Cited by 653
Deep blending for free-viewpoint image-based rendering
Hedman, P., Philip, J., Price, T., Frahm, J.-M., Drettakis, G., & Brostow, G. (2018) · ACM Transactions on Graphics · doi:10.1145/3272127.3275084 · Cited by 617
View Synthesis by Appearance Flow
Zhou, T., Tulsiani, S., Sun, W., Malik, J., & Efros, A. A. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_18 · Cited by 579
Deep Stereo: Learning to Predict New Views from the World's Imagery
Flynn, J., Neulander, I., Philbin, J., & Snavely, N. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.595 · Cited by 574
Multi-view Supervision for Single-View Reconstruction via Differentiable Ray Consistency
Tulsiani, S., Zhou, T., Efros, A. A., & Malik, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.30 · Cited by 519
Neural scene representation and rendering
Eslami, S. M. A., Jimenez Rezende, D., Besse, F., Viola, F., Morcos, A. S., Garnelo, M., Ruderman, A., Rusu, A. A., Danihelka, I., Gregor, K., et al. (2018) · Science · doi:10.1126/science.aar6170 · Cited by 495
Kim, H., Garrido, P., Tewari, A., Xu, W., Thies, J., Niessner, M., Pérez, P., Richardt, C., Zollhöfer, M., & Theobalt, C. (2018) · ACM Transactions on Graphics · doi:10.1145/3197517.3201283 · Cited by 475
Self-citationUnsupervised Learning of 3D Structure from ImagesPreprint
Rezende, D. J., Eslami, S. M. A., Mohamed, S., Battaglia, P., Jaderberg, M., & Heess, N. (2016) · Neural Information Processing Systems · doi:10.48550/arxiv.1607.00662 · Cited by 414
Review of image-based rendering techniques
Shum, H., & Kang, S. B. (2000) · SPIE Proceedings · doi:10.1117/12.386541 · Cited by 385
Learning Efficient Point Cloud Generation for Dense 3D Object Reconstruction
Lin, C.-H., Kong, C., & Lucey, S. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12278 · Cited by 385
Soft 3D reconstruction for view synthesis
Penner, E., & Zhang, L. (2017) · ACM Transactions on Graphics · doi:10.1145/3130800.3130855 · Cited by 358
Transformation-Grounded Image Generation Network for Novel 3D View Synthesis
Park, E., Yang, J., Yumer, E., Ceylan, D., & Berg, A. C. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.82 · Cited by 279
Perspective Transformer Nets: Learning Single-View 3D Object Reconstruction without 3D Supervision
Yan, X., Yang, J., Yumer, E., Guo, Y., & Lee, H. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.00814 · Cited by 267
Weakly-supervised Disentangling with Recurrent Transformations for 3D View Synthesis
Yang, J., Reed, S., Yang, M., & Lee, H. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1601.00706 · Cited by 246
Learning to Generate Chairs, Tables and Cars with Convolutional Networks
Dosovitskiy, A., Springenberg, J., Tatarchenko, M., & Brox, T. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2567384 · Cited by 226
Unsupervised Geometry-Aware Representation for 3D Human Pose Estimation
Rhodin, H., Salzmann, M., & Fua, P. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01249-6_46 · Cited by 218
Learning a Multi-View Stereo Machine
Kar, A., Häne, C., & Malik, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.05375 · Cited by 209
Layer-Structured 3D Scene Inference via View Synthesis
Tulsiani, S., Tucker, R. P., & Snavely, N. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01234-2_19 · Cited by 156
Scalable inside-out image-based rendering
Hedman, P., Ritschel, T., George Drettakis, B., & Brostow, G. (2016) · ACM Transactions on Graphics · doi:10.1145/2980179.2982420 · Cited by 155
MapNet: An Allocentric Spatial Memory for Mapping Environments
Henriques, J. F., & Vedaldi, A. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00884 · Cited by 155
Explorations in Homeomorphic Variational Auto-EncodingPreprint
Falorsi, L., de Haan, P., Davidson, T. R., Cao, N. D., Weiler, M., Forré, P. D., & Cohen, T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.04689 · Cited by 108
RenderNet: A deep convolutional network for differentiable rendering from 3D shapesPreprint
Nguyen-Phuoc, T., Li, C., Balaban, S., & Yang, Y. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.06575 · Cited by 106
DeepStereo: Learning to Predict New Views from the World's ImageryPreprint
Flynn, J., Neulander, I., Philbin, J. F., & Snavely, N. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.06825 · Cited by 92
(2007) · ITNOW · doi:10.1093/combul/bwl126 · Cited by 69
Transformation Properties of Learned Visual RepresentationsPreprint
Cohen, T., & Welling, M. (2014) · UvA-DARE (University of Amsterdam) · doi:10.48550/arxiv.1412.7659 · Cited by 61
Single-view to Multi-view: Reconstructing Unseen Views with a Convolutional NetworkPreprint
Tatarchenko, M., Dosovitskiy, A., & Brox, T. (2015) · arXiv (Cornell University) · Cited by 58
View Extrapolation of Human Body from a Single Image
Zhu, H., Su, H., Wang, P., Cao, X., & Yang, R. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00468 · Cited by 50
Cited by
NeRF: representing scenes as neural radiance fields for view synthesis
Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, J. T., Ramamoorthi, R., & Ng, R. (2022) · Communications of the ACM · doi:10.1145/3503250 · Cited by 6,852
3D Gaussian Splatting for Real-Time Radiance Field Rendering
Kerbl, B., Kopanas, G., Leimkuehler, T., & Drettakis, G. (2023) · ACM Transactions on Graphics · doi:10.1145/3592433 · Cited by 6,025
NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis
Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, J. T., Ramamoorthi, R., & Ng, R. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58452-8_24 · Cited by 5,351
Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields
Barron, J. T., Mildenhall, B., Tancik, M., Hedman, P., Martin-Brualla, R., & Srinivasan, P. P. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00580 · Cited by 1,921
NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections
Martin-Brualla, R., Radwan, N., Sajjadi, M. S. M., Barron, J. T., Dosovitskiy, A., & Duckworth, D. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00713 · Cited by 1,374
Metadata
- Authors
- Sitzmann, V., Thies, J., Heide, F., Nießner, M., Wetzstein, G., & Zollhöfer, M.
- Venue
- 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2019-06
- Pages
- 2432-2441
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2903200123
- Cited by
- 558
- References
- 61
- FWCI
- 90.91
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.