The DOI matches its Crossref record; 52 of the 52 references it cites were found in a registry or an index.
D-NeRF: Neural Radiance Fields for Dynamic Scenes
Open accessAbstract
Neural rendering techniques combining machine learning with geometric reasoning have arisen as one of the most promising approaches for synthesizing novel views of a scene from a sparse set of images. Among these, stands out the Neural radiance fields (NeRF) [31], which trains a deep network to map 5D input coordinates (representing spatial location and viewing direction) into a volume density and view-dependent emitted radiance. However, despite achieving an unprecedented level of photorealism on the generated images, NeRF is only applicable to static scenes, where the same spatial location can be queried from different images. In this paper we introduce D-NeRF, a method that extends neural radiance fields to a dynamic domain, allowing to reconstruct and render novel images of objects under rigid and non-rigid motions from a single camera moving around the scene. For this purpose we consider time as an additional input to the system, and split the learning process in two main stages: one that encodes the scene into a canonical space and another that maps this canonical representation into the deformed scene at a particular time. Both mappings are simultaneously learned using fully-connected networks. Once the networks are trained, D-NeRF can render novel images, controlling both the camera view and the time variable, and thus, the object movement. We demonstrate the effectiveness of our approach on scenes with objects under rigid, articulated and non-rigid motions. Code, model weights and the dynamic scenes dataset will be available at [1].
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892
Multiple View Geometry in Computer Vision
Hartley, R., & Zisserman, A. (2004) · doi:10.1017/cbo9780511811685 · Cited by 21,070
The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
Zhang, R., Isola, P., Efros, A. A., Shechtman, E., & Wang, O. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00068 · Cited by 14,623
Marching cubes: A high resolution 3D surface construction algorithm
Lorensen, W. E., & Cline, H. E. (1987) · Proceedings of the 14th annual conference on Computer graphics and interactive techniques · doi:10.1145/37401.37422 · Cited by 10,104
Marching cubes: A high resolution 3D surface construction algorithm
Lorensen, W. E., & Cline, H. E. (1987) · ACM SIGGRAPH Computer Graphics · doi:10.1145/37402.37422 · Cited by 8,647
Bundle Adjustment — A Modern Synthesis
Triggs, B., McLauchlan, P. F., Hartley, R. I., & Fitzgibbon, A. W. (2000) · Lecture Notes in Computer Science · doi:10.1007/3-540-44480-7_21 · Cited by 3,988
Levoy, M., & Hanrahan, P. (1996) · Proceedings of the 23rd annual conference on Computer graphics and interactive techniques · doi:10.1145/237170.237199 · Cited by 3,755
DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation
Park, J. J., Florence, P., Straub, J., Newcombe, R., & Lovegrove, S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00025 · Cited by 3,422
Shape and motion from image streams under orthography: a factorization method
Tomasi, C., & Kanade, T. (1992) · International Journal of Computer Vision · doi:10.1007/bf00129684 · Cited by 2,878
Photo tourism: exploring photo collections in 3D
Snavely, N., Seitz, S. M., & Szeliski, R. (2006) · ACM Transactions on Graphics · doi:10.1145/1141911.1141964 · Cited by 2,866
Occupancy Networks: Learning 3D Reconstruction in Function Space
Mescheder, L., Oechsle, M., Niemeyer, M., Nowozin, S., & Geiger, A. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00459 · Cited by 2,708
A Point Set Generation Network for 3D Object Reconstruction from a Single Image
Fan, H., Su, H., & Guibas, L. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.264 · Cited by 2,402
On the Spectral Bias of Neural NetworksPreprint
Rahaman, N., Baratin, A., Arpit, D., Draxler, F., Lin, M., Hamprecht, F. A., Bengio, Y., & Courville, A. C. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.08734 · Cited by 1,915
Learning Implicit Fields for Generative Shape Modeling
Chen, Z., & Zhang, H. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00609 · Cited by 1,443
High-quality video view interpolation using a layered representation
Zitnick, C. L., Kang, S. B., Uyttendaele, M., Winder, S., & Szeliski, R. (2004) · ACM Transactions on Graphics · doi:10.1145/1015706.1015766 · Cited by 1,385
NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections
Martin-Brualla, R., Radwan, N., Sajjadi, M. S. M., Barron, J. T., Dosovitskiy, A., & Duckworth, D. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00713 · Cited by 1,374
Local light field fusion: practical view synthesis with prescriptive sampling guidelines
Mildenhall, B., Srinivasan, P. P., Ortiz-Cayon, R., Kalantari, N. K., Ramamoorthi, R., Ng, R., & Kar, A. (2019) · ACM Transactions on Graphics · doi:10.1145/3306346.3322980 · Cited by 1,175
Multiview Neural Surface Reconstruction by Disentangling Geometry and AppearancePreprint
Yariv, L., Kasten, Y., Moran, D., Galun, M., Atzmon, M., Basri, R., & Lipman, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.09852 · Cited by 990
O-CNN: octree-based convolutional neural networks for 3D shape analysis
Wang, P.-S., Liu, Y., Guo, Y.-X., Sun, C.-Y., & Tong, X. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073608 · Cited by 884
Unstructured lumigraph rendering
Buehler, C., Bosse, M., McMillan, L., Gortler, S., & Cohen, M. (2001) · Proceedings of the 28th annual conference on Computer graphics and interactive techniques · doi:10.1145/383259.383309 · Cited by 881
Differentiable Volumetric Rendering: Learning Implicit 3D Representations Without 3D Supervision
Niemeyer, M., Mescheder, L., Oechsle, M., & Geiger, A. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00356 · Cited by 834
Stereo magnification: learning view synthesis using multiplane images
Zhou, T., Tucker, R., Flynn, J., Fyffe, G., & Snavely, N. (2018) · ACM Transactions on Graphics · doi:10.1145/3197517.3201323 · Cited by 776
Neural Sparse Voxel FieldsPreprint
Liu, L., Gu, J., Lin, K. Z., Chua, T., & Theobalt, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.11571 · Cited by 749
Octree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputs
Tatarchenko, M., Dosovitskiy, A., & Brox, T. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.230 · Cited by 742
Neural volumes: learning dynamic renderable volumes from images
Lombardi, S., Simon, T., Saragih, J., Schwartz, G., Lehrmann, A., & Sheikh, Y. (2019) · ACM Transactions on Graphics · doi:10.1145/3306346.3323020 · Cited by 693
Learning a Predictable and Generative Vector Representation for Objects
Girdhar, R., Fouhey, D. F., Rodriguez, M., & Gupta, A. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46466-4_29 · Cited by 646
Deep Stereo: Learning to Predict New Views from the World's Imagery
Flynn, J., Neulander, I., Philbin, J., & Snavely, N. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.595 · Cited by 574
NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis
Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, J. T., Ramamoorthi, R., & Ng, R. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.08934 · Cited by 524
Implicit Functions in Feature Space for 3D Shape Reconstruction and Completion
Chibane, J., Alldieck, T., & Pons-Moll, G. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00700 · Cited by 509
Self-citationDeepView: View Synthesis With Learned Gradient Descent
Flynn, J., Broxton, M., Debevec, P., DuVall, M., Fyffe, G., Overbeck, R., Snavely, N., & Tucker, R. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00247 · Cited by 424
Local Deep Implicit Functions for 3D Shape
Genova, K., Cole, F., Sud, A., Sarna, A., & Funkhouser, T. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00491 · Cited by 370
Shen, L., Zhao, W., & Xing, L. (2019) · Nature Biomedical Engineering · doi:10.1038/s41551-019-0466-4 · Cited by 313
Vedula, S., Rander, P., Collins, R., & Kanade, T. (2005) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2005.63 · Cited by 281
Perspective Transformer Nets: Learning Single-View 3D Object Reconstruction without 3D Supervision
Yan, X., Yang, J., Yumer, E., Guo, Y., & Lee, H. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.00814 · Cited by 267
Occupancy Flow: 4D Reconstruction by Learning Particle Dynamics
Niemeyer, M., Mescheder, L., Oechsle, M., & Geiger, A. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00548 · Cited by 246
Novel View Synthesis of Dynamic Scenes With Globally Coherent Depths From a Monocular Camera
Yoon, J. S., Kim, K., Gallo, O., Park, H. S., & Kautz, J. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00538 · Cited by 231
Learning a Multi-View Stereo Machine
Kar, A., Häne, C., & Malik, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.05375 · Cited by 209
Learning to Infer Implicit Surfaces without 3D SupervisionPreprint
Liu, S., Saito, S., Chen, W., & Li, H. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.00767 · Cited by 186
NASA Neural Articulated Shape Approximation
Deng, B., Lewis, J. P., Jeruzalski, T., Pons-Moll, G., Hinton, G., Norouzi, M., & Tagliasacchi, A. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58571-6_36 · Cited by 185
Self-citationChoi, I., Gallo, O., Troccoli, A., Kim, M. H., & Kautz, J. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00787 · Cited by 181
X-Fields: implicit neural view-, light- and time-image interpolation
Bemana, M., Myszkowski, K., Seidel, H., & Ritschel, T. (2020) · ACM Transactions on Graphics · doi:10.1145/3414685.3417827 · Cited by 110
DeepStereo: Learning to Predict New Views from the World's ImageryPreprint
Flynn, J., Neulander, I., Philbin, J. F., & Snavely, N. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.06825 · Cited by 103
Bartoli, A., Gérard, Y., Chadebecq, F., Collins, T., & Pizarro, D. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2392759 · Cited by 102
C-Flow: Conditional Generative Flow Models for Images and 3D Point Clouds
Pumarola, A., Popov, S., Moreno-Noguer, F., & Ferrari, V. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00797 · Cited by 95
Self-citation4D Visualization of Dynamic Events From Unconstrained Multi-View Videos
Bansal, A., Vo, M., Sheikh, Y. A., Ramanan, D., & Narasimhan, S. G. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00541 · Cited by 90
Learning Implicit Surface Light Fields
Oechsle, M., Niemeyer, M., Reiser, C., Mescheder, L., Strauss, T., & Geiger, A. (2020) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv50981.2020.00055 · Cited by 49
Neural Voxel Renderer: Learning an Accurate and Controllable Rendering Tool
Rematas, K., & Ferrari, V. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00546 · Cited by 40
3D Reconstruction of Dynamic Scenes with Multiple Handheld Cameras
Jiang, H., Liu, H., Tan, P., Zhang, G., & Bao, H. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33709-3_43 · Cited by 27
Plane-based multi-view inpainting for image-based rendering in large scenes
Philip, J., & George Drettakis, B. (2018) · ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games (I3D) · doi:10.1145/3190834.3190846 · Cited by 22
Stochastic Exploration of Ambiguities for Nonrigid Shape Recovery
Moreno-Noguer, F., & Fua, P. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.102 · Cited by 21
Self-citation
Cited by
Efficient Geometry-aware 3D Generative Adversarial Networks
Chan, E. R., Lin, C. Z., Chan, M. A., Nagano, K., Pan, B., de Mello, S., Gallo, O., Guibas, L., Tremblay, J., Khamis, S., et al. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01565 · Cited by 1,093
Nerfies: Deformable Neural Radiance Fields
Park, K., Sinha, U., Barron, J. T., Bouaziz, S., Goldman, D. B., Seitz, S. M., & Martin-Brualla, R. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00581 · Cited by 1,059
Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstruction
Sun, C., Sun, M., & Chen, H.-T. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00538 · Cited by 939
Depth-supervised NeRF: Fewer Views and Faster Training for Free
Deng, K., Liu, A., Zhu, J.-Y., & Ramanan, D. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01254 · Cited by 841
4D Gaussian Splatting for Real-Time Dynamic Scene Rendering
Wu, G., Yi, T., Fang, J., Xie, L., Zhang, X., Wei, W., Liu, W., Tian, Q., & Wang, X. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01920 · Cited by 806
Metadata
- Authors
- Pumarola, A., Corona, E., Pons-Moll, G., & Moreno-Noguer, F.
- Venue
- 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2021-06
- Pages
- 10313-10322
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W3109476562
- Cited by
- 1,355
- References
- 52
- FWCI
- 156.77
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.