CoChat

NeRF: representing scenes as neural radiance fields for view synthesis

Open access

The DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.

Abstract

We present a method that achieves state-of-the-art results for synthesizing novel views of complex scenes by optimizing an underlying continuous volumetric scene function using a sparse set of input views. Our algorithm represents a scene using a fully connected (nonconvolutional) deep network, whose input is a single continuous 5D coordinate (spatial location ( x , y , z ) and viewing direction ( θ, ϕ )) and whose output is the volume density and view-dependent emitted radiance at that spatial location. We synthesize views by querying 5D coordinates along camera rays and use classic volume rendering techniques to project the output colors and densities into an image. Because volume rendering is naturally differentiable, the only input required to optimize our representation is a set of images with known camera poses. We describe how to effectively optimize neural radiance fields to render photorealistic novel views of scenes with complicated geometry and appearance, and demonstrate results that outperform prior work on neural rendering and view synthesis.

Overview

Integrity signals

Registry

Crossref · 10.1145/3503250

Cites

13 of 13 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Research works printed but not linked: 12

  1. Chang , A.X. , Fhnkhouser , T. , Guibas , L. , Hanrahan , P. , Huang , Q. , Li , Z. , Savarese , S. , Savva , M. , Song , S. , Su , H. , ShapeNet: An information-rich 3D model repository. arXiv:1512.03012 ( 2015 ). Chang, A.X., Fhnkhouser, T., Guibas, L., Hanrahan, P., Huang, Q., Li, Z., Savarese, S., Savva, M., Song, S., Su, H., et al. ShapeNet: An information-rich 3D model repository. arXiv:1512.03012 (2015). · arXiv
  2. Kingma , D.P. , Ba , J. Adam: A method for stochastic optimization . In ICLR ( 2015 ). Kingma, D.P., Ba, J. Adam: A method for stochastic optimization. In ICLR (2015).
  3. Li , T.-M. , Aittala , M. , Durand , F. , Lehtinen , J. Differentiable monte carlo ray tracing through edge sampling. ACM Trans. Graph. (SIGGRAPH Asia) ( 2018 ). Li, T.-M., Aittala, M., Durand, F., Lehtinen, J. Differentiable monte carlo ray tracing through edge sampling. ACM Trans. Graph. (SIGGRAPH Asia) (2018).
  4. Lombardi , S. , Simon , T. , Saragih , J. , Schwartz , G. , Lehrmann , A. , Sheikh , Y. Neural volumes: Learning dynamic renderable volumes from images. ACM Trans. Graph. (SIGGRAPH) ( 2019 ). Lombardi, S., Simon, T., Saragih, J., Schwartz, G., Lehrmann, A., Sheikh, Y. Neural volumes: Learning dynamic renderable volumes from images. ACM Trans. Graph. (SIGGRAPH) (2019).
  5. Mildenhall , B. , Srinivasan , P.P. , Ortiz-Cayon , R. , Kalantari , N.K. , Ramamoorthi , R. , Ng , R. , Kar , A. Local light field fusion: Practical view synthesis with prescriptive sampling guidelines. ACM Trans. Graph. (SIGGRAPH) ( 2019 ). Mildenhall, B., Srinivasan, P.P., Ortiz-Cayon, R., Kalantari, N.K., Ramamoorthi, R., Ng, R., Kar, A. Local light field fusion: Practical view synthesis with prescriptive sampling guidelines. ACM Trans. Graph. (SIGGRAPH) (2019).
  6. Niemeyer , M. , Mescheder , L. , Oechsle , M. , Geiger , A. Differentiable volumetric rendering: Learning implicit 3D representations without 3D supervision . In CVPR ( 2019 ). Niemeyer, M., Mescheder, L., Oechsle, M., Geiger, A. Differentiable volumetric rendering: Learning implicit 3D representations without 3D supervision. In CVPR (2019).
  7. Porter , T. , Duff , T. Compositing digital images. Comput. Graph. (SIGGRAPH) ( 1984 ). Porter, T., Duff, T. Compositing digital images. Comput. Graph. (SIGGRAPH) (1984).
  8. Rahaman , N. , Baratin , A. , Arpit , D. , Dräxler , F. , Lin , M. , Hamprecht , F.A. , Bengio , Y. , Courville , A.C. On the spectral bias of neural networks . In ICML ( 2018 ). Rahaman, N., Baratin, A., Arpit, D., Dräxler, F., Lin, M., Hamprecht, F.A., Bengio, Y., Courville, A.C. On the spectral bias of neural networks. In ICML (2018).
  9. Seitz , S.M. , Dyer , C.R. Photorealistic scene reconstruction by voxel coloring. Int. J. Comput. Vision ( 1999 ). Seitz, S.M., Dyer, C.R. Photorealistic scene reconstruction by voxel coloring. Int. J. Comput. Vision (1999).
  10. Sitzmann , V. , Zollhoefer , M. , Wetzstein , G. Scene representation networks: Continuous 3D-structure-aware neural scene representations . In NeurIPS ( 2019 ). Sitzmann, V., Zollhoefer, M., Wetzstein, G. Scene representation networks: Continuous 3D-structure-aware neural scene representations. In NeurIPS (2019).
  11. Tancik , M. , Srinivasan , P.P. , Mildenhall , B. , Fridovich-Keil , S. , Raghavan , N. , Singhal , U. , Ramamoorthi , R. , Barron , J.T. , Ng , R. Fourier features let networks learn high frequency functions in low dimensional domains . In NeurIPS ( 2020 ). Tancik, M., Srinivasan, P.P., Mildenhall, B., Fridovich-Keil, S., Raghavan, N., Singhal, U., Ramamoorthi, R., Barron, J.T., Ng, R. Fourier features let networks learn high frequency functions in low dimensional domains. In NeurIPS (2020).
  12. Zhou , T. , Tucker , R. , Flynn , J. , Fyffe , G. , Snavely , N. Stereo magnification: Learning view synthesis using multiplane images. ACM Trans. Graph. (SIGGRAPH) ( 2018 ). Zhou, T., Tucker, R., Flynn, J., Fyffe, G., Snavely, N. Stereo magnification: Learning view synthesis using multiplane images. ACM Trans. Graph. (SIGGRAPH) (2018).

Cited by

See all 6,852

Metadata

Authors
Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, J. T., Ramamoorthi, R., & Ng, R.
Affiliations
  • University of California, Berkeley (US)
  • Google (United States) (US)
  • University of California San Diego (US)
Venue
Communications of the ACM
Published
2021-12-17
Volume
65
Issue
1
Pages
99-106
ISSN
  • 0001-0782 (ISSN-L)
  • 1557-7317
OpenAlex
W4200150166
Cited by
6,852
References
13
Citations by year
2026: 1802 · 2025: 2474 · 2024: 1675 · 2023: 756 · 2022: 134 · 2021: 11
FWCI
365.48
Type
journal article
Language
en
Open access
Locations
Communications of the ACM (publishedVersion, cc-by)
Funders
  • Office of Naval Research (N000141712687, N000141912293, N000142012529)
  • National Science Foundation (1730158)
Topics
  • Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › 3D Shape Modeling and Analysis
  • Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Computer Graphics and Visualization Techniques
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
Keywords
neural radiance fields, view synthesis, novel view synthesis, neural rendering, volume rendering, differentiable rendering, photorealistic rendering, scene representation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.