CoChat
Sign Up
PreprintCited by 1183D Shape Modeling and AnalysisComputer Graphics and Visualization TechniquesAdvanced Vision and Imaging

Light Field Networks: Neural Scene Representations with Single-Evaluation Rendering

Open access
Sitzmann, Vincent, Rezchikov, Semon, Freeman, William T., Tenenbaum, Joshua B., Durand, Fredo·2021·arXiv
DOI 10.48550/arxiv.2106.02634arXiv 2106.02634
Open this paper in CoChat — free accountOpen PDF

Abstract

Inferring representations of 3D scenes from 2D observations is a fundamental problem of computer graphics, computer vision, and artificial intelligence. Emerging 3D-structured neural scene representations are a promising approach to 3D scene understanding. In this work, we propose a novel neural scene representation, Light Field Networks or LFNs, which represent both geometry and appearance of the underlying 3D scene in a 360-degree, four-dimensional light field parameterized via a neural implicit representation. Rendering a ray from an LFN requires only a single network evaluation, as opposed to hundreds of evaluations per ray for ray-marching or volumetric based renderers in 3D-structured neural scene representations. In the setting of simple scenes, we leverage meta-learning to learn a prior over LFNs that enables multi-view consistent light field reconstruction from as little as a single image observation. This results in dramatic reductions in time and memory complexity, and enables real-time rendering. The cost of storing a 360-degree light field via an LFN is two orders of magnitude lower than conventional methods such as the Lumigraph. Utilizing the analytical differentiability of neural implicit representations and a novel parameterization of light space, we further demonstrate the extraction of sparse depth maps from LFNs.

Integrity signals

Registry

DataCite · 10.48550/arxiv.2106.02634

Cites

5 of 5 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Target cameras are represented as Plücker raymaps, where pixels are mapped to 6D ray representations and subsequently patchified.

    “Target cameras are represented as Plücker raymaps (Sitzmann et al., 2021), where pixels are mapped to 6D ray representations and subsequently patchified.”

    From World Simulation with Video Foundation Models for Physical AI (2025) · cites this paper as Sitzmann et al., 2021

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Related

Authors
Sitzmann, V., Rezchikov, S., Freeman, W. T., Tenenbaum, J. B., & Durand, F.
Affiliations
  • MIT Computer Science and Artificial Intelligence Laboratory (US)
  • Columbia University (US)
  • Massachusetts Institute of Technology (US)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2021
Volume
34
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2106.02634
arXiv
2106.02634
OpenAlex
W3170017212
Cited by
117
References
5
Citations by year
2026: 1 · 2025: 15 · 2024: 38 · 2023: 46 · 2022: 16 · 2021: 1
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
  • neural information processing systems
Funders
  • National Science Foundation (2019786)
  • Office of Naval Research
Topics
  • Physical Sciences › Engineering › Computational Mechanics › 3D Shape Modeling and Analysis
  • Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Computer Graphics and Visualization Techniques
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
Keywords
neural scene representation, light field, implicit neural representations, meta-learning, light field reconstruction, multi-view consistency, real-time rendering, 3D scene understanding

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review