The DOI matches its Crossref record; 213 of the 213 references it cites were found in a registry or an index.
Advances in Neural Rendering
Abstract
Synthesizing photo‐realistic images and videos is at the heart of computer graphics and has been the focus of decades of research. Traditionally, synthetic images of a scene are generated using rendering algorithms such as rasterization or ray tracing, which take specifically defined representations of geometry and material properties as input. Collectively, these inputs define the actual scene and what is rendered, and are referred to as the scene representation (where a scene consists of one or more objects). Example scene representations are triangle meshes with accompanied textures (e.g., created by an artist), point clouds (e.g., from a depth sensor), volumetric grids (e.g., from a CT scan), or implicit surface functions (e.g., truncated signed distance fields). The reconstruction of such a scene representation from observations using differentiable rendering losses is known as inverse graphics or inverse rendering. Neural rendering is closely related, and combines ideas from classical computer graphics and machine learning to create algorithms for synthesizing images from real‐world observations. Neural rendering is a leap forward towards the goal of synthesizing photo‐realistic image and video content. In recent years, we have seen immense progress in this field through hundreds of publications that show different ways to inject learnable components into the rendering pipeline. This state‐of‐the‐art report on advances in neural rendering focuses on methods that combine classical rendering principles with learned 3D scene representations, often now referred to as neural scene representations. A key advantage of these methods is that they are 3D‐consistent by design, enabling applications such as novel viewpoint synthesis of a captured scene. In addition to methods that handle static scenes, we cover neural scene representations for modeling non‐rigidly deforming objects and scene editing and composition. While most of these approaches are scene‐specific, we also discuss techniques that generalize across object classes and can be used for generative tasks. In addition to reviewing these state‐of‐the‐art methods, we provide an overview of fundamental concepts and definitions used in the current literature. We conclude with a discussion on open challenges and social implications.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Items 201–213 of 213
Semantic Implicit Neural Scene Representations With Semi-Supervised Training
Kohli, A., Sitzmann, V., & Wetzstein, G. (2020) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv50981.2020.00052 · Cited by 46
Self-citationFiG-NeRF: Figure-Ground Neural Radiance Fields for 3D Object Category Modelling
Xie, C., Park, K., Martin-Brualla, R., & Brown, M. (2021) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv53792.2021.00104 · Cited by 46
Self-citationNeRF‐Tex: Neural Reflectance Field Textures
Baatz, H., Granskog, J., Papas, M., et al. (2022) · Computer Graphics Forum · doi:10.1111/cgf.14449 · Cited by 41
High Performance I/O For Large Scale Deep Learning
Aizman, A., Maltby, G., & Breuel, T. M. (2019) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata47090.2019.9005703 · Cited by 40
ADOP: Approximate Differentiable One-Pixel Point RenderingPreprint
Rückert, D., Franke, L., & Stamminger, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.06635 · Cited by 38
Neural Synthesis of Binaural Speech From Mono Audio
Richard, A., Marković, D., Gebru, I. D., Krenn, S., Butler, G. A., la Torre, F. D., & Sheikh, Y. A. (2020) · International Conference on Learning Representations · Cited by 37
Coupling Explicit and Implicit Surface Representations for Generative 3D Modeling
Poursaeed, O., Fisher, M., Aigerman, N., et al. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58607-2_39 · Cited by 36
Geometry-Consistent Neural Shape Representation with Implicit Displacement FieldsPreprint
Wang, Y., Rahmann, L., & Sorkine‐Hornung, O. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.05187 · Cited by 35
Systematically differentiating parametric discontinuities
Bangaru, S. P., Michel, J., Mu, K., Bernstein, G., Li, T., & Ragan‐Kelley, J. (2021) · ACM Transactions on Graphics · doi:10.1145/3450626.3459775 · Cited by 35
Animatable Neural Radiance Fields for Human Body Modeling.
Peng, S., Dong, J., Wang, Q., Zhang, S., Shuai, Q., Bao, H., & Zhou, X. (2021) · Cited by 31
A Shading-Guided Generative Implicit Model for Shape-Accurate 3D-Aware Image SynthesisPreprint
Pan, X., Xu, X., Loy, C. C., Theobalt, C., & Dai, B. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.15678 · Cited by 31
Self-citationEditable free-viewpoint video using a layered neural representation
Zhang, J., Liu, X., Ye, X., Zhao, F., Zhang, Y., Wu, M., Zhang, Y., Xu, L., & Jingyi Yu, A. (2021) · ACM Transactions on Graphics · doi:10.1145/3476576.3476729 · Cited by 22
Pushing the Boundaries of View Extrapolation with Multiplane ImagesPreprint
Srinivasan, P. P., Tucker, R. P., Barron, J. T., Ramamoorthi, R., Ng, R., & Snavely, N. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.00413 · Cited by 9
Self-citation
Cited by
3D Gaussian Splatting for Real-Time Radiance Field Rendering
Kerbl, B., Kopanas, G., Leimkuehler, T., & Drettakis, G. (2023) · ACM Transactions on Graphics · doi:10.1145/3592433 · Cited by 6,025
“Recent neural rendering algorithms [Tewari et al. 2022] vastly reduce such artifacts and avoid the overwhelming cost of storing all input images on the GPU, outperforming these methods on most fronts.”
MethodPixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstruction
Charatan, D., Li, S. L., Tagliasacchi, A., & Sitzmann, V. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01840 · Cited by 420
Self-citationInstruct-NeRF2NeRF: Editing 3D Scenes with Instructions
Haque, A., Tancik, M., Efros, A. A., Holynski, A., & Kanazawa, A. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01808 · Cited by 314
Sajjadi, M. S. M., Meyer, H., Pot, E., Bergmann, U., Greff, K., Radwan, N., Vora, S., Lucic, M., Duckworth, D., Dosovitskiy, A., et al. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00613 · Cited by 143
Metadata
- Authors
- Tewari, A., Thies, J., Mildenhall, B., Srinivasan, P., Tretschk, E., Yifan, W., Lassner, C., Sitzmann, V., Martin‐Brualla, R., Lombardi, S., Simon, T., Theobalt, C., Nießner, M., Barron, J. T., Wetzstein, G., Zollhöfer, M., & Golyanik, V.
- Venue
- Computer Graphics Forum
- Publisher
- Wiley
- Published
- 2022-05
- Volume
- 41
- Issue
- 2
- Pages
- 703-735
- ISSN
- 0167-7055 (ISSN-L)
- 1467-8659
- OpenAlex
- W3214712237
- Cited by
- 339
- References
- 213
- FWCI
- 41.16
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.