The DOI matches its Crossref record; 57 of the 57 references it cites were found in a registry or an index.
RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputs
Open accessAbstract
Neural Radiance Fields (NeRF) have emerged as a powerful representation for the task of novel view synthesis due to their simplicity and state-of-the-art performance. Though NeRF can produce photorealistic renderings of unseen viewpoints when many input views are available, its performance drops significantly when this number is reduced. We observe that the majority of artifacts in sparse input scenarios are caused by errors in the estimated scene geometry, and by divergent behavior at the start of training. We address this by regularizing the geometry and appearance of patches rendered from unobserved viewpoints, and annealing the ray sampling space during training. We additionally use a normalizing flow model to regularize the color of unobserved viewpoints. Our model outperforms not only other methods that optimize over a single scene, but in many cases also conditional models that are extensively pre-trained on large multi-view datasets.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293
The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
Zhang, R., Isola, P., Efros, A. A., Shechtman, E., & Wang, O. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00068 · Cited by 14,623
NeRF: representing scenes as neural radiance fields for view synthesis
Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, J. T., Ramamoorthi, R., & Ng, R. (2022) · Communications of the ACM · doi:10.1145/3503250 · Cited by 6,852
Self-citationNeRF: Representing Scenes as Neural Radiance Fields for View Synthesis
Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, J. T., Ramamoorthi, R., & Ng, R. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58452-8_24 · Cited by 5,351
Self-citationIn-Datacenter Performance Analysis of a Tensor Processing Unit
Jouppi, N. P., Young, C., Patil, N., Patterson, D., Agrawal, G., Bajwa, R., Bates, S., Bhatia, S., Boden, N., Borchers, A., et al. (2017) · Proceedings of the 44th Annual International Symposium on Computer Architecture · doi:10.1145/3079856.3080246 · Cited by 4,537
DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation
Park, J. J., Florence, P., Straub, J., Newcombe, R., & Lovegrove, S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00025 · Cited by 3,422
Occupancy Networks: Learning 3D Reconstruction in Function Space
Mescheder, L., Oechsle, M., Niemeyer, M., Nowozin, S., & Geiger, A. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00459 · Cited by 2,708
Self-citationRevisiting Unreasonable Effectiveness of Data in Deep Learning Era
Sun, C., Shrivastava, A., Singh, S., & Gupta, A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.97 · Cited by 2,484
Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields
Barron, J. T., Mildenhall, B., Tancik, M., Hedman, P., Martin-Brualla, R., & Srinivasan, P. P. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00580 · Cited by 1,921
Self-citationpixelNeRF: Neural Radiance Fields from One or Few Images
Yu, A., Ye, V., Tancik, M., & Kanazawa, A. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00455 · Cited by 1,560
Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains
Tancik, M., Srinivasan, P. P., Mildenhall, B., Fridovich-Keil, S., Raghavan, N., Singhal, U., Ramamoorthi, R., Barron, J. T., & Ng, R. (2020) · arXiv · doi:10.48550/arxiv.2006.10739 · Cited by 1,459
Self-citationLearning Implicit Fields for Generative Shape Modeling
Chen, Z., & Zhang, H. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00609 · Cited by 1,443
NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections
Martin-Brualla, R., Radwan, N., Sajjadi, M. S. M., Barron, J. T., Dosovitskiy, A., & Duckworth, D. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00713 · Cited by 1,374
Self-citationPIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization
Saito, S., Huang, Z., Natsume, R., Morishima, S., Li, H., & Kanazawa, A. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00239 · Cited by 1,228
Local light field fusion: practical view synthesis with prescriptive sampling guidelines
Mildenhall, B., Srinivasan, P. P., Ortiz-Cayon, R., Kalantari, N. K., Ramamoorthi, R., Ng, R., & Kar, A. (2019) · ACM Transactions on Graphics · doi:10.1145/3306346.3322980 · Cited by 1,175
Self-citationDepth-supervised NeRF: Fewer Views and Faster Training for Free
Deng, K., Liu, A., Zhu, J.-Y., & Ramanan, D. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01254 · Cited by 841
Differentiable Volumetric Rendering: Learning Implicit 3D Representations Without 3D Supervision
Niemeyer, M., Mescheder, L., Oechsle, M., & Geiger, A. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00356 · Cited by 834
Self-citationConvolutional Occupancy Networks
Peng, S., Niemeyer, M., Mescheder, L., Pollefeys, M., & Geiger, A. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58580-8_31 · Cited by 801
Self-citationMVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereo
Chen, A., Xu, Z., Zhao, F., Zhang, X., Xiang, F., Yu, J., & Su, H. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01386 · Cited by 797
GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields
Niemeyer, M., & Geiger, A. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01129 · Cited by 731
Self-citationLarge Scale Multi-view Stereopsis Evaluation
Jensen, R., Dahl, A., Vogiatzis, G., Tola, E., & Aanaes, H. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.59 · Cited by 729
IBRNet: Learning Multi-View Image-Based Rendering
Wang, Q., Wang, Z., Genova, K., Srinivasan, P., Zhou, H., Barron, J. T., Martin-Brualla, R., Snavely, N., & Funkhouser, T. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00466 · Cited by 725
Self-citationpi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis
Chan, E. R., Monteiro, M., Kellnhofer, P., Wu, J., & Wetzstein, G. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00574 · Cited by 689
UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstruction
Oechsle, M., Peng, S., & Geiger, A. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00554 · Cited by 622
Self-citationVolume Rendering of Neural Implicit SurfacesPreprint
Yariv, L., Gu, J., Kasten, Y., & Lipman, Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.12052 · Cited by 551
Putting NeRF on a Diet: Semantically Consistent Few-Shot View Synthesis
Jain, A., Tancik, M., & Abbeel, P. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00583 · Cited by 463
Baking Neural Radiance Fields for Real-Time View Synthesis
Hedman, P., Srinivasan, P. P., Mildenhall, B., Barron, J. T., & Debevec, P. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00582 · Cited by 451
Self-citation(2021) · International Conference on Learning Representations · Cited by 449
Occupancy Flow: 4D Reconstruction by Learning Particle Dynamics
Niemeyer, M., Mescheder, L., Oechsle, M., & Geiger, A. (2019) · IEEE International Conference on Computer Vision · doi:10.1109/iccv.2019.00548 · Cited by 326
Self-citationLearning Shape Templates With Structured Implicit Functions
Genova, K., Cole, F., Vlasic, D., Sarna, A., Freeman, W., & Funkhouser, T. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00725 · Cited by 323
GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis
Schwarz, K., Liao, Y., Niemeyer, M., & Geiger, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.02442 · Cited by 314
Self-citationTexture Fields: Learning Texture Representations in Function Space
Oechsle, M., Mescheder, L., Niemeyer, M., Strauss, T., & Geiger, A. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00463 · Cited by 252
Self-citationCodeNeRF: Disentangled Neural Radiance Fields for Object Categories
Jang, W., & Agapito, L. (2021) · IEEE International Conference on Computer Vision · doi:10.1109/iccv48922.2021.01271 · Cited by 252
Implicit Surface Representations As Layers in Neural Networks
Michalkiewicz, M., Pontes, J. K., Jack, D., Baktashmotlagh, M., & Eriksson, A. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00484 · Cited by 227
Stereo Radiance Fields (SRF): Learning View Synthesis for Sparse Views of Novel Scenes
Chibane, J., Bansal, A., Lazova, V., & Pons‐Moll, G. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00782 · Cited by 226
Learned Initializations for Optimizing Coordinate-Based Neural Representations
Tancik, M., Mildenhall, B., Wang, T., Schmidt, D., Srinivasan, P. P., Barron, J. T., & Ng, R. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00287 · Cited by 218
Self-citationSDFDiff: Differentiable Rendering of Signed Distance Fields for 3D Shape Optimization
Jiang, Y., Ji, D., Han, Z., & Zwicker, M. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00133 · Cited by 208
GRF: Learning a General Radiance Field for 3D Representation and Rendering
Trevithick, A., & Yang, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01490 · Cited by 202
Neural Rays for Occlusion-aware Image-based Rendering
Liu, Y., Peng, S., Liu, L., Wang, Q., Wang, P., Theobalt, C., Zhou, X., & Wang, W. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00767 · Cited by 185
StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesis
Gu, J., Liu, L., Wang, P., & Theobalt, C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.08985 · Cited by 178
GNeRF: GAN-based Neural Radiance Field without Posed Camera
Meng, Q., Chen, A., Luo, H., Wu, M., Su, H., Xu, L., He, X., & Yu, J. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00629 · Cited by 170
MINE: Towards Continuous Depth MPI with NeRF for Novel View Synthesis
Li, J., Feng, Z., She, Q., Ding, H., Wang, C., & Lee, G. H. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01235 · Cited by 158
CIPS-3D: A 3D-Aware Generator of GANs Based on Conditionally-Independent Pixel SynthesisPreprint
Zhou, P., Xie, L., Ni, B., & Tian, Q. C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.09788 · Cited by 148
Huang, J., Lee, A. B., & Mumford, D. (2002) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2000.855836 · Cited by 147
Kellnhofer, P., Jebe, L., Jones, A., Spicer, R. P., Pulli, K., & Wetzstein, G. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00427 · Cited by 133
Learning to Infer Implicit Surfaces without 3D SupervisionPreprint
Liu, S., Saito, S., Chen, W., & Li, H. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.00767 · Cited by 131
Unconstrained Scene Generation with Locally Conditioned Radiance Fields
DeVries, T., Bautista, M. Á., Srivastava, N., Taylor, G. W., & Susskind, J. M. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01404 · Cited by 119
Local Light Field Fusion: Practical View Synthesis with Prescriptive Sampling GuidelinesPreprint
Mildenhall, B., Srinivasan, P. P., Ortiz-Cayon, R., Kalantari, N. K., Ramamoorthi, R., Ng, R., & Kar, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.00889 · Cited by 108
Self-citationGANcraft: Unsupervised 3D Neural Rendering of Minecraft Worlds
Hao, Z., Mallya, A., Belongie, S., & Liu, M.-Y. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01381 · Cited by 99
ShaRF: Shape-conditioned Radiance Fields from a Single ViewPreprint
Rematas, K., Martin-Brualla, R., & Ferrari, V. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.08860 · Cited by 90
Portrait Neural Radiance Fields from a Single ImagePreprint
Gao, C., Shih, Y., Lai, W., Liang, C., & Huang, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.05903 · Cited by 83
Controlling Neural Level SetsPreprint
Atzmon, M., Haim, N., Yariv, L., Israelov, O., Maron, H., & Lipman, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.11911 · Cited by 74
Stochastic Neural Radiance Fields: Quantifying Uncertainty in Implicit 3D Representations
Shen, J., Ruiz, A., Agudo, A., et al. (2021) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv53792.2021.00105 · Cited by 60
CAMPARI: Camera-Aware Decomposed Generative Neural Radiance Fields
Niemeyer, M., & Geiger, A. (2021) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv53792.2021.00103 · Cited by 54
Self-citation
Metadata
- Authors
- Niemeyer, M., Barron, J. T., Mildenhall, B., Sajjadi, M. S. M., Geiger, A., & Radwan, N.
- Venue
- 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2022-06
- Pages
- 5470-5480
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W3216476011
- Cited by
- 609
- References
- 57
- FWCI
- 98.94
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
