The DOI matches its Crossref record; 47 of the 47 references it cites were found in a registry or an index.
NeRV: Neural Reflectance and Visibility Fields for Relighting and View Synthesis
Open accessAbstract
We present a method that takes as input a set of images of a scene illuminated by unconstrained known lighting, and produces as output a 3D representation that can be rendered from novel viewpoints under arbitrary lighting conditions. Our method represents the scene as a continuous volumetric function parameterized as MLPs whose inputs are a 3D location and whose outputs are the following scene properties at that input location: volume density, surface normal, material parameters, distance to the first surface intersection in any direction, and visibility of the external environment in any direction. Together, these allow us to render novel views of the object under arbitrary lighting, including indirect illumination effects. The predicted visibility and surface intersection fields are critical to our model’s ability to simulate direct and indirect illumination during training, because the brute-force techniques used by prior work are intractable for lighting conditions outside of controlled setups with a single light. Our method outperforms alternative approaches for recovering relightable 3D scene representations, and performs well in complex lighting settings that have posed a significant challenge to prior work.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Multiple View Geometry in Computer Vision
Hartley, R., & Zisserman, A. (2004) · doi:10.1017/cbo9780511811685 · Cited by 21,070
Marching cubes: A high resolution 3D surface construction algorithm
Lorensen, W. E., & Cline, H. E. (1987) · Proceedings of the 14th annual conference on Computer graphics and interactive techniques · doi:10.1145/37401.37422 · Cited by 10,104
PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
Charles, R. Q., Su, H., Kaichun, M., & Guibas, L. J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.16 · Cited by 9,237
Marching cubes: A high resolution 3D surface construction algorithm
Lorensen, W. E., & Cline, H. E. (1987) · ACM SIGGRAPH Computer Graphics · doi:10.1145/37402.37422 · Cited by 8,647
NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis
Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, J. T., Ramamoorthi, R., & Ng, R. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58452-8_24 · Cited by 5,351
Self-citationLand, E. H., & McCann, J. J. (1971) · Journal of the Optical Society of America · doi:10.1364/josa.61.000001 · Cited by 4,095
DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation
Park, J. J., Florence, P., Straub, J., Newcombe, R., & Lovegrove, S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00025 · Cited by 3,422
Photo tourism: exploring photo collections in 3D
Snavely, N., Seitz, S. M., & Szeliski, R. (2006) · ACM Transactions on Graphics · doi:10.1145/1141911.1141964 · Cited by 2,866
Occupancy Networks: Learning 3D Reconstruction in Function Space
Mescheder, L., Oechsle, M., Niemeyer, M., Nowozin, S., & Geiger, A. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00459 · Cited by 2,708
(2017) · doi:10.1016/c2013-0-15557-2 · Cited by 1,474
Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains
Tancik, M., Srinivasan, P. P., Mildenhall, B., Fridovich-Keil, S., Raghavan, N., Singhal, U., Ramamoorthi, R., Barron, J. T., & Ng, R. (2020) · arXiv · doi:10.48550/arxiv.2006.10739 · Cited by 1,459
Self-citationNeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections
Martin-Brualla, R., Radwan, N., Sajjadi, M. S. M., Barron, J. T., Dosovitskiy, A., & Duckworth, D. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00713 · Cited by 1,374
Self-citationOptical models for direct volume rendering
Max, N. (1995) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/2945.468400 · Cited by 1,059
Differentiable Volumetric Rendering: Learning Implicit 3D Representations Without 3D Supervision
Niemeyer, M., Mescheder, L., Oechsle, M., & Geiger, A. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00356 · Cited by 834
Sloan, P., Kautz, J., & Snyder, J. (2002) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/566570.566612 · Cited by 796
RECOVERING INTRINSIC SCENE CHARACTERISTICS FROM IMAGES
Barrow, H. G., & Tenenbaum, J. M. (1978) · Cited by 725
Kajiya, J. T., & Von Herzen, B. P. (1984) · ACM SIGGRAPH Computer Graphics · doi:10.1145/964965.808594 · Cited by 706
Shape, Illumination, and Reflectance from Shading
Barron, J. T., & Malik, J. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2014.2377712 · Cited by 656
Self-citationDetermining lightness from an image
Horn, B. K. P. (1974) · Computer Graphics and Image Processing · doi:10.1016/0146-664x(74)90022-7 · Cited by 635
A signal-processing framework for inverse rendering
Ramamoorthi, R., & Hanrahan, P. (2001) · Proceedings of the 28th annual conference on Computer graphics and interactive techniques · doi:10.1145/383259.383271 · Cited by 577
SHAPE FROM SHADING: A METHOD FOR OBTAINING THE SHAPE OF A SMOOTH OPAQUE OBJECT FROM ONE VIEW
Horn, B. K. P. (1970) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 534
Photorealistic Scene Reconstruction by Voxel Coloring
Seitz, S. M., & Dyer, C. R. (1999) · International Journal of Computer Vision · doi:10.1023/a:1008176507526 · Cited by 492
Learning to reconstruct shape and spatially-varying reflectance from a single image
Li, Z., Xu, Z., Ramamoorthi, R., Sunkavalli, K., & Chandraker, M. (2018) · ACM Transactions on Graphics · doi:10.1145/3272127.3275055 · Cited by 342
Learning Shape Templates With Structured Implicit Functions
Genova, K., Cole, F., Vlasic, D., Sarna, A., Freeman, W., & Funkhouser, T. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00725 · Cited by 323
Li, Z., Shafiei, M., Ramamoorthi, R., Sunkavalli, K., & Chandraker, M. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00255 · Cited by 243
InverseRenderNet: Learning Single Image Inverse Rendering
Yu, Y., & Smith, W. A. P. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00327 · Cited by 200
Imperfect shadow maps for efficient computation of indirect illumination
Ritschel, T., Grosch, T., Kim, M. H., Seidel, H., Dachsbacher, C., & Kautz, J. (2008) · ACM Transactions on Graphics · doi:10.1145/1409060.1409082 · Cited by 185
Neural Inverse Rendering of an Indoor Scene From a Single Image
Sengupta, S., Gu, J., Kim, K., Liu, G., Jacobs, D. W., & Kautz, J. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00869 · Cited by 184
Neural Rerendering in the Wild
Meshry, M., Goldman, D. B., Khamis, S., Hoppe, H., Pandey, R., Snavely, N., & Martin-Brualla, R. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00704 · Cited by 183
Neural Reflectance Fields for Appearance AcquisitionPreprint
Bi, S., Xu, Z., Srinivasan, P. P., Mildenhall, B., Sunkavalli, K., Hašan, M., Hold-Geoffroy, Y., Kriegman, D. J., & Ramamoorthi, R. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.03824 · Cited by 181
Self-citationPractical SVBRDF acquisition of 3D objects with unstructured flash photography
Nam, G., Lee, J. H., Gutiérrez, D., & Kim, M. H. (2018) · ACM Transactions on Graphics · doi:10.1145/3272127.3275017 · Cited by 171
Deep Reflectance Volumes: Relightable Reconstructions from Multi-view Photometric Images
Bi, S., Xu, Z., Sunkavalli, K., Hašan, M., Hold-Geoffroy, Y., Kriegman, D. J., & Ramamoorthi, R. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58580-8_18 · Cited by 138
Sample-based Monte Carlo denoising using a kernel-splatting network
Gharbi, M., Li, T., Aittala, M., Lehtinen, J., & Durand, F. (2019) · ACM Transactions on Graphics · doi:10.1145/3306346.3322954 · Cited by 93
Micro-rendering for scalable, parallel final gathering
Ritschel, T., Engelhardt, T., Grosch, T., Seidel, H., Kautz, J., & Dachsbacher, C. (2009) · ACM Transactions on Graphics · doi:10.1145/1618452.1618478 · Cited by 88
Crowdsampling the Plenoptic Function
Li, Z., Xian, W., Davis, A., & Snavely, N. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58452-8_11 · Cited by 88
Single-Shot Neural Relighting and SVBRDF Estimation
Sang, S., & Chandraker, M. K. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58529-7_6 · Cited by 79
Recovering shape and spatially-varying surface reflectance under unknown illumination
Xia, R., Dong, Y., Peers, P., & Tong, X. (2016) · ACM Transactions on Graphics · doi:10.1145/2980179.2980248 · Cited by 72
Advances in geometry and reflectance acquisition (course notes)
Weinmann, M., & Klein, R. (2015) · SIGGRAPH Asia Courses · doi:10.1145/2818143.2818165 · Cited by 62
On Joint Estimation of Pose, Geometry and svBRDF From a Handheld Scanner
Schmitt, C., Donné, S., Riegler, G., Koltun, V., & Geiger, A. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00355 · Cited by 62
Multiview Shape and Reflectance from Natural Illumination
Oxholm, G., & Nishino, K. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.277 · Cited by 58
Dong, Y., Chen, G., Peers, P., Zhang, J.-W., & Tong, X. (2014) · ACM Transactions on Graphics · doi:10.1145/2661229.2661283 · Cited by 55
A Neural Rendering Framework for Free-Viewpoint Relighting
Chen, Z., Chen, A., Zhang, G., Wang, C., Ji, Y., Kutulakos, K. N., & Yu, J. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00564 · Cited by 54
Gao, D., Chen, G., Dong, Y., Peers, P., Xu, K., & Tong, X. (2020) · ACM Transactions on Graphics · doi:10.1145/3414685.3417767 · Cited by 51
Efficient Reflectance and Visibility Approximations for Environment Map Rendering
Green, P. E., Kautz, J., & Durand, F. (2007) · Computer Graphics Forum · doi:10.1111/j.1467-8659.2007.01072.x · Cited by 47
Seeing the World in a Bag of Chips
Park, J. J., Holynski, A., & Seitz, S. M. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00149 · Cited by 42
Interactive illumination with coherent shadow maps
Ritschel, T., Grosch, T., Kautz, J., & Müeller, S. (2007) · UCL Discovery (University College London) · doi:10.5555/2383847.2383858 · Cited by 23
Cited by
Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields
Barron, J. T., Mildenhall, B., Verbin, D., Srinivasan, P. P., & Hedman, P. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00539 · Cited by 1,935
Self-citationMip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields
Barron, J. T., Mildenhall, B., Tancik, M., Hedman, P., Martin-Brualla, R., & Srinivasan, P. P. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00580 · Cited by 1,921
Self-citationEfficient Geometry-aware 3D Generative Adversarial Networks
Chan, E. R., Lin, C. Z., Chan, M. A., Nagano, K., Pan, B., de Mello, S., Gallo, O., Guibas, L., Tremblay, J., Khamis, S., et al. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01565 · Cited by 1,093
Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstruction
Sun, C., Sun, M., & Chen, H.-T. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00538 · Cited by 939
PlenOctrees for Real-time Rendering of Neural Radiance Fields
Yu, A., Li, R., Tancik, M., Li, H., Ng, R., & Kanazawa, A. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00570 · Cited by 854
Self-citation
Metadata
- Authors
- Srinivasan, P. P., Deng, B., Zhang, X., Tancik, M., Mildenhall, B., & Barron, J. T.
- Venue
- 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2021-06
- Pages
- 7491-7500
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W3113005628
- Cited by
- 515
- References
- 47
- FWCI
- 76.95
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.