The DOI matches its Crossref record; 50 of the 50 references it cites were found in a registry or an index.
NeRF in the Dark: High Dynamic Range View Synthesis from Noisy Raw Images
Open accessAbstract
Neural Radiance Fields (NeRF) is a technique for high quality novel view synthesis from a collection of posed input images. Like most view synthesis methods, NeRF uses tonemapped low dynamic range (LDR) as input; these images have been processed by a lossy camera pipeline that smooths detail, clips highlights, and distorts the simple noise distribution of raw sensor data. We modify NeRF to instead train directly on linear raw images, preserving the scene's full dynamic range. By rendering raw output images from the resulting NeRF, we can perform novel high dynamic range (HDR) view synthesis tasks. In addition to changing the camera viewpoint, we can manipulate focus, exposure, and tonemapping after the fact. Although a single raw image appears significantly more noisy than a postprocessed one, we show that NeRF is highly robust to the zeromean distribution of raw noise. When optimized over many noisy raw inputs (25–200), NeRF produces a scene representation so accurate that its rendered novel views outperform dedicated single and multi-image deep raw denoisers run on the same wide baseline input images. As a result, our method, which we call RawNeRF, can reconstruct scenes from extremely noisy images captured in near-darkness.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Image denoising by sparse 3-D transform-domain collaborative filtering
Dabov, K., Foi, A., Katkovnik, V., & Egiazarian, K. (2007) · IEEE Transactions on Image Processing · doi:10.1109/tip.2007.901238 · Cited by 9,283
Structure-from-Motion Revisited
Schonberger, J. L., & Frahm, J.-M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.445 · Cited by 6,433
NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis
Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, J. T., Ramamoorthi, R., & Ng, R. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58452-8_24 · Cited by 5,351
Self-citationLevoy, M., & Hanrahan, P. (1996) · Proceedings of the 23rd annual conference on Computer graphics and interactive techniques · doi:10.1145/237170.237199 · Cited by 3,755
Gortler, S. J., Grzeszczuk, R., Szeliski, R., & Cohen, M. F. (1996) · Proceedings of the 23rd annual conference on Computer graphics and interactive techniques · doi:10.1145/237170.237200 · Cited by 2,458
Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields
Barron, J. T., Mildenhall, B., Tancik, M., Hedman, P., Martin-Brualla, R., & Srinivasan, P. P. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00580 · Cited by 1,921
Self-citationRecovering high dynamic range radiance maps from photographs
Debevec, P. E., & Malik, J. (1997) · Proceedings of the 24th annual conference on Computer graphics and interactive techniques - SIGGRAPH '97 · doi:10.1145/258734.258884 · Cited by 1,775
Chen, C., Chen, Q., Xu, J., & Koltun, V. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00347 · Cited by 1,376
NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections
Martin-Brualla, R., Radwan, N., Sajjadi, M. S. M., Barron, J. T., Dosovitskiy, A., & Duckworth, D. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00713 · Cited by 1,374
Self-citationNoise2Void - Learning Denoising From Single Noisy Images
Krull, A., Buchholz, T.-O., & Jug, F. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00223 · Cited by 1,366
Noise2Noise: Learning Image Restoration without Clean Data
Lehtinen, J., Munkberg, J., Hasselgren, J., Laine, S., Karras, T., Aittala, M., & Aila, T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.04189 · Cited by 944
Practical Poissonian-Gaussian Noise Modeling and Fitting for Single-Image Raw-Data
Foi, A., Trimeche, M., Katkovnik, V., & Egiazarian, K. (2008) · IEEE Transactions on Image Processing · doi:10.1109/tip.2008.2001399 · Cited by 928
Unstructured lumigraph rendering
Buehler, C., Bosse, M., McMillan, L., Gortler, S., & Cohen, M. (2001) · Proceedings of the 28th annual conference on Computer graphics and interactive techniques · doi:10.1145/383259.383309 · Cited by 881
Depth-supervised NeRF: Fewer Views and Faster Training for Free
Deng, K., Liu, A., Zhu, J.-Y., & Ramanan, D. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01254 · Cited by 841
Deep bilateral learning for real-time image enhancement
Gharbi, M., Chen, J., Barron, J. T., Hasinoff, S. W., & Durand, F. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073592 · Cited by 800
Self-citationIBRNet: Learning Multi-View Image-Based Rendering
Wang, Q., Wang, Z., Genova, K., Srinivasan, P., Zhou, H., Barron, J. T., Martin-Brualla, R., Snavely, N., & Funkhouser, T. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00466 · Cited by 725
Self-citationNeural volumes: learning dynamic renderable volumes from images
Lombardi, S., Simon, T., Saragih, J., Schwartz, G., Lehrmann, A., & Sheikh, Y. (2019) · ACM Transactions on Graphics · doi:10.1145/3306346.3323020 · Cited by 693
Benchmarking Denoising Algorithms with Real Photographs
Plotz, T., & Roth, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.294 · Cited by 655
BARF: Bundle-Adjusting Neural Radiance Fields
Lin, C.-H., Ma, W.-C., Torralba, A., & Lucey, S. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00569 · Cited by 628
Deep blending for free-viewpoint image-based rendering
Hedman, P., Philip, J., Price, T., Frahm, J.-M., Drettakis, G., & Brostow, G. (2018) · ACM Transactions on Graphics · doi:10.1145/3272127.3275084 · Cited by 617
Self-citationHDR image reconstruction from a single exposure using deep CNNs
Eilertsen, G., Kronander, J., Denes, G., Mantiuk, R. K., & Unger, J. (2017) · ACM Transactions on Graphics · doi:10.1145/3130800.3130816 · Cited by 598
Deep high dynamic range imaging of dynamic scenes
Kalantari, N. K., & Ramamoorthi, R. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073609 · Cited by 596
Deep Stereo: Learning to Predict New Views from the World's Imagery
Flynn, J., Neulander, I., Philbin, J., & Snavely, N. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.595 · Cited by 574
Learning photographic global tonal adjustment with a database of input/output image pairs
Bychkovsky, V., Paris, S., Chan, E., & Durand, F. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995413 · Cited by 536
Burst photography for high dynamic range and low-light imaging on mobile cameras
Hasinoff, S. W., Sharlet, D., Geiss, R., Adams, A., Barron, J. T., Kainz, F., Chen, J., & Levoy, M. (2016) · ACM Transactions on Graphics · doi:10.1145/2980179.2980254 · Cited by 483
Self-citationUnprocessing Images for Learned Raw Denoising
Brooks, T., Mildenhall, B., Xue, T., Chen, J., Sharlet, D., & Barron, J. T. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01129 · Cited by 473
Self-citationImage demosaicing: a systematic survey
Li, X., Gunturk, B., & Zhang, L. (2008) · SPIE Proceedings · doi:10.1117/12.766768 · Cited by 419
Burst Denoising with Kernel Prediction Networks
Mildenhall, B., Barron, J. T., Chen, J., Sharlet, D., Ng, R., & Carroll, R. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00265 · Cited by 404
Self-citationIn-Place Scene Labelling and Understanding with Implicit Scene Representation
Zhi, S., Laidlow, T., Leutenegger, S., & Davison, A. J. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01554 · Cited by 396
Zhang, Y. (2017) · De Gruyter eBooks · doi:10.1515/9783110524116 · Cited by 329
Chen, C., Chen, Q., Do, M., & Koltun, V. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00328 · Cited by 282
Exposure: A White-Box Photo Post-Processing Framework
Hu, Y., He, H., Xu, C., Wang, B., & Lin, S. (2018) · ACM Transactions on Graphics · doi:10.1145/3181974 · Cited by 279
NerfingMVS: Guided Optimization of Neural Radiance Fields for Indoor Multi-view Stereo
Wei, Y., Liu, S., Rao, Y., Zhao, W., Lu, J., & Zhou, J. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00556 · Cited by 250
NeRF--: Neural Radiance Fields Without Known Camera Parameters
Wang, Z., Wu, S., Xie, W., Chen, M., & Prisacariu, V. A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.07064 · Cited by 245
Self-Calibrating Neural Radiance Fields
Jeong, Y., Ahn, S., Choy, C., Anandkumar, A., Cho, M., & Park, J. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00579 · Cited by 183
Riegler, G., & Koltun, V. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01204 · Cited by 177
Synthetic depth-of-field with a single-camera mobile phone
Wadhwa, N., Garg, R., Jacobs, D. E., Feldman, B. E., Kanazawa, N., Carroll, R., Movshovitz-Attias, Y., Barron, J. T., Pritch, Y., & Levoy, M. (2018) · ACM Transactions on Graphics · doi:10.1145/3197517.3201329 · Cited by 173
Self-citationA Software Platform for Manipulating the Camera Imaging Pipeline
Karaimer, H. C., & Brown, M. S. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46448-0_26 · Cited by 166
Point‐Based Neural Rendering with Per‐View Optimization
Kopanas, G., Philip, J., Leimkühler, T., & Drettakis, G. (2021) · Computer Graphics Forum · doi:10.1111/cgf.14339 · Cited by 158
Stereo Magnification: Learning View Synthesis using Multiplane Images
Zhou, T., Tucker, R., Flynn, J., Fyffe, G., & Snavely, N. (2018) · arXiv · doi:10.48550/arxiv.1805.09817 · Cited by 150
Fast bilateral-space stereo for synthetic defocus
Barron, J. T., Adams, A., Shih, Y., & Hernández, C. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7299076 · Cited by 146
Self-citationSupervised Raw Video Denoising With a Benchmark Dataset on Dynamic Scenes
Yue, H., Cao, C., Liao, L., Chu, R., & Yang, J. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00237 · Cited by 126
Mobile Computational Photography: A Tour
Delbracio, M., Kelly, D., Brown, M. S., & Milanfar, P. (2021) · Annual Review of Vision Science · doi:10.1146/annurev-vision-093019-115521 · Cited by 115
Model-Blind Video Denoising via Frame-To-Frame Training
Ehret, T., Davy, A., Morel, J., Facciolo, G., & Arias, P. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01163 · Cited by 93
Unsupervised Deep Video Denoising
Sheth, D. Y., Mohan, S., Vincent, J., Manzorro, R., Crozier, P. A., Khapra, M. M., Simoncelli, E. P., & Fernandez‐Granda, C. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00178 · Cited by 74
Aperture Supervision for Monocular Depth Estimation
Srinivasan, P. P., Garg, R. D., Wadhwa, N., Ng, R., & Barron, J. T. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00669 · Cited by 63
Self-citationSynthetic defocus and look-ahead autofocus for casual videography
Zhang, X., Matzen, K., Nguyen, V., Yao, D., Zhang, Y., & Ng, R. (2019) · ACM Transactions on Graphics · doi:10.1145/3306346.3323015 · Cited by 51
NAN: Noise-Aware NeRFs for Burst-Denoising
Pearl, N., Treibitz, T., & Korman, S. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01234 · Cited by 45
ADOP: Approximate Differentiable One-Pixel Point RenderingPreprint
Rückert, D., Franke, L., & Stamminger, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.06635 · Cited by 38
Cited by
Mip-Splatting: Alias-Free 3D Gaussian Splatting
Yu, Z., Chen, A., Huang, B., Sattler, T., & Geiger, A. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01839 · Cited by 564
HexPlane: A Fast Representation for Dynamic Scenes
Cao, A., & Johnson, J. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00021 · Cited by 435
Tewari, A., Thies, J., Mildenhall, B., Srinivasan, P., Tretschk, E., Yifan, W., Lassner, C., Sitzmann, V., Martin‐Brualla, R., Lombardi, S., et al. (2022) · Computer Graphics Forum · doi:10.1111/cgf.14507 · Cited by 339
Self-citationFreeNeRF: Improving Few-Shot Neural Rendering with Free Frequency Regularization
Yang, J., Pavone, M., & Wang, Y. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00798 · Cited by 316
Instruct-NeRF2NeRF: Editing 3D Scenes with Instructions
Haque, A., Tancik, M., Efros, A. A., Holynski, A., & Kanazawa, A. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01808 · Cited by 314
Metadata
- Authors
- Mildenhall, B., Hedman, P., Martin-Brualla, R., Srinivasan, P. P., & Barron, J. T.
- Venue
- 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2022-06
- Pages
- 16169-16178
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W4312480149
- Cited by
- 349
- References
- 50
- FWCI
- 29.26
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.