The DOI matches its Crossref record; 48 of the 48 references it cites were found in a registry or an index.
Seeing Motion in the Dark
Abstract
Deep learning has recently been applied with impressive results to extreme low-light imaging. Despite the success of single-image processing, extreme low-light video processing is still intractable due to the difficulty of collecting raw video data with corresponding ground truth. Collecting long-exposure ground truth, as was done for single-image processing, is not feasible for dynamic scenes. In this paper, we present deep processing of very dark raw videos: on the order of one lux of illuminance. To support this line of work, we collect a new dataset of raw low-light videos, in which high-resolution raw data is captured at video rate. At this level of darkness, the signal-to-noise ratio is extremely low (negative if measured in dB) and the traditional image processing pipeline generally breaks down. A new method is presented to address this challenging problem. By carefully designing a learning-based pipeline and introducing a new loss function to encourage temporal stability, we train a siamese network on static raw videos, for which ground truth is available, such that the network generalizes to videos of dynamic scenes at test time. Experimental results demonstrate that the presented approach outperforms state-of-the-art models for burst processing, per-frame processing, and blind temporal consistency.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892
Nonlinear total variation based noise removal algorithms
Rudin, L. I., Osher, S., & Fatemi, E. (1992) · Physica D: Nonlinear Phenomena · doi:10.1016/0167-2789(92)90242-f · Cited by 15,704
TensorFlow: A system for large-scale machine learning
Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.08695 · Cited by 11,822
Image denoising by sparse 3-D transform-domain collaborative filtering
Dabov, K., Foi, A., Katkovnik, V., & Egiazarian, K. (2007) · IEEE Transactions on Image Processing · doi:10.1109/tip.2007.901238 · Cited by 9,283
Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising
Zhang, K., Zuo, W., Chen, Y., Meng, D., & Zhang, L. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2662206 · Cited by 8,983
Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries
Elad, M., & Aharon, M. (2006) · IEEE Transactions on Image Processing · doi:10.1109/tip.2006.881969 · Cited by 5,321
LIME: Low-Light Image Enhancement via Illumination Map Estimation
Guo, X., Li, Y., & Ling, H. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2016.2639450 · Cited by 2,998
PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume
Sun, D., Yang, X., Liu, M.-Y., & Kautz, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00931 · Cited by 2,648
SIGNATURE VERIFICATION USING A “SIAMESE” TIME DELAY NEURAL NETWORK
BROMLEY, J., BENTZ, J. W., BOTTOU, L., GUYON, I., LECUN, Y., MOORE, C., SÄCKINGER, E., & SHAH, R. (1994) · Series in Machine Perception and Artificial Intelligence · doi:10.1142/9789812797926_0003 · Cited by 2,350
Weighted Nuclear Norm Minimization with Application to Image Denoising
Gu, S., Zhang, L., Zuo, W., & Feng, X. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.366 · Cited by 2,316 · cited 2×
“Researchers have developed many techniques to reduce noise for low-light imaging [40, 38, 13, 33, 16, 11, 20, 45, 46, 3, 8, 29, 22, 47].”
BackgroundImage denoising using scale mixtures of gaussians in the wavelet domain
Portilla, J., Strela, V., Wainwright, M. J., & Simoncelli, E. P. (2003) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.818640 · Cited by 2,255
LLNet: A deep autoencoder approach to natural low-light image enhancement
Lore, K. G., Akintayo, A., & Sarkar, S. (2017) · Pattern Recognition · doi:10.1016/j.patcog.2016.06.008 · Cited by 1,951
Non-local sparse models for image restoration
Mairal, J., Bach, F., Ponce, J., Sapiro, G., & Zisserman, A. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459452 · Cited by 1,666
Chen, C., Chen, Q., Xu, J., & Koltun, V. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00347 · Cited by 1,376
Self-citationChen, Y., & Pock, T. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2596743 · Cited by 1,344
Image denoising: Can plain neural networks compete with BM3D?
Burger, H. C., Schuler, C. J., & Harmeling, S. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247952 · Cited by 1,325
Image Denoising and Inpainting with Deep Neural Networks
Xie, J., Xu, L., & Chen, E. (2012) · Cited by 1,237
Noise2Noise: Learning Image Restoration without Clean Data
Lehtinen, J., Munkberg, J., Hasselgren, J., Laine, S., Karras, T., Aittala, M., & Aila, T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.04189 · Cited by 944
A High-Quality Denoising Dataset for Smartphone Cameras
Abdelhamed, A., Lin, S., & Brown, M. S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00182 · Cited by 867
(2008) · Cited by 838
Benchmarking Denoising Algorithms with Real Photographs
Plotz, T., & Roth, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.294 · Cited by 655
MBLLEN: Low-Light Image/Video Enhancement Using CNNs.
Lv, F., Lu, F., Wu, J., & Lim, C. (2018) · Cited by 543
Burst photography for high dynamic range and low-light imaging on mobile cameras
Hasinoff, S. W., Sharlet, D., Geiss, R., Adams, A., Barron, J. T., Kainz, F., Chen, J., & Levoy, M. (2016) · ACM Transactions on Graphics · doi:10.1145/2980179.2980254 · Cited by 483
Deep joint demosaicking and denoising
Gharbi, M., Chaurasia, G., Paris, S., & Durand, F. (2016) · ACM Transactions on Graphics · doi:10.1145/2980179.2982399 · Cited by 473
Unprocessing Images for Learned Raw Denoising
Brooks, T., Mildenhall, B., Xue, T., Chen, J., Sharlet, D., & Barron, J. T. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01129 · Cited by 473
Learning Blind Video Temporal Consistency
Lai, W.-S., Huang, J.-B., Wang, O., Shechtman, E., Yumer, E., & Yang, M.-H. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01267-0_11 · Cited by 405
Burst Denoising with Kernel Prediction Networks
Mildenhall, B., Barron, J. T., Chen, J., Sharlet, D., Ng, R., & Carroll, R. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00265 · Cited by 404
Maggioni, M., Boracchi, G., Foi, A., & Egiazarian, K. (2012) · IEEE Transactions on Image Processing · doi:10.1109/tip.2012.2199324 · Cited by 388
Fast efficient algorithm for enhancement of low lighting video
Dong, X., Wang, G., Pang, Y., Li, W., Wen, J., Meng, W., & Lu, Y. (2011) · 2011 IEEE International Conference on Multimedia and Expo · doi:10.1109/icme.2011.6012107 · Cited by 364
A Nonlocal Bayesian Image Denoising Algorithm
Lebrun, M., Buades, A., & Morel, J. M. (2013) · SIAM Journal on Imaging Sciences · doi:10.1137/120874989 · Cited by 318
Fast efficient algorithm for enhancement of low lighting video
Dong, X., Pang, Y. (., & Wen, J. (. (2010) · ACM SIGGRAPH 2010 Posters · doi:10.1145/1836845.1836920 · Cited by 310
FlexISP: a flexible camera image processing framework
Heide, F., Steinberger, M., Tsai, Y.-T., Rouf, M., Pająk, D., Reddy, D., Gallo, O., Liu, J., Heidrich, W., Egiazarian, K., et al. (2014) · ACM Transactions on Graphics · doi:10.1145/2661229.2661260 · Cited by 298
DeepISP: Toward Learning an End-to-End Image Processing Pipeline
Schwartz, E., Giryes, R., & Bronstein, A. M. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2872858 · Cited by 265
Adaptive Multi-Column Deep Neural Networks with Application to Robust Image Denoising
Agostinelli, F., Anderson, M. R., & Lee, H. (2013) · neural information processing systems · Cited by 249
Low-light image enhancement using variational optimization-based retinex model
Park, S., Yu, S., Moon, B., Ko, S., & Paik, J. (2017) · IEEE Transactions on Consumer Electronics · doi:10.1109/tce.2017.014847 · Cited by 231
RENOIR – A dataset for real low-light image noise reduction
Anaya, J., & Barbu, A. (2018) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2018.01.012 · Cited by 221
Learning Recursive Filters for Low-Level Vision via a Hybrid Neural Network
Liu, S., Pan, J., & Yang, M. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46493-0_34 · Cited by 178
Blind video temporal consistency
Bonneel, N., Tompkin, J., Sunkavalli, K., Sun, D., Paris, S., & Pfister, H. (2015) · ACM Transactions on Graphics · doi:10.1145/2816795.2818107 · Cited by 171
Automatic contrast enhancement of low-light images based on local statistics of wavelet coefficients
Łoza, A., Bull, D., Hill, P., & Achim, A. (2013) · Digital Signal Processing · doi:10.1016/j.dsp.2013.06.002 · Cited by 152
Godard, C., Matzen, K., & Uyttendaele, M. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01267-0_33 · Cited by 140
Joint demosaicing and denoising
Hirakawa, K., & Parks, T. W. (2006) · IEEE Transactions on Image Processing · doi:10.1109/tip.2006.875241 · Cited by 138
Liu, Z., Yuan, L., Tang, X., Uyttendaele, M., & Sun, J. Z. (2014) · ACM Transactions on Graphics · doi:10.1145/2661229.2661277 · Cited by 110
Adaptive enhancement and noise reduction in very low light-level video
Malm, H., Oskarsson, M., Warrant, E. J., Clarberg, P., Hasselgren, J., & Lejdfors, C. (2007) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2007.4409007 · Cited by 109
Wen, B., Li, Y., Pfister, L., & Bresler, Y. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.35 · Cited by 49
Cited by
Retinexformer: One-stage Retinex-based Transformer for Low-light Image Enhancement
Cai, Y., Bian, H., Lin, J., Wang, H., Timofte, R., & Zhang, Y. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01149 · Cited by 896
Learning to Enhance Low-Light Image via Zero-Reference Deep Curve Estimation
Li, C., Guo, C., & Chen, C. L. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3063604 · Cited by 719
Self-citationSNR-Aware Low-light Image Enhancement
Xu, X., Wang, R., Fu, C.-W., & Jia, J. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01719 · Cited by 688
Low-Light Image and Video Enhancement Using Deep Learning: A Survey
Li, C., Guo, C., Han, L., Jiang, J., Cheng, M.-M., Gu, J., & Loy, C. C. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3126387 · Cited by 657
Benchmarking Low-Light Image Enhancement and Beyond
Liu, J., Xu, D., Yang, W., Fan, M., & Huang, H. (2021) · International Journal of Computer Vision · doi:10.1007/s11263-020-01418-8 · Cited by 349
Metadata
- Authors
- Chen, C., Chen, Q., Do, M., & Koltun, V.
- Venue
- 2019 IEEE/CVF International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2019-10
- Pages
- 3184-3193
- OpenAlex
- W2986422266
- Cited by
- 282
- References
- 48
- FWCI
- 7.25
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.