The DOI matches its Crossref record; 50 of the 50 references it cites were found in a registry or an index.
Deep Video Deblurring for Hand-Held Cameras
Abstract
Motion blur from camera shake is a major problem in videos captured by hand-held devices. Unlike single-image deblurring, video-based approaches can take advantage of the abundant information that exists across neighboring frames. As a result the best performing methods rely on the alignment of nearby frames. However, aligning images is a computationally expensive and fragile procedure, and methods that aggregate information must therefore be able to identify which regions have been accurately aligned and which have not, a task that requires high level scene understanding. In this work, we introduce a deep learning solution to video deblurring, where a CNN is trained end-to-end to learn how to accumulate information across frames. To train this network, we collected a dataset of real videos recorded with a high frame rate camera, which we use to generate synthetic motion blur for supervision. We show that the features learned from this dataset extend to deblurring motion blur that arises due to camera shake in a wide range of videos, and compare the quality of results to a number of other baselines.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Shi, W., Caballero, J., Huszar, F., Totz, J., Aitken, A. P., Bishop, R., Rueckert, D., & Wang, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.207 · Cited by 7,570
Context Encoders: Feature Learning by Inpainting
Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T., & Efros, A. A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.278 · Cited by 5,691
Learning a Deep Convolutional Network for Image Super-Resolution
Dong, C., Loy, C. C., He, K., & Tang, X. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10593-2_13 · Cited by 5,494
Flownet: Learning optical flow with convolutional networks
Dosovitskiy, A., Fischer, P., Ilg, E., Hausser, P., Hazirbas, C., Golkov, V., Smagt, P. V. D., Cremers, D., & Brox, T. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.316 · Cited by 4,313
MLESAC: A New Robust Estimator with Application to Estimating Image Geometry
Torr, P. H. S., & Zisserman, A. (2000) · Computer Vision and Image Understanding · doi:10.1006/cviu.1999.0832 · Cited by 2,396
Removing camera shake from a single photograph
Fergus, R., Singh, B., Hertzmann, A., Roweis, S. T., & Freeman, W. T. (2006) · ACM Transactions on Graphics · doi:10.1145/1141911.1141956 · Cited by 1,815
A Unified Multi-scale Deep Convolutional Neural Network for Fast Object Detection
Cai, Z., Fan, Q., Feris, R. S., & Vasconcelos, N. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_22 · Cited by 1,605
High-quality motion deblurring from a single image
Shan, Q., Jia, J., & Agarwala, A. (2008) · ACM Transactions on Graphics · doi:10.1145/1360612.1360672 · Cited by 1,386
Kundur, D., & Hatzinakos, D. (1996) · IEEE Signal Processing Magazine · doi:10.1109/79.489268 · Cited by 1,275
Mao, X.-J., Shen, C., & Yang, Y.-B. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.09056 · Cited by 1,263
Image Denoising and Inpainting with Deep Neural Networks
Xie, J., Xu, L., & Chen, E. (2012) · Cited by 1,237
Blind deconvolution using a normalized sparsity measure
Krishnan, D., Tay, T., & Fergus, R. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995521 · Cited by 1,119
Unnatural L0 Sparse Representation for Natural Image Deblurring
Xu, L., Zheng, S., & Jia, J. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.147 · Cited by 1,079
Digital photography with flash and no-flash image pairs
Petschnigg, G., Szeliski, R., Agrawala, M., Cohen, M., Hoppe, H., & Toyama, K. (2004) · ACM Transactions on Graphics · doi:10.1145/1015706.1015777 · Cited by 1,056
Two-Phase Kernel Estimation for Robust Motion Deblurring
Xu, L., & Jia, J. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15549-9_12 · Cited by 1,027
Cho, S., & Lee, S. (2009) · ACM Transactions on Graphics · doi:10.1145/1618452.1618491 · Cited by 942
Learning a convolutional neural network for non-uniform motion blur removal
Sun, J., Cao, W., Xu, Z., & Ponce, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298677 · Cited by 906
Iizuka, S., Simo-Serra, E., & Ishikawa, H. (2016) · ACM Transactions on Graphics · doi:10.1145/2897824.2925974 · Cited by 814
Image deblurring with blurred/noisy image pairs
Yuan, L., Sun, J., Quan, L., & Shum, H.-Y. (2007) · ACM SIGGRAPH 2007 papers · doi:10.1145/1275808.1276379 · Cited by 764
Video Super-Resolution With Convolutional Neural Networks
Kappeler, A., Yoo, S., Dai, Q., & Katsaggelos, A. K. (2016) · IEEE Transactions on Computational Imaging · doi:10.1109/tci.2016.2532323 · Cited by 647
Full-frame video stabilization with motion inpainting
Matsushita, Y., Ofek, E., Ge, W., Tang, X., & Shum, H.-Y. (2006) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2006.141 · Cited by 542
Laplacian Pyramid Reconstruction and Refinement for Semantic Segmentation
Ghiasi, G., & Fowlkes, C. C. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46487-9_32 · Cited by 502
Recording and Playback of Camera Shake: Benchmarking Blind Deconvolution with a Real-World Database
Köhler, R., Hirsch, M., Mohler, B., Schölkopf, B., & Harmeling, S. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33786-4_3 · Cited by 424
Sánchez Pérez, J., Meinhardt-Llopis, E., & Facciolo, G. (2013) · Image Processing On Line · doi:10.5201/ipol.2013.26 · Cited by 416
Blind Deblurring Using Internal Patch Recurrence
Michaeli, T., & Irani, M. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10578-9_51 · Cited by 403
A Neural Approach to Blind Motion Deblurring
Chakrabarti, A. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46487-9_14 · Cited by 400
Image deblurring with blurred/noisy image pairs
Yuan, L., Sun, J., Quan, L., & Shum, H.-Y. (2007) · ACM Transactions on Graphics · doi:10.1145/1276377.1276379 · Cited by 374
Phase-based frame interpolation for video
Meyer, S., Wang, O., Zimmer, H., Grosse, M., & Sorkine-Hornung, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298747 · Cited by 294
Self-citationRobust Single Image Super-Resolution via Deep Networks With Sparse Prior
Liu, D., Wang, Z., Wen, B., Yang, J., Han, W., & Huang, T. S. (2016) · IEEE Transactions on Image Processing · doi:10.1109/tip.2016.2564643 · Cited by 266
Lucky imaging: high angular resolution imaging in the visible from the ground
Law, N. M., Mackay, C. D., & Baldwin, J. E. (2006) · Astronomy & Astrophysics · doi:10.1051/0004-6361:20053695 · Cited by 249
Bidirectional recurrent convolutional networks for multi-frame super-resolution
Huang, Y., Wang, W., & Wang, L. (2015) · neural information processing systems · Cited by 222
Learning to simplify: fully convolutional networks for rough sketch cleanup
Simo-Serra, E., Iizuka, S., Sasaki, K., & Ishikawa, H. (2016) · ACM Transactions on Graphics · doi:10.1145/2897824.2925972 · Cited by 211
A High-Quality Video Denoising Algorithm Based on Reliable Motion Estimation
Liu, C., & Freeman, W. T. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15558-1_51 · Cited by 173
Generalized video deblurring for dynamic scenes
Kim, T. H., & Lee, K. M. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7299181 · Cited by 165
Video deblurring for hand-held cameras using patch-based synthesis
Cho, S., Wang, J., & Lee, S. (2012) · ACM Transactions on Graphics · doi:10.1145/2185520.2185560 · Cited by 161
Self-citationImage/video deblurring using a hybrid camera
Tai, Y., Du, H. Y., Brown, M. S., & Lin, S. C.-F. (2008) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2008.4587507 · Cited by 116
Liu, Z., Yuan, L., Tang, X., Uyttendaele, M., & Sun, J. Z. (2014) · ACM Transactions on Graphics · doi:10.1145/2661229.2661277 · Cited by 110
Recent Progress in Image DeblurringPreprint
Wang, R., & Tao, D. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1409.6838 · Cited by 108
Seeing Mt. Rainier: Lucky imaging for multi-image denoising, sharpening, and haze removal
Joshi, N. S., & Cohen, M. F. (2010) · IEEE International Conference on Computational Photography (ICCP) · doi:10.1109/iccphot.2010.5585096 · Cited by 97
Modeling Blurred Video with Layers
Wulff, J., & Black, M. J. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10599-4_16 · Cited by 95
Burst deblurring: Removing camera shake through fourier burst accumulation
Delbracio, M., & Sapiro, G. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298852 · Cited by 79
Self-citationGenerating sharp panoramas from motion-blurred videos
Li, Y., Kang, S. B., Joshi, N. S., et al. (2010) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2010.5539938 · Cited by 72
Hand-Held Video Deblurring Via Efficient Fourier Aggregation
Delbracio, M., & Sapiro, G. (2015) · IEEE Transactions on Computational Imaging · doi:10.1109/tci.2015.2501245 · Cited by 70
Self-citationNon-uniform Motion Deblurring for Bilayer Scenes
Chandramouli, P., & Rajagopalan, A. N. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.148 · Cited by 58
Rolling shutter motion deblurring
Su, S., & Heidrich, W. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298760 · Cited by 52
Self-citationSellent, A., Rother, C., & Roth, S. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46475-6_35 · Cited by 51
Sampling based scene-space video processing
Klose, F., Wang, O., Bazin, J., Magnor, M. A., & Sorkine‐Hornung, A. (2015) · ACM Transactions on Graphics · doi:10.1145/2766920 · Cited by 40
Self-citationEnd-to-End Learning for Image Burst Deblurring
Wieschollek, P., Schölkopf, B., Lensch, H. P. A., & Hirsch, M. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-54190-7_3 · Cited by 34
Cited by
The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
Zhang, R., Isola, P., Efros, A. A., Shechtman, E., & Wang, O. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00068 · Cited by 14,623
Self-citationEdvr: video restoration with enhanced deformable convolutional networks
Wang, X., Chan, K. C. K., Yu, K., Dong, C., & Loy, C. C. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2019.00247 · Cited by 1,272
DeblurGAN-v2: Deblurring (Orders-of-Magnitude) Faster and Better
Kupyn, O., Martyniuk, T., Wu, J., & Wang, Z. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00897 · Cited by 1,148
Artificial intelligence in the creative industries: a review
Anantrasirichai, N., & Bull, D. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-021-10039-7 · Cited by 793
NTIRE 2019 Challenge on Video Deblurring and Super-Resolution: Dataset and Study
Nah, S., Baik, S., Hong, S., Moon, G., Son, S., Timofte, R., & Lee, K. M. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2019.00251 · Cited by 543
Metadata
- Authors
- Su, S., Delbracio, M., Wang, J., Sapiro, G., Heidrich, W., & Wang, O.
- Venue
- 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2017-07
- Pages
- 237-246
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2738579427
- Cited by
- 577
- References
- 50
- FWCI
- 13.91
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.