The DOI matches its Crossref record; 80 of the 80 references it cites were found in a registry or an index.
Fast Image Processing with Fully-Convolutional Networks
Open accessAbstract
We present an approach to accelerating a wide variety of image processing operators. Our approach uses a fully-convolutional network that is trained on input-output pairs that demonstrate the operator's action. After training, the original operator need not be run at all. The trained network operates at full resolution and runs in constant time. We investigate the effect of network architecture on approximation accuracy, runtime, and memory footprint, and identify a specific architecture that balances these considerations. We evaluate the presented approach on ten advanced image processing operators, including multiple variational models, multiscale tone and detail manipulation, photographic style transfer, nonlocal dehazing, and nonphoto-realistic stylization. All operators are approximated by the same model. Experiments demonstrate that the presented approach is significantly more accurate than prior approximation schemes. It increases approximation accuracy as measured by PSNR across the evaluated operators by 8.5 dB on the MIT-Adobe dataset (from 27.5 to 36 dB) and reduces DSSIM by a multiplicative factor of 3 compared to the most accurate prior approximation scheme, while being the fastest. We show that our models generalize across datasets and across resolutions, and investigate a number of extensions of the presented approach.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818
Image-to-Image Translation with Conditional Adversarial Networks
Isola, P., Zhu, J.-Y., Zhou, T., & Efros, A. A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.632 · Cited by 22,930
Nonlinear total variation based noise removal algorithms
Rudin, L. I., Osher, S., & Fatemi, E. (1992) · Physica D: Nonlinear Phenomena · doi:10.1016/0167-2789(92)90242-f · Cited by 15,704
Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network
Ledig, C., Theis, L., Huszar, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., Wang, Z., et al. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.19 · Cited by 12,696
Backpropagation Applied to Handwritten Zip Code Recognition
LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., & Jackel, L. D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.4.541 · Cited by 11,891
Perceptual Losses for Real-Time Style Transfer and Super-Resolution
Johnson, J., Alahi, A., & Fei-Fei, L. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46475-6_43 · Cited by 11,283
Image Super-Resolution Using Deep Convolutional Networks
Dong, C., Loy, C. C., He, K., & Tang, X. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2439281 · Cited by 10,105
Accurate Image Super-Resolution Using Very Deep Convolutional Networks
Kim, J., Lee, J. K., & Lee, K. M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.182 · Cited by 7,870
Multi-scale context aggregation by dilated convolutions
Yu, F., & Koltun, V. (2015) · arXiv · doi:10.48550/arxiv.1511.07122 · Cited by 7,738
Self-citationSingle image haze removal using dark channel prior
He, K., Sun, J., & Tang, X. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.168 · Cited by 7,601
Image Style Transfer Using Convolutional Neural Networks
Gatys, L. A., Ecker, A. S., & Bethge, M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.265 · Cited by 6,025
Han, S., Mao, H., & Dally, W. J. (2015) · arXiv · doi:10.48550/arxiv.1510.00149 · Cited by 4,662
A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging
Chambolle, A., & Pock, T. (2011) · Journal of Mathematical Imaging and Vision · doi:10.1007/s10851-010-0251-1 · Cited by 4,336
Deeply-Recursive Convolutional Network for Image Super-Resolution
Kim, J., Lee, J. K., & Lee, K. M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.181 · Cited by 3,025
Fattal, R. (2008) · ACM Transactions on Graphics · doi:10.1145/1360612.1360671 · Cited by 2,114
Single Image Dehazing via Multi-scale Convolutional Neural Networks
Ren, W., Liu, S., Zhang, H., Pan, J., Cao, X., & Yang, M.-H. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46475-6_10 · Cited by 1,759
Tanks and temples: benchmarking large-scale scene reconstruction
Knapitsch, A., Park, J., Zhou, Q.-Y., & Koltun, V. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073599 · Cited by 1,707
Self-citationBerman, D., Treibitz, T., & Avidan, S. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.185 · Cited by 1,642
Fast bilateral filtering for the display of high-dynamic-range images
Durand, F., & Dorsey, J. (2002) · Proceedings of the 29th annual conference on Computer graphics and interactive techniques · doi:10.1145/566570.566574 · Cited by 1,547
Edge-preserving decompositions for multi-scale tone and detail manipulation
Farbman, Z., Fattal, R., Lischinski, D., & Szeliski, R. (2008) · ACM Transactions on Graphics · doi:10.1145/1360612.1360666 · Cited by 1,443
Image denoising: Can plain neural networks compete with BM3D?
Burger, H. C., Schuler, C. J., & Harmeling, S. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247952 · Cited by 1,325
Fattal, R. (2014) · ACM Transactions on Graphics · doi:10.1145/2651362 · Cited by 1,167
Fast bilateral filtering for the display of high-dynamic-range images
Durand, F., & Dorsey, J. (2002) · ACM Transactions on Graphics · doi:10.1145/566654.566574 · Cited by 1,003
Photographic Image Synthesis with Cascaded Refinement Networks
Chen, Q., & Koltun, V. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.168 · Cited by 887
Self-citationImage smoothing via L0gradient minimization
Xu, L., Lu, C., Xu, Y., & Jia, J. (2011) · Proceedings of the 2011 SIGGRAPH Asia Conference · doi:10.1145/2024156.2024208 · Cited by 873
Kopf, J., Cohen, M. F., Lischinski, D., & Uyttendaele, M. (2007) · ACM SIGGRAPH 2007 papers · doi:10.1145/1275808.1276497 · Cited by 872
Structure extraction from texture via relative total variation
Xu, L., Yan, Q., Xia, Y., & Jia, J. (2012) · ACM Transactions on Graphics · doi:10.1145/2366145.2366158 · Cited by 867
(2008) · Cited by 838
Deep Convolutional Neural Network for Image Deconvolution
Li, X., Ren, J., Liu, C., & Jia, J. (2014) · Rare & Special e-Zone (The Hong Kong University of Science and Technology) · Cited by 836
A Fast Approximation of the Bilateral Filter Using a Signal Processing Approach
Paris, S., & Durand, F. (2009) · International Journal of Computer Vision · doi:10.1007/s11263-007-0110-8 · Cited by 797
Image smoothing via L0gradient minimization
Xu, L., Lu, C., Xu, Y., & Jia, J. (2011) · ACM Transactions on Graphics · doi:10.1145/2070781.2024208 · Cited by 786
Regularizing Flows for Constrained Matrix-Valued Images
Chefd'hotel, C., Tschumperlé, D., Deriche, R., & Faugeras, O. (2004) · Journal of Mathematical Imaging and Vision · doi:10.1023/b:jmiv.0000011920.58935.9c · Cited by 648
Structure-Texture Image Decomposition—Modeling, Algorithms, and Parameter Selection
Aujol, J.-F., Gilboa, G., Chan, T., & Osher, S. (2006) · International Journal of Computer Vision · doi:10.1007/s11263-006-4331-z · Cited by 630
Domain transform for edge-aware image and video processing
Gastal, E. S. L., & Oliveira, M. M. (2011) · ACM Transactions on Graphics · doi:10.1145/2010324.1964964 · Cited by 620
A Tour of Modern Image Filtering: New Insights and Methods, Both Practical and Theoretical
Milanfar, P. (2013) · IEEE Signal Processing Magazine · doi:10.1109/msp.2011.2179329 · Cited by 605
Domain transform for edge-aware image and video processing
Gastal, E. S. L., & Oliveira, M. M. (2011) · ACM SIGGRAPH 2011 papers · doi:10.1145/1964921.1964964 · Cited by 591
Zhang, Q., Shen, X., Xu, L., & Jia, J. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10578-9_53 · Cited by 580
Texture Networks: Feed-forward Synthesis of Textures and Stylized Images
Ulyanov, D., Lebedev, V., Vedaldi, A., & Lempitsky, V. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.03417 · Cited by 531
Learning photographic global tonal adjustment with a database of input / output image pairs
Bychkovsky, V., Paris, S., Chan, E., & Durand, F. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995332 · Cited by 529
RAISE: a raw images dataset for digital image forensics
Dang-Nguyen, D.-T., Pasquini, C., Conotter, V., & Boato, G. (2015) · Proceedings of the 6th ACM Multimedia Systems Conference · doi:10.1145/2713168.2713194 · Cited by 508
Deep joint demosaicking and denoising
Gharbi, M., Chaurasia, G., Paris, S., & Durand, F. (2016) · ACM Transactions on Graphics · doi:10.1145/2980179.2982399 · Cited by 473
Restoring an Image Taken through a Window Covered with Dirt or Rain
Eigen, D., Krishnan, D., & Fergus, R. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.84 · Cited by 454
Real-time edge-aware image processing with the bilateral grid
Chen, J., Paris, S., & Durand, F. (2007) · ACM Transactions on Graphics · doi:10.1145/1276377.1276506 · Cited by 395
Fast High‐Dimensional Filtering Using the Permutohedral Lattice
Adams, A., Baek, J., & Davis, M. A. (2010) · Computer Graphics Forum · doi:10.1111/j.1467-8659.2009.01645.x · Cited by 394
Real-time edge-aware image processing with the bilateral grid
Chen, J., Paris, S., & Durand, F. (2007) · ACM SIGGRAPH 2007 papers · doi:10.1145/1275808.1276506 · Cited by 394
Local Laplacian filters: edge-aware image processing with a Laplacian pyramid
Paris, S., Hasinoff, S. W., & Kautz, J. (2011) · ACM Transactions on Graphics · doi:10.1145/2010324.1964963 · Cited by 384
Unrolled Generative Adversarial Networks
Metz, L., Poole, B., Pfau, D., & Sohl‐Dickstein, J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.02163 · Cited by 382
Fast median and bilateral filtering
Weiss, B. (2006) · ACM Transactions on Graphics · doi:10.1145/1141911.1141918 · Cited by 346
Two-scale tone management for photographic look
Bae, S., Paris, S., & Durand, F. (2006) · ACM Transactions on Graphics · doi:10.1145/1141911.1141935 · Cited by 332
On learning optimized reaction diffusion processes for effective image restoration
Chen, Y., Yu, W., & Pock, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299163 · Cited by 323
Local Laplacian filters: edge-aware image processing with a Laplacian pyramid
Paris, S., Hasinoff, S. W., & Kautz, J. (2011) · ACM SIGGRAPH 2011 papers · doi:10.1145/1964921.1964963 · Cited by 322
Median Filtering in Constant Time
Perreault, S., & Hebert, P. (2007) · IEEE Transactions on Image Processing · doi:10.1109/tip.2007.902329 · Cited by 317
Barron, J. T., & Poole, B. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46487-9_38 · Cited by 309
Edge-preserving multiscale image decomposition based on local extrema
Subr, K., Soler, C., & Durand, F. (2009) · ACM Transactions on Graphics · doi:10.1145/1618452.1618493 · Cited by 306
Edge-avoiding wavelets and their applications
Fattal, R. (2009) · ACM Transactions on Graphics · doi:10.1145/1531326.1531328 · Cited by 294
Constant Time Weighted Median Filtering for Stereo Matching and Beyond
Ma, Z., He, K., Wei, Y., Sun, J., & Wu, E. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.13 · Cited by 292
Automatic Photo Adjustment Using Deep Neural Networks
Yan, Z., Zhang, H., Wang, B., Paris, S., & Yu, Y. (2016) · ACM Transactions on Graphics · doi:10.1145/2790296 · Cited by 271
Li, Y., Huang, J.-B., Ahuja, N., & Yang, M.-H. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_10 · Cited by 269
Video Super-Resolution via Deep Draft-Ensemble Learning
Liao, R., Tao, X., Li, R., Ma, Z., & Jia, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.68 · Cited by 258
Decoupling algorithms from schedules for easy optimization of image processing pipelines
Ragan-Kelley, J., Adams, A., Paris, S., Levoy, M., Amarasinghe, S., & Durand, F. (2012) · ACM Transactions on Graphics · doi:10.1145/2185520.2185528 · Cited by 251
Adaptive manifolds for real-time high-dimensional filtering
Gastal, E. S. L., & Oliveira, M. M. (2012) · ACM Transactions on Graphics · doi:10.1145/2185520.2185529 · Cited by 241
100+ Times Faster Weighted Median Filter (WMF)
Zhang, Q., Xu, L., & Jia, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.362 · Cited by 225
Gaussian KD-trees for fast high-dimensional filtering
Adams, A., Gelfand, N., Dolson, J., & Levoy, M. (2009) · ACM Transactions on Graphics · doi:10.1145/1531326.1531327 · Cited by 213
Adaptive Multi-Column Deep Neural Networks with Application to Robust Image Denoising
Agostinelli, F., Anderson, M. R., & Lee, H. (2013) · Cited by 210
Fast Local Laplacian Filters: Theory and Applications
Aubry, M., Paris, S., Hasinoff, S. W., Kautz, J., & Durand, F. (2014) · ACM Transactions on Graphics · doi:10.1145/2629645 · Cited by 197
Li, X., Ren, J., Yan, Q., Liao, R., & Jia, J. (2015) · Cited by 175
Learning Recursive Filters for Low-Level Vision via a Hybrid Neural Network
Liu, S., Pan, J., & Yang, M. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46493-0_34 · Cited by 160
Chen, J., Adams, A., Wadhwa, N., & Hasinoff, S. W. (2016) · ACM Transactions on Graphics · doi:10.1145/2980179.2982423 · Cited by 131
Efficient preconditioning of laplacian matrices for computer graphics
Krishnan, D., Fattal, R., & Szeliski, R. (2013) · ACM Transactions on Graphics · doi:10.1145/2461912.2461992 · Cited by 112
Fast Guided FilterPreprint
He, K., & Sun, J. D. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.00996 · Cited by 106
Hegarty, J., Daly, R., DeVito, Z., Ragan‐Kelley, J., Horowitz, M., & Hanrahan, P. (2016) · ACM Transactions on Graphics · doi:10.1145/2897824.2925892 · Cited by 69
Farbman, Z., Fattal, R., & Lischinski, D. (2011) · ACM Transactions on Graphics · doi:10.1145/2070781.2024209 · Cited by 57
Fast and exact solution of Total Variation models on the GPU
Pock, T., Unger, M., Cremers, D., & Bischof, H. (2008) · 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2008.4563099 · Cited by 37
Transform recipes for efficient cloud photo enhancement
Gharbi, M., Shih, Y., Chaurasia, G., Ragan‐Kelley, J., Paris, S., & Durand, F. (2015) · ACM Transactions on Graphics · doi:10.1145/2816795.2818127 · Cited by 30
Cited by
Chen, C., Chen, Q., Xu, J., & Koltun, V. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00347 · Cited by 1,376
Self-citationScale-Recurrent Network for Deep Image Deblurring
Tao, X., Gao, H., Shen, X., Wang, J., & Jia, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00853 · Cited by 1,316
Learning to Optimize: Training Deep Neural Networks for Interference Management
Sun, H., Chen, X., Shi, Q., Hong, M., Fu, X., & Sidiropoulos, N. D. (2018) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2018.2866382 · Cited by 874
Gibson Env: Real-World Perception for Embodied Agents
Xia, F., Zamir, A. R., He, Z., Sax, A., Malik, J., & Savarese, S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00945 · Cited by 763
Low-Light Image Enhancement via a Deep Hybrid Network
Ren, W., Liu, S., Ma, L., Xu, Q., Xu, X., Cao, X., Du, J., & Yang, M.-H. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2910412 · Cited by 508
Metadata
- Authors
- Chen, Q., Xu, J., & Koltun, V.
- Venue
- 2017 IEEE International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2017-10
- Pages
- 2516-2525
- OpenAlex
- W2963452532
- Cited by
- 341
- References
- 80
- FWCI
- 10.76
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.