The DOI matches its Crossref record; 60 of the 60 references it cites were found in a registry or an index.
EdgeConnect: Structure Guided Image Inpainting using Edge Prediction
Abstract
In recent years, many deep learning techniques have been applied to the image inpainting problem: the task of filling incomplete regions of an image. However, these models struggle to recover and/or preserve image structure especially when significant portions of the image are missing. We propose a two-stage model that separates the inpainting problem into structure prediction and image completion. Similar to sketch art, our model first predicts the image structure of the missing region in the form of edge maps. Predicted edge maps are passed to the second stage to guide the inpainting process. We evaluate our model end-to-end over publicly available datasets CelebA, CelebHQ, Places2, and Paris StreetView on images up to a resolution of 512 × 512. We demonstrate that this approach outperforms current state-of-the-art techniques quantitatively and qualitatively.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
A Computational Approach to Edge Detection
Canny, J. (1986) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.1986.4767851 · Cited by 30,004
Unpaired image-to-image translation using cycle-consistent adversarial networks
Zhu, J.-Y., Park, T., Isola, P., & Efros, A. A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.244 · Cited by 23,602
Image-to-Image Translation with Conditional Adversarial Networks
Isola, P., Zhu, J.-Y., Zhou, T., & Efros, A. A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.632 · Cited by 22,930
Labaca-Castro, R. (2023) · Machine Learning under Malware Attack · doi:10.1007/978-3-658-40442-0_9 · Cited by 19,272
GAN(Generative Adversarial Nets)
(2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,270
The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
Zhang, R., Isola, P., Efros, A. A., Shechtman, E., & Wang, O. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00068 · Cited by 14,623
Perceptual Losses for Real-Time Style Transfer and Super-Resolution
Johnson, J., Alahi, A., & Fei-Fei, L. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46475-6_43 · Cited by 11,283
Conditional Generative Adversarial Nets
Mirza, M., & Osindero, S. (2014) · arXiv · doi:10.48550/arxiv.1411.1784 · Cited by 8,863
GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., & Hochreiter, S. (2017) · arXiv · doi:10.48550/arxiv.1706.08500 · Cited by 8,365
Deep Learning Face Attributes in the Wild
Liu, Z., Luo, P., Wang, X., & Tang, X. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.425 · Cited by 7,747
Multi-scale context aggregation by dilated convolutions
Yu, F., & Koltun, V. (2015) · arXiv · doi:10.48550/arxiv.1511.07122 · Cited by 7,738 · cited 1×
“To preserve the effective receptive field from the reduction of downsampling layers, the channel-wise fully connected layer was replaced by a series of dilated convolution layers [54] with varying dilation factors.”
MethodImage Style Transfer Using Convolutional Neural Networks
Gatys, L. A., Ecker, A. S., & Bethge, M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.265 · Cited by 6,025
Context Encoders: Feature Learning by Inpainting
Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T., & Efros, A. A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.278 · Cited by 5,691
High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs
Wang, T.-C., Liu, M.-Y., Zhu, J.-Y., Tao, A., Kautz, J., & Catanzaro, B. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00917 · Cited by 4,332
Places: A 10 Million Image Database for Scene Recognition
Zhou, B., Lapedriza, A., Khosla, A., Oliva, A., & Torralba, A. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2723009 · Cited by 4,042
Bertalmio, M., Sapiro, G., Caselles, V., & Ballester, C. (2000) · Proceedings of the 27th annual conference on Computer graphics and interactive techniques - SIGGRAPH '00 · doi:10.1145/344779.344972 · Cited by 3,580
Spectral Normalization for Generative Adversarial Networks
Miyato, T., Kataoka, T., Koyama, M., & Yoshida, Y. (2018) · arXiv · doi:10.48550/arxiv.1802.05957 · Cited by 3,438
Self-Attention Generative Adversarial Networks
Zhang, H., Goodfellow, I., Metaxas, D., & Odena, A. (2018) · arXiv · doi:10.48550/arxiv.1805.08318 · Cited by 3,353
Holistically-Nested Edge Detection
Xie, S., & Tu, Z. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.164 · Cited by 3,096
PatchMatch: a randomized correspondence algorithm for structural image editing
Barnes, C., Shechtman, E., Finkelstein, A., & Goldman, D. B. (2009) · ACM Transactions on Graphics · doi:10.1145/1531326.1531330 · Cited by 2,880
Generative Image Inpainting with Contextual Attention
Yu, J., Lin, Z., Yang, J., Shen, X., Lu, X., & Huang, T. S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00577 · Cited by 2,527
Image Inpainting for Irregular Holes Using Partial Convolutions
Liu, G., Reda, F. A., Shih, K. J., Wang, T.-C., Tao, A., & Catanzaro, B. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01252-6_6 · Cited by 2,419 · cited 5×
“We implemented PConv based on the guidelines in [31] using the PConv layer that is publicly available.”
BackgroundMethodInfluentialArjovsky, M., Chintala, S., & Bottou, L. (2017) · arXiv · doi:10.48550/arxiv.1701.07875 · Cited by 2,225
Globally and locally consistent image completion
Iizuka, S., Simo-Serra, E., & Ishikawa, H. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073659 · Cited by 2,203 · cited 2×
“[24] Yu et al .”
BackgroundFree-Form Image Inpainting With Gated Convolution
Yu, J., Lin, Z., Yang, J., Shen, X., Lu, X., & Huang, T. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00457 · Cited by 1,928
Deconvolution and Checkerboard Artifacts
Odena, A., Dumoulin, V., & Olah, C. (2016) · Distill · doi:10.23915/distill.00003 · Cited by 1,640
Semantic Image Inpainting with Deep Generative Models
Yeh, R. A., Chen, C., Lim, T. Y., Schwing, A. G., Hasegawa-Johnson, M., & Do, M. N. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.728 · Cited by 1,206
Simultaneous structure and texture image inpainting
Bertalmio, M., Vese, L., Sapiro, G., & Osher, S. (2003) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.815261 · Cited by 1,079
EnhanceNet: Single Image Super-Resolution Through Automated Texture Synthesis
Sajjadi, M. S. M., Scholkopf, B., & Hirsch, M. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.481 · Cited by 1,058
Fast Edge Detection Using Structured Forests
Dollar, P., & Zitnick, C. L. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2014.2377715 · Cited by 962
Filling-in by joint interpolation of vector fields and gray levels
Ballester, C., Bertalmio, M., Caselles, V., Sapiro, G., & Verdera, J. (2001) · IEEE Transactions on Image Processing · doi:10.1109/83.935036 · Cited by 962
Are GANs Created Equal? A Large-Scale StudyPreprint
Lučić, M., Kurach, K., Michalski, M., Gelly, S., & Bousquet, O. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.10337 · Cited by 928
High-Resolution Image Inpainting Using Multi-scale Neural Patch Synthesis
Yang, C., Lu, X., Lin, Z., Shechtman, E., Wang, O., & Li, H. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.434 · Cited by 877
Eitz, M., Hays, J., & Alexa, M. (2012) · ACM Transactions on Graphics · doi:10.1145/2185520.2185540 · Cited by 868
Ulyanov, D., Vedaldi, A., & Lempitsky, V. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.437 · Cited by 815
Richer Convolutional Features for Edge Detection
Liu, Y., Cheng, M.-M., Hu, X., Wang, K., & Bai, X. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.622 · Cited by 780
Richer Convolutional Features for Edge Detection
Liu, Y., Cheng, M.-M., Hu, X., Bian, J.-W., Zhang, L., Bai, X., & Tang, J. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2878849 · Cited by 751
What makes Paris look like Paris?
Doersch, C., Singh, S., Gupta, A., Sivic, J., & Efros, A. A. (2012) · ACM Transactions on Graphics · doi:10.1145/2185520.2185597 · Cited by 663
Image melding: combining inconsistent images using patch-based synthesis
Darabi, S., Shechtman, E., Barnes, C., Goldman, D. B., & Sen, P. (2012) · ACM Transactions on Graphics · doi:10.1145/2185520.2185578 · Cited by 550
Image completion with structure propagation
Sun, J., Yuan, L., Jia, J., & Shum, H.-Y. (2005) · ACM Transactions on Graphics · doi:10.1145/1073204.1073274 · Cited by 526
DeepEdge: A multi-scale bifurcated deep network for top-down contour detection
Bertasius, G., Shi, J., & Torresani, L. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299067 · Cited by 525
Scribbler: Controlling Deep Image Synthesis with Sketch and Color
Sangkloy, P., Lu, J., Fang, C., Yu, F., & Hays, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.723 · Cited by 497
Supervised Learning of Edges and Object Boundaries
Dollar, P., Tu, Z., & Belongie, S. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.298 · Cited by 424
Digital inpainting based on the Mumford–Shah–Euler image model
ESEDOGLU, S., & SHEN, J. (2002) · European Journal of Applied Mathematics · doi:10.1017/s0956792502004904 · Cited by 418
Image completion using planar structure guidance
Huang, J.-B., Kang, S. B., Ahuja, N., & Kopf, J. (2014) · ACM Transactions on Graphics · doi:10.1145/2601097.2601205 · Cited by 322
Structured Learning and Prediction in Computer Vision
Nowozin, S. (2010) · now publishers, Inc. eBooks · doi:10.1561/9781601984579 · Cited by 239
Contextual-Based Image Inpainting: Infer, Match, and Translate
Song, Y., Yang, C., Lin, Z., Liu, X., Huang, Q., Li, H., & Kuo, C.-C. J. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01216-8_1 · Cited by 234
SPG-Net: Segmentation Prediction and Guidance Network for Image InpaintingPreprint
Song, Y., Yang, C., Shen, Y., Wang, P., Huang, Q., & Kuo, C. ‐. J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.03356 · Cited by 203
Image Compression With Edge-Based Inpainting
Liu, D., Sun, X., Wu, F., Li, S., & Zhang, Y.-Q. (2007) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2007.903663 · Cited by 192
Structured Learning and Prediction in Computer Vision
Nowozin, S., & Lampert, C. H. (2011) · Foundations and Trends in Computer Graphics and Vision · doi:10.1561/0600000033 · Cited by 178
Eye In-painting with Exemplar Generative Adversarial Networks
Dolhansky, B., & Ferrer, C. C. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00824 · Cited by 155
Is Generator Conditioning Causally Related to GAN Performance?Preprint
Odena, A., Buckman, J., Olsson, C., Brown, T. B., Olah, C., Raffel, C., & Goodfellow, I. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.08768 · Cited by 138
Unsupervised Learning of Edges
Li, Y., Paluri, M., Rehg, J. M., & Dollár, P. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.179 · Cited by 107
Bertalmı́o, M., Sapiro, G., Caselles, V., & Ballester, C. (2005) · doi:10.21236/ada437378 · Cited by 66
The Unreasonable Effectiveness of Texture Transfer for Single Image Super-Resolution
Gondal, M. W., Schölkopf, B., & Hirsch, M. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-11021-5_6 · Cited by 52
Cited by
Palette: Image-to-Image Diffusion Models
Saharia, C., Chan, W., Chang, H., Lee, C., Ho, J., Salimans, T., Fleet, D., & Norouzi, M. (2022) · Special Interest Group on Computer Graphics and Interactive Techniques Conference Proceedings · doi:10.1145/3528233.3530757 · Cited by 1,717
RePaint: Inpainting using Denoising Diffusion Probabilistic Models
Lugmayr, A., Danelljan, M., Romero, A., Yu, F., Timofte, R., & Van Gool, L. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01117 · Cited by 1,778
Recurrent Feature Reasoning for Image Inpainting
Li, J., Wang, N., Zhang, L., Du, B., & Tao, D. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00778 · Cited by 484
M2TR: Multi-modal Multi-scale Transformers for Deepfake Detection
Wang, J., Wu, Z., Ouyang, W., Han, X., Chen, J., Jiang, Y.-G., & Li, S.-N. (2022) · Proceedings of the 2022 International Conference on Multimedia Retrieval · doi:10.1145/3512527.3531415 · Cited by 368
Image Inpainting via Conditional Texture and Structure Dual Generation
Guo, X., Yang, H., & Huang, D. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01387 · Cited by 335
Metadata
- Authors
- Nazeri, K., Ng, E., Joseph, T., Qureshi, F., & Ebrahimi, M.
- Venue
- 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)
- Publisher
- IEEE
- Published
- 2019-10
- Pages
- 3265-3274
- OpenAlex
- W2991377405
- Cited by
- 562
- References
- 60
- FWCI
- 18.47
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.