The DOI matches its Crossref record; 69 of the 69 references it cites were found in a registry or an index.
Resolution-robust Large Mask Inpainting with Fourier Convolutions
Open accessAbstract
Modern image inpainting systems, despite the significant progress, often struggle with large missing areas, complex geometric structures, and high-resolution images. We find that one of the main reasons for that is the lack of an effective receptive field in both the inpainting network and the loss function. To alleviate this issue, we propose a new method called large mask inpainting (LaMa). LaMa is based on i) a new inpainting network architecture that uses fast Fourier convolutions (FFCs), which have the image-wide receptive field; ii) a high receptive field perceptual loss; iii) large training masks, which unlocks the potential of the first two components. Our inpainting network improves the state-of-the-art across a range of datasets and achieves excellent performance even in challenging scenarios, e.g. completion of periodic structures. Our model generalizes surprisingly well to resolutions that are higher than those seen at train time, and achieves this at lower parameter&time costs than the competitive baselines. The code is available at https://github.com/saic-mdal/lama.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
PyTorch: An Imperative Style, High-Performance Deep Learning Library
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al. (2019) · arXiv · doi:10.48550/arxiv.1912.01703 · Cited by 31,473
Image-to-Image Translation with Conditional Adversarial Networks
Isola, P., Zhu, J.-Y., Zhou, T., & Efros, A. A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.632 · Cited by 22,930
The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
Zhang, R., Isola, P., Efros, A. A., Shechtman, E., & Wang, O. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00068 · Cited by 14,623
A Style-Based Generator Architecture for Generative Adversarial Networks
Karras, T., Laine, S., & Aila, T. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00453 · Cited by 11,714
Perceptual Losses for Real-Time Style Transfer and Super-Resolution
Johnson, J., Alahi, A., & Fei-Fei, L. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46475-6_43 · Cited by 11,283
GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., & Hochreiter, S. (2017) · arXiv · doi:10.48550/arxiv.1706.08500 · Cited by 8,365
Multi-scale context aggregation by dilated convolutions
Yu, F., & Koltun, V. (2015) · arXiv · doi:10.48550/arxiv.1511.07122 · Cited by 7,738
Context Encoders: Feature Learning by Inpainting
Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T., & Efros, A. A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.278 · Cited by 5,691
Progressive Growing of GANs for Improved Quality, Stability, and Variation
Karras, T., Aila, T., Laine, S., & Lehtinen, J. (2017) · arXiv · doi:10.48550/arxiv.1710.10196 · Cited by 4,727
Generative Adversarial Networks
Goodfellow, I. J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1406.2661 · Cited by 4,477
High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs
Wang, T.-C., Liu, M.-Y., Zhu, J.-Y., Tao, A., Kautz, J., & Catanzaro, B. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00917 · Cited by 4,332
Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2014) · arXiv · doi:10.48550/arxiv.1412.7062 · Cited by 4,213
Places: A 10 Million Image Database for Scene Recognition
Zhou, B., Lapedriza, A., Khosla, A., Oliva, A., & Torralba, A. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2723009 · Cited by 4,042
Generative Image Inpainting with Contextual Attention
Yu, J., Lin, Z., Yang, J., Shen, X., Lu, X., & Huang, T. S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00577 · Cited by 2,527
Taming Transformers for High-Resolution Image Synthesis
Esser, P., Rombach, R., & Ommer, B. (2020) · arXiv · doi:10.48550/arxiv.2012.09841 · Cited by 2,514
Taming Transformers for High-Resolution Image Synthesis
Esser, P., Rombach, R., & Ommer, B. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01268 · Cited by 2,514
Image Inpainting for Irregular Holes Using Partial Convolutions
Liu, G., Reda, F. A., Shih, K. J., Wang, T.-C., Tao, A., & Catanzaro, B. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01252-6_6 · Cited by 2,419 · cited 5×
“The subject has been studied in pre-deep learning era [1, 5, 13], and the progress accelerated in recent years through the use of deep and wide neural networks [26, 30, 25] and adversarial learning [34, 18, 56, 44, 57, 32, 54, 61].”
BackgroundMethodInfluentialGenerative Adversarial Networks
(2022) · The Science of Deep Learning · doi:10.1017/9781108891530.013 · Cited by 2,402
Globally and locally consistent image completion
Iizuka, S., Simo-Serra, E., & Ishikawa, H. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073659 · Cited by 2,203
Free-Form Image Inpainting With Gated Convolution
Yu, J., Lin, Z., Yang, J., Shen, X., Lu, X., & Huang, T. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00457 · Cited by 1,928
Understanding the Effective Receptive Field in Deep Convolutional Neural Networks
Luo, W., Li, Y., Urtasun, R., & Zemel, R. (2017) · arXiv · doi:10.48550/arxiv.1701.04128 · Cited by 1,575
Geirhos, R., Rubisch, P., Michaelis, C., Bethge, M., Wichmann, F. A., & Brendel, W. (2018) · arXiv · doi:10.48550/arxiv.1811.12231 · Cited by 1,490
Semantic Image Inpainting with Deep Generative Models
Yeh, R. A., Chen, C., Lim, T. Y., Schwing, A. G., Hasegawa-Johnson, M., & Do, M. N. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.728 · Cited by 1,206
High-Resolution Image Inpainting Using Multi-scale Neural Patch Synthesis
Yang, C., Lu, X., Lin, Z., Shechtman, E., Wang, O., & Li, H. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.434 · Cited by 877
Object removal by exemplar-based inpainting
Criminisi, A., Perez, P., & Toyama, K. (2003) · 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings. · doi:10.1109/cvpr.2003.1211538 · Cited by 847
Which Training Methods for GANs do actually Converge?
Mescheder, L., Geiger, A., & Nowozin, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.04406 · Cited by 768
Scene completion using millions of photographs
Hays, J., & Efros, A. A. (2007) · ACM SIGGRAPH 2007 papers · doi:10.1145/1275808.1276382 · Cited by 687
Ross, A., & Doshi-Velez, F. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11504 · Cited by 589
Brigham, E. O., & Morrow, R. E. (1967) · IEEE Spectrum · doi:10.1109/mspec.1967.5217220 · Cited by 588
EdgeConnect: Generative Image Inpainting with Adversarial Edge Learning
Nazeri, K., Ng, E., Joseph, T., Qureshi, F. Z., & Ebrahimi, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.00212 · Cited by 538
Scene completion using millions of photographs
Hays, J., & Efros, A. A. (2007) · ACM Transactions on Graphics · doi:10.1145/1276377.1276382 · Cited by 534
Zheng, C., Cham, T.-J., & Cai, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00153 · Cited by 522
FNet: Mixing Tokens with Fourier Transforms
Lee-Thorp, J., Ainslie, J., Eckstein, I., & Ontanon, S. (2022) · Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2022.naacl-main.319 · Cited by 519
Shift-Net: Image Inpainting via Deep Feature Rearrangement
Yan, Z., Li, X., Li, M., Zuo, W., & Shan, S. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01264-9_1 · Cited by 511
Learning Pyramid-Context Encoder Network for High-Quality Image Inpainting
Zeng, Y., Fu, J., Chao, H., & Guo, B. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00158 · Cited by 500
Recurrent Feature Reasoning for Image Inpainting
Li, J., Wang, N., Zhang, L., Du, B., & Tao, D. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00778 · Cited by 484
Global Filter Networks for Image ClassificationPreprint
Rao, Y., Zhao, W., Zhu, Z., Lu, J., & Zhou, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.00645 · Cited by 418
GANs Trained by a Two Time-Scale Update Rule Converge to a Nash EquilibriumPreprint
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., Klambauer, G., & Hochreiter, S. (2017) · arXiv (Cornell University) · Cited by 407
Foreground-Aware Image Inpainting
Xiong, W., Yu, J., Lin, Z., Yang, J., Lu, X., Barnes, C., & Luo, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00599 · Cited by 402
StructureFlow: Image Inpainting via Structure-Aware Appearance Flow
Ren, Y., Yu, X., Zhang, R., Li, T. H., Liu, S., & Li, G. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00027 · Cited by 396
Coherent Semantic Attention for Image Inpainting
Liu, H., Jiang, B., Xiao, Y., & Yang, C. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00427 · Cited by 383
Generative Adversarial Networks
Salvaris, M., Dean, D., & Tok, W. H. (2018) · Deep Learning with Azure · doi:10.1007/978-1-4842-3679-6_8 · Cited by 375
Contextual Residual Aggregation for Ultra High-Resolution Image Inpainting
Yi, Z., Tang, Q., Azizi, S., Jang, D., & Xu, Z. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00753 · Cited by 374
Chi, L., Jiang, B., & Mu, Y. (2020) · neural information processing systems · Cited by 374
Image Inpainting via Generative Multi-column Convolutional Neural NetworksPreprint
Wang, Y., Tao, X., Qi, X., Shen, X., & Jia, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.08771 · Cited by 295
Rethinking Image Inpainting via a Mutual Encoder-Decoder with Feature Equalizations
Liu, H., Jiang, B., Song, Y., Huang, W., & Yang, C. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58536-5_43 · Cited by 290
Image Inpainting With Learnable Bidirectional Attention Maps
Xie, C., Liu, S., Li, C., Cheng, M.-M., Zuo, W., Liu, X., Wen, S., & Ding, E. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00895 · Cited by 282
Simultaneous structure and texture image inpainting
Bertalmı́o, M., Vese, L. A., Sapiro, G., et al. (2003) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2003.1211536 · Cited by 251
Contextual-Based Image Inpainting: Infer, Match, and Translate
Song, Y., Yang, C., Lin, Z., Liu, X., Huang, Q., Li, H., & Kuo, C. ‐. J. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01216-8_1 · Cited by 250
Improving generalization performance using double backpropagation
Drucker, H., & LeCun, Y. (1992) · IEEE Transactions on Neural Networks · doi:10.1109/72.165600 · Cited by 238
Large Scale Image Completion via Co-Modulated Generative Adversarial Networks
Zhao, S., Cui, J., Sheng, Y., Dong, Y., Liang, X., Chang, E. I., & Xu, Y. (2021) · arXiv · doi:10.48550/arxiv.2103.10428 · Cited by 224
Region Normalization for Image Inpainting
Yu, T., Guo, Z., Jin, X., Wu, S., Chen, Z., Li, W., Zhang, Z., & Liu, S. (2020) · AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) · doi:10.1609/aaai.v34i07.6967 · Cited by 198
High-Resolution Image Inpainting with Iterative Confidence Feedback and Guided Upsampling
Zeng, Y., Lin, Z., Yang, J., Zhang, J., Shechtman, E., & Lu, H. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58529-7_1 · Cited by 175
SPG-Net: Segmentation Prediction and Guidance Network for Image InpaintingPreprint
Song, Y., Yang, C., Shen, Y., Wang, P., Huang, Q., & Kuo, C. ‐. J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.03356 · Cited by 168
Image Inpainting by End-to-End Cascaded Refinement With Mask Awareness
Zhu, M., He, D., Li, X., Li, C., Li, F., Liu, X., Ding, E., & Zhang, Z. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3076310 · Cited by 148
Semantic Image Inpainting with Progressive Generative Networks
Zhang, H., Hu, Z., Luo, C., Zuo, W., & Wang, M. (2018) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3240508.3240625 · Cited by 142
PEPSI : Fast Image Inpainting With Parallel Decoding Network
Sagong, M.-C., Shin, Y.-G., Kim, S., Park, S., & Ko, S.-J. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01162 · Cited by 138
Learning to Incorporate Structure Knowledge for Image Inpainting
Yang, J., Qi, Z., & Shi, Y. G. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6951 · Cited by 134
PEPSI++: Fast and Lightweight Network for Image Inpainting
Shin, Y.-G., Sagong, M.-C., Yeo, Y.-J., Kim, S., & Ko, S.-J. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2978501 · Cited by 123
Progressive Image Inpainting with Full-Resolution Residual Network
Guo, Z., Chen, Z., Yu, T., Chen, J., & Liu, S. (2019) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3343031.3351022 · Cited by 122
Image Fine-grained InpaintingPreprint
Zheng, H., Li, J., Wang, X., & Gao, X. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.02609 · Cited by 72
Regionwise Generative Adversarial Image Inpainting for Large Missing Areas
Ma, Y., Liu, X., Bai, S., Wang, L., Liu, A., Tao, D., & Hancock, E. R. (2022) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2022.3194149 · Cited by 60
SDC – Stacked Dilated Convolution: A Unified Descriptor Network for Dense Matching Tasks
Schuster, R., Wasenmüller, O., Unger, C., & Stricker, D. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00266 · Cited by 53
Image Inpainting with Learnable Feature Imputation
Hukkelås, H., Lindseth, F., & Mester, R. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-71278-5_28 · Cited by 16
Research works printed but not linked: 2
- yadan, 2019, A framework for elegantly configuring complex applications
- falcon, 2019, Pytorch lightning, Github, 3
Cited by
High-Resolution Image Synthesis with Latent Diffusion Models
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01042 · Cited by 20,302
RePaint: Inpainting using Denoising Diffusion Probabilistic Models
Lugmayr, A., Danelljan, M., Romero, A., Yu, F., Timofte, R., & Van Gool, L. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01117 · Cited by 1,778
DiffIR: Efficient Diffusion Model for Image Restoration
Xia, B., Zhang, Y., Wang, S., Wang, Y., Wu, X., Tian, Y., Yang, W., & Van Gool, L. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01204 · Cited by 412
The Stable Signature: Rooting Watermarks in Latent Diffusion Models
Fernandez, P., Couairon, G., Jégou, H., Douze, M., & Furon, T. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.02053 · Cited by 251
Towards efficient and scale-robust ultra-high-definition image demoiréing
Yu, X., Dai, P., Li, W., Ma, L., Shen, J., Li, J., & Qi, X. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-19797-0_37 · Cited by 93
Metadata
- Authors
- Suvorov, R., Logacheva, E., Mashikhin, A., Remizova, A., Ashukha, A., Silvestrov, A., Kong, N., Goka, H., Park, K., & Lempitsky, V.
- Venue
- 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)
- Publisher
- IEEE
- Published
- 2022-01
- Pages
- 3172-3182
- OpenAlex
- W3199003182
- Cited by
- 1,123
- References
- 69
- FWCI
- 85.57
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
