The DOI matches its Crossref record; 69 of the 69 references it cites were found in a registry or an index.
MAT: Mask-Aware Transformer for Large Hole Image Inpainting
Open accessAbstract
Recent studies have shown the importance of modeling long-range interactions in the inpainting problem. To achieve this goal, existing approaches exploit either standalone attention techniques or transformers, but usually under a low resolution in consideration of computational cost. In this paper, we present a novel transformer-based model for large hole inpainting, which unifies the merits of transformers and convolutions to efficiently process high-resolution images. We carefully design each component of our framework to guarantee the high fidelity and diversity of recovered images. Specifically, we customize an inpainting-oriented transformer block, where the attention module aggregates non-local information only from partial valid tokens, indicated by a dynamic mask. Extensive experiments demonstrate the state-of-the-art performance of the new model on multiple benchmark datasets. Code is released at https://github.com/fenglinglwb/MAT.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Exploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534
Labaca-Castro, R. (2023) · Machine Learning under Malware Attack · doi:10.1007/978-3-658-40442-0_9 · Cited by 19,272
The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
Zhang, R., Isola, P., Efros, A. A., Shechtman, E., & Wang, O. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00068 · Cited by 14,623
Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network
Ledig, C., Theis, L., Huszar, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., Wang, Z., et al. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.19 · Cited by 12,696
A Style-Based Generator Architecture for Generative Adversarial Networks
Karras, T., Laine, S., & Aila, T. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00453 · Cited by 11,714
Wang, X., Girshick, R., Gupta, A., & He, K. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00813 · Cited by 11,596
Perceptual Losses for Real-Time Style Transfer and Super-Resolution
Johnson, J., Alahi, A., & Fei-Fei, L. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46475-6_43 · Cited by 11,283
GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., & Hochreiter, S. (2017) · arXiv · doi:10.48550/arxiv.1706.08500 · Cited by 8,365
A Non-Local Algorithm for Image Denoising
Buades, A., Coll, B., & Morel, J.-M. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.38 · Cited by 7,616
Analyzing and Improving the Image Quality of StyleGAN
Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J., & Aila, T. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00813 · Cited by 6,332
Context Encoders: Feature Learning by Inpainting
Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T., & Efros, A. A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.278 · Cited by 5,691
SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers
Xie, E., Wang, W., Yu, Z., Anandkumar, A., Alvarez, J. M., & Luo, P. (2021) · arXiv · doi:10.48550/arxiv.2105.15203 · Cited by 5,090
Places: A 10 Million Image Database for Scene Recognition
Zhou, B., Lapedriza, A., Khosla, A., Oliva, A., & Torralba, A. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2723009 · Cited by 4,042
Region Filling and Object Removal by Exemplar-Based Image Inpainting
Criminisi, A., Perez, P., & Toyama, K. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2004.833105 · Cited by 3,258
PatchMatch: a randomized correspondence algorithm for structural image editing
Barnes, C., Shechtman, E., Finkelstein, A., & Goldman, D. B. (2009) · ACM Transactions on Graphics · doi:10.1145/1531326.1531330 · Cited by 2,880
Generative Image Inpainting with Contextual Attention
Yu, J., Lin, Z., Yang, J., Shen, X., Lu, X., & Huang, T. S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00577 · Cited by 2,527
Self-citationCvT: Introducing Convolutions to Vision Transformers
Wu, H., Xiao, B., Codella, N., Liu, M., Dai, X., Yuan, L., & Zhang, L. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00009 · Cited by 2,444
Image Inpainting for Irregular Holes Using Partial Convolutions
Liu, G., Reda, F. A., Shih, K. J., Wang, T.-C., Tao, A., & Catanzaro, B. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01252-6_6 · Cited by 2,419 · cited 1×
“Besides, more sophisticated networks or learning strategies are proposed to generate high-quality images, including global and local discrimination [22], contextual attention [35, 61, 65, 66], partial [33] and gated [67] convolution, etc.”
BackgroundGlobally and locally consistent image completion
Iizuka, S., Simo-Serra, E., & Ishikawa, H. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073659 · Cited by 2,203
Free-Form Image Inpainting With Gated Convolution
Yu, J., Lin, Z., Yang, J., Shen, X., Lu, X., & Huang, T. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00457 · Cited by 1,928
Self-citationNon-local sparse models for image restoration
Mairal, J., Bach, F., Ponce, J., Sapiro, G., & Zisserman, A. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459452 · Cited by 1,666
Berman, D., Treibitz, T., & Avidan, S. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.185 · Cited by 1,642
Resolution-robust Large Mask Inpainting with Fourier Convolutions
Suvorov, R., Logacheva, E., Mashikhin, A., Remizova, A., Ashukha, A., Silvestrov, A., Kong, N., Goka, H., Park, K., & Lempitsky, V. (2022) · 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv51458.2022.00323 · Cited by 1,123
EnhanceNet: Single Image Super-Resolution Through Automated Texture Synthesis
Sajjadi, M. S. M., Scholkopf, B., & Hirsch, M. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.481 · Cited by 1,058
Filling-in by joint interpolation of vector fields and gray levels
Ballester, C., Bertalmio, M., Caselles, V., Sapiro, G., & Verdera, J. (2001) · IEEE Transactions on Image Processing · doi:10.1109/83.935036 · Cited by 962
Object removal by exemplar-based inpainting
Criminisi, A., Perez, P., & Toyama, K. (2003) · 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings. · doi:10.1109/cvpr.2003.1211538 · Cited by 847
Which Training Methods for GANs do actually Converge?
Mescheder, L., Geiger, A., & Nowozin, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.04406 · Cited by 768
Ross, A., & Doshi-Velez, F. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11504 · Cited by 589
EdgeConnect: Generative Image Inpainting with Adversarial Edge Learning
Nazeri, K., Ng, E., Joseph, T., Qureshi, F. Z., & Ebrahimi, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.00212 · Cited by 538
Scene completion using millions of photographs
Hays, J., & Efros, A. A. (2007) · ACM Transactions on Graphics · doi:10.1145/1276377.1276382 · Cited by 534
Zheng, C., Cham, T.-J., & Cai, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00153 · Cited by 522
Shift-Net: Image Inpainting via Deep Feature Rearrangement
Yan, Z., Li, X., Li, M., Zuo, W., & Shan, S. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01264-9_1 · Cited by 511
Learning Pyramid-Context Encoder Network for High-Quality Image Inpainting
Zeng, Y., Fu, J., Chao, H., & Guo, B. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00158 · Cited by 500
Recurrent Feature Reasoning for Image Inpainting
Li, J., Wang, N., Zhang, L., Du, B., & Tao, D. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00778 · Cited by 484
StructureFlow: Image Inpainting via Structure-Aware Appearance Flow
Ren, Y., Yu, X., Zhang, R., Li, T. H., Liu, S., & Li, G. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00027 · Cited by 396
Coherent Semantic Attention for Image Inpainting
Liu, H., Jiang, B., Xiao, Y., & Yang, C. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00427 · Cited by 383
Contextual Residual Aggregation for Ultra High-Resolution Image Inpainting
Yi, Z., Tang, Q., Azizi, S., Jang, D., & Xu, Z. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00753 · Cited by 374
SC-FEGAN: Face Editing Generative Adversarial Network With User’s Sketch and Color
Jo, Y., & Park, J. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00183 · Cited by 346
Early Convolutions Help Transformers See Better
Xiao, T., Singh, M., Mintun, E., Darrell, T. J., Dollár, P., & Girshick, R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.14881 · Cited by 339
Inpainting and Zooming Using Sparse Representations
Fadili, M. J., Starck, J.-L., & Murtagh, F. (2008) · The Computer Journal · doi:10.1093/comjnl/bxm055 · Cited by 333
Learning how to inpaint from global image statistics
Levin, Zomet, & Weiss (2003) · Proceedings Ninth IEEE International Conference on Computer Vision · doi:10.1109/iccv.2003.1238360 · Cited by 324
High-Fidelity Pluralistic Image Completion with Transformers
Wan, Z., Zhang, J., Chen, D., & Liao, J. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00465 · Cited by 299
Rethinking Image Inpainting via a Mutual Encoder-Decoder with Feature Equalizations
Liu, H., Jiang, B., Song, Y., Huang, W., & Yang, C. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58536-5_43 · Cited by 290
Image Inpainting With Learnable Bidirectional Attention Maps
Xie, C., Liu, S., Li, C., Cheng, M.-M., Zuo, W., Liu, X., Wen, S., & Ding, E. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00895 · Cited by 282
Bringing Old Photos Back to Life
Wan, Z., Zhang, B., Chen, D., Zhang, P., Chen, D., Liao, J., & Wen, F. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00282 · Cited by 231
Image Inpainting via Generative Multi-column Convolutional Neural Networks
Wang, Y., Tao, X., Qi, X., Shen, X., & Jia, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.08771 · Cited by 228
Self-citationLarge Scale Image Completion via Co-Modulated Generative Adversarial Networks
Zhao, S., Cui, J., Sheng, Y., Dong, Y., Liang, X., Chang, E. I., & Xu, Y. (2021) · arXiv · doi:10.48550/arxiv.2103.10428 · Cited by 224
Diverse Image Inpainting with Bidirectional and Autoregressive Transformers
Yu, Y., Zhan, F., Wu, R., Pan, J., Cui, K., Lu, S., Ma, F., Xie, X., & Miao, C. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3475436 · Cited by 207
SPG-Net: Segmentation Prediction and Guidance Network for Image InpaintingPreprint
Song, Y., Yang, C., Shen, Y., Wang, P., Huang, Q., & Kuo, C. ‐. J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.03356 · Cited by 203
UCTGAN: Diverse Image Inpainting Based on Unsupervised Cross-Space Translation
Zhao, L., Mo, Q., Lin, S., Wang, Z., Zuo, Z., Chen, H., Xing, W., & Lu, D. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00578 · Cited by 198
Image Inpainting Using Nonlocal Texture Matching and Nonlinear Filtering
Ding, D., Ram, S., & Rodriguez, J. J. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2880681 · Cited by 169
Image Inpainting by End-to-End Cascaded Refinement With Mask Awareness
Zhu, M., He, D., Li, X., Li, C., Li, F., Liu, X., Ding, E., & Zhang, Z. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3076310 · Cited by 167
High-Resolution Image Inpainting with Iterative Confidence Feedback and Guided Upsampling
Zeng, Y., Lin, Z., Yang, J., Zhang, J., Shechtman, E., & Lu, H. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58529-7_1 · Cited by 160
Self-citationSemantic Image Inpainting with Progressive Generative Networks
Zhang, H., Hu, Z., Luo, C., Zuo, W., & Wang, M. (2018) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3240508.3240625 · Cited by 156
Progressive Image Inpainting with Full-Resolution Residual Network
Guo, Z., Chen, Z., Yu, T., Chen, J., & Liu, S. (2019) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3343031.3351022 · Cited by 139
Onion-Peel Networks for Deep Video Completion
Oh, S. W., Lee, S.-H., Lee, J., & Kim, S. J. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00450 · Cited by 136
Examplar-based inpainting based on local geometry
Meur, O. L., Gautier, J., & Guillemot, C. M. (2011) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2011.6116441 · Cited by 133
On Self Modulation for Generative Adversarial NetworksPreprint
Chen, T., Lučić, M., Houlsby, N., & Gelly, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.01365 · Cited by 91
A Combined PDE and Texture Synthesis Approach to Inpainting
Grossauer, H. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-24671-8_17 · Cited by 88
Strong-continuation, contrast-invariant inpainting with a third-order optimal PDE
Bertalmı́o, M. (2006) · IEEE Transactions on Image Processing · doi:10.1109/tip.2006.877067 · Cited by 87
Weakly- and Self-Supervised Learning for Content-Aware Deep Image Retargeting
Cho, D., Park, J., Oh, T.-H., Tai, Y., & Kweon, I. S. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.488 · Cited by 86
Regionwise Generative Adversarial Image Inpainting for Large Missing Areas
Ma, Y., Liu, X., Bai, S., Wang, L., Liu, A., Tao, D., & Hancock, E. R. (2022) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2022.3194149 · Cited by 71
Bertalmı́o, M., Sapiro, G., Caselles, V., & Ballester, C. (2005) · doi:10.21236/ada437378 · Cited by 66
Multiple Pyramids Based Image Inpainting Using Local Patch Statistics and Steering Kernel Feature
Ghorai, M., Samanta, S., Mandal, S., & Chanda, B. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2920528 · Cited by 44
Cited by
DiffIR: Efficient Diffusion Model for Image Restoration
Xia, B., Zhang, Y., Wang, S., Wang, Y., Wu, X., Tian, Y., Yang, W., & Van Gool, L. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01204 · Cited by 412
Self-citationRahman, M. A., Paul, B., Sarker, N. H., Hakim, Z. I. A., & Fattah, S. A. (2023) · 2023 IEEE International Conference on Image Processing (ICIP) · doi:10.1109/icip49359.2023.10222083 · Cited by 34
Metadata
- Authors
- Li, W., Lin, Z., Zhou, K., Qi, L., Wang, Y., & Jia, J.
- Venue
- 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2022-06
- Pages
- 10748-10758
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W4312383858
- Cited by
- 449
- References
- 69
- FWCI
- 34.57
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.