The DOI matches its Crossref record; 82 of the 82 references it cites were found in a registry or an index.
Image Inpainting: A Review
Open accessAbstract
Although image inpainting, or the art of repairing the old and deteriorated images, has been around for many years, it has recently gained even more popularity, because of the recent development in image processing techniques. With the improvement of image processing tools and the flexibility of digital image editing, automatic image inpainting has found important applications in computer vision and has also become an important and challenging topic of research in image processing. This paper reviews the existing image inpainting approaches, that were classified into three subcategories, sequential-based, CNN-based, and GAN-based methods. In addition, for each category, a list of methods for different types of distortion on images are presented. Furthermore, the paper also presents available datasets. Last but not least, we present the results of real evaluations of the three categories of image inpainting methods performed on the used datasets, for different types of image distortion. We also present the evaluations metrics and discuss the performance of these methods in terms of these metrics. This overview can be used as a reference for image inpainting researchers, and it can also facilitate the comparison of the methods as well as the datasets used. The main contribution of this paper is the presentation of the three categories of image inpainting methods along with a list of available datasets that the researchers can use to evaluate their proposed methodology against.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
The Cityscapes Dataset for Semantic Urban Scene Understanding
Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., & Schiele, B. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.350 · Cited by 12,433 · cited 1×
“Cityscapes dataset (12)[69] is large scale dataset of stereo videos of street scene of 50 cities.”
BackgroundMartin, D., Fowlkes, C., Tal, D., & Malik, J. (2002) · Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001 · doi:10.1109/iccv.2001.937655 · Cited by 8,216
Context Encoders: Feature Learning by Inpainting
Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T., & Efros, A. A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.278 · Cited by 5,691
Places: A 10 Million Image Database for Scene Recognition
Zhou, B., Lapedriza, A., Khosla, A., Oliva, A., & Torralba, A. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2723009 · Cited by 4,042
3D Object Representations for Fine-Grained Categorization
Krause, J., Stark, M., Deng, J., & Fei-Fei, L. (2013) · 2013 IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2013.77 · Cited by 3,496
ICDAR 2013 Robust Reading Competition
Karatzas, D., Shafait, F., Uchida, S., Iwamura, M., Bigorda, L. G. I., Mestre, S. R., Mas, J., Mota, D. F., Almazan, J. A., & de las Heras, L. P. (2013) · 2013 12th International Conference on Document Analysis and Recognition · doi:10.1109/icdar.2013.221 · Cited by 1,360
High-Resolution Stereo Datasets with Subpixel-Accurate Ground Truth
Scharstein, D., Hirschmüller, H., Kitajima, Y., Krathwohl, G., Nešić, N., Wang, X., & Westling, P. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-11752-2_3 · Cited by 1,305
Semantic Image Inpainting with Deep Generative Models
Yeh, R. A., Chen, C., Lim, T. Y., Schwing, A. G., Hasegawa-Johnson, M., & Do, M. N. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.728 · Cited by 1,206
Evaluation of Cost Functions for Stereo Matching
Hirschmuller, H., & Scharstein, D. (2007) · 2007 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2007.383248 · Cited by 1,122
Multimodal Deep Autoencoder for Human Pose Recovery
Hong, C., Yu, J., Wan, J., Tao, D., & Wang, M. (2015) · IEEE Transactions on Image Processing · doi:10.1109/tip.2015.2487860 · Cited by 568
Shift-Net: Image Inpainting via Deep Feature Rearrangement
Yan, Z., Li, X., Li, M., Zuo, W., & Shan, S. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01264-9_1 · Cited by 511
Learning Pyramid-Context Encoder Network for High-Quality Image Inpainting
Zeng, Y., Fu, J., Chao, H., & Guo, B. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00158 · Cited by 500
Hierarchical Deep Click Feature Prediction for Fine-Grained Image Recognition
Yu, J., Tan, M., Zhang, H., Rui, Y., & Tao, D. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2932058 · Cited by 498
Learning to Rank Using User Clicks and Visual Features for Image Retrieval
Yu, J., Tao, D., Wang, M., & Rui, Y. (2015) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2336697 · Cited by 445
Foreground-Aware Image Inpainting
Xiong, W., Yu, J., Lin, Z., Yang, J., Lu, X., Barnes, C., & Luo, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00599 · Cited by 402
Coherent Semantic Attention for Image Inpainting
Liu, H., Jiang, B., Xiao, Y., & Yang, C. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00427 · Cited by 383
Perceptual Adversarial Networks for Image-to-Image Transformation
Wang, C., Xu, C., Wang, C., & Tao, D. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2836316 · Cited by 381
Deep Multimodal Distance Metric Learning Using Click Constraints for Image Ranking
Yu, J., Yang, X., Gao, F., & Tao, D. (2017) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2016.2591583 · Cited by 364
Spatial Pyramid-Enhanced NetVLAD With Weighted Triplet Loss for Place Recognition
Yu, J., Zhu, C., Zhang, J., Huang, Q., & Tao, D. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2908982 · Cited by 352
iPrivacy: Image Privacy Protection by Identifying Sensitive Objects via Deep Multi-Task Learning
Yu, J., Zhang, B., Kuang, Z., Lin, D., & Fan, J. (2017) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2016.2636090 · Cited by 349
Multimodal Face-Pose Estimation With Multitask Manifold Deep Learning
Hong, C., Yu, J., Zhang, J., Jin, X., & Lee, K.-H. (2019) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2018.2884211 · Cited by 313
Local Deep-Feature Alignment for Unsupervised Dimension Reduction
Zhang, J., Yu, J., & Tao, D. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2804218 · Cited by 210
Context-Aware Patch-Based Image Inpainting Using Markov Random Field Modeling
Ruzic, T., & Pizurica, A. (2015) · IEEE Transactions on Image Processing · doi:10.1109/tip.2014.2372479 · Cited by 186
A deep learning approach to patch-based image inpainting forensics
Zhu, X., Qian, Y., Zhao, X., Sun, B., & Sun, Y. (2018) · Signal Processing Image Communication · doi:10.1016/j.image.2018.05.015 · Cited by 182
Annihilating Filter-Based Low-Rank Hankel Matrix Approach for Image Inpainting
Jin, K. H., & Ye, J. C. (2015) · IEEE Transactions on Image Processing · doi:10.1109/tip.2015.2446943 · Cited by 177
Image Inpainting Using Nonlocal Texture Matching and Nonlinear Filtering
Ding, D., Ram, S., & Rodriguez, J. J. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2880681 · Cited by 169
A Novel Joint Data-Hiding and Compression Scheme Based on SMVQ and Image Inpainting
Qin, C., Chang, C.-C., & Chiu, Y.-P. (2014) · IEEE Transactions on Image Processing · doi:10.1109/tip.2013.2260760 · Cited by 166
Localization of Diffusion-Based Inpainting in Digital Images
Li, H., Luo, W., & Huang, J. (2017) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2017.2730822 · Cited by 163
PEPSI : Fast Image Inpainting With Parallel Decoding Network
Sagong, M.-C., Shin, Y.-G., Kim, S., Park, S., & Ko, S.-J. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01162 · Cited by 162
Patch-Based Image Inpainting via Two-Stage Low Rank Approximation
Guo, Q., Gao, S., Zhang, X., et al. (2017) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2017.2702738 · Cited by 159
What makes Paris look like Paris?
Doersch, C., Singh, S., Gupta, A., Šivic, J., & Efros, A. A. (2015) · Communications of the ACM · doi:10.1145/2830541 · Cited by 153
Progressive Image Inpainting with Full-Resolution Residual Network
Guo, Z., Chen, Z., Yu, T., Chen, J., & Liu, S. (2019) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3343031.3351022 · Cited by 139
Blind inpainting using the fully convolutional neural network
Cai, N., Su, Z., Lin, Z., Wang, H., Yang, Z., & Ling, B. W. (2015) · The Visual Computer · doi:10.1007/s00371-015-1190-z · Cited by 132
Depth Image Inpainting: Improving Low Rank Matrix Completion With Low Gradient Regularization
Xue, H., Zhang, S., & Cai, D. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2718183 · Cited by 126
Image Inpainting Based on Fractional-Order Nonlinear Diffusion for Image Reconstruction
Sridevi, G., & Kumar, S. S. (2019) · Circuits Systems and Signal Processing · doi:10.1007/s00034-019-01029-w · Cited by 117
FiNet: Compatible and Diverse Fashion Image Inpainting
Han, X., Wu, Z., Huang, W., Scott, M., & Davis, L. S. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00458 · Cited by 112
Inpainting of Remote Sensing SST Images With Deep Convolutional Generative Adversarial Network
Dong, J., Yin, R., Sun, X., Li, Q., Yang, Y., & Qin, X. (2018) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2018.2870880 · Cited by 111
Diminished Reality Based on Image Inpainting Considering Background Geometry
Kawai, N., Sato, T., & Yokoya, N. (2015) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2015.2462368 · Cited by 105
Adversarial Inpainting of Medical Image Modalities
Armanious, K., Mecky, Y., Gatidis, S., & Yang, B. (2019) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2019.8682677 · Cited by 96
Nakamura, T., Zhu, A., Yanai, ., & Uchida, S. (2017) · IAPR International Conference on Document Analysis and Recognition (ICDAR) · doi:10.1109/icdar.2017.141 · Cited by 86
Structure-Guided Image Inpainting Using Homography Transformation
Liu, J., Yang, S., Fang, Y., et al. (2018) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2018.2831636 · Cited by 76
Coarse-to-Fine Image Inpainting via Region-wise Convolutions and Non-Local Correlation
Ma, Y., Liu, X., Bai, S., Wang, L., He, D., & Liu, A. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/433 · Cited by 73
A robust forgery detection algorithm for object removal by exemplar-based image inpainting
Zhang, D., Liang, Z., Yang, G., Li, Q., Li, L., & Sun, X. (2017) · Multimedia Tools and Applications · doi:10.1007/s11042-017-4829-0 · Cited by 72
Peeking behind objects: Layered depth prediction from a single image
Dhamo, H., Tateno, K., Laina, I., Navab, N., & Tombari, F. (2019) · Pattern Recognition Letters · doi:10.1016/j.patrec.2019.05.007 · Cited by 71
Image inpainting algorithm based on TV model and evolutionary algorithm
Li, K., Wei, Y., Yang, Z., et al. (2014) · Soft Computing · doi:10.1007/s00500-014-1547-7 · Cited by 70
Blind Visual Motif Removal From a Single Image
Hertz, A., Fogel, S., Hanocka, R., Giryes, R., & Cohen‐Or, D. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00702 · Cited by 62
An Improved Method for Semantic Image Inpainting with GANs: Progressive Inpainting
Chen, Y., & Hu, H. (2018) · Neural Processing Letters · doi:10.1007/s11063-018-9877-6 · Cited by 52
Guided Image Inpainting: Replacing an Image Region by Pulling Content From Another Image
Zhao, Y., Price, B. L., Cohen, S. D., & Gurari, D. (2019) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2019.00166 · Cited by 50
Fast algorithm for color texture image inpainting using the non-local CTV model
Duan, J., Pan, Z., Zhang, B., et al. (2015) · Journal of Global Optimization · doi:10.1007/s10898-015-0290-7 · Cited by 48
Wali, S., Zhang, H., Chang, H., & Wu, C. (2019) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2018.12.047 · Cited by 46
A novel patch matching algorithm for exemplar-based image inpainting
Fan, Q., & Zhang, L. (2017) · Multimedia Tools and Applications · doi:10.1007/s11042-017-5077-z · Cited by 45
Multiple Pyramids Based Image Inpainting Using Local Patch Statistics and Steering Kernel Feature
Ghorai, M., Samanta, S., Mandal, S., & Chanda, B. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2920528 · Cited by 44
Image Inpainting For Random Areas Using Dense Context Features
Su, Y.-Z., Liu, T.-J., Liu, K.-H., et al. (2019) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2019.8803450 · Cited by 39
Just-in-Time Reconstruction: Inpainting Sparse Maps Using Single View Depth Predictors as Priors
Weerasekera, C. S., Dharmasiri, T., Garg, R., Drummond, T. W., & Reid, I. D. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8460549 · Cited by 37
Multi-scale semantic image inpainting with residual learning and GAN
Jiao, L., Wu, H., Wang, H., & Bie, R. (2018) · Neurocomputing · doi:10.1016/j.neucom.2018.11.045 · Cited by 36
Image Inpainting Based on Patch-GANs
Yuan, L., Ruan, C., Hu, H., & Chen, D. (2019) · IEEE Access · doi:10.1109/access.2019.2909553 · Cited by 36
Towards an Inpainting Framework for Visual Cultural Heritage
Jboor, N. H., Belhi, A., Al-Ali, A. K., Bouras, A., & Jaoua, A. (2019) · IEEE Jordan International Joint Conference on Electrical Engineering and Information Technology (JEEIT) · doi:10.1109/jeeit.2019.8717470 · Cited by 34
Visual object tracking via the local soft cosine similarity
Moujahid, D., Elharrouss, O., & Tairi, H. (2018) · Pattern Recognition Letters · doi:10.1016/j.patrec.2018.03.026 · Cited by 32
Self-citationJin, X. H., Su, Y., Zou, L., Wang, Y., Jing, P., & Wang, Z. J. (2018) · IEEE Access · doi:10.1109/access.2018.2866089 · Cited by 29
Gradient-based low rank method and its application in image inpainting
Lu, H., Liu, Q., Zhang, M., et al. (2017) · Multimedia Tools and Applications · doi:10.1007/s11042-017-4509-0 · Cited by 28
Mask Optimization for Image Inpainting
Isogawa, M., Mikami, D., Iwai, D., Kimata, H., & Sato, K. (2018) · IEEE Access · doi:10.1109/access.2018.2877401 · Cited by 28
New Inpainting Algorithm Based on Simplified Context Encoders and Multi-Scale Adversarial Network
Wang, H., Jiao, L., Wu, H., et al. (2019) · Procedia Computer Science · doi:10.1016/j.procs.2019.01.250 · Cited by 25
Image Inpainting Based on Generative Adversarial Networks
Liu, H., Lu, G., Bi, X., Yan, J., & Wang, W. (2018) · International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD) · doi:10.1109/fskd.2018.8686914 · Cited by 24
Spatio-temporal consistent depth-image-based rendering using layered depth image and inpainting
Muddala, S. M., Olsson, R., & Sjöström, M. (2016) · EURASIP Journal on Image and Video Processing · doi:10.1186/s13640-016-0109-6 · Cited by 23
Image Inpainting Algorithm Based on Saliency Map and Gray Entropy
Zeng, J., Fu, X., Leng, L., et al. (2018) · Arabian Journal for Science and Engineering · doi:10.1007/s13369-018-3592-5 · Cited by 23
Video Summarization based on Motion Detection for Surveillance Systems
Elharrouss, O., Almaadeed, N., & Al-Maadeed, S. A. (2019) · doi:10.1109/iwcmc.2019.8766541 · Cited by 23
Self-citationExemplar-based image inpainting using svd-based approximation matrix and multi-scale analysis
Alilou, V. K., & Yaghmaee, F. (2016) · Multimedia Tools and Applications · doi:10.1007/s11042-016-3366-6 · Cited by 21
Artist-Net: Decorating the Inferred Content With Unified Style for Image Inpainting
Liao, L., Hu, R., Xiao, J., & Wang, Z. (2019) · IEEE Access · doi:10.1109/access.2019.2905268 · Cited by 21
Generative Image Inpainting with Submanifold Alignment
Li, A., Qi, J., Zhang, R., Ma, X., & Ramamohanarao, K. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/114 · Cited by 21
Domain-based structure-aware image inpainting
Wei, Y., & Liu, S. (2016) · Signal Image and Video Processing · doi:10.1007/s11760-015-0840-y · Cited by 20
Application of MEEMD in post‐processing of dimensionality reduction methods for face recognition
Abbad, A., Elharrouss, O., Abbad, K., & Tairi, H. (2018) · IET Biometrics · doi:10.1049/iet-bmt.2018.5033 · Cited by 20
Self-citationDamaged region filling and evaluation by symmetrical exemplar-based image inpainting for Thangka
Wang, W., & Jia, Y. (2017) · EURASIP Journal on Image and Video Processing · doi:10.1186/s13640-017-0186-1 · Cited by 18
Moving object detection zone using a block‐based background model
Elharrouss, O., Abbad, A., Moujahid, D., et al. (2017) · IET Computer Vision · doi:10.1049/iet-cvi.2017.0136 · Cited by 17
Self-citationHigh-Resolution Image Inpainting through Multiple Deep Networks
Hsu, C., Chen, F., & Wang, G. (2017) · International Conference on Vision, Image and Signal Processing (ICVISP) · doi:10.1109/icvisp.2017.27 · Cited by 17
Boosted GAN with Semantically Interpretable Information for Image Inpainting
Li, A., Qi, J., Zhang, R., & Ramamohanarao, K. (2019) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2019.8851926 · Cited by 16
Semantic object removal with convolutional neural network feature-based inpainting approach
Cai, X., & Song, B. (2018) · Multimedia Systems · doi:10.1007/s00530-018-0585-x · Cited by 13
Semantic Image Inpainting Using Self-Learning Encoder-Decoder and Adversarial Loss
Salem, N. M., Mahdi, H. M. K., & Abbas, H. M. (2018) · International Conference on Computer Engineering & Systems (ICCES) · doi:10.1109/icces.2018.8639258 · Cited by 11
A deep network architecture for image inpainting
Xiang, P., Wang, L., Cheng, J., Zhang, B., & Wu, J.-J. (2017) · 2017 3rd IEEE International Conference on Computer and Communications (ICCC) · doi:10.1109/compcomm.2017.8322859 · Cited by 9
Image Restoration using Prioritized Exemplar Inpainting with Automatic Patch Optimization
Borole, R. P., & Bonde, S. V. (2016) · Journal of The Institution of Engineers (India) Series B · doi:10.1007/s40031-016-0268-y · Cited by 4
Interleaved Zooming Network for Image Inpainting
Liu, S., Guo, Z., Chen, J., et al. (2019) · IEEE International Conference on Multimedia & Expo Workshops (ICMEW) · doi:10.1109/icmew.2019.00131 · Cited by 4
Rate-distortion optimized image compression based on image inpainting
Jiang, W. T. (2014) · Multimedia Tools and Applications · doi:10.1007/s11042-014-2332-4 · Cited by 1
Metadata
- Authors
- Elharrouss, O., Almaadeed, N., Al-Maadeed, S., & Akbari, Y.
- Venue
- Neural Processing Letters
- Published
- 2019-12-06
- Volume
- 51
- Issue
- 2
- Pages
- 2007-2028
- ISSN
- 1370-4621 (ISSN-L)
- 1573-773X
- OpenAlex
- W2992838650
- Cited by
- 341
- References
- 82
- FWCI
- 11.28
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.