The DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
Semantic Image Inpainting with Deep Generative Models
Open accessAbstract
Semantic image inpainting is a challenging task where large missing regions have to be filled based on the available visual data. Existing methods which extract information from only a single image generally produce unsatisfactory results due to the lack of high level context. In this paper, we propose a novel method for semantic image inpainting, which generates the missing content by conditioning on the available data. Given a trained generative model, we search for the closest encoding of the corrupted image in the latent image manifold using our context and prior losses. This encoding is then passed through the generative model to infer the missing content. In our method, inference is possible irrespective of how the missing content is structured, while the state-of-the-art learning based method requires specific information about the holes in the training phase. Experiments on three datasets show that our method successfully predicts information in large missing regions and achieves pixel-level photorealism, significantly outperforming the state-of-the-art methods.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892
Auto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067
Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network
Ledig, C., Theis, L., Huszar, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., Wang, Z., et al. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.19 · Cited by 12,696
Unsupervised representation learning with deep convolutional generative adversarial networks
Radford, A., Metz, L., & Chintala, S. (2015) · arXiv · doi:10.48550/arxiv.1511.06434 · Cited by 12,235
Context Encoders: Feature Learning by Inpainting
Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T., & Efros, A. A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.278 · Cited by 5,691
Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
Simonyan, K., Vedaldi, A., & Zisserman, A. (2013) · arXiv · doi:10.48550/arxiv.1312.6034 · Cited by 5,370
Understanding deep image representations by inverting them
Mahendran, A., & Vedaldi, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299155 · Cited by 1,827
Sparse Representation for Color Image Restoration
Mairal, J., Elad, M., & Sapiro, G. (2008) · IEEE Transactions on Image Processing · doi:10.1109/tip.2007.911828 · Cited by 1,716
Cited by
Image Inpainting for Irregular Holes Using Partial Convolutions
Liu, G., Reda, F. A., Shih, K. J., Wang, T.-C., Tao, A., & Catanzaro, B. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01252-6_6 · Cited by 2,419
Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery
Schlegl, T., Seeböck, P., Waldstein, S. M., Schmidt-Erfurth, U., & Langs, G. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-59050-9_12 · Cited by 2,627
A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications
Gui, J., Sun, Z., Wen, Y., Tao, D., & Ye, J. (2023) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3130191 · Cited by 1,231
Resolution-robust Large Mask Inpainting with Fourier Convolutions
Suvorov, R., Logacheva, E., Mashikhin, A., Remizova, A., Ashukha, A., Silvestrov, A., Kong, N., Goka, H., Park, K., & Lempitsky, V. (2022) · 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv51458.2022.00323 · Cited by 1,123
Pros and cons of GAN evaluation measures
Borji, A. (2019) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2018.10.009 · Cited by 963
Occlusion Aware Facial Expression Recognition Using CNN With Attention Mechanism
Li, Y., Zeng, J., Shan, S., & Chen, X. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2886767 · Cited by 916
Null-text Inversion for Editing Real Images using Guided Diffusion Models
Mokady, R., Hertz, A., Aberman, K., Pritch, Y., & Cohen-Or, D. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00585 · Cited by 747
FSGAN: Subject Agnostic Face Swapping and Reenactment
Nirkin, Y., Keller, Y., & Hassner, T. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00728 · Cited by 706
Loyola-Gonzalez, O. (2019) · IEEE Access · doi:10.1109/access.2019.2949286 · Cited by 559
Scaffolding protein functional sites using deep learning
Wang, J., Lisanza, S., Juergens, D., Tischer, D., Watson, J. L., Castro, K. M., Ragotte, R., Saragovi, A., Milles, L. F., Baek, M., et al. (2022) · Science · doi:10.1126/science.abn2100 · Cited by 487
Durall, R., Keuper, M., & Keuper, J. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00791 · Cited by 26
LOGAN: Membership Inference Attacks Against Generative Models
Hayes, J., Melis, L., Danezis, G., & De Cristofaro, E. (2019) · Proceedings on Privacy Enhancing Technologies · doi:10.2478/popets-2019-0008 · Cited by 433
Generative Adversarial Networks and Its Applications in Biomedical Informatics
Lan, L., You, L., Zhang, Z., Fan, Z., Zhao, W., Zeng, N., Chen, Y., & Zhou, X. (2020) · Frontiers in Public Health · doi:10.3389/fpubh.2020.00164 · Cited by 237
Defect-GAN: High-Fidelity Defect Synthesis for Automated Defect Inspection
Zhang, G., Cui, K., Hung, T.-Y., & Lu, S. (2021) · 2021 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv48630.2021.00257 · Cited by 200
Cascade EF-GAN: Progressive Facial Expression Editing With Local Focuses
Wu, R., Zhang, G., Lu, S., & Chen, T. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00507 · Cited by 121
Mok, T. C. W., & Chung, A. C. S. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-11723-8_7 · Cited by 95
Towards efficient and scale-robust ultra-high-definition image demoiréing
Yu, X., Dai, P., Li, W., Ma, L., Shen, J., Li, J., & Qi, X. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-19797-0_37 · Cited by 93
Wang, J., Chen, Y., Gu, Y., Xiao, Y., & Pan, H. (2018) · 2018 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2018.8489106 · Cited by 92
Lewis, J. K., Toubal, I. E., Chen, H., Sandesera, V., Lomnitz, M., Hampel-Arias, Z., Prasad, C., & Palaniappan, K. (2020) · 2020 IEEE Applied Imagery Pattern Recognition Workshop (AIPR) · doi:10.1109/aipr50011.2020.9425167 · Cited by 80
3D Reconstruction of Incomplete Archaeological Objects Using a Generative Adversarial Network
Hermoza, R., & Sipiran, I. (2018) · Proceedings of Computer Graphics International 2018 · doi:10.1145/3208159.3208173 · Cited by 56
Machine learning in NextG networks via generative adversarial networks
Ayanoglu, E., Davaslioglu, K., & Sagduyu, Y. E. (2022) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2022.3153004 · Cited by 55
Quantum Hamiltonian-Based Models and the Variational Quantum Thermalizer Algorithm
Verdon, G., Marks, J., Nanda, S., Leichenauer, S., & Hidary, J. (2019) · arXiv · doi:10.48550/arxiv.1910.02071 · Cited by 45
Detecting CNN-Generated Facial Images in Real-World Scenarios
Hulzebosch, N., Ibrahimi, S., & Worring, M. (2020) · arXiv · doi:10.48550/arxiv.2005.05632 · Cited by 18
(2020) · doi:10.1109/cvprw50498.2020.00329 · Cited by 18
DeepFake Detection Algorithms: A Meta-Analysis
Zotov, S., Dremliuga, R., Borshevnikov, A., & Krivosheeva, K. (2020) · 2020 2nd Symposium on Signal Processing Systems · doi:10.1145/3421515.3421532 · Cited by 12
Deep Learning-Based X-ray Inpainting for Improving Spinal 2d-3d Registration
Esfandiari, H., Weidert, S., Kövesházi, I., Anglin, C., Street, J., & Hodgson, A. J. (2021) · The International Journal of Medical Robotics and Computer Assisted Surgery · doi:10.1002/rcs.2228 · Cited by 12
Metadata
- Authors
- Yeh, R. A., Chen, C., Lim, T. Y., Schwing, A. G., Hasegawa-Johnson, M., & Do, M. N.
- Venue
- 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2017-07
- Pages
- 6882-6890
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2963917315
- Cited by
- 1,206
- References
- 9
- FWCI
- 47.81
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
