CoChat
Sign Up
Cited by 105Generative Adversarial Networks and Image SynthesisFace recognition and analysisDigital Media Forensic Detection

Face Generation and Editing With StyleGAN: A Survey

Open access
Melnik, Andrew, Miasayedzenkau, Maksim, Makaravets, Dzianis, Pirshtuk, Dzianis, Akbulut, Eren, Holzmann, Dennis, Renusch, Tarek, Reichert, Gustav +1 more·2024·IEEE Transactions on Pattern Analysis and Machine Intelligence
DOI 10.1109/tpami.2024.3350004
Open this paper in CoChat — free accountOpen PDF

Abstract

Our goal with this survey is to provide an overview of the state of the art deep learning methods for face generation and editing using StyleGAN. The survey covers the evolution of StyleGAN, from PGGAN to StyleGAN3, and explores relevant topics such as suitable metrics for training, different latent representations, GAN inversion to latent spaces of StyleGAN, face image editing, cross-domain face stylization, face restoration, and even Deepfake applications. We aim to provide an entry point into the field for readers that have basic knowledge about the field of deep learning and are looking for an accessible introduction and overview.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/tpami.2024.3350004

Cites

18 of 18 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Prior to diffusion models, GANs and auto-regressive transformer models have been popular choices for generative video tasks.

    “Prior to diffusion models, Generative Adversarial Networks (GANs, Goodfellow et al. 2014, Melnik et al. 2024) and auto-regressive transformer models (Vaswani et al., 2017) have been popular choices for generative video tasks.”

    From Video Diffusion Models: A Survey (2024) · cites this paper as Melnik et al. 2024

    Melnik et al. supports the GAN portion of the sentence: its Generative Adversarial Networks section describes GANs as suitable for face generation, and its References list earlier GAN-based video and talking-face generation work. The survey focuses on faces and does not establish that GANs were popular across generative video tasks generally. The separate citation to Vaswani et al. is attached to the transformer portion, which is not assessed here. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 118

  1. T. Karras, S. Laine, M. Aittala, J. Hellsten, J. Lehtinen, and T. Aila, "Analyzing and improving the image quality of styleGAN," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2020, pp. 8110-8119.
  2. T. Karras, M. Aittala, J. Hellsten, S. Laine, J. Lehtinen, and T. Aila, "Training generative adversarial networks with limited data," in Proc. Adv. Neural Inf. Process. Syst., 2020, pp. 12104-12114.
  3. O. Patashnik, Z. Wu, E. Shechtman, D. Cohen-Or, and D. Lischinski, "StyleCLIP: Text-driven manipulation of styleGAN imagery," in Proc. IEEE/CVF Int. Conf. Comput. Vis., 2021, pp. 2065-2074.
  4. R. Abdal, Y. Qin, and P. Wonka, "Image2StyleGAN : How to edit the embedded images?," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2020, pp. 8296-8305.
  5. X. Wang, Y. Li, H. Zhang, and Y. Shan, "Towards real-world blind face restoration with generative facial prior," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2021, pp. 9164-9174.
  6. Y. Nitzan et al., "Mystyle: A personalized generative prior," 2022, arXiv:2203.17272. · arXiv
  7. R. Gal, O. Patashnik, H. Maron, G. Chechik, and D. Cohen-Or, "StyleGAN-NADA: Clip-guided domain adaptation of image genera- tors," ACM Trans. Graph., vol. 41, 2022, Art. no. 141.
  8. A. Tewari et al., "StyleRig: Rigging StyleGAN for 3D control over por- trait images," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2020, Art. no. 6141-6150.
  9. Y. Zhu, Q. Li, J. Wang, C. Xu, and Z. Sun, "One shot face swapping on megapixels," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2021, pp. 4832-4842.
  10. H. Luo et al., "Normalized avatar synthesis using styleGAN and per- ceptual refinement," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2021, pp. 11657-11667.
  11. A. Melnik et al., "Systems, subjects, sessions: To what extent do these factors influence EEG data?," Front. Hum. Neurosci., vol. 11, 2017, Art. no. 150.
  12. H. Li, X. Hou, Z. Huang, and L. Shen, "StyleGene: Crossover and mutation of region-level facial genes for kinship face synthe- sis," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2023, pp. 20960-20969.
  13. C.-H. Lin, H.-C. Chen, L.-C. Cheng, S.-C. Hsu, J.-C. Chen, and C.-Y. Wang, "StyleDNA: A high-fidelity age and gender aware kinship face synthesizer," in Proc. IEEE 16th Int. Conf. Autom. Face Gesture Recog- nit., 2021, pp. 1-8.
  14. X. Qin, X. Tan, and S. Chen, "Tri-subject kinship verification: Under- standing the core of a family," IEEE Trans. Multimedia, vol. 17, no. 10, pp. 1855-1867, Oct. 2015.
  15. X. Huang and S. Belongie, "Arbitrary style transfer in real-time with adaptive instance normalization," in Proc. IEEE Int. Conf. Comput. Vis., 2017, pp. 1510-1519.
  16. Y. Alaluf et al., "Third time's the charm? Image and video editing with StyleGAN3," 2022, arXiv:2201.13433. · arXiv
  17. T. Karras et al., "Alias-free generative adversarial networks," in Proc. Adv. Neural Inf. Process. Syst., 2021, pp. 852-863.
  18. A. Melnik, E. Akbulut, J. Sheikh, K. Loos, M. Buettner, and T. Lenze, "Faces: Ai blitz XIII solutions," 2022, arXiv:2204.01081. · arXiv
  19. A. Radford et al., "Learning transferable visual models from natu- ral language supervision," in Proc. Int. Conf. Mach. Learn., 2021, pp. 8748-8763.
  20. T. Yang, P. Ren, X. Xie, and L. Zhang, "GAN prior embedded network for blind face restoration in the wild," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2021, pp. 672-681.
  21. Y. Poirier-Ginter and J.-F. Lalonde, "Robust unsupervised styleGAN image restoration," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recog- nit., 2023, pp. 22292-22301.
  22. M. Zhang, N. Wang, Y. Li, and X. Gao, "Neural probabilistic graphical model for face sketch synthesis," IEEE Trans. Neural Netw. Learn. Syst., vol. 31, no. 7, pp. 2623-2637, Jul. 2020.
  23. M. Zhang, N. Wang, Y. Li, and Gao, "Deep latent low-rank representation for face sketch synthesis," IEEE Trans. Neural Netw. Learn. Syst., vol. 30, no. 10, Oct. 2019.
  24. M. Zhang, N. Wang, Y. Li, and X. Gao, "Bionic face sketch generator," IEEE Trans. Cybern., vol. 50, no. 6, pp. 2701-2714, Jun. 2020.
  25. M. Zhang, R. Wang, X. Gao, J. Li, and D. Tao, "Dual-transfer face sketch- photo synthesis," IEEE Trans. Image Process., vol. 28, no. 2, pp. 642-657, Feb. 2019.
  26. M. Zhang, Y. Li, N. Wang, Y. Chi, and X. Gao, "Cascaded face sketch synthesis under various illuminations," IEEE Trans. Image Process., vol. 29, pp. 1507-1521, 2020.
  27. W. Jang, G. Ju, Y. Jung, J. Yang, X. Tong, and S. Lee, "StylecariGAN: Caricature generation via styleGAN feature map modulation," ACM Trans. Graph., vol. 40, no. 4, pp. 1-16, 2021.
  28. H. Wang, G. Lin, S. C. Hoi, and C. Miao, "3D cartoon face gen- eration with controllable expressions from a single GAN image," 2022, arXiv:2207.14425. · arXiv
  29. J. Back, "Fine-tuning StyleGAN2 for cartoon face generation," 2021, arXiv:2106.12445. · arXiv
  30. L. Verdoliva, "Media forensics and deepFakes: An overview," IEEE J. Sel. Topics Signal Process., vol. 14, no. 5, pp. 910-932, Aug. 2020.
  31. C. Yang and S.-N. Lim, "Unconstrained facial expression transfer using style-based generator," 2019, arXiv: 1912.06253. · arXiv
  32. E. Burkov, I. Pasechnik, A. Grigorev, and V. Lempitsky, "Neural head reenactment with latent pose descriptors," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2020, pp. 13783-13792.
  33. J. R. A. Moniz, C. Beckham, S. Rajotte, S. Honari, and C. Pal, "Unsu- pervised depth estimation, 3D face rotation and replacement," in Proc. Adv. Neural Inf. Process. Syst., 2018, pp. 9759-9769.
  34. A. G. Howard et al., "MobileNets: Efficient convolutional neural net- works for mobile vision applications," 2017, arXiv: 1704.04861. · arXiv
  35. M. Li, J. Lin, Y. Ding, Z. Liu, J.-Y. Zhu, and S. Han, "GAN com- pression: Efficient architectures for interactive conditional GANs," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2020, pp. 5283-5293.
  36. Y. Viazovetskyi, V. Ivashkin, and E. Kashin, "StyleGAN2 distillation for feed-forward image manipulation," in Proc. Eur. Conf. Comput. vis., 2020, pp. 170-186.
  37. Z. Liu, P. Luo, X. Wang, and X. Tang, "Deep learning face attributes in the wild," in Proc. IEEE Int. Conf. Comput. Vis., 2015, pp. 3730-3738.
  38. T. Karras, T. Aila, S. Laine, and J. Lehtinen, "Progressive growing of GANs for improved quality, stability, and variation," in Proc. Int. Conf. Learn. Representations, 2018.
  39. D. E. King, "Dlib-ml: A machine learning toolkit," J. Mach. Learn. Res., vol. 10, pp. 1755-1758, 2009.
  40. I. Goodfellow et al., "Generative adversarial networks," Commun. ACM, vol. 63, no. 11, pp. 139-144, 2020.
  41. N. Kodali, J. Abernethy, J. Hays, and Z. Kira, "On convergence and stability of GANs," 2017, arXiv: 1705.07215. · arXiv
  42. R. Xu, X. Wang, K. Chen, B. Zhou, and C. C. Loy, "Positional encoding as spatial inductive bias in GANs," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2021, pp. 13569-13578.
  43. M. Arjovsky, S. Chintala, and L. Bottou, "Wasserstein generative adver- sarial networks," in Proc. Int. Conf. Mach. Learn., 2017, pp. 214-223.
  44. J. H. Lim and J. C. Ye, "Geometric GAN," 2017, arXiv: 1705.02894. · arXiv
  45. Y. Blau and T. Michaeli, "The perception-distortion tradeoff," in Proc. IEEE Conf. Comput. Vis. pattern Recognit., 2018, pp. 6228-6237.
  46. A. Dosovitskiy and T. Brox, "Generating images with perceptual simi- larity metrics based on deep networks," in Proc. Adv. Neural Inf. Process. Syst., 2016.
  47. R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang, "The unreasonable effectiveness of deep features as a perceptual metric," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2018, pp. 586-595.
  48. K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," 2014, arXiv:1409.1556. · arXiv
  49. C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, "Rethinking the inception architecture for computer vision," in Proc. IEEE Conf. Comput. Vis. pattern Recognit., 2016, pp. 2818-2826.
  50. J. Zhu, D. Zhao, B. Zhang, and B. Zhou, "Disentangled inference for GANs with latently invertible autoencoder," Int. J. Comput. Vis., vol. 130, pp. 1259-127, 2022.
  51. A. H. Bermano et al., "State-of-the-art in the architecture, methods and applications of styleGAN," in Computer Graphics Forum. Hoboken, NJ, USA: Wiley, 2022, pp. 591-611.
  52. Z. Wu, D. Lischinski, and E. Shechtman, "Stylespace analysis: Disentan- gled controls for styleGAN image generation," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2021, pp. 12858-12867.
  53. E. Richardson et al., "Encoding in style: A styleGAN encoder for image- to-image translation," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2021, pp. 2287-2296.
  54. O. Tov, Y. Alaluf, Y. Nitzan, O. Patashnik, and D. Cohen-Or, "Designing an encoder for styleGAN image manipulation," ACM Trans. Graph., vol. 40, pp. 1-14, 2021.
  55. Y. Alaluf, O. Patashnik, and D. Cohen-Or, "Restyle: A residual-based styleGAN encoder via iterative refinement," in Proc. IEEE/CVF Int. Conf. Comput. Vis., 2021, pp. 6711-6720.
  56. D. Roich, R. Mokady, A. H. Bermano, and D. Cohen-Or, "Pivotal tuning for latent-based editing of real images," ACM Trans. Graph., vol. 42, no. 1, pp. 1-13, 2022.
  57. Y. Alaluf, O. Tov, R. Mokady, R. Gal, and A. Bermano, "Hyper- Style: StyleGAN inversion with hypernetworks for real image edit- ing," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2022, pp. 18 511-18 521.
  58. Z. C. Lipton and S. Tripathi, "Precise recovery of latent vectors from generative adversarial networks," 2017, arXiv: 1702.04782. · arXiv
  59. J. Johnson, A. Alahi, and L. Fei-Fei, "Perceptual losses for real-time style transfer and super-resolution," in Proc. Eur. Conf. Comput. Vis., 2016, pp. 694-711.
  60. S. Guan, Y. Tai, B. Ni, F. Zhu, F. Huang, and X. Yang, "Collaborative learning for faster styleGAN embedding," 2020, arXiv: 2007.01758. · arXiv
  61. T. Wei et al., "E2Style: Improve the efficiency and effectiveness of style- GAN inversion," IEEE Trans. Image Process., vol. 31, pp. 3267-3280, 2022.
  62. J. Zhu, Y. Shen, D. Zhao, and B. Zhou, "In-domain GAN inversion for real image editing," in Proc. 16th Eur. Conf. Comput. Vis., Glasgow, U.K., 2020, pp. 592-608.
  63. J. Donahue, P. Krähenbühl, and T. Darrell, "Adversarial feature learning," 2016, arXiv:1605.09782. · arXiv
  64. Y. Shen, J. Gu, X. Tang, and B. Zhou, "Interpreting the latent space of GANs for semantic face editing," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2020, pp. 9240-9249.
  65. E. Härkönen, A. Hertzmann, J. Lehtinen, and S. Paris, "Ganspac: Dis- covering interpretable GAN controls," in Proc. Adv. Neural Inf. Process. Syst., 2020, pp. 9841-9850.
  66. Z. Wu, Y. Nitzan, E. Shechtman, and D. Lischinski, "StyleAlign: Anal- ysis and applications of aligned styleGAN models," in Proc. Int. Conf. Learn. Representations, 2022. [Online]. Available: https://openreview. net/forum?id=Qg2vi4ZbHM9
  67. J. N. Pinkney and D. Adler, "Resolution dependent GAN interpola- tion for controllable image synthesis between domains," arXiv preprint 2020, arXiv: 2010.05334. · arXiv
  68. M. Liu, Q. Li, Z. Qin, G. Zhang, P. Wan, and W. Zheng, "Blendgan: Implicitly GAN blending for arbitrary stylized face generation," in Proc. Int. Conf. Neural Inf. Process. Syst., 2021, pp. 29710-29722.
  69. S. Zhou, K. C. Chan, C. Li, and C. C. Loy, "Towards robust blind face restoration with codebook lookup transformer," in Proc. Int. Conf. Neural Inf. Process. Syst., 2022, pp. 30599-30611.
  70. Y. Gu et al., "VQFR: Blind face restoration with vector-quantized dic- tionary and parallel decoder," in Proc. Eur. Conf. Comput. Vis., 2022, pp. 126-143.
  71. A. van den Oord, O. Vinyals, and K. Kavukcuoglu, "Neural discrete representation learning," 2017, arXiv: 1711.00937. · arXiv
  72. J. Thies, M. Zollhofer, M. Stamminger, C. Theobalt, and M. Nießner, "Face2Face: Real-time face capture and reenactment of RGB videos," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2016, pp. 2387-2395.
  73. S. Tulyakov, M.-Y. Liu, X. Yang, and J. Kautz, "MoCoGAN: Decompos- ing motion and content for video generation," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2018, pp. 1526-1535.
  74. Y. Song, J. Zhu, D. Li, A. Wang, and H. Qi, "Talking face generation by conditional recurrent adversarial network," in Proc. 28th Int. Joint Conf. Artif. Intell., 2019, pp. 919-925.
  75. L. Yu, J. Yu, and Q. Ling, "Mining audio, text and visual information for talking face generation," in Proc. IEEE Int. Conf. Data Mining, 2019, pp. 787-795.
  76. D. Kononenko, Y. Ganin, D. Sungatullina, and V. Lempitsky, "Pho- torealistic monocular gaze redirection using machine learning," IEEE Trans. Pattern Anal. Mach. Intell., vol. 40, no. 11, pp. 2696-2710, Nov. 2018.
  77. L. Tran, X. Yin, and X. Liu, "Representation learning by rotating your faces," IEEE Trans. Pattern Anal. Mach. Intell., vol. 41, pp. 3007-3021, Dec. 2019.
  78. J. Zhang, X. Zeng, Y. Pan, Y. Liu, Y. Ding, and C. Fan, "FaceSwapNet: Landmark guided many-to-many face reenactment, 2019, arXiv:1905.11805. · arXiv
  79. L. Li, J. Bao, H. Yang, D. Chen, and F. Wen, "Advancing high fidelity identity swapping for forgery detection," in Proc. IEEE/CVF Conf. Com- put. Vis. Pattern Recognit., 2020, pp. 5073-5082.
  80. Y. Liu and J. Chen, "Unsupervised face frontalization using disentangled representation-learning CycleGAN," Comput. Vis. Image Understanding, vol. 222, 2022, Art. no. 103526.
  81. C. Limberg, A. Melnik, A. Harter, and H. J. Ritter, "Yolo-you only look 10647 times," 2022, arXiv:2201.06159. · arXiv
  82. K. Vougioukas, S. Petridis, and M. Pantic, "End-to-end speech-driven facial animation with temporal GANs," in Proc. Brit. Mach. Vis. Conf., 2018.
  83. H. Li, J. Liu, Y. Bai, H. Wang, and K. Mueller, "Transforming the latent space of styleGAN for real face editing," 2022, arXiv:2105.14230. · arXiv
  84. Y. Ma et al., "Styletalk: One-shot talking head generation with control- lable speaking styles," 2023, arXiv:2301.01081. · arXiv
  85. J. Bromley, I. Guyon, Y. LeCun, E. Säckinger, and R. Shah, "Signature verification using a "siamese" time delay neural network," in Proc. 6th Int. Conf. Neural Inf. Process. Syst., San Francisco, CA, USA: Morgan Kaufmann Publishers Inc., 1993, pp. 737-744.
  86. J. Zhu, T. Park, P. Isola, and A. A. Efros, "Unpaired image-to-image translation using cycle-consistent adversarial networks," 2017, arXiv: 1703.10593. · arXiv
  87. J. Gu, L. Liu, P. Wang, and C. Theobalt, "StyleNeRF: A style- based 3D-aware generator for high-resolution image synthesis," 2021, arXiv:2110.08985. · arXiv
  88. R. Or-El, X. Luo, M. Shan, E. Shechtman, J. J. Park, and I. Kemelmacher- Shlizerman, "StyleSDF: High-resolution 3D-consistent image and ge- ometry generation," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2022, pp. 13493-13503.
  89. D. Bigioi, S. Basak, H. Jordan, R. McDonnell, and P. Corcoran, "Speech driven video editing via an audio-conditioned diffusion model," 2023, arXiv:2301.04474. · arXiv
  90. S. Shen et al., "DiffTalk: Crafting diffusion models for generalized talking head synthesis," 2023, arXiv:2301.03786. · arXiv
  91. T. Brooks, A. Holynski, and A. A. Efros, "Instructpix2pix: Learning to follow image editing instructions," 2022, arXiv:2211.09800. · arXiv
  92. Z. Huang, K. C. Chan, Y. Jiang, and Z. Liu, "Collaborative diffusion for multi-modal face generation and editing," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2023, pp. 6080-6090.
  93. N. G. Nair, W. G. C. Bandara, and V. M. Patel, "Unite and conquer: Plug & play multi-modal synthesis using diffusion models," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2023, pp. 6070-6079.
  94. R. Zhang et al., "Llama-adapter: Efficient fine-tuning of language models with zero-init attention," 2023, arXiv:2303.16199. · arXiv
  95. A. Sevastopolsky, Y. Malkov, N. Durasov, L. Verdoliva, and M. Nießner, "How to boost face recognition with StyleGAN?," in Proc. Int. Conf. Comput. Vis., 2023, pp. 20924-20934.
  96. M. Wu, H. Zhu, L. Huang, Y. Zhuang, Y. Lu, and X. Cao, "High- fidelity 3D face generation from natural language descriptions," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2023, pp. 4521-4530.
  97. A. Sauer, K. Schwarz, and A. Geiger, "StyleGAN-XL: Scaling styleGAN to large diverse datasets," in Proc. ACM SIGGRAPH 2022 Conf. Proc., 2022, Art. no. 49.
  98. X. Pan, A. Tewari, T. Leimkühler, L. Liu, A. Meka, and C. Theobalt, "Drag your GAN: Interactive point-based manipulation on the gener- ative image manifold," in Proc. ACM SIGGRAPH 2023 Conf., 2023, pp. 1-11.
  99. H. Jia et al., "BlazeStyleGAN: A real-time on-device styleGAN," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2023, pp. 4689-4693.
  100. P. Melzi et al., "GANDiffFace: Controllable generation of syn- thetic datasets for face recognition with realistic variations," 2023, arXiv:2305.19962. · arXiv
  101. M. Kansy et al., "Controllable inversion of black-box face-recognition models via diffusion," 2023, arXiv:2303.13006. · arXiv
  102. M. Kim, F. Liu, A. Jain, and X. Liu, "DCFace: Synthetic face generation with dual condition diffusion model," in Proc. IEEE/CVF Comput. Vis. Pattern Recognit., 2023, pp. 3078-3087.
  103. J. Wang, Z. Yue, S. Zhou, K. C. Chan, and C. C. Loy, "Ex- ploiting diffusion prior for real-world image super-resolution," arXiv preprint arXiv:2305.07015, 2023. · arXiv
  104. Z. Liang, C. Li, S. Zhou, R. Feng, and C. C. Loy, "Itera- tive prompt learning for unsupervised backlit image enhancement," 2023, arXiv:2303.17569. · arXiv
  105. Z. Wang et al., "DR2: Diffusion-based robust degradation remover for blind face restoration," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2023, pp. 1704-1713.
  106. G. Kim, H. Shim, H. Kim, Y. Choi, J. Kim, and E. Yang, "Diffusion video autoencoders: Toward temporally consistent face video editing via disentangled video encoding," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2023, pp. 6091-6100.
  107. O. Avrahami, O. Fried, and D. Lischinski, "Blended latent diffusion," ACM Trans. Graph., vol. 42, no. 4, pp. 1-11, 2023.
  108. A. Sauer, T. Karras, S. Laine, A. Geiger, and T. Aila, "StyleGAN-T: Un- locking the power of GANs for fast large-scale text-to-image synthesis," 2023, arXiv:2301.09515. · arXiv
  109. M. Kang et al., "Scaling up GANs for text-to-image synthesis," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2023, pp. 10124-10134.
  110. M. Zheng, H. Zhang, H. Yang, and D. Huang, "NeuFace: Realistic 3D neural face rendering from multi-view images," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2023, pp. 16868-16 877.
  111. E. R. Chan et al., "Efficient geometry-aware 3D generative adversarial networks," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2022, pp. 16102-16112.
  112. R. Abdal et al., "3DAvatarGAN: Bridging domains for personalized ed- itable avatars," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2023, pp. 4552-4562.
  113. J. Sun et al., "Next3D: Generative neural texture rasterization for 3D- aware head avatars," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2023, pp. 20991-21002.
  114. Z. Ma, X. Zhu, G.-J. Qi, Z. Lei, and L. Zhang, "OTAvatar: One-shot talk- ing face avatar with controllable tri-plane rendering," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2023, pp. 16901-16910.
  115. Y. Xu, Z. Shu, C. Smith, J.-B. Huang, and S. W. Oh, "In-n-out: Face video inversion and editing with volumetric decomposition," arXiv preprint 2023, arXiv:2302.04871. · arXiv
  116. F. Yin et al., "3D GAN inversion with facial symmetry prior," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2023, pp. 342-351.
  117. S. Yang, L. Jiang, Z. Liu, and C. C. Loy, "Vtoonify: Controllable high- resolution portrait video style transfer," ACM Trans. Graph., vol. 41, no. 6, pp. 1-15, 2022.
  118. S. Yang, L. Jiang, Z. Liu, and C. C. Loy, "StyleGANEX: StyleGAN-based manipulation beyond cropped aligned faces," in Proc. IEEE Int. Conf. Comput. Vis., 2023.

Cited by

Sorted by citations

Authors
Melnik, A., Miasayedzenkau, M., Makaravets, D., Pirshtuk, D., Akbulut, E., Holzmann, D., Renusch, T., Reichert, G., & Ritter, H.
Affiliations
  • Bielefeld University (DE)
  • Banuba, Minsk, Belarus
ORCID
Venue
IEEE Transactions on Pattern Analysis and Machine Intelligence
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Published
2024-05
Volume
46
Issue
5
Pages
3557-3576
ISSN
  • 0162-8828 (ISSN-L)
  • 2160-9292
  • 1939-3539
DOI
10.1109/tpami.2024.3350004
OpenAlex
W4390873137
Cited by
105
References
18
Citations by year
2026: 29 · 2025: 55 · 2024: 19 · 2023: 2
FWCI
14.49
Type
journal article
Language
en
Open access
Licence
Locations
  • IEEE Transactions on Pattern Analysis and Machine Intelligence (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • PUB – Publications at Bielefeld University (Bielefeld University) (submittedVersion, cc-by-nc-nd)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
Keywords
face synthesis, face editing, StyleGAN, StyleGAN3, latent representations, GAN inversion, latent space, face restoration, deepfakes, generative adversarial networks

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review