Thin-Plate Spline Motion Model for Image Animation
Open accessThe DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
Abstract
Image animation brings life to the static object in the source image according to the driving video. Recent works attempt to perform motion transfer on arbitrary objects through unsupervised methods without using a priori knowledge. However, it remains a significant challenge for current unsupervised methods when there is a large pose gap between the objects in the source and driving images. In this paper, a new end-to-end unsupervised motion transfer framework is proposed to overcome such issues. Firstly, we propose thin-plate spline motion estimation to produce a more flexible optical flow, which warps the feature maps of the source image to the feature domain of the driving image. Secondly, in order to restore the missing regions more realistically, we leverage multi-resolution occlusion masks to achieve more effective feature fusion. Finally, additional auxiliary loss functions are designed to ensure that there is a clear division of labor in the network modules, encouraging the network to generate high-quality images. Our method11Our source code is publicly available: https://github.com/yoyo-nb/Thin-Plate-Spline-Motion-Model. can animate a variety of objects, including talking faces, human bodies, and pixel animations. Experiments demonstrate that our method performs better on most benchmarks than the state of the art with visible improvements in motion-related metrics.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Perceptual Losses for Real-Time Style Transfer and Super-Resolution
Johnson, J., Alahi, A., & Fei-Fei, L. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46475-6_43 · Cited by 11,283
Realtime Multi-person 2D Pose Estimation Using Part Affinity Fields
Cao, Z., Simon, T., Wei, S.-E., & Sheikh, Y. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.143 · Cited by 7,539 · cited 2×
“We use the same pretrained detectors for bodies [5] and faces [4] as MRAA [28] to extract keypoints from the generated and driving images.”
BackgroundMethodImproving neural networks by preventing co-adaptation of feature detectors
Hinton, G. E., Srivastava, N., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. R. (2012) · arXiv · doi:10.48550/arxiv.1207.0580 · Cited by 6,382
Deep High-Resolution Representation Learning for Human Pose Estimation
Sun, K., Xiao, B., Liu, D., & Wang, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00584 · Cited by 5,984
StarGAN: Unified Generative Adversarial Networks for Multi-domain Image-to-Image Translation
Choi, Y., Choi, M., Kim, M., Ha, J.-W., Kim, S., & Choo, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00916 · Cited by 4,163
VoxCeleb: A Large-Scale Speaker Identification Dataset
Nagrani, A., Chung, J. S., & Zisserman, A. (2017) · Interspeech 2017 · doi:10.21437/interspeech.2017-950 · Cited by 2,210 · cited 1×
“• VoxCeleb [20]: Videos of different celebrities talking downloaded from youtube cropped to 256*256 resolution according to the bounding boxes of the faces.”
Bulat, A., & Tzimiropoulos, G. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.116 · Cited by 1,464
Chan, C., Ginosar, S., Zhou, T., & Efros, A. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00603 · Cited by 737
Few-Shot Adversarial Learning of Realistic Neural Talking Head Models
Zakharov, E., Shysheya, A., Burkov, E., & Lempitsky, V. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00955 · Cited by 687
Research works printed but not linked: 24
- yin, 2017, Multi-task convolutional neural network for pose-invariant face recognition, IEEE Transactions on Image Processing
- wiles, 0, X2face: A network for controlling face generation using images, audio, and pose codes, Proceedings of the European Conference on Computer Vision
- 10.1109/CVPR46437.2021.00991 · DOI · Paper page
- 10.1145/3292039 · DOI · Paper page
- 10.1109/CVPR.2019.00245 · DOI · Paper page
- 10.1109/CVPR.2017.660 · DOI · Paper page
- yu, 2021, Deep learning for face anti-spoofing A survey
- ekman, 1997, What the Face Reveals Basic and Applied Studies of Spontaneous Expression using the Facial Action Coding System (FACS)
- hermans, 0, Defense of the Triplet Loss for Person Re-Identification
- 10.1109/CVPR42600.2020.00782 · DOI · Paper page
- 10.1109/CVPR.2017.618 · DOI · Paper page
- liang, 0, High-resolution photo-realistic image translation in real-time: A laplacian pyramid translation network, CVPR
- 10.1109/ICCV.2019.00600 · DOI · Paper page
- 10.1109/CVPR.2015.7298965 · DOI · Paper page
- 10.1109/CVPR46437.2021.01344 · DOI · Paper page
- 10.1109/CVPR.2018.00359 · DOI · Paper page
- 10.1109/ICIP.2015.7351280 · DOI · Paper page
- denton, 0, Deep generative image models using a laplacian pyramid of adversarial networks, NeurIPS
- 10.1109/34.24792 · DOI · Paper page
- amos, 2016, Openface: A general-purpose face recognition library with mobile applications, CMU School of Computer Science
- pumarola, 0, Ganimation: Anatomically-aware facial animation from a single image, Proceedings of the European Conference on Computer Vision
- 10.1111/cgf.14062 · DOI · Paper page
- 10.1109/CVPR42600.2020.00771 · DOI · Paper page
- 10.1109/CVPR.2019.00248 · DOI · Paper page
Cited by
Zhang, W., Cun, X., Wang, X., Zhang, Y., Shen, X., Guo, Y., Shan, Y., & Wang, F. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00836 · Cited by 336
“Previous works also learn a shared intermediate motion representation from the source image and the target, which can be roughly divided into the landmark [39] and the unsupervised landmark-based meth-ods [14,34,40,48], 3DMM based methods [7,31,45] and the latent animation [23,42].”
BackgroundDreamPose: Fashion Image-to-Video Synthesis via Stable Diffusion
Karras, J., Holynski, A., Wang, T.-C., & Kemelmacher-Shlizerman, I. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.02073 · Cited by 137
X-Portrait: Expressive Portrait Animation with Hierarchical Motion Attention
Xie, Y., Xu, H., Song, G., Wang, C., Shi, Y., & Luo, L. (2024) · Special Interest Group on Computer Graphics and Interactive Techniques Conference Conference Papers 24 · doi:10.1145/3641519.3657459 · Cited by 45
LivePhoto: Real Image Animation with Text-guided Motion Control
Chen, X., Liu, Z., Chen, M., Feng, Y., Liu, Y., Shen, Y., & Zhao, H. (2023) · arXiv · doi:10.48550/arxiv.2312.02928 · Cited by 18
A Dual-Branch Fusion Model for Deepfake Detection Using Video Frames and Microexpression Features
Petmezas, G., Vanian, V., Rufete, M. P., Almaloglou, E. E. I., & Zarpalas, D. (2025) · Journal of Imaging · doi:10.3390/jimaging11070231 · Cited by 8
“We further benchmarked our method on ReenactFaces, a new dataset comprising 10,000 real and 10,000 re-enacted videos created using three state-of-the-art facial re-enactment methods, namely the First-Order Motion Model [19], Thin-Plate Spline Motion [20], and FSRT [21].”
Cross-dataset video deepfake detection using Transformer and CNN architectures
Petmezas, G. (2026) · Machine Vision and Applications · doi:10.1007/s00138-026-01809-w · Cited by 6
ReenactFaces: A Specialized Dataset for Reenactment-Based Deepfake Detection
Vanian, V., Petmezas, G., Konstantoudakis, K., & Zarpalas, D. (2025) · Lecture Notes on Data Engineering and Communications Technologies · doi:10.1007/978-3-031-87784-1_35 · Cited by 2
Metadata
- Authors
- Zhao, J., & Zhang, H.
- Venue
- 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2022-06
- Pages
- 3647-3656
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W4312473638
- Cited by
- 222
- References
- 13
- FWCI
- 18.06
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.