The DOI matches its Crossref record; 53 of the 53 references it cites were found in a registry or an index.
MixSTE: Seq2seq Mixed Spatio-Temporal Encoder for 3D Human Pose Estimation in Video
Open accessAbstract
Recent transformer-based solutions have been introduced to estimate 3D human pose from 2D keypoint sequence by considering body joints among all frames globally to learn spatio-temporal correlation. We observe that the motions of different joints differ significantly. However, the previous methods cannot efficiently model the solid inter-frame correspondence of each joint, leading to insufficient learning of spatial-temporal correlation. We propose MixSTE (Mixed Spatio-Temporal Encoder), which has a temporal transformer block to separately model the temporal motion of each joint and a spatial transformer block to learn inter-joint spatial correlation. These two blocks are utilized alternately to obtain better spatio-temporal feature encoding. In addition, the network output is extended from the central frame to entire frames of the input video, thereby improving the coherence between the input and output sequences. Extensive experiments are conducted on three benchmarks (i.e. Human3.6M, MPI-INF-3DHP, and HumanEva). The results show that our model outperforms the state-of-the-art approach by 10.9% P-MPJPE and 7.6% MPJPE. The code is available at https://github.com/JinluZhang1126/MixSTE.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Exploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472
He, K., Gkioxari, G., Dollar, P., & Girshick, R. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.322 · Cited by 29,137
End-to-End Object Detection with Transformers
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58452-8_13 · Cited by 18,285
End-to-End Object Detection with Transformers
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. (2020) · arXiv · doi:10.48550/arxiv.2005.12872 · Cited by 18,253
Deep High-Resolution Representation Learning for Human Pose Estimation
Sun, K., Xiao, B., Liu, D., & Wang, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00584 · Cited by 5,984
Stacked Hourglass Networks for Human Pose Estimation
Newell, A., Yang, K., & Deng, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46484-8_29 · Cited by 5,634
Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments
Ionescu, C., Papava, D., Olaru, V., & Sminchisescu, C. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.248 · Cited by 3,653
Cascaded Pyramid Network for Multi-person Pose Estimation
Chen, Y., Wang, Z., Peng, Y., Zhang, Z., Yu, G., & Sun, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00742 · Cited by 1,799
Self-citationA Simple Yet Effective Baseline for 3d Human Pose Estimation
Martinez, J., Hossain, R., Romero, J., & Little, J. J. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.288 · Cited by 1,462
3D Human Pose Estimation in Video With Temporal Convolutions and Semi-Supervised Training
Pavllo, D., Feichtenhofer, C., Grangier, D., & Auli, M. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00794 · Cited by 1,297
Sigal, L., Balan, A. O., & Black, M. J. (2010) · International Journal of Computer Vision · doi:10.1007/s11263-009-0273-6 · Cited by 1,164
Monocular 3D Human Pose Estimation in the Wild Using Improved CNN Supervision
Mehta, D., Rhodin, H., Casas, D., Fua, P., Sotnychenko, O., Xu, W., & Theobalt, C. (2017) · 2017 International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2017.00064 · Cited by 1,142
VNect: real-time 3D human pose estimation with a single RGB camera
Mehta, D., Sridhar, S., Sotnychenko, O., Rhodin, H., Shafiei, M., Seidel, H.-P., Xu, W., Casas, D., & Theobalt, C. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073596 · Cited by 1,113
Coarse-to-Fine Volumetric Prediction for Single-Image 3D Human Pose
Pavlakos, G., Zhou, X., Derpanis, K. G., & Daniilidis, K. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.139 · Cited by 884
Integral Human Pose Regression
Sun, X., Xiao, B., Wei, F., Liang, S., & Wei, Y. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01231-1_33 · Cited by 874
End-to-End Human Pose and Mesh Reconstruction with Transformers
Lin, K., Wang, L., & Liu, Z. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00199 · Cited by 740
3D Human Pose Estimation with Spatial and Temporal Transformers
Zheng, C., Zhu, S., Mendieta, M., Yang, T., Chen, C., & Ding, Z. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01145 · Cited by 711
Towards 3D Human Pose Estimation in the Wild: A Weakly-Supervised Approach
Zhou, X., Huang, Q., Sun, X., Xue, X., & Wei, Y. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.51 · Cited by 599
Semantic Graph Convolutional Networks for 3D Human Pose Regression
Zhao, L., Peng, X., Tian, Y., Kapadia, M., & Metaxas, D. N. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00354 · Cited by 522
Exploiting Spatial-Temporal Relationships for 3D Pose Estimation via Graph Convolutional Networks
Cai, Y., Ge, L., Liu, J., Cai, J., Cham, T.-J., Yuan, J., & Thalmann, N. M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00236 · Cited by 519
Self-citationOrdinal Depth Supervision for 3D Human Pose Estimation
Pavlakos, G., Zhou, X., & Daniilidis, K. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00763 · Cited by 411
Anatomy-Aware 3D Human Pose Estimation With Bone-Based Pose Decomposition
Chen, T., Fang, C., Shen, X., Zhu, Y., Chen, Z., & Luo, J. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3057267 · Cited by 338
Exploiting Temporal Contexts With Strided Transformer for 3D Human Pose Estimation
Li, W., Liu, H., Ding, R., Liu, M., Wang, P., & Yang, W. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3141231 · Cited by 314
Graph Stacked Hourglass Networks for 3D Human Pose Estimation
Xu, T., & Takano, W. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01584 · Cited by 302
Reconstructing 3D Human Pose from 2D Image Landmarks
Ramakrishna, V., Kanade, T., & Sheikh, Y. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33765-9_41 · Cited by 296
Optimizing Network Structure for 3D Human Pose Estimation
Ci, H., Wang, C., Ma, X., & Wang, Y. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00235 · Cited by 276
Attention Mechanism Exploits Temporal Contexts: Real-Time 3D Human Pose Reconstruction
Liu, R., Shen, J., Wang, H., Chen, C., Cheung, S.-C., & Asari, V. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00511 · Cited by 266
Latent structured models for human pose estimation
Ionescu, C., Li, F., & Sminchisescu, C. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126500 · Cited by 258
Motion Guided 3D Pose Estimation from Videos
Wang, J., Yan, S., Xiong, Y., & Lin, D. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58601-0_45 · Cited by 225
Direct Prediction of 3D Body Poses from Motion Compensated Sequences
Tekin, B., Rozantsev, A., Lepetit, V., & Fua, P. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.113 · Cited by 215
He, Y., Yan, R., Fragkiadaki, K., & Yu, S.-I. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00780 · Cited by 183
Cascaded Deep Monocular 3D Human Pose Estimation With Evolutionary Training Data
Li, S., Ke, L., Pratama, K., Tai, Y.-W., Tang, C.-K., & Cheng, K.-T. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00621 · Cited by 175
PoseAug: A Differentiable Pose Augmentation Framework for 3D Human Pose Estimation
Gong, K., Zhang, J., & Feng, J. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00847 · Cited by 175
Self-citationLearning Skeletal Graph Neural Networks for Hard 3D Pose Estimation
Zeng, A., Sun, X., Yang, L., Zhao, N., Liu, M., & Xu, Q. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01124 · Cited by 170
A Comprehensive Study of Weight Sharing in Graph Networks for 3D Human Pose Estimation
Liu, K., Ding, R., Zou, Z., Wang, L., & Tang, W. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58607-2_19 · Cited by 155
Probabilistic Monocular 3D Human Pose Estimation with Normalizing Flows
Wehrbein, T., Rudolph, M., Rosenhahn, B., & Wandt, B. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01101 · Cited by 144
A spatial attentive and temporal dilated (SATD) GCN for skeleton‐based action recognition
Zhang, J., Ye, G., Tu, Z., Qin, Y., Qin, Q., Zhang, J., & Liu, J. (2021) · CAAI Transactions on Intelligence Technology · doi:10.1049/cit2.12012 · Cited by 126
Self-citationModel-based 3D Hand Reconstruction via Self-Supervised Learning
Chen, Y., Tu, Z., Kang, D., Bao, L., Zhang, Y., Zhe, X., Chen, R., & Yuan, J. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01031 · Cited by 122
Self-citationRecurrent 3D Pose Sequence Machines
Lin, M., Lin, L., Liang, X., Wang, K., & Cheng, H. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.588 · Cited by 110
Collaborative Refining for Person Re-Identification With Label Noise
Ye, M., He, L., Du, B., Shen, J., Shao, L., & Hoi, S. C. H. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3131937 · Cited by 108
A Beamforming Method Based on Polarization Matching
Yang, X., Gu, J., Jiang, J., & Li, Y. (2020) · IEEE International Conference on Advanced Infocomm Technology · doi:10.1109/icait51223.2020.9315404 · Cited by 107
Iterated Second-Order Label Sensitive Pooling for 3D Human Pose Estimation
Ionescu, C., Carreira, J., & Sminchisescu, C. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.215 · Cited by 99
SO-HandNet: Self-Organizing Network for 3D Hand Pose Estimation With Semi-Supervised Learning
Chen, Y., Tu, Z., Ge, L., Zhang, D., Chen, R., & Yuan, J. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00706 · Cited by 94
Self-citationContext Modeling in 3D Human Pose Estimation: A Unified Perspective
Ma, X., Su, J., Wang, C., Ci, H., & Wang, Y. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00617 · Cited by 89
Joint Hand-Object 3D Reconstruction From a Single Image With Cross-Branch Feature Fusion
Chen, Y., Tu, Z., Kang, D., Chen, R., Bao, L., Zhang, Z., & Yuan, J. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3068645 · Cited by 75
Self-citationPose-Guided Human Animation from a Single Image in the Wild
Yoon, J. S., Liu, L., Golyanik, V., Sarkar, K., Park, H. S., & Theobalt, C. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01479 · Cited by 73
Unsupervised Pre-training for Temporal Action Localization Tasks
Zhang, C., Yang, T., Weng, J., Cao, M., Wang, J., & Zou, Y. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01364 · Cited by 64
Meta Agent Teaming Active Learning for Pose Estimation
Gong, J., Fan, Z., Ke, Q., Rahmani, H., & Liu, J. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01080 · Cited by 62
REMOTE: Reinforced Motion Transformation Network for Semi-supervised 2D Pose Estimation in Videos
Ma, X., Rahmani, H., Fan, Z., Yang, B., Chen, J., & Liu, J. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i2.20089 · Cited by 10
Chen, T., Wang, Z., Li, L., Li, D., Li, Z., Chang, X., Bian, P., Liang, G., Wang, Q., & Xie, T. (2025) · Schloss Dagstuhl – Leibniz-Zentrum für Informatik · doi:10.4230/lipics.ecoop.2025.6
Cited by
MotionBERT: A Unified Perspective on Learning Human Motion Representations
Zhu, W., Ma, X., Liu, Z., Liu, L., Wu, W., & Wang, Y. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01385 · Cited by 348
PoseFormerV2: Exploring Frequency Domain for Efficient and Robust 3D Human Pose Estimation
Zhao, Q., Zheng, C., Liu, M., Wang, P., & Chen, C. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00857 · Cited by 260
SkeletonMAE: Spatial-Temporal Masked Autoencoders for Self-Supervised Skeleton Action Recognition
Wu, W., Hua, Y., Zheng, C., Wu, S., Chen, C., & Lu, A. (2023) · 2023 IEEE International Conference on Multimedia and Expo Workshops (ICMEW) · doi:10.1109/icmew59549.2023.00045 · Cited by 60
A Systematic Review of Recent Deep Learning Approaches for 3D Human Pose Estimation
El Kaid, A., & Baïna, K. (2023) · Journal of Imaging · doi:10.3390/jimaging9120275 · Cited by 44
Metadata
- Authors
- Zhang, J., Tu, Z., Yang, J., Chen, Y., & Yuan, J.
- Venue
- 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2022-06
- Pages
- 13222-13232
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W4312417903
- Cited by
- 426
- References
- 53
- FWCI
- 31.87
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.