EHPE: Skeleton Cues-Based Gaussian Coordinate Encoding for Efficient Human Pose Estimation
Open accessThe DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
Abstract
Human pose estimation (HPE) has many wide applications such as multimedia processing, behavior understanding and human-computer interaction. Most previous studies have encountered many constraints, such as restricted scenarios and RGB inputs. To mitigate constraints to estimating the human poses in general scenarios, we present an efficient human pose estimation model (i.e., EHPE) with joint direction cues and Gaussian coordinate encoding. Specifically, we propose an anisotropic Gaussian coordinate coding method to describe the skeleton direction cues among adjacent keypoints. To the best of our knowledge, this is the first time that the skeleton direction cues is introduced to the heatmap encoding in HPE task. Then, a multi-loss function is proposed to constrain the output to prevent the overfitting. The Kullback-Leibler divergence is introduced to measure the predication label and its ground truth one. The performance of EHPE is evaluated on two HPE datasets: MS COCO and MPII. Experimental results demonstrate that EHPE can obtain robust results, and it significantly outperforms existing state-of-the-art HPE methods. Lastly, we extend the experiments on infrared images captured by our research group. The experiments achieved the impressive results regardless of insufficient color and texture information.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
He, K., Gkioxari, G., Dollar, P., & Girshick, R. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.322 · Cited by 29,137
Deep High-Resolution Representation Learning for Human Pose Estimation
Sun, K., Xiao, B., Liu, D., & Wang, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00584 · Cited by 5,984
Stacked Hourglass Networks for Human Pose Estimation
Newell, A., Yang, K., & Deng, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46484-8_29 · Cited by 5,634 · cited 2×
“Forexample,thesimplebaselinemodel[27]proposedbyXiao et al. is more intuitive and simpler than the stacked hourglass model [28].”
MethodOpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields
Cao, Z., Hidalgo, G., Simon, T., Wei, S.-E., & Sheikh, Y. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2929257 · Cited by 5,536 · cited 4×
“(cid:2) OpenPose [42]: Part affinity fields are introduced into the bottom-up HPE task.”
BackgroundInfluentialSpatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition
Yan, S., Xiong, Y., & Lin, D. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12328 · Cited by 5,410
DeepPose: Human Pose Estimation via Deep Neural Networks
Toshev, A., & Szegedy, C. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.214 · Cited by 3,185 · cited 3×
“It combines rich multiscale features and improves the robustness of keypoint location through cross-scale feature matching in comparison with [19], [21], [23].”
BackgroundWei, S.-E., Ramakrishna, V., Kanade, T., & Sheikh, Y. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.511 · Cited by 2,966 · cited 1×
“For heatmap-based regression, the convolutional pose machine [33] performs multistage regression on the heatmap.”
Method2D Human Pose Estimation: New Benchmark and State of the Art Analysis
Andriluka, M., Pishchulin, L., Gehler, P., & Schiele, B. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.471 · Cited by 2,688 · cited 2×
“MPII dataset [40]: It contains about 25K images, a total of 40K human body instances, and 16 keypoints for each image.”
BackgroundMethodSimple Baselines for Human Pose Estimation and Tracking
Xiao, B., Wu, H., & Wei, Y. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01231-1_29 · Cited by 2,149
RMPE: Regional Multi-person Pose Estimation
Fang, H.-S., Xie, S., Tai, Y.-W., & Lu, C. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.256 · Cited by 1,884 · cited 2×
“(cid:2) RMPE [24]: A symmetric spatial transformation network is proposed to strengthen human object instances.”
BackgroundMethodCascaded Pyramid Network for Multi-person Pose Estimation
Chen, Y., Wang, Z., Peng, Y., Zhang, Z., Yu, G., & Sun, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00742 · Cited by 1,799
Self-citationDeeperCut: A Deeper, Stronger, and Faster Multi-person Pose Estimation Model
Insafutdinov, E., Pishchulin, L., Andres, B., Andriluka, M., & Schiele, B. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46466-4_3 · Cited by 1,130 · cited 1×
“As an upstream task of behavior monitoring [4], human-computer interaction [5], [6], and [7], action recognition [8],[9],andonlinelearning[10],HPEmustbeimprovedinterms ofaccuracyandrobustness[11],[12].”
BackgroundRegion Covariance: A Fast Descriptor for Detection and Classification
Tuzel, O., Porikli, F., & Meer, P. (2006) · Lecture Notes in Computer Science · doi:10.1007/11744047_45 · Cited by 1,115 · cited 1×
“Second-order statistics, such as the covariance, are deemed to be better regional descriptors than first-order statistics [37].”
BackgroundTowards Accurate Multi-person Pose Estimation in the Wild
Papandreou, G., Zhu, T., Kanazawa, N., Toshev, A., Tompson, J., Bregler, C., & Murphy, K. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.395 · Cited by 860 · cited 1×
“As an upstream task of behavior monitoring [4], human-computer interaction [5], [6], and [7], action recognition [8],[9],andonlinelearning[10],HPEmustbeimprovedinterms ofaccuracyandrobustness[11],[12].”
BackgroundHuman Pose Estimation with Iterative Error Feedback
Carreira, J., Agrawal, P., Fragkiadaki, K., & Malik, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.512 · Cited by 805 · cited 1×
“Carreira et al. [31] used this approach of direct regression in their respective studies.”
ResultMulti-context Attention for Human Pose Estimation
Chu, X., Yang, W., Ouyang, W., Ma, C., Yuille, A. L., & Wang, X. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.601 · Cited by 737
CrowdPose: Efficient Crowded Scenes Pose Estimation and a New Benchmark
Li, J., Wang, C., Zhu, H., Mao, Y., Fang, H.-S., & Lu, C. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01112 · Cited by 613 · cited 2×
“To solve and study human frame inaccuracy and pose estimation in crowded scenes, several approaches have been proposed, such as the regional multi-person pose estimation (RMPE) [24] method, and the CrowdPose [25] method .”
MethodDistribution-Aware Coordinate Representation for Human Pose Estimation
Zhang, F., Zhu, X., Dai, H., Ye, M., & Zhu, C. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00712 · Cited by 588
Learning Graph Convolutional Network for Skeleton-Based Human Action Recognition by Neural Searching
Peng, W., Hong, X., Chen, H., & Zhao, G. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i03.5652 · Cited by 361
Is Second-Order Information Helpful for Large-Scale Visual Recognition?
Li, P., Xie, J., Wang, Q., & Zuo, W. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.228 · Cited by 294 · cited 1×
“Therefore, end-to-end learning [38] is referred to calculate the gradients.”
BackgroundLiu, H., Liu, T., Zhang, Z., Sangaiah, A. K., Yang, B., & Li, Y. (2022) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2022.3143605 · Cited by 288 · cited 2×
“The main contribution of Fastpose [29] proposed by Zhang et al.”
BackgroundSelf-citationThe Devil Is in the Details: Delving Into Unbiased Data Processing for Human Pose Estimation
Huang, J., Zhu, Z., Guo, F., & Huang, G. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00574 · Cited by 281
Multi-Scale Structure-Aware Network for Human Pose Estimation
Ke, L., Chang, M.-C., Qi, H., & Lyu, S. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01216-8_44 · Cited by 257
PaStaNet: Toward Human Activity Knowledge Engine
Li, Y., Xu, L., Liu, X., Huang, X., Xu, Y., Wang, S., Fang, H.-S., Ma, Z., Chen, M., & Lu, C. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00046 · Cited by 198
Self-citationA Coarse-Fine Network for Keypoint Localization
Huang, S., Gong, M., & Tao, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.329 · Cited by 193 · cited 2×
“(cid:2) CFN [43]: The network leverages multi-level supervision to realize the keypoint location function.”
BackgroundUniPose: Unified Human Pose Estimation in Single Images and Videos
Artacho, B., & Savakis, A. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00706 · Cited by 188
Rethinking the Heatmap Regression for Bottom-up Human Pose Estimation
Luo, Z., Wang, Z., Huang, Y., Wang, L., Tan, T., & Zhou, E. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01306 · Cited by 181 · cited 1×
“As an upstream task of behavior monitoring [4], humancomputer interaction [5-7], action recognition [8, 9], and online learning [10], HPE must be improved in terms of accuracy and robustness [11, 12].”
BackgroundKnowledge-Guided Deep Fractal Neural Networks for Human Pose Estimation
Ning, G., Zhang, Z., & He, Z. (2018) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2017.2762010 · Cited by 177
Self-citationOrientation Cues-Aware Facial Relationship Representation for Head Pose Estimation via Transformer
Liu, H., Zhang, C., Deng, Y., Liu, T., Zhang, Z., & Li, Y.-F. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3331309 · Cited by 166
Self-citationLiu, H., Nie, H., Zhang, Z., & Li, Y.-F. (2021) · Neurocomputing · doi:10.1016/j.neucom.2020.09.068 · Cited by 164 · cited 1×
“As an upstream task of behavior monitoring [4], humancomputer interaction [5-7], action recognition [8, 9], and online learning [10], HPE must be improved in terms of accuracy and robustness [11, 12].”
BackgroundSelf-citationFace-from-Depth for Head Pose Estimation on Depth Images
Borghi, G., Fabbri, M., Vezzani, R., Calderara, S., & Cucchiara, R. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2885472 · Cited by 115 · cited 1×
“Although CNNs are powerful in learning image features and have been successfully leveraged to numerous of computer vision tasks [18], three fundamental issues for CNN-based HPE remain.”
BackgroundHuman Pose Estimation with Parsing Induced Learner
Nie, X., Feng, J., Zuo, Y., & Yan, S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00224 · Cited by 106 · cited 1×
“Nie et al. [26] proposed a novel analytically induced learner, which assists HPE by effectively applying analytical information of body parts.”
BackgroundTowards High Performance Human Keypoint Detection
Zhang, J., Chen, Z., & Tao, D. (2021) · International Journal of Computer Vision · doi:10.1007/s11263-021-01482-8 · Cited by 105 · cited 2×
“[17] designed an efficient network structure named cascaded context mixer, with three useful training strategies and four effective post-processing techniques.”
MethodMulti-Person Pose Estimation Using Bounding Box Constraint and LSTM
Li, M., Zhou, Z., & Liu, X. (2019) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2019.2903455 · Cited by 91 · cited 2×
“H UMAN pose estimation (HPE) is a long-standing and challenging problem in the field of computer vision, and it aims to locate the keypoints of each person in an RGB image [1], [2], and [3].”
BackgroundMethodInfluentialHybrid Refinement-Correction Heatmaps for Human Pose Estimation
Kamel, A., Sheng, B., Li, P., Kim, J., & Feng, D. D. (2020) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2020.2999181 · Cited by 86 · cited 1×
“UMAN pose estimation (HPE) is a long-standing and challenging problem in the field of computer vision, and it aims to locate the keypoints of each person in an RGB image [1-3].”
BackgroundGaze Estimation via the Joint Modeling of Multiple Cues
Chen, W., Xu, H. F., Zhu, C., Liu, X., Lu, Y., Zheng, C., & Kong, J. (2021) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3071621 · Cited by 46 · cited 1×
“As an upstream task of behavior monitoring [4], human-computer interaction [5], [6], and [7], action recognition [8],[9],andonlinelearning[10],HPEmustbeimprovedinterms ofaccuracyandrobustness[11],[12].”
BackgroundLearning Representations From Skeletal Self-Similarities for Cross-View Action Recognition
Shao, Z., Li, Y., & Zhang, H. (2020) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2020.2965574 · Cited by 41 · cited 1×
“In our previous work [16], a light multi-stream neural network is proposed to learn the view-invariant representations from skeletal self-similarities of varying scales for the human action recognition.”
BackgroundSelf-citation
Cited by
Convolutional Neural Networks: A Survey
Krichen, M. (2023) · Computers · doi:10.3390/computers12080151 · Cited by 623
“By understanding the role of each layer, researchers and practitioners can design more effective CNN architectures for a wide range of computer vision tasks [62–64].”
Metadata
- Authors
- Liu, H., Liu, T., Chen, Y., Zhang, Z., & Li, Y.-F.
- Venue
- IEEE Transactions on Multimedia
- Published
- 2024
- Volume
- 26
- Pages
- 8464-8475
- ISSN
- 1520-9210 (ISSN-L)
- 1941-0077
- OpenAlex
- W4290717512
- Cited by
- 213
- References
- 38
- FWCI
- 14.65
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.