A Comprehensive Survey of Vision-Based Human Action Recognition Methods
Open accessThe DOI matches its Crossref record; 119 of the 119 references it cites were found in a registry or an index.
Abstract
Although widely used in many applications, accurate and efficient human action recognition remains a challenging area of research in the field of computer vision. Most recent surveys have focused on narrow problems such as human action recognition methods using depth data, 3D-skeleton data, still image data, spatiotemporal interest point-based methods, and human walking motion recognition. However, there has been no systematic survey of human action recognition. To this end, we present a thorough review of human action recognition methods and provide a comprehensive overview of recent approaches in human action recognition research, including progress in hand-designed action features in RGB and depth data, current deep learning-based action feature representation methods, advances in human–object interaction recognition methods, and the current prominent research topic of action detection methods. Finally, we present several analysis recommendations for researchers. This survey paper provides an essential reference for those interested in further research on human action recognition.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“videobased activity recognition is naturally complementary to other sensing modalities Zhang et al. (2019);”
From Opportunity++: A Multimodal Dataset for Video- and Wearable, Object and Ambient Sensors-Based Human Activity Recognition (2021) · cites this paper as Zhang et al. (2019)
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Representation Learning: A Review and New Perspectives
Bengio, Y., Courville, A., & Vincent, P. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.50 · Cited by 13,001 · cited 1×
“Feature representation and selection is a classic problem in computer vision and machine learning [8].”
BackgroundLearning Spatiotemporal Features with 3D Convolutional Networks
Tran, D., Bourdev, L., Fergus, R., Torresani, L., & Paluri, M. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.510 · Cited by 9,593 · cited 4×
“Depending on the structure of the deep learning network, the main representative works can be summarized as discussing methods based on two-stream convolutional networks [19], those based on 3D convolutional networks [20], and those based on long short-term memory (LSTM) [83].”
BackgroundInfluentialQuo Vadis, Action Recognition? A New Model and the Kinetics Dataset
Carreira, J., & Zisserman, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.502 · Cited by 9,570
Realtime Multi-person 2D Pose Estimation Using Part Affinity Fields
Cao, Z., Simon, T., Wei, S.-E., & Sheikh, Y. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.143 · Cited by 7,539
Two-Stream Convolutional Networks for Action Recognition in Videos
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1406.2199 · Cited by 6,756 · cited 4×
“Depending on the structure of the deep learning network, the main representative works can be summarized as discussing methods based on two-stream convolutional networks [19], those based on 3D convolutional networks [20], and those based on long short-term memory (LSTM) [83].”
BackgroundInfluentialSpatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition
Yan, S., Xiong, Y., & Lin, D. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12328 · Cited by 5,410
Proceedings European Conference on Computer Vision 2002
Heyden, A., Sparr, G., Nielsen, M., & Johansen, P. (2002) · Lund University Publications (Lund University) · Cited by 5,153
Temporal Segment Networks: Towards Good Practices for Deep Action Recognition
Wang, L., Xiong, Y., Wang, Z., Qiao, Y., Lin, D., Tang, X., & Van Gool, L. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46484-8_2 · Cited by 3,958
Action Recognition with Improved Trajectories
Wang, H., & Schmid, C. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.441 · Cited by 3,407
NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis
Shahroudy, A., Liu, J., Ng, T.-T., & Wang, G. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.115 · Cited by 3,148 · cited 2×
“However, with the development of large-scale and challenging RGBD and skeleton datasets, for example, the NTU RGB+D dataset, deep-leaning-based methods for depth and skeleton data are becoming the primary research topic in the field of human action recognition.”
Research works printed but not linked: 9
- Yu, K., and Yun, F. (arXiv, 2018). Human Action Recognition and Prediction: A Survey, arXiv.
- Diba, A., Fayyaz, M., Sharma, V., Karami, A.H., Arzani, M.M., Yousefzadeh, R., and Van Gool, L. (arXiv, 2017). Temporal 3D ConvNets: New Architecture and Transfer Learning for Video Classification, arXiv.
- Wang, P., Li, W., Gao, Z., Zhang, J., Tang, C., and Ogunbona, P. (arXiv, 2015). Deep Convolutional Neural Networks for Action Recognition Using Depth Map Sequences, arXiv.
- Zhu, Y., Lan, Z., Newsam, S., and Hauptmann, A.G. (arXiv, 2017). Hidden two-stream convolutional networks for action recognition, arXiv.
- Soomro, K., Zamir, A.R., and Shah, M. (arXiv, 2012). UCF101: A Dataset of 101 Human Actions Classes from Videos in The Wild, arXiv.
- Kay, W., Carreira, J., Simonyan, K., Zhang, B., Hillier, C., Vijayanarasimhan, S., Viola, F., Green, T., Back, T., and Natsev, P. (arXiv, 2017). The Kinetics Human Action Video Dataset, arXiv.
- Gupta, S., and Malik, J. (arXiv, 2015). Visual Semantic Role Labeling, arXiv.
- Yu, G., Liu, Z., and Yuan, J. (2014). Discriminative Orderlet Mining for Real-Time Recognition of Human-Object Interaction, Springer.
- Gorban, A., Idrees, H., Jiang, Y., Zamir, A.R., Laptev, I., Shah, M., and Sukthankar, R. (2015, January 7–12). THUMOS challenge: Action recognition with a large number of classes. Proceedings of the CVPR Workshop, Boston, MA, USA.
Cited by
Items 501–512 of 512
Paul, A. T., & Damahe, L. B. (2026) · African Journal Of Applied Research · doi:10.26437/ytetf928
“Vision-based systems do not require wearable devices to analyse posture, use RGB or RGB-D data, and are applicable in real-world settings (Beddiar et al., 2020; Zhang et al., 2019).”
Zhang, Z., Mao, Y., Ma, Y., & Gui, X. (2026) · Sensors · doi:10.3390/s26154770
“In parallel, vision-based educational sensing systems have been widely explored for classroom analytics [25–27].”
Vision-based ballast particle motion monitoring using semantic and instance segmentation
Xue, M., Pang, T., Wang, R., Chen, R., Liu, K., Lü, T., & Wang, P. (2026) · Computer-Aided Civil and Infrastructure Engineering · doi:10.1016/j.cacaie.2026.100177
IMU- and Vision-Based Measurement Techniques for Joint Kinematics: A Narrative Review
Ceriola, L., Molinaro, L., Taborri, J., Patanè, F., & Mileti, I. (2026) · Sensors · doi:10.3390/s26165063
“…which involves recognizing activities from a series of observations; this has been comprehensively explored in reviews by Ke et al. (2013) [32] and Zhang et al. (2019) [33], with Ke et al. (2013) [32] detailing the recognition of activities crucial for applications like video surveillance,…”
InfluentialAI-based vision techniques for human activity recognition in surveillance videos
Deotale, D., Verma, M., Suresh, P., Singal, G., & Sharma, L. K. (2026) · Discover Artificial Intelligence · doi:10.1007/s44163-026-01758-3
Nguyen, Q. H., & Duong, L. G. K. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-25620-1_28
Sparse frame selection for graph-based deadlift form assessment
shoaib, Attia, N. A., Yu, Z., Khoriba, G. A., & Arafa, T. (2026) · Scientific Reports · doi:10.1038/s41598-026-67912-0
“Early approaches to human action recognition employed hand-crafted features and traditional machine learning methods 4,5 .”
Activity recognition via Riemannian optimization and multimodal Transformers
Soleimani, F., Khodabandelou, G., Chibani, A., & Amirat, Y. (2026) · Machine Learning with Applications · doi:10.1016/j.mlwa.2026.101021
Weakly supervised temporal action localization in a video using phase correlation
Phathong, P., & Siriborvornratanakul, T. (2026) · Soft Computing · doi:10.1007/s00500-026-11477-7
“According to Xia and Zhan (2020), Zhu et al. (2020) and Zhang et al. (2019), the challenges in the action localizationtaskare:(a)Theactionisambiguous,thereareno clear boundaries of start and end time of the action (Schindler and Van Gool 2008; Satkin and Hebert 2010).”
Deep Learning Methods for Video Classification and Tagging
Tuyet, V. T. H., & Bình, N. T. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-2600-1393-9.ch004
Arya, V., & Maji, S. (2026) · IEEE Access · doi:10.1109/access.2026.3747756
Real-time Recognition of Contextual Composite Activities from Asynchronous Multi-IMU Sensor Streams
Mondal, A., Khan, A., Chugh, G., & Chakraborty, S. (2026) · ACM Transactions on Sensor Networks · doi:10.1145/3860888
Metadata
- Authors
- Zhang, H.-B., Zhang, Y.-X., Zhong, B., Lei, Q., Yang, L., Du, J.-X., & Chen, D.-S.
- Affiliations
- Department of Computer Science and Technology, Huaqiao University, Xiamen 361000, China
- Huaqiao University (CN)
- Corresponding author
- Du, Ji-Xiang
- Venue
- Sensors
- Publisher
- MDPI AG
- Published
- 2019-02-27
- Volume
- 19
- Issue
- 5
- Pages
- 1005
- ISSN
- 1424-8220 (ISSN-L)
- PMC
- PMC6427144
- OpenAlex
- W2917819557
- Cited by
- 503
- References
- 119
- Citations by year
- 2026: 33 · 2025: 55 · 2024: 79 · 2023: 97 · 2022: 71 · 2021: 93 · 2020: 62 · 2019: 12
- FWCI
- 6.84
- Type
- journal article
- Language
- en
- Open access
- gold
- dx.doi.org
- Licence
- Locations
- Sensors (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Physical Sciences › Computer Science › Human-Computer Interaction › Hand Gesture Recognition Systems
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Keywords
- human action recognition, depth information, 3D skeleton data, human-object interaction recognition, action detection
- MeSH terms
- Algorithms, Humans, Human Activities, Motion, Pattern Recognition, Automated, Surveys and Questionnaires, Skeleton, Vision, Ocular, Visual Perception
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
