Vision-based human action recognition: An overview and real world challenges
The DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index.
Abstract
Within a large range of applications in computer vision, Human Action Recognition has become one of the most attractive research fields. Ambiguities in recognizing actions does not only come from the difficulty to define the motion of body parts, but also from many other challenges related to real world problems such as camera motion, dynamic background, and bad weather conditions. There has been little research work in the real world conditions of human action recognition systems, which encourages us to seriously search in this application domain. Although a plethora of robust approaches have been introduced in the literature, they are still insufficient to fully cover the challenges. To quantitatively and qualitatively compare the performance of these methods, public datasets that present various actions under several conditions and constraints are recorded. In this paper, we investigate an overview of the existing methods according to the kind of issue they address. Moreover, we present a comparison of the existing datasets introduced for the human action recognition field.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Histograms of Oriented Gradients for Human Detection
Dalal, N., & Triggs, B. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.177 · Cited by 33,584
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., & Adam, H. (2017) · arXiv · doi:10.48550/arxiv.1704.04861 · Cited by 19,015
Nearest neighbor pattern classification
Cover, T., & Hart, P. (1967) · IEEE Transactions on Information Theory · doi:10.1109/tit.1967.1053964 · Cited by 16,936
3D Convolutional Neural Networks for Human Action Recognition
Ji, S., Xu, W., Yang, M., & Yu, K. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.59 · Cited by 6,235
Hmdb: a large video database for human motion recognition
Kuehne, H., Jhuang, H., Garrote, E., Poggio, T., & Serre, T. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126543 · Cited by 3,965
Action Recognition with Improved Trajectories
Wang, H., & Schmid, C. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.441 · Cited by 3,407
NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis
Shahroudy, A., Liu, J., Ng, T.-T., & Wang, G. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.115 · Cited by 3,148
Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition
Ordóñez, F., & Roggen, D. (2016) · Sensors · doi:10.3390/s16010115 · Cited by 2,761
A Survey on Human Activity Recognition using Wearable Sensors
Lara, O. D., & Labrador, M. A. (2013) · IEEE Communications Surveys & Tutorials · doi:10.1109/surv.2012.110112.00192 · Cited by 2,662
Research works printed but not linked: 103
- Aggarwal, 2011, Human activity analysis: a review, ACM Comput. Surv., 43, 16, 10.1145/1922649.1922653 · DOI · Paper page
- Aggarwal, 2014, Human activity recognition from 3d data: a review, Pattern Recognit. Lett., 48, 70, 10.1016/j.patrec.2014.04.011 · DOI · Paper page
- Ang, 2012, Wavelet mach filter for omnidirectional human activity recognition, Intl. Journal of Innovative Computing, Information and Control, 8, 1
- Anitha, 2019, Posture based health monitoring and unusual behavior recognition system for elderly using dynamic bayesian network, Cluster Computing, 13583, 10.1007/s10586-018-2010-9 · doi:10.1007/s10586-018-2010-9
- Baek, 2008, A sequence-action recognition applying state machine for user interface, IEEE Trans. Consum. Electron., 54, 719, 10.1109/TCE.2008.4560153 · doi:10.1109/tce.2008.4560153
- F. Baradel, C. Wolf, J. Mille, Pose-conditioned Spatio-Temporal Attention for Human Action Recognition, arXiv preprint arXiv:1703.10106. · arXiv
- Barnachon, 2014, Ongoing human action recognition with motion capture, Pattern Recognit., 47, 238, 10.1016/j.patcog.2013.06.020 · DOI
- Bobick, 1996, An appearance-based representation of action, vol. 1, 307
- Bobick, 2001, The recognition of human movement using temporal templates, IEEE Trans. Pattern Anal. Mach. Intell., 23, 257, 10.1109/34.910878 · DOI · Paper page
- Chaaraoui, 2013, Silhouette-based human action recognition using sequences of key poses, Pattern Recognit. Lett., 34, 1799, 10.1016/j.patrec.2013.01.021 · DOI
- Chakraborty, 2017, Review of constraints on vision-based gesture recognition for human–computer interaction, IET Comput. Vis., 12, 3, 10.1049/iet-cvi.2017.0052 · DOI · Paper page
- Chang, 2016, Nonparametric feature matching based conditional random fields for gesture recognition from multi-modal video, IEEE Trans. Pattern Anal. Mach. Intell., 38, 1612, 10.1109/TPAMI.2016.2519021 · DOI
- Chen, 2006, Meng Joo Er, Shiqian Wu, Illumination compensation and normalization for robust face recognition using discrete cosine transform in logarithm domain, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 36, 458, 10.1109/TSMCB.2005.857353 · DOI · Paper page
- Debes, 2016, Monitoring activities of daily living in smart homes: understanding human behavior, IEEE Signal Process. Mag., 33, 81, 10.1109/MSP.2015.2503881 · DOI · Paper page
- Duckworth, 2016, Unsupervised activity recognition using latent semantic analysis on a mobile robot, vol. 285, 1062
- Duta, 2017, Efficient human action recognition using histograms of motion gradients and vlad with descriptor shape information, Multimed. Tools Appl., 76, 22445, 10.1007/s11042-017-4795-6 · DOI
- Eraqi, 2019, Driver distraction identification with an ensemble of convolutional neural networks, Journal of Advanced Transportation, 2019, 1, 10.1155/2019/4125865 · DOI · Paper page
- Forsyth, 2006, Computational studies of human motion: part 1, tracking and motion synthesis, Found. Trends® Comput. Graph. Vis., 1, 77
- Gorelick, 2007, Actions as space-time shapes, IEEE Trans. Pattern Anal. Mach. Intell., 29, 2247, 10.1109/TPAMI.2007.70711 · DOI · Paper page
- Gorelick, 2007, Actions as space-time shapes, IEEE Trans. Pattern Anal. Mach. Intell., 29, 2247, 10.1109/TPAMI.2007.70711 · DOI · Paper page
- Guha, 2012, Learning sparse representations for human action recognition, IEEE Trans. Pattern Anal. Mach. Intell., 34, 1576, 10.1109/TPAMI.2011.253 · DOI · Paper page
- Guo, 2014, A survey on still image based human action recognition, Pattern Recognit., 47, 3343, 10.1016/j.patcog.2014.04.018 · DOI · Paper page
- Guo, 2013, Action recognition from video using feature covariance matrices, IEEE Trans. Image Process., 22, 2479, 10.1109/TIP.2013.2252622 · DOI
- Hadfield, 2017, Hollywood 3d: what are the best 3d features for action recognition?, Int. J. Comput. Vis., 121, 95, 10.1007/s11263-016-0917-2 · DOI
- He, 2017, Multi-view action recognition method based on regularized extreme learning machine, vol. 1, 854
- Hong, 2018, Not all frames are equal: aggregating salient features for dynamic texture classification, Multidimens. Syst. Signal Process., 29, 279, 10.1007/s11045-016-0463-7 · DOI
- Hu, 2017, Jointly learning heterogeneous features for rgb-d activity recognition, IEEE Trans. Pattern Anal. Mach. Intell., 39, 2186, 10.1109/TPAMI.2016.2640292 · DOI · Paper page
- Ikizler-Cinbis, 2012, Web-based classifiers for human action recognition, IEEE Trans. Multimed., 14, 1031, 10.1109/TMM.2012.2187180 · DOI
- Ishan Misra, 2016, Unsupervised learning using sequential verification for action recognition, Computer Science, Published in ArXiv
- Ji, 2010, Advances in view-invariant human motion analysis: a review, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 40, 13, 10.1109/TSMCC.2009.2027608 · DOI · Paper page
- Ji, 2013, Hidden markov model-based human action recognition using mixed features, J. Comput. Inf. Syst., 9, 3659
- Ji, 2019, A survey of human action analysis in hri applications, IEEE Trans. Circuits Syst. Video Technol.
- Jiang, 2006, Successive convex matching for action detection, vol. 2, 1646
- Jiang, 2012, Recognizing human actions by learning and matching shape-motion prototype trees, IEEE Trans. Pattern Anal. Mach. Intell., 34, 533, 10.1109/TPAMI.2011.147 · DOI
- Jun, 2013, Local transform features and hybridization for accurate face and human detection, IEEE Trans. Pattern Anal. Mach. Intell., 35, 1423, 10.1109/TPAMI.2012.219 · DOI
- Junejo, 2011, View-independent action recognition from temporal self-similarities, IEEE Trans. Pattern Anal. Mach. Intell., 33, 172, 10.1109/TPAMI.2010.68 · DOI · Paper page
- Kliper-Gross, 2012, The action similarity labeling challenge, IEEE Trans. Pattern Anal. Mach. Intell., 34, 615, 10.1109/TPAMI.2011.209 · DOI · Paper page
- Koppula, 2013, Learning human activities and object affordances from rgb-d videos, Int. J. Robot. Res., 32, 951, 10.1177/0278364913478446 · DOI · Paper page
- Kuehne, 2013, Hmdb51: a large video database for human motion recognition, vol. 12, 571
- Laptev, 2005, On space-time interest points, Int. J. Comput. Vis., 64, 107, 10.1007/s11263-005-1838-7 · DOI · Paper page
- Lavee, 2013, Propagating certainty in petri nets for activity recognition, IEEE Trans. Circuits Syst. Video Technol., 23, 326, 10.1109/TCSVT.2012.2203742 · doi:10.1109/tcsvt.2012.2203742
- Lei, 2018, A hierarchical representation for human action recognition in realistic scenes, Multimed. Tools Appl., 77, 11403, 10.1007/s11042-018-5626-0 · DOI
- Liu, 2010, Human action recognition using boosted eigenactions, Image Vis Comput., 28, 825, 10.1016/j.imavis.2009.07.009 · DOI
- Liu, 2015, Single/multi-view human action recognition via regularized multi-task learning, Neurocomputing, 151, 544, 10.1016/j.neucom.2014.04.090 · DOI
- Liu, 2015, Coupled hidden conditional random fields for rgb-d human action recognition, Signal Process., 112, 74, 10.1016/j.sigpro.2014.08.038 · DOI
- Liu, 2015, Multiple/single-view human action recognition via part-induced multitask structural learning, IEEE Transactions on Cybernetics, 45, 1194, 10.1109/TCYB.2014.2347057 · DOI
- Liu, 2016, Convolutional neural random fields for action recognition, Pattern Recognit., 59, 213, 10.1016/j.patcog.2016.03.019 · DOI
- Liu, 2017, Benchmarking a multimodal and multiview and interactive dataset for human action recognition, IEEE Transactions on Cybernetics, 47, 1781, 10.1109/TCYB.2016.2582918 · DOI · Paper page
- Lu, 2012, A human action recognition method based on tchebichef moment invariants and temporal templates, vol. 2, 76
- Lucas, 1981, An Iterative Image Registration Technique with an Application to Stereo Vision, vol. 2, 674
- Mansur, 2013, Inverse dynamics for action recognition, IEEE Transactions on Cybernetics, 43, 1226, 10.1109/TSMCB.2012.2226879 · DOI
- Minhas, 2010, Human action recognition using extreme learning machine based on visual vocabularies, Neurocomputing, 73, 1906, 10.1016/j.neucom.2010.01.020 · DOI
- Moeslund, 2006, A survey of advances in vision-based human motion capture and analysis, Comput. Vis. Image Understand., 104, 90, 10.1016/j.cviu.2006.08.002 · DOI · Paper page
- Nasiri, 2017, Human action recognition by fuzzy hidden markov model, Int. J. Comput. Vis. Robot., 7, 538, 10.1504/IJCVR.2017.086279 · doi:10.1504/ijcvr.2017.086279
- Nazir, 2018, Evaluating a bag-of-visual features approach using spatio-temporal features for action recognition, Comput. Electr. Eng., 72, 660, 10.1016/j.compeleceng.2018.01.037 · DOI
- Nez, 2018, Convolutional neural networks and long short-term memory for skeleton-based human activity and hand gesture recognition, Pattern Recognit., 76, 80, 10.1016/j.patcog.2017.10.033 · DOI · Paper page
- Niebles, 2008, Unsupervised learning of human action categories using spatial-temporal words, Int. J. Comput. Vis., 79, 299, 10.1007/s11263-007-0122-4 · DOI · Paper page
- Oikonomopoulos, 2006, Spatiotemporal salient points for visual recognition of human actions, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 36, 710, 10.1109/TSMCB.2005.861864 · DOI · Paper page
- Park, 2004, A hierarchical bayesian network for event recognition of human actions and interactions, Multimed. Syst., 10, 164, 10.1007/s00530-004-0148-1 · DOI · Paper page
- Peng, 2013, Exploring motion boundary based sampling and spatial-temporal context descriptors for action recognition, BMVC, 20, 93
- Quattoni, 2007, Hidden conditional random fields, IEEE Trans. Pattern Anal. Mach. Intell., 29, 1848, 10.1109/TPAMI.2007.1124 · DOI · Paper page
- Rahman, 2012, Recognising human actions by analysing negative spaces, IET Comput. Vis., 6, 197, 10.1049/iet-cvi.2011.0185 · DOI
- H. Rahmani, A. Mahmood, D. Huynh, A. Mian, Action Classification with Locality-Constrained Linear Coding, arXiv preprint arXiv:1408.3810. · arXiv
- Rahmani, 2016, Discriminative human action classification using locality-constrained linear coding, Pattern Recognit. Lett., 72, 62, 10.1016/j.patrec.2015.07.015 · DOI
- Rahmani, 2016, Histogram of oriented principal components for cross-view action recognition, IEEE Trans. Pattern Anal. Mach. Intell., 38, 2430, 10.1109/TPAMI.2016.2533389 · DOI · Paper page
- Rahmani, 2018, Learning a deep model for human action recognition from novel viewpoints, IEEE Trans. Pattern Anal. Mach. Intell., 40, 667, 10.1109/TPAMI.2017.2691768 · DOI · Paper page
- Ramanathan, 2014, Human action recognition with video data: research and evaluation challenges, IEEE Transactions on Human-Machine Systems, 44, 650, 10.1109/THMS.2014.2325871 · DOI · Paper page
- Reddy, 2013, Recognizing 50 human action categories of web videos, Mach. Vis. Appl., 24, 971, 10.1007/s00138-012-0450-4 · DOI · Paper page
- Schuldt, 2004, Recognizing human actions: a local svm approach, vol. 3, 32
- Shahroudy, 2016, Multimodal multipart learning for action recognition in depth videos, IEEE Trans. Pattern Anal. Mach. Intell., 38, 2123, 10.1109/TPAMI.2015.2505295 · DOI · Paper page
- Shao, 2012, Human action segmentation and recognition via motion and shape analysis, Pattern Recognit. Lett., 33, 438, 10.1016/j.patrec.2011.05.015 · DOI
- Shechtman, 2007, Space-time behavior-based correlation-or-how to tell if two underlying motion fields are similar without computing them?, IEEE Trans. Pattern Anal. Mach. Intell., 29, 2045, 10.1109/TPAMI.2007.1119 · DOI
- Sigal, 2010, Synchronized video and motion capture dataset and baseline algorithm for evaluation of articulated human motion, Int. J. Comput. Vis., 87, 4, 10.1007/s11263-009-0273-6 · DOI · Paper page
- Singh, 2008, Human activity recognition based on silhouette directionality, IEEE Trans. Circuits Syst. Video Technol., 18, 1280, 10.1109/TCSVT.2008.928888 · DOI
- Sipiran, 2011, Harris 3d: a robust extension of the harris operator for interest point detection on 3d meshes, Vis. Comput., 27, 963, 10.1007/s00371-011-0610-y · DOI
- Song, 2014, Body surface context: a new robust feature for action recognition from depth videos, IEEE Trans. Circuits Syst. Video Technol., 24, 952, 10.1109/TCSVT.2014.2302558 · DOI
- K. Soomro, A. R. Zamir, M. Shah, Ucf101: A Dataset of 101 Human Actions Classes from Videos in the Wild, arXiv preprint arXiv:1212.0402. · arXiv
- Subramanian, 2012, Human action recognition using meta-cognitive neuro-fuzzy inference system, Int. J. Neural Syst., 22, 1250028, 10.1142/S0129065712500281 · DOI
- Sunny, 2015, Applications and challenges of human activity recognition using sensors in a smart environment, IJIRST Int. J. Innov. Res. Sci. Technol, 2, 50
- Suykens, 1999, Least squares support vector machine classifiers, Neural Process. Lett., 9, 293, 10.1023/A:1018628609742 · DOI · Paper page
- Turaga, 2008, Machine recognition of human activities: a survey, IEEE Trans. Circuits Syst. Video Technol., 18, 1473, 10.1109/TCSVT.2008.2005594 · DOI · Paper page
- Varol, 2018, Long-term temporal convolutions for action recognition, IEEE Trans. Pattern Anal. Mach. Intell., 40, 1510, 10.1109/TPAMI.2017.2712608 · DOI · Paper page
- Wang, 2013, Semantic context based refinement for news video annotation, Multimed. Tools Appl., 67, 607, 10.1007/s11042-012-1060-x · DOI
- Wang, 2018, Rgb-d-based human motion recognition with deep learning: a survey, Comput. Vis. Image Understand., 171, 118, 10.1016/j.cviu.2018.04.007 · DOI · Paper page
- Weinland, 2006, Free viewpoint action recognition using motion history volumes, Comput. Vis. Image Understand., 104, 249, 10.1016/j.cviu.2006.07.013 · DOI · Paper page
- Weinland, 2011, A survey of vision-based methods for action representation, segmentation and recognition, Comput. Vis. Image Understand., 115, 224, 10.1016/j.cviu.2010.10.002 · DOI · Paper page
- Wu, 2013, Silhouette analysis-based action recognition via exploiting human poses, IEEE Trans. Circuits Syst. Video Technol., 23, 236, 10.1109/TCSVT.2012.2203731 · DOI
- Wu, 2013, Action recognition using multilevel features and latent structural svm, IEEE Trans. Circuits Syst. Video Technol., 23, 1422, 10.1109/TCSVT.2013.2244794 · DOI
- Xiao, 2018, Action recognition based on hierarchical dynamic bayesian network, Multimed. Tools Appl., 77, 6955, 10.1007/s11042-017-4614-0 · DOI
- Xie, 2005, Face recognition under varying illumination based on a 2d face shape model, Pattern Recognit., 38, 221, 10.1016/S0031-3203(04)00275-4 · doi:10.1016/s0031-3203(04)00275-4
- Yang, 2013, Discovering motion primitives for unsupervised grouping and one-shot learning of human actions, gestures, and expressions, IEEE Trans. Pattern Anal. Mach. Intell., 35, 1635, 10.1109/TPAMI.2012.253 · DOI
- Yang, 2013, Rgb-depth feature for 3d human activity recognition, China Communications, 10, 93, 10.1109/CC.2013.6571292 · DOI
- Yeung, 2018, Every moment counts: dense detailed labeling of actions in complex videos, Int. J. Comput. Vis., 126, 375, 10.1007/s11263-017-1013-y · DOI · Paper page
- Yilmaz, 2005, Actions sketch: a novel action representation, vol. 1, 984
- Yilmaz, 2005, Recognizing human actions in videos acquired by uncalibrated moving cameras, vol. 1, 150
- Zaidenberg, 2014, Towards unsupervised sudden group movement discovery for video surveillance, vol. 2, 388
- Zelnik-Manor, 2001, Event-based analysis of video, vol. 2
- Zhang, 2017, A review on human activity recognition using vision-based method, Journal of healthcare engineering, 1
- Zhang, 2013, Auto learning temporal atomic actions for activity classification, Pattern Recognit., 46, 1789, 10.1016/j.patcog.2012.10.016 · doi:10.1016/j.patcog.2012.10.016
- Zhang, 2013, Modeling temporal interactions with interval temporal bayesian networks for complex activity recognition, IEEE Trans. Pattern Anal. Mach. Intell., 35, 2468, 10.1109/TPAMI.2013.33 · DOI
- Zhang, 2016, Rgb-d-based action recognition datasets: a survey, Pattern Recognit., 60, 86, 10.1016/j.patcog.2016.05.019 · DOI · Paper page
- Zhang, 2017, Semi-supervised image-to-video adaptation for video action recognition, IEEE Transactions on Cybernetics, 47, 960, 10.1109/TCYB.2016.2535122 · DOI
- H. Zhao, Z. Yan, H. Wang, L. Torresani, A. Torralba, Slac: A Sparsely Labeled Dataset for Action Classification and Localization, arXiv preprint arXiv:1712.09374. · arXiv
Cited by
Boulahia, S. Y., Amamra, A., Madi, M. R., & Daikh, S. (2021) · Machine Vision and Applications · doi:10.1007/s00138-021-01249-8 · Cited by 236
Metadata
- Authors
- Jegham, I., Ben Khalifa, A., Alouani, I., & Mahjoub, M. A.
- Venue
- Forensic Science International: Digital Investigation
- Publisher
- Elsevier BV
- Published
- 2020-03
- Volume
- 32
- Pages
- 200901
- ISSN
- 2666-2817 (ISSN-L)
- OpenAlex
- W3003646123
- Cited by
- 181
- References
- 18
- FWCI
- 11.00
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
- www.elsevier.com/legal/tdmrep-license (tdm)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Vision-based human action recognition: An overview and real world challenges. https://app.cochat.ai/papers/10.1016/j.fsidi.2019.200901
- The paper
- Jegham, I., Ben Khalifa, A., Alouani, I., & Mahjoub, M. A. (2020). Vision-based human action recognition: An overview and real world challenges. Forensic Science International: Digital Investigation. https://doi.org/10.1016/j.fsidi.2019.200901
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.