Multi-Modality Multi-Task Recurrent Neural Network for Online Action Detection
Abstract
Online action detection is a brand new challenge and plays a critical role in visual surveillance analytics. It goes one step further than a conventional action recognition task, which recognizes human actions from well-segmented clips. Online action detection is desired to identify the action type and localize action positions on the fly from the untrimmed stream data. In this paper, we propose a multi-modality multi-task recurrent neural network, which incorporates both RGB and Skeleton networks. We design different temporal modeling networks to capture specific characteristics from various modalities. Then, a deep long short-term memory subnetwork is utilized effectively to capture the complex long-range temporal dynamics, naturally avoiding the conventional sliding window design and thus ensuring high computational efficiency. Constrained by a multi-task objective function in the training phase, this network achieves superior detection performance and is capable of automatically localizing the start and end points of actions more accurately. Furthermore, embedding subtask of regression provides the ability to forecast the action prior to its occurrence. We evaluate the proposed method and several other methods in action detection and forecasting on the online action detection data set and gaming action data set datasets. Experimental results demonstrate that our model achieves the state-of-the-art performance on both tasks.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
By using them with RGB images as input, cross-modal perceiving has attracted increasing attention in real-world applications, e.g., scene parsing for autonomous driving [181], [85], object detection and tracking in low-light scenarios [182], [183], and action recognition [184].
“By using them with RGB images as input, cross-modal perceiving has attracted increasing attention in real-world applications, e.g., scene parsing for autonomous driving [181], [85], object detection and tracking in low-light scenarios [182], [183], and action recognition [184].”
From Empowering things with intelligence: a survey of the progress, challenges, and opportunities in artificial intelligence of things (2021) · cites this paper as [184]
There are three ways of fusing multi-modal data: at the input level [181], at the feature level [185], [186], [85], [182], [183], and at the output level [184].
“There are three ways of fusing multi-modal data, i.e., at the input level [181], at the feature level [185], [186], [85], [182], [183], and at the output level [184], respectively.”
From Empowering things with intelligence: a survey of the progress, challenges, and opportunities in artificial intelligence of things (2021) · cites this paper as [184]
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · see citing works
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 74,248 · see citing works
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562 · see citing works
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818 · see citing works
The pascal visual object classes (VOC) challenge
Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2010) · International Journal of Computer Vision · doi:10.1007/s11263-009-0275-4 · Cited by 19,599 · see citing works
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766 · see citing works
Experiments with a new boosting algorithm
Freund, Y., & Schapire, R. E. (1996) · Cited by 8,189 · see citing works
Shi, J., & Tomasi (1994) · Proceedings of IEEE Conference on Computer Vision and Pattern Recognition CVPR-94 · doi:10.1109/cvpr.1994.323794 · Cited by 7,724 · cited 1× · see citing works
“Bag-of-Feature (BOF) technique [36] tracks feature points in consecutive frames and encodes the extracted trajectories to capture the motion of objects in the video.”
BackgroundTwo-Stream Convolutional Networks for Action Recognition in Videos
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1406.2199 · Cited by 6,756 · see citing works
Long-term recurrent convolutional networks for visual recognition and description
Donahue, J., Hendricks, L. A., Guadarrama, S., Rohrbach, M., Venugopalan, S., Darrell, T., & Saenko, K. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298878 · Cited by 5,228 · see citing works
Backpropagation through time: what it does and how to do it
Werbos, P. J. (1990) · Proceedings of the IEEE · doi:10.1109/5.58337 · Cited by 4,879 · see citing works
Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning
Szegedy, C., Ioffe, S., Vanhoucke, V., & Alemi, A. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.11231 · Cited by 4,636 · see citing works
Ucf101: A dataset of 101 human actions classes from videos in the wild
Soomro, K., Zamir, A. R., & Shah, M. (2012) · arXiv · doi:10.48550/arxiv.1212.0402 · Cited by 4,575 · see citing works
Visual perception of biological motion and a model for its analysis
Johansson, G. (1973) · Perception & Psychophysics · doi:10.3758/bf03212378 · Cited by 4,547 · see citing works
Temporal Segment Networks: Towards Good Practices for Deep Action Recognition
Wang, L., Xiong, Y., Wang, Z., Qiao, Y., Lin, D., Tang, X., & Gool, L. V. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46484-8_2 · Cited by 4,164 · see citing works
Real-time human pose recognition in parts from single depth images
Shotton, J., Fitzgibbon, A. W., Cook, M., Sharp, T., Finocchio, M., Moore, R., Kipman, A., & Blake, A. (2011) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2011.5995316 · Cited by 3,545 · see citing works
Action Recognition with Improved Trajectories
Wang, H., & Schmid, C. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.441 · Cited by 3,407 · see citing works
NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis
Shahroudy, A., Liu, J., Ng, T.-T., & Wang, G. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.115 · Cited by 3,148 · see citing works
Self-citationConvolutional Two-Stream Network Fusion for Video Action Recognition
Feichtenhofer, C., Pinz, A. J., & Zisserman, A. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.213 · Cited by 2,842 · see citing works
Improving the Fisher Kernel for Large-Scale Image Classification
Perronnin, F., Sánchez, J., & Mensink, T. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15561-1_11 · Cited by 2,588 · see citing works
ActivityNet: A large-scale video benchmark for human activity understanding
Heilbron, F. C., Escorcia, V., Ghanem, B., & Niebles, J. C. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298698 · Cited by 2,578 · see citing works
Beyond short snippets: Deep networks for video classification
Ng, J. Y.-H., Hausknecht, M., Vijayanarasimhan, S., Vinyals, O., Monga, R., & Toderici, G. D. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7299101 · Cited by 2,121 · see citing works
Hierarchical recurrent neural network for skeleton based action recognition
Du, Y., Wang, W., & Wang, L. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298714 · Cited by 1,962 · see citing works
Mining actionlet ensemble for action recognition with depth cameras
Wang, J., Liu, Z., Wu, Y. N., & Yuan, J. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6247813 · Cited by 1,556 · see citing works
Human Action Recognition by Representing 3D Skeletons as Points in a Lie Group
Vemulapalli, R., Arrate, F., & Chellappa, R. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.82 · Cited by 1,506 · see citing works
Action recognition based on a bag of 3D points
Li, W., Zhang, Z., & Liu, Z. (2010) · 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2010.5543273 · Cited by 1,478 · see citing works
Spatio-Temporal LSTM with Trust Gates for 3D Human Action Recognition
Liu, J., Shahroudy, A., Xu, D., & Wang, G. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46487-9_50 · Cited by 1,353 · see citing works
Self-citationGradient Flow in Recurrent Nets: The Difficulty of Learning LongTerm Dependencies
(2009) · Institute of Electrical and Electronics Engineers eBooks · doi:10.1109/9780470544037.ch14 · Cited by 1,312 · see citing works
Mehta, D., Sridhar, S., Sotnychenko, O., Rhodin, H., Shafiei, M., Seidel, H., Xu, W., Casas, D., & Theobalt, C. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073596 · Cited by 1,156 · see citing works
An End-to-End Spatio-Temporal Attention Model for Human Action Recognition from Skeleton Data
Song, S., Lan, C., Xing, J., Zeng, W., & Liu, J. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.11212 · Cited by 1,011 · see citing works
Self-citationA survey of vision-based methods for action representation, segmentation and recognition
Weinland, D., Ronfard, R. P., & Boyer, E. (2010) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2010.10.002 · Cited by 987 · see citing works
Zhu, W., Lan, C., Xing, J., Zeng, W.-J., Li, Y.-H., Shen, L., & Xie, X.-H. (2016) · AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10451 · Cited by 934 · see citing works
Self-citationEnd-to-End Learning of Action Detection from Frame Glimpses in Videos
Yeung, S., Russakovsky, O., Mori, G., & Fei-Fei, L. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.293 · Cited by 585 · see citing works
Dynamic Image Networks for Action Recognition
Bilen, H., Fernando, B., Gavves, E., Vedaldi, A., & Gould, S. J. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.331 · Cited by 571 · see citing works
Human activity prediction: Early recognition of ongoing activities from streaming videos
Ryoo, M. S. (2011) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2011.6126349 · Cited by 565 · see citing works
Graves, A. (2012) · Studies in computational intelligence · doi:10.1007/978-3-642-24797-2_2 · Cited by 560 · see citing works
Two-person interaction detection using body-pose features and multiple instance learning
Yun, K., Honorio, J., Chattopadhyay, D., Berg, T. L., & Samaras, D. (2012) · 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2012.6239234 · Cited by 539 · see citing works
Human action recognition using a temporal hierarchy of covariance descriptors on 3D joint locations
Hussein, M. E., Torki, M., Gowayyed, M. A., & El-Saban, M. (2013) · Cited by 507 · see citing works
Lifting from the Deep: Convolutional 3D Pose Estimation from a Single Image
Tomè, D., Russell, C., & Agapito, L. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.603 · Cited by 505 · see citing works
Learning Actionlet Ensemble for 3D Human Action Recognition
Wang, J., Liu, Z., Wu, Y. N., & Yuan, J. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.198 · Cited by 503 · see citing works
Modeling Spatial-Temporal Clues in a Hybrid Deep Learning Framework for Video Classification
Wu, Z., Wang, X., Jiang, Y., Ye, H., & Xue, X. (2015) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2733373.2806222 · Cited by 483 · see citing works
An End-to-End Spatio-Temporal Attention Model for Human Action Recognition from Skeleton DataPreprint
Song, S., Lan, C., Xing, J., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.06067 · Cited by 481 · see citing works
Self-citationZanfir, M., Leordeanu, M., & Sminchisescu, C. (2013) · IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.342 · Cited by 462 · see citing works
A Multi-stream Bi-directional Recurrent Neural Network for Fine-Grained Action Detection
Singh, B., Marks, T. K., Jones, M., Tuzel, O., & Shao, M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.216 · Cited by 461 · see citing works
Skeleton based action recognition with convolutional neural network
Du, Y., Fu, Y., & Wang, L. (2015) · 2015 3rd IAPR Asian Conference on Pattern Recognition (ACPR) · doi:10.1109/acpr.2015.7486569 · Cited by 445 · see citing works
DAPs: Deep Action Proposals for Action Understanding
Escorcia, V. A., Heilbron, F. C., Niebles, J. C., & Ghanem, B. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46487-9_47 · Cited by 401 · see citing works
Learning Activity Progression in LSTMs for Activity Detection and Early Detection
Ma, S., Sigal, L., & Sclaroff, S. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.214 · Cited by 382 · see citing works
Zhu, W., Lan, C., Xing, J., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.07772 · Cited by 287 · see citing works
Self-citationJoint Distance Maps Based Action Recognition With Convolutional Neural Networks
Li, C., Hou, Y., Wang, P., et al. (2017) · IEEE Signal Processing Letters · doi:10.1109/lsp.2017.2678539 · Cited by 284 · see citing works
Multimodal Gesture Recognition Using 3-D Convolution and Convolutional LSTM
Zhu, G., Zhang, L., Shen, P., & Song, J. (2017) · IEEE Access · doi:10.1109/access.2017.2684186 · Cited by 269 · see citing works
Action Localization with Tubelets from Motion
Jain, M., van Gemert, J. C., Jeǵou, H., Bouthémy, P., & Snoek, C. G. M. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.100 · Cited by 250 · see citing works
Spatiotemporal Deformable Part Models for Action Detection
Tian, Y., Sukthankar, R., & Shah, M. A. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.341 · Cited by 243 · see citing works
G3D: A gaming action dataset and real time action recognition evaluation framework
Bloom, V., Makris, D., & Argyriou, V. (2012) · 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2012.6239175 · Cited by 232 · see citing works
Rolling Rotations for Recognizing Human Actions from 3D Skeletal Data
Vemulapalli, R., & Chellappa, R. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.484 · Cited by 225 · see citing works
Max-Margin Early Event Detectors
Hoai, M., & la Torre, F. D. (2013) · International Journal of Computer Vision · doi:10.1007/s11263-013-0683-3 · Cited by 220 · see citing works
Wu, D., & Shao, L. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.98 · Cited by 214 · see citing works
Online Human Action Detection Using Joint Classification-Regression Recurrent Neural Networks
Li, Y., Lan, C., Xing, J., et al. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46478-7_13 · Cited by 206 · see citing works
Self-citationEfficient Nonlinear Markov Models for Human Motion
Lehrmann, A., Gehler, P., & Nowozin, S. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.171 · Cited by 195 · see citing works
Fast action proposals for human action detection and search
Yu, G., & Yuan, J. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298735 · Cited by 190 · see citing works
Regularizing Long Short Term Memory with 3D Human-Skeleton Sequences for Action Recognition
Mahasseni, B., & Todorović, S. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.333 · Cited by 169 · see citing works
Multimodal Multipart Learning for Action Recognition in Depth Videos
Shahroudy, A., Ng, T.-T., Yang, Q., & Wang, G. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2505295 · Cited by 132 · see citing works
Graph Based Skeleton Motion Representation and Similarity Measurement for Action Recognition
Wang, P., Yuan, C., Hu, W., et al. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46478-7_23 · Cited by 119 · see citing works
Discriminative Hierarchical Rank Pooling for Activity Recognition
Fernando, B., Anderson, P., Hütter, M., & Gould, S. J. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.212 · Cited by 108 · see citing works
Realistic Human Action Recognition With Multimodal Feature Selection and Fusion
Wu, Q., Wang, Z., Deng, F., Chi, Z., & Feng, D. D. (2013) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmca.2012.2226575 · Cited by 92 · see citing works
Effective Active Skeleton Representation for Low Latency Human Action Recognition
Cai, X., Zhou, W., Wu, L., et al. (2015) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2015.2505089 · Cited by 92 · see citing works
Garcia-Hernando, G., & Kim, T. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.51 · Cited by 89 · see citing works
Predicting the Where and What of Actors and Actions through Online Action Localization
Soomro, K., Idrees, H., & Shah, M. A. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.290 · Cited by 87 · see citing works
Two streams Recurrent Neural Networks for Large-Scale Continuous Gesture Recognition
Chai, X., Liu, Z., Yin, F., Liu, Z., & Chen, X. (2016) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2016.7899603 · Cited by 87 · see citing works
Joint Classification-Regression Forests for Spatially Structured Multi-object Segmentation
Glocker, B., Pauly, O., Konukoğlu, E., et al. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33765-9_62 · Cited by 83 · see citing works
VLAD3: Encoding Dynamics of Deep Features for Action Recognition
Li, Y., Li, W., Mahadevan, V., & Vasconcelos, N. M. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.215 · Cited by 81 · see citing works
Self-citationConcurrent Action Detection with Structural Prediction
Wei, P., Zheng, N., Zhao, Y., & Zhu, S. C. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.389 · Cited by 71 · see citing works
The LEAR submission at Thumos 2014
Oneaţă, D., Verbeek, J. J., & Schmid, C. (2014) · Cited by 68 · see citing works
Efficient Action Localization with Approximately Normalized Fisher Vectors
Oneaţă, D., Verbeek, J. J., & Schmid, C. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.326 · Cited by 67 · see citing works
Weakly Supervised Action Detection
Siva, P., & Xiang, T. (2011) · British Machine Vision Conference (BMVC) · doi:10.5244/c.25.65 · Cited by 59 · see citing works
Tao, L., & Vidal, R. (2015) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2015.48 · Cited by 55 · see citing works
Real-Time Multi-scale Action Detection from 3D Skeleton Data
Sharaf, A., Torki, M., Hussein, M. E., et al. (2015) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2015.138 · Cited by 53 · see citing works
Accurate Object Detection with Joint Classification-Regression Random Forests
Schulter, S., Leistner, C., Wohlhart, P., Roth, P. M., & Bischof, H. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.123 · Cited by 38 · see citing works
Linear-time online action detection from 3D skeletal data using bags of gesturelets
Meshry, M., Hussein, M. E., & Torki, M. (2016) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2016.7477587 · Cited by 30 · see citing works
Propagative Hough Voting for Human Activity Detection and Recognition
Yu, G., Yuan, J., & Liu, Z. (2014) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2014.2319594 · Cited by 28 · see citing works
Real-Time Online Action Detection Forests Using Spatio-Temporal Contexts
Baek, S., Kim, K. I., & Kim, T. (2017) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2017.25 · Cited by 25 · see citing works
Action Recognition by Time Series of Retinotopic Appearance and Motion Features
Barrett, D. P., & Siskind, J. M. (2015) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2015.2502839 · Cited by 23 · see citing works
A Multiattribute Sparse Coding Approach for Action Recognition From a Single Unknown Viewpoint
Su, T.-F., Chiang, C.-K., & Lai, S. (2015) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2015.2409012 · Cited by 16 · see citing works
Goutsu, Y., Takano, W., & Nakamura, Y. (2015) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2015.50 · Cited by 12 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Zhang, J., & Tao, D. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3039359 · Cited by 774 · see citing works
A Comprehensive Survey of Vision-Based Human Action Recognition Methods
Zhang, H.-B., Zhang, Y.-X., Zhong, B., Lei, Q., Yang, L., Du, J.-X., & Chen, D.-S. (2019) · Sensors · doi:10.3390/s19051005 · Cited by 503 · see citing works
“[30] proposed a multimodality multitask recurrent neural network to model action in RGB and skeleton data, and integrated action classification and localization into a unified network to recognize action.”
BackgroundVideo Coding for Machines: A Paradigm of Collaborative Compression and Intelligent Analytics
Duan, L., Liu, J., Yang, W., Huang, T., & Gao, W. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.3016485 · Cited by 260 · see citing works
Self-citationHachaj, T., & Piekarczyk, M. (2019) · Sensors · doi:10.3390/s19245408 · Cited by 30 · see citing works
- Authors
- Liu, J., Li, Y., Song, S., Xing, J., Lan, C., & Zeng, W.
- Affiliations
- Peking University (CN)
- Chinese Academy of Sciences (CN)
- Microsoft Research Asia (China) (CN)
- Venue
- IEEE Transactions on Circuits and Systems for Video Technology
- Published
- 2019-09
- Volume
- 29
- Issue
- 9
- Pages
- 2667-2682
- ISSN
- 1051-8215 (ISSN-L)
- 1558-2205
- OpenAlex
- W2792312609
- Cited by
- 62
- References
- 84
- Citations by year
- 2026: 2 · 2025: 3 · 2024: 6 · 2023: 12 · 2022: 11 · 2021: 2 · 2020: 12 · 2019: 9 · 2018: 5
- FWCI
- 3.50
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Transactions on Circuits and Systems for Video Technology (publishedVersion)
- Funders
- National Natural Science Foundation of China (61772043, 61672519)
- Microsoft Research (FY17-RESTHEME-013)
- CCF-Tencent Open Research Fund
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Physical Sciences › Computer Science › Human-Computer Interaction › Hand Gesture Recognition Systems
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Keywords
- online action detection, multimodal fusion, multi-task learning, recurrent neural networks, long short-term memory, temporal modeling, action localization, video surveillance
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
