The DOI matches its Crossref record; 53 of the 53 references it cites were found in a registry or an index.
Hierarchical Clustering Multi-Task Learning for Joint Human Action Grouping and Recognition
Abstract
This paper proposes a hierarchical clustering multi-task learning (HC-MTL) method for joint human action grouping and recognition. Specifically, we formulate the objective function into the group-wise least square loss regularized by low rank and sparsity with respect to two latent variables, model parameters and grouping information, for joint optimization. To handle this non-convex optimization, we decompose it into two sub-tasks, multi-task learning and task relatedness discovery. First, we convert this non-convex objective function into the convex formulation by fixing the latent grouping information. This new objective function focuses on multi-task learning by strengthening the shared-action relationship and action-specific feature learning. Second, we leverage the learned model parameters for the task relatedness measure and clustering. In this way, HC-MTL can attain both optimal action models and group discovery by alternating iteratively. The proposed method is validated on three kinds of challenging datasets, including six realistic action datasets (Hollywood2, YouTube, UCF Sports, UCF50, HMDB51 & UCF101), two constrained datasets (KTH & TJU), and two multi-view datasets (MV-TJU & IXMAS). The extensive experimental results show that: 1) HC-MTL can produce competing performances to the state of the arts for action recognition and grouping; 2) HC-MTL can overcome the difficulty in heuristic action grouping simply based on human knowledge; 3) HC-MTL can avoid the possible inconsistency between the subjective action grouping depending on human knowledge and objective action grouping based on the feature subspace distributions of multiple actions. Comparison with the popular clustered multi-task learning further reveals that the discovered latent relatedness by HC-MTL aids inducing the group-wise multi-task learning and boosts the performance. To the best of our knowledge, ours is the first work that breaks the assumption that all actions are either independent for individual learning or correlated for joint modeling and proposes HC-MTL for automated, joint action grouping and modeling.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Boyd, S., & Vandenberghe, L. (2004) · doi:10.1017/cbo9780511804441 · Cited by 56,601
On Spectral Clustering: Analysis and an algorithm
Ng, A. Y., Jordan, M. I., & Weiss, Y. (2001) · Cited by 8,159
3D Convolutional Neural Networks for Human Action Recognition
Ji, S., Xu, W., Yang, M., & Yu, K. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.59 · Cited by 6,235
Ucf101: A dataset of 101 human actions classes from videos in the wild
Soomro, K., Zamir, A. R., & Shah, M. (2012) · arXiv · doi:10.48550/arxiv.1212.0402 · Cited by 4,575
Hmdb: a large video database for human motion recognition
Kuehne, H., Jhuang, H., Garrote, E., Poggio, T., & Serre, T. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126543 · Cited by 3,965
Introductory Lectures on Convex Optimization: A Basic Course
Nesterov, Y. E. (2014) · Medical Entomology and Zoology · Cited by 3,557
Learning realistic human actions from movies
Laptev, I., Marszalek, M., Schmid, C., & Rozenfeld, B. (2008) · 2008 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2008.4587756 · Cited by 3,444
Action Recognition with Improved Trajectories
Wang, H., & Schmid, C. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.441 · Cited by 3,407
Sparse Subspace Clustering: Algorithm, Theory, and Applications
Elhamifar, E., & Vidal, R. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.57 · Cited by 3,106
Dense Trajectories and Motion Boundary Descriptors for Action Recognition
Wang, H., Kläser, A., Schmid, C., & Liu, C.-L. (2013) · International Journal of Computer Vision · doi:10.1007/s11263-012-0594-8 · Cited by 1,669
Convex multi-task feature learning
Argyriou, A., Evgeniou, T., & Pontil, M. (2008) · Machine Learning · doi:10.1007/s10994-007-5040-8 · Cited by 1,528
A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data
Ando, R. K., & Zhang, T. (2005) · doi:10.5555/1046920.1194905 · Cited by 1,311
Laptev, & Lindeberg (2003) · Proceedings Ninth IEEE International Conference on Computer Vision · doi:10.1109/iccv.2003.1238378 · Cited by 1,300
Evaluation of local spatio-temporal features for action recognition
Wang, H., Ullah, M. M., Klaser, A., Laptev, I., & Schmid, C. (2009) · Procedings of the British Machine Vision Conference 2009 · doi:10.5244/c.23.124 · Cited by 1,296
Action MACH a spatio-temporal Maximum Average Correlation Height filter for action recognition
Rodriguez, M. D., Ahmed, J., & Shah, M. (2008) · 2008 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2008.4587727 · Cited by 1,223
Marszalek, M., Laptev, I., & Schmid, C. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206557 · Cited by 1,168
Recognizing realistic actions from videos “in the wild”
Liu, J., Luo, J., & Shah, M. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206744 · Cited by 1,075
Free viewpoint action recognition using motion history volumes
Weinland, D., Ronfard, R., & Boyer, E. (2006) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2006.07.013 · Cited by 896
Action bank: A high-level representation of activity in video
Sadanand, S., & Corso, J. J. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247806 · Cited by 691
Bag of visual words and fusion methods for action recognition: Comprehensive study and good practice
Peng, X., Wang, L., Wang, X., & Qiao, Y. (2016) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2016.03.013 · Cited by 677
Recognizing 50 human action categories of web videos
Reddy, K. K., & Shah, M. (2013) · Machine Vision and Applications · doi:10.1007/s00138-012-0450-4 · Cited by 655
Modeling video evolution for action recognition
Fernando, B., Gavves, E., Jose Oramas, M., Ghodrati, A., & Tuytelaars, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299176 · Cited by 569
Recognizing human actions by attributes
Liu, J., Kuipers, B., & Savarese, S. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995353 · Cited by 563
Spatio-temporal Depth Cuboid Similarity Feature for Activity Recognition Using Depth Camera
Xia, L., & Aggarwal, J. K. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.365 · Cited by 433
View-Independent Action Recognition from Temporal Self-Similarities
Junejo, I. N., Dexter, E., Laptev, I., & Pérez, P. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.68 · Cited by 392
Action Recognition with Stacked Fisher Vectors
Peng, X., Zou, C., Qiao, Y., & Peng, Q. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_38 · Cited by 338
Human Activity Recognition with Metric Learning
Tran, D., & Sorokin, A. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-88682-2_42 · Cited by 332
Integrating low-rank and group-sparse structures for robust multi-task learning
Chen, J., Zhou, J., & Ye, J. (2011) · Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/2020408.2020423 · Cited by 311
Clustered Multi-Task Learning Via Alternating Structure Optimization.
Zhou, J., Chen, J., & Ye, J. (2011) · PubMed · Cited by 260
Action recognition using context and appearance distribution features
Wu, X., Xu, D., Duan, L., & Luo, J. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995624 · Cited by 222
Trajectory-Based Modeling of Human Actions with Motion Reference Points
Jiang, Y.-G., Dai, Q., Xue, X., Liu, W., & Ngo, C.-W. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33715-4_31 · Cited by 203
Motionlets: Mid-level 3D Parts for Human Motion Recognition
Wang, L., Qiao, Y., & Tang, X. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.345 · Cited by 180
Soft Margin Multiple Kernel Learning
Xu, X., Tsang, I. W.-H., & Xu, D. (2013) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2012.2237183 · Cited by 139
DL-SFA: Deeply-Learned Slow Feature Analysis for Action Recognition
Sun, L., Jia, K., Chan, T., Fang, Y., Wang, G., & Yan, S. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.336 · Cited by 119
Multiple/Single-View Human Action Recognition via Part-Induced Multitask Structural Learning
Liu, A.-A., Su, Y., Jia, P.-P., Gao, Z., Hao, T., & Yang, Z. (2014) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2347057 · Cited by 113
Self-citationImproving Human Action Recognition Using Score Distribution and Ranking
Hoai, M., & Zisserman, A. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-16814-2_1 · Cited by 79
Convex Discriminative Multitask Clustering
Zhang, X.-L. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2014.2343221 · Cited by 73
Action Recognition Using Multilevel Features and Latent Structural SVM
Wu, X., Xu, D., Duan, L., Luo, J., & Jia, Y. (2013) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2013.2244794 · Cited by 59
Single/multi-view human action recognition via regularized multi-task learning
Liu, A.-A., Xu, N., Su, Y., Lin, H., Hao, T., & Yang, Z. (2014) · Neurocomputing · doi:10.1016/j.neucom.2014.04.090 · Cited by 57
Self-citationLearning to Share Latent Tasks for Action Recognition
Zhou, Q., Wang, G., Jia, K., & Zhao, Q. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.281 · Cited by 54
Action Recognition Using Super Sparse Coding Vector with Spatio-temporal Awareness
Yang, X., & Tian, Y. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10605-2_47 · Cited by 51
Mining Compact Bag-of-Patterns for Low Bit Rate Mobile Visual Search
Ji, R., Duan, L., Chen, J., Huang, T., & Gao, W. (2014) · IEEE Transactions on Image Processing · doi:10.1109/tip.2014.2324291 · Cited by 51
Multi-task Sparse Learning with Beta Process Prior for Action Recognition
Yuan, C., Hu, W., Tian, G., Yang, S., & Wang, H. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.61 · Cited by 44
Unsupervised Spectral Dual Assignment Clustering of Human Actions in Context
Jones, S., & Shao, L. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.84 · Cited by 44
Attribute Regularization Based Human Action Recognition
Zhang, Z., Wang, C., Xiao, B., Zhou, W., & Liu, S. (2013) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2013.2258152 · Cited by 43
A Cause and Effect Analysis of Motion Trajectories for Modeling Actions
Narayan, S., & Ramakrishnan, K. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.337 · Cited by 42
Simplex-Based 3D Spatio-temporal Feature Description for Action Recognition
Zhang, H., Zhou, W., Reardon, C., et al. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.265 · Cited by 42
Weakly Supervised Visual Dictionary Learning by Harnessing Image Attributes
Gao, Y., Ji, R., Liu, W., Dai, Q., & Hua, G. (2014) · IEEE Transactions on Image Processing · doi:10.1109/tip.2014.2364536 · Cited by 37
Learning Human Actions by Combining Global Dynamics and Local Appearance
Luo, G., Yang, S., Tian, G., Yuan, C., Hu, W., & Maybank, S. J. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2014.2329301 · Cited by 32
Learning Effective Event Models to Recognize a Large Number of Human Actions
Wu, J., & Hu, D. (2013) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2013.2283846 · Cited by 20
Multi-Max-Margin Support Vector Machine for multi-source human action recognition
Wu, D., & Shao, L. (2013) · Neurocomputing · doi:10.1016/j.neucom.2013.08.038 · Cited by 12
Xu, Z., Zhu, L., Yang, Y., & Hauptmann, A. G. (2015) · Cited by 9
Low-Rank and Sparse Multi-task Learning
Chen, J.-H., Zhou, J.-Y., & Ye, J. (2014) · Low-Rank and Sparse Modeling for Visual Analysis · doi:10.1007/978-3-319-12000-3_8 · Cited by 3
Cited by
A Survey on Multi-Task Learning
Zhang, Y., & Yang, Q. (2017) · arXiv · doi:10.48550/arxiv.1707.08114 · Cited by 2,910
A Deep Learning Approach for Intrusion Detection Using Recurrent Neural Networks
Yin, C., Zhu, Y., Fei, J., & He, X. (2017) · IEEE Access · doi:10.1109/access.2017.2762418 · Cited by 2,100
A Comprehensive Survey of Vision-Based Human Action Recognition Methods
Zhang, H.-B., Zhang, Y.-X., Zhong, B., Lei, Q., Yang, L., Du, J.-X., & Chen, D.-S. (2019) · Sensors · doi:10.3390/s19051005 · Cited by 503
Automated pulmonary nodule detection in CT images using deep convolutional neural networks
Xie, H., Yang, D., Sun, N., Chen, Z., & Zhang, Y. (2019) · Pattern Recognition · doi:10.1016/j.patcog.2018.07.031 · Cited by 459
TSDL: A Two-Stage Deep Learning Model for Efficient Network Intrusion Detection
Khan, F. A., Gumaei, A., Derhab, A., & Hussain, A. (2019) · IEEE Access · doi:10.1109/access.2019.2899721 · Cited by 353
Metadata
- Authors
- Liu, A.-A., Su, Y.-T., Nie, W.-Z., & Kankanhalli, M.
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2017-01-01
- Volume
- 39
- Issue
- 1
- Pages
- 102-114
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- 1939-3539
- OpenAlex
- W2322020277
- Cited by
- 375
- References
- 53
- FWCI
- 38.79
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.