The DOI matches its Crossref record; 61 of the 61 references it cites were found in a registry or an index.
Jointly Learning Heterogeneous Features for RGB-D Activity Recognition
Open accessAbstract
In this paper, we focus on heterogeneous features learning for RGB-D activity recognition. We find that features from different channels (RGB, depth) could share some similar hidden structures, and then propose a joint learning model to simultaneously explore the shared and feature-specific components as an instance of heterogeneous multi-task learning. The proposed model formed in a unified framework is capable of: 1) jointly mining a set of subspaces with the same dimensionality to exploit latent shared features across different feature channels, 2) meanwhile, quantifying the shared and feature-specific components of features in the subspaces, and 3) transferring feature-specific intermediate transforms (i-transforms) for learning fusion of heterogeneous features across datasets. To efficiently train the joint model, a three-step iterative optimization algorithm is proposed, followed by a simple inference model. Extensive experimental results on four activity datasets have demonstrated the efficacy of the proposed method. A new RGB-D activity dataset focusing on human-object interaction is further contributed, which presents more challenges for RGB-D activity benchmarking.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Pan, S. J., & Yang, Q. (2010) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2009.191 · Cited by 23,387
Real-time human pose recognition in parts from single depth images
Shotton, J., Fitzgibbon, A., Cook, M., Sharp, T., Finocchio, M., Moore, R., Kipman, A., & Blake, A. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995316 · Cited by 3,491
Action recognition by dense trajectories
Wang, H., Klaser, A., Schmid, C., & Liu, C.-L. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995407 · Cited by 2,154
Real-time human pose recognition in parts from single depth images
Shotton, J., Sharp, T., Kipman, A., Fitzgibbon, A., Finocchio, M., Blake, A., Cook, M., & Moore, R. (2013) · Communications of the ACM · doi:10.1145/2398356.2398381 · Cited by 2,044
Hierarchical recurrent neural network for skeleton based action recognition
Du, Y., Wang, W., & Wang, L. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298714 · Cited by 1,895
Human Action Recognition by Representing 3D Skeletons as Points in a Lie Group
Vemulapalli, R., Arrate, F., & Chellappa, R. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.82 · Cited by 1,517
View invariant human action recognition using histograms of 3D joints
Xia, L., Chen, C.-C., & Aggarwal, J. K. (2012) · 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2012.6239233 · Cited by 1,452
Action recognition based on a bag of 3D points
Li, W., Zhang, Z., & Liu, Z. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops · doi:10.1109/cvprw.2010.5543273 · Cited by 1,428
A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data
Ando, R. K., & Zhang, T. (2005) · doi:10.5555/1046920.1194905 · Cited by 1,311
HON4D: Histogram of Oriented 4D Normals for Activity Recognition from Depth Sequences
Oreifej, O., & Liu, Z. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.98 · Cited by 956
A feasible method for optimization with orthogonality constraints
Wen, Z., & Yin, W. (2013) · Mathematical Programming · doi:10.1007/s10107-012-0584-1 · Cited by 849
Sharing Visual Features for Multiclass and Multiview Object Detection
Torralba, A., Murphy, K. P., & Freeman, W. T. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.1055 · Cited by 700
Learning human activities and object affordances from RGB-D videos
Koppula, H. S., Gupta, R., & Saxena, A. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913478446 · Cited by 683
Human action recognition using a temporal hierarchy of covariance descriptors on 3D joint locations
Hussein, M. E., Torki, M., Gowayyed, M. A., & El-Saban, M. (2013) · Cited by 507
Learning Actionlet Ensemble for 3D Human Action Recognition
Wang, J., Liu, Z., Wu, Y., & Yuan, J. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.198 · Cited by 502
Robust 3D Action Recognition with Random Occupancy Patterns
Wang, J., Liu, Z., Chorowski, J., Chen, Z., & Wu, Y. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33709-3_62 · Cited by 471
Activity recognition using the velocity histories of tracked keypoints
Messing, R., Pal, C., & Kautz, H. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459154 · Cited by 455
Learning With Augmented Features for Supervised and Semi-Supervised Heterogeneous Domain Adaptation
Li, W., Duan, L., Xu, D., & Tsang, I. W. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.167 · Cited by 449
EigenJoints-based action recognition using Naïve-Bayes-Nearest-Neighbor
Yang, X., & Tian, Y. L. (2012) · 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2012.6239232 · Cited by 439
Zanfir, M., Leordeanu, M., & Sminchisescu, C. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.342 · Cited by 438
Spatio-temporal Depth Cuboid Similarity Feature for Activity Recognition Using Depth Camera
Xia, L., & Aggarwal, J. K. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.365 · Cited by 433
Super Normal Vector for Activity Recognition Using Depth Sequences
Yang, X., & Tian, Y. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.108 · Cited by 388
Jointly learning heterogeneous features for RGB-D activity recognition
Hu, J.-F., Zheng, W.-S., Lai, J., & Zhang, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299172 · Cited by 358
Self-citationRecognition and Segmentation of 3-D Human Action Using HMM and Multi-class AdaBoost
Lv, F., & Nevatia, R. (2006) · Lecture Notes in Computer Science · doi:10.1007/11744085_28 · Cited by 351
Learning Sparse Representations for Human Action Recognition
Guha, T., & Ward, R. K. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2011.253 · Cited by 349
A convex formulation for learning task relationships in multi-task learning
Zhang, Y., & Yeung, D. (2010) · Cited by 338
Ofli, F., Chaudhry, R., Kurillo, G., Vidal, R., & Bajcsy, R. (2014) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2013.04.007 · Cited by 318
Human Activity Detection from RGBD ImagesPreprint
Sung, J., Ponce, C., Selman, B., & Saxena, A. (2011) · arXiv (Cornell University) · Cited by 282
Uncovering shared structures in multiclass classification
Amit, Y., Fink, M., Srebro, N., & Ullman, S. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273499 · Cited by 278
Learning discriminative representations from RGB-D video data
Liu, L., & Shao, L. (2013) · Cited by 263
Fusing Spatiotemporal Features and Joints for 3D Action Recognition
Zhu, Y., Chen, W., & Guo, G. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2013.78 · Cited by 226
Yao, B., & Fei-Fei, L. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.67 · Cited by 217
Multi-view Super Vector for Action Recognition
Cai, Z., Wang, L., Peng, X., & Qiao, Y. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.83 · Cited by 205
Group Sparsity and Geometry Constrained Dictionary Learning for Action Recognition from Depth Maps
Luo, J., Wang, W., & Qi, H. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.227 · Cited by 204
Multitask Linear Discriminant Analysis for View Invariant Action Recognition
Yan, Y., Ricci, E., Subramanian, R., Liu, G., & Sebe, N. (2014) · IEEE Transactions on Image Processing · doi:10.1109/tip.2014.2365699 · Cited by 165
Towards Open-World Person Re-Identification by One-Shot Group-Based Verification
Zheng, W.-S., Gong, S., & Xiang, T. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2453984 · Cited by 159
Self-citationFine-grained kitchen activity recognition using RGB-D
Lei, J., Ren, X., & Fox, D. (2012) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/2370216.2370248 · Cited by 135
Range-Sample Depth Feature for Action Recognition
Lu, C., Jia, J., & Tang, C. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.104 · Cited by 123
Supervised class-specific dictionary learning for sparse modeling in action recognition
Wang, H., Yuan, C., Hu, W., & Sun, C. (2012) · Pattern Recognition · doi:10.1016/j.patcog.2012.04.024 · Cited by 122
Modeling 4D Human-Object Interactions for Event and Object Recognition
Wei, P., Zhao, Y., Zheng, N., et al. (2013) · IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.406 · Cited by 117
Evaluating spatiotemporal interest point features for depth-based action recognition
Zhu, Y. M., Chen, W., & Guo, G. (2014) · Image and Vision Computing · doi:10.1016/j.imavis.2014.04.005 · Cited by 114
Structure-Preserving Binary Representations for RGB-D Action Recognition
Yu, M., Liu, L., & Shao, L. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2491925 · Cited by 107
Action Recognition with Exemplar Based 2.5D Graph Matching
Yao, B., & Fei-Fei, L. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33765-9_13 · Cited by 102
Real-Time RGB-D Activity Prediction by Soft Regression
Hu, J., Zheng, W., Ma, L., Wang, G., & Lai, J. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46448-0_17 · Cited by 99
Self-citationMMSS: Multi-modal Sharable and Specific Feature Learning for RGB-D Object Recognition
Wang, A., Cai, J., Lu, J., & Cham, T. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.134 · Cited by 98
Fusion of Skeletal and Silhouette-Based Features for Human Action Recognition with RGB-D Devices
Chaaraoui, A. A., Padilla-López, J. R., & Flórez‐Revuelta, F. (2013) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2013.19 · Cited by 97
Combing RGB and Depth Map Features for human activity recognition
Zhao, Y., Liu, Z., Yang, L., & Cheng, H. (2012) · Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) · Cited by 89
Discriminative Hierarchical Modeling of Spatio-temporally Composable Human Activities
Lillo, I., Soto, Á., & Niebles, J. C. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.109 · Cited by 88
Ofli, F., Chaudhry, R., Kurillo, G., Vidal, R., & Bajcsy, R. K. (2012) · 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2012.6239231 · Cited by 88
Heterogeneous feature machines for visual recognition
Cao, L., Luo, J., Liang, F., & Huang, T. S. (2009) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2009.5459401 · Cited by 84
Multi-view latent variable discriminative models for action recognition
Song, Y., Morency, L., & Davis, R. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6247918 · Cited by 83
Bilinear heterogeneous information machine for RGB-D action recognition
Kong, Y., & Fu, Y. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298708 · Cited by 82
Heterogeneous multitask learning with joint sparsity constraints
Yang, X., Kim, S. Y., & Xing, E. P. (2009) · Cited by 76
Learning to Share Latent Tasks for Action Recognition
Zhou, Q., Wang, G., Jia, K., & Zhao, Q. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.281 · Cited by 54
Multi-modal feature fusion for action recognition in RGB-D sequences
Shahroudy, A., Wang, G., & Ng, T.-T. (2014) · International Symposium on Communications, Control and Signal Processing · doi:10.1109/isccsp.2014.6877819 · Cited by 53
Multi-Task Learning in Heterogeneous Feature Spaces
Zhang, Y., & Yeung, D. (2011) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v25i1.7909 · Cited by 50
A Convex Formulation for Learning a Shared Predictive Structure from Multiple Tasks
Chen, J., Tang, L., Liu, J., & Ye, J. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.189 · Cited by 48
Action recognition by exploring data distribution and feature correlation
Wang, S., Yang, Y., Ma, Z., Li, X., Pang, C., & Hauptmann, A. G. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6247823 · Cited by 44
Order-Preserving Sparse Coding for Sequence Classification
Ni, B., Moulin, P., & Yan, S. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33709-3_13 · Cited by 40
Exemplar-Based Recognition of Human–Object Interactions
Hu, J., Zheng, W., Lai, J., Gong, S., & Xiang, T. (2015) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2015.2397200 · Cited by 31
Self-citationCross-Domain Multitask Learning with Latent Probit ModelsPreprint
Han, S., Liao, X., & Carin, L. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1206.6419 · Cited by 15
Cited by
NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis
Shahroudy, A., Liu, J., Ng, T.-T., & Wang, G. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.115 · Cited by 3,148
NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding
Liu, J., Shahroudy, A., Perez, M., Wang, G., Duan, L.-Y., & Kot, A. C. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2916873 · Cited by 1,818
Spatio-Temporal LSTM with Trust Gates for 3D Human Action Recognition
Liu, J., Shahroudy, A., Xu, D., & Wang, G. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46487-9_50 · Cited by 1,351
An End-to-End Spatio-Temporal Attention Model for Human Action Recognition from Skeleton Data
Song, S., Lan, C., Xing, J., Zeng, W., & Liu, J. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.11212 · Cited by 977
Enhanced skeleton visualization for view invariant human action recognition
Liu, M., Liu, H., & Chen, C. (2017) · Pattern Recognition · doi:10.1016/j.patcog.2017.02.030 · Cited by 842
Metadata
- Authors
- Hu, J.-F., Zheng, W.-S., Lai, J., & Zhang, J.
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2017-11-01
- Volume
- 39
- Issue
- 11
- Pages
- 2186-2200
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- OpenAlex
- W1893516992
- Cited by
- 383
- References
- 61
- FWCI
- 44.95
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.