Early Action Prediction by Soft Regression
Open accessThe DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.
Abstract
We propose a novel approach for predicting on-going action with the assistance of a low-cost depth camera. Our approach introduces a soft regression-based early prediction framework. In this framework, we estimate soft labels for the subsequences at different progress levels, jointly learned with an action predictor. Our formulation of soft regression framework 1) overcomes a usual assumption in existing early action prediction systems that the progress level of on-going sequence is given in the testing stage; and 2) presents a theoretical framework to better resolve the ambiguity and uncertainty of subsequences at early performing stage. The proposed soft regression framework is further enhanced in order to take the relationships among subsequences and the discrepancy of soft labels over different classes into consideration, so that a Multiple Soft labels Recurrent Neural Network (MSRNN) is finally developed. For real-time performance, we also introduce a new RGB-D feature called "local accumulative frame feature (LAFF)", which can be computed efficiently by constructing an integral feature map. Our experiments on three RGB-D benchmark datasets and an unconstrained RGB action set demonstrate that the proposed regression-based early action prediction model outperforms existing models significantly and also show that the early action prediction on RGB-D sequence is more accurate than that on RGB channel.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Histograms of Oriented Gradients for Human Detection
Dalal, N., & Triggs, B. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.177 · Cited by 33,584
Speech recognition with deep recurrent neural networks
Graves, A., Mohamed, A.-R., & Hinton, G. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6638947 · Cited by 8,893
Two-Stream Convolutional Networks for Action Recognition in Videos
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1406.2199 · Cited by 6,756
Deep visual-semantic alignments for generating image descriptions
Karpathy, A., & Fei-Fei, L. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298932 · Cited by 4,902
Backpropagation through time: what it does and how to do it
Werbos, P. J. (1990) · Proceedings of the IEEE · doi:10.1109/5.58337 · Cited by 4,879
Ucf101: A dataset of 101 human actions classes from videos in the wild
Soomro, K., Zamir, A. R., & Shah, M. (2012) · arXiv · doi:10.48550/arxiv.1212.0402 · Cited by 4,575
Action Recognition with Improved Trajectories
Wang, H., & Schmid, C. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.441 · Cited by 3,407
NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis
Shahroudy, A., Liu, J., Ng, T.-T., & Wang, G. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.115 · Cited by 3,148
Self-citationAction recognition with trajectory-pooled deep-convolutional descriptors
Wang, L., Qiao, Y., & Tang, X. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299059 · Cited by 1,194
Kitani, K. M., Ziebart, B. D., Bagnell, J. A., & Hebert, M. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33765-9_15 · Cited by 701
Cited by
NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding
Liu, J., Shahroudy, A., Perez, M., Wang, G., Duan, L.-Y., & Kot, A. C. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2916873 · Cited by 1,818
Self-citationSensor-based and vision-based human activity recognition: A comprehensive survey
Minh Dang, L., Min, K., Wang, H., Jalil Piran, M., Hee Lee, C., & Moon, H. (2020) · Pattern Recognition · doi:10.1016/j.patcog.2020.107561 · Cited by 730
Decoupling GCN with DropGraph Module for Skeleton-Based Action Recognition
Cheng, K., Zhang, Y., Cao, C., Shi, L., Cheng, J., & Lu, H. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58586-0_32 · Cited by 384
Richly Activated Graph Convolutional Network for Robust Skeleton-Based Action Recognition
Song, Y.-F., Zhang, Z., Shan, C., & Wang, L. (2021) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2020.3015051 · Cited by 254
3D Human Action Representation Learning via Cross-View Consistency Pursuit
Li, L., Wang, M., Ni, B., Wang, H., Yang, J., & Zhang, W. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00471 · Cited by 236
Rao, H., Xu, S., Hu, X., Cheng, J., & Hu, B. (2021) · Information Sciences · doi:10.1016/j.ins.2021.04.023 · Cited by 235
Skeleton Image Representation for 3D Action Recognition Based on Tree Structure and Reference Joints
Caetano, C., Bremond, F., & Schwartz, W. R. (2019) · 2019 32nd SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI) · doi:10.1109/sibgrapi.2019.00011 · Cited by 149
“(b) Chains Ct considering 25 Kinect joints: [2, 21, 3, 4, 3, 21, 5, 6, 7, 8, 22, 23, 22, 8, 7, 6, 5, 21, 9, 10, 11, 12, 24, 25, 24, 12, 11, 10, 9, 21, 2, 1, 13, 14, 15, 16, 15, 14, 13, 1, 17, 18, 19, 20, 19, 18, 17, 1, 2], as defined in [18].”
BackgroundResult
Metadata
- Authors
- Hu, J.-F., Zheng, W.-S., Ma, L., Wang, G., Lai, J., & Zhang, J.
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2019-11-01
- Volume
- 41
- Issue
- 11
- Pages
- 2568-2583
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- 1939-3539
- OpenAlex
- W2885940446
- Cited by
- 125
- References
- 12
- FWCI
- 7.98
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.