The DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
What do 15,000 object categories tell us about classifying and localizing actions?
Open accessAbstract
This paper contributes to automatic classification and localization of human actions in video. Whereas motion is the key ingredient in modern approaches, we assess the benefits of having objects in the video representation. Rather than considering a handful of carefully selected and localized objects, we conduct an empirical study on the benefit of encoding 15,000 object categories for action using 6 datasets totaling more than 200 hours of video and covering 180 action classes. Our key contributions are i) the first in-depth study of encoding objects for actions, ii) we show that objects matter for actions, and are often semantically relevant as well. iii) We establish that actions have object preferences. Rather than using all objects, selection is advantageous for action recognition. iv)We reveal that object-action relations are generic, which allows to transferring these relationships from the one domain to the other. And, v) objects, when combined with motion, improve the state-of-the-art for both action classification and localization.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Hmdb: a large video database for human motion recognition
Kuehne, H., Jhuang, H., Garrote, E., Poggio, T., & Serre, T. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126543 · Cited by 3,965
Action Recognition with Improved Trajectories
Wang, H., & Schmid, C. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.441 · Cited by 3,407
Improving the Fisher Kernel for Large-Scale Image Classification
Perronnin, F., Sánchez, J., & Mensink, T. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15561-1_11 · Cited by 2,588
Research works printed but not linked: 54
- 10.1109/ICPR.2004.1334462 · DOI · Paper page
- 10.1109/CVPR.2012.6247806 · DOI · Paper page
- peng, 2014, Action recognition with stacked fisher vectors, ECCV
- peng, 2014, Boosting vlad with supervised dictionary learning and high-order statistics, ECCV
- oneata, 2014, The LEAR submission at Thumos 2014, ECCV workshop
- 10.1109/ICCV.2013.228 · DOI · Paper page
- 10.1109/CVPR.2008.4587727 · DOI · Paper page
- ramanan, 2003, Automatic annotation of everyday movements, NIPS
- 10.1109/TPAMI.2011.158 · DOI · Paper page
- 10.1109/ICCV.2013.442 · DOI
- 10.1109/ICCVW.2009.5457659 · DOI · Paper page
- 10.1109/CVPR.2009.5206557 · DOI · Paper page
- oneata, 2014, Spatiotemporal object detection proposals, ECCV
- 10.1109/CVPR.2010.5539875 · DOI · Paper page
- 10.1109/CVPR.2014.83 · DOI · Paper page
- lan, 2011, Discriminative figure-centric models for joint action localization and recognition, ICCV
- 10.1109/CVPR.2008.4587756 · DOI · Paper page
- 10.1109/ICCV.2003.1238378 · DOI · Paper page
- 10.1109/CVPR.2011.5995353 · DOI · Paper page
- li, 2012, Recognizing activities by attribute dynamics, NIPS
- 10.1109/CVPR.2011.5995407 · DOI · Paper page
- zeiler, 2014, Visualizing and understanding convolutional networks, ECCV
- 10.1109/ICCV.2011.6126386 · DOI · Paper page
- 10.1109/CVPR.2010.5540235 · DOI · Paper page
- 10.1109/CVPR.2011.5995624 · DOI · Paper page
- 10.1109/CVPR.2014.330 · DOI
- willems, 2008, An efficient dense and scale-invariant spatio-temporal interest point detector, ECCV
- 10.1109/ICCV.2013.333 · DOI
- wang, 2013, LEAR-INRIA submission for the THUMOS workshop, ICCV Workshop
- 10.1109/CVPR.2007.383331 · DOI · Paper page
- han, 2009, Selection and context for action recognition, ICCV
- 10.1109/CVPR.2013.335 · DOI
- hoai, 2014, Improving human action recognition using score distribution and ranking, ACCV
- ikizler-cinbis, 2010, Object, scene and actions: Combining multiple features for human action recognition, ECCV
- 10.1109/CVPR.2013.330 · DOI · Paper page
- 10.1109/CVPR.2014.100 · DOI · Paper page
- jain, 2014, University of Amsterdam at THUMOS Challenge 2014, ECCV workshop
- jiang, 2012, Trajectory-based modeling of human actions with motion reference points, ECCV
- khiser, 2008, A spatio-temporal descriptor based on 3d-gradients, BMVC
- dalal, 0, Human detection using oriented histograms of flow and appearance, ECCV 2006
- chatfield, 2014, Return of the devil in the details: Delving deep into convolutional nets, BMVC
- deng, 2010, What does classifying more than 10,000 image categories tell us, ECCV
- 10.1109/CVPR.2015.7299176 · DOI · Paper page
- 10.1109/TIP.2014.2302677 · DOI
- 10.1007/978-3-319-10602-1_28 · DOI
- 10.5244/C.26.30 · DOI
- 10.5244/C.24.95 · DOI
- 10.5244/C.22.30 · DOI
- vig, 2012, Space-variant descriptor sampling for action recognition based on saliency and eye movements, ECCV
- varol, 0, Extreme Learning Machine for Large-Scale Action Recognition, ECCV workshop 2014
- 10.1007/978-3-319-10599-4_27 · DOI
- soomro, 2012, UCF 101: A dataset of 101 human actions classes from videos in the wild, CoRR
- 10.1109/CVPR.2013.341 · DOI · Paper page
- 10.1109/ICCV.2013.453 · DOI
Cited by
Revisiting Unreasonable Effectiveness of Data in Deep Learning Era
Sun, C., Shrivastava, A., Singh, S., & Gupta, A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.97 · Cited by 2,484
Temporal Convolutional Networks for Action Segmentation and Detection
Lea, C., Flynn, M. D., Vidal, R., Reiter, A., & Hager, G. D. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.113 · Cited by 2,238
“Large-scale Recognition: There has been substantial work on spatiotemporal models for large scale video classification and detection [31, 11, 12, 27, 33, 23, 19].”
BackgroundTemporal Action Localization in Untrimmed Videos via Multi-stage CNNs
Shou, Z., Wang, D., & Chang, S.-F. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.119 · Cited by 968
Temporal Segment Networks for Action Recognition in Videos
Wang, L., Xiong, Y., Wang, Z., Qiao, Y., Lin, D., Tang, X., & Van Gool, L. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2868668 · Cited by 918
Shou, Z., Chan, J., Zareian, A., Miyazawa, K., & Chang, S.-F. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.155 · Cited by 574
Modeling video evolution for action recognition
Fernando, B., Gavves, E., Jose Oramas, M., Ghodrati, A., & Tuytelaars, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299176 · Cited by 569
A Review of Human Activity Recognition Methods
Vrigkas, M., Nikou, C., & Kakadiaris, I. A. (2015) · Frontiers in Robotics and AI · doi:10.3389/frobt.2015.00028 · Cited by 566
UntrimmedNets for Weakly Supervised Action Recognition and Detection
Wang, L., Xiong, Y., Lin, D., & Van Gool, L. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.678 · Cited by 546
VideoLSTM convolves, attends and flows for action recognition
Li, Z., Gavrilyuk, K., Gavves, E., Jain, M., & Snoek, C. G. M. (2018) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2017.10.011 · Cited by 466
Self-citationA review of Convolutional-Neural-Network-based action recognition
Yao, G., Lei, T., & Zhong, J. (2019) · Pattern Recognition Letters · doi:10.1016/j.patrec.2018.05.018 · Cited by 437
Real-Time Action Recognition with Enhanced Motion Vector CNNs
Zhang, B., Wang, L., Wang, Z., Qiao, Y., & Wang, H. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.297 · Cited by 433
Prompting Visual-Language Models for Efficient Video Understanding
Ju, C., Han, T., Zheng, K., Zhang, Y., & Xie, W. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-19833-5_7 · Cited by 391
Tube Convolutional Neural Network (T-CNN) for Action Detection in Videos
Hou, R., Chen, C., & Shah, M. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.620 · Cited by 375
W-TALC: Weakly-Supervised Temporal Activity Localization and Classification
Paul, S., Roy, S., & Roy-Chowdhury, A. K. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01225-0_35 · Cited by 340
Rank Pooling for Action Recognition
Fernando, B., Gavves, E., Oramas M., J., Ghodrati, A., & Tuytelaars, T. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2558148 · Cited by 304
Action Recognition using Visual Attention
Sharma, S., Kiros, R., & Salakhutdinov, R. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.04119 · Cited by 298
Segmental Spatiotemporal CNNs for Fine-Grained Action Segmentation
Lea, C., Reiter, A., Vidal, R., & Hager, G. D. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46487-9_3 · Cited by 246
“Large-scale Action Classification: Recent efforts to extend CNN models to video [2,24,26,27,28,29,30] improve over holistic methods by encoding spatial and temporal information.”
BackgroundMethodTemporal Action Detection Using a Statistical Language Model
Richard, A., & Gall, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.341 · Cited by 215
Objects2action: Classifying and Localizing Actions without Any Video Example
Jain, M., Gemert, J. C. V., Mensink, T., & Snoek, C. G. M. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.521 · Cited by 142
Self-citationWeakly supervised learning of actions from transcripts
Kuehne, H., Richard, A., & Gall, J. (2017) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2017.06.004 · Cited by 119
Metadata
- Authors
- Jain, M., van Gemert, J. C., & Snoek, C. G. M.
- Venue
- 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2015-06
- Pages
- 46-55
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2095242101
- Cited by
- 185
- References
- 5
- FWCI
- 16.49
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.