The DOI matches its Crossref record; 76 of the 76 references it cites were found in a registry or an index.
Rank Pooling for Action Recognition
Open accessAbstract
We propose a function-based temporal pooling method that captures the latent structure of the video sequence data - e.g., how frame-level features evolve over time in a video. We show how the parameters of a function that has been fit to the video data can serve as a robust new video representation. As a specific example, we learn a pooling function via ranking machines. By learning to rank the frame-level features of a video in chronological order, we obtain a new representation that captures the video-wide temporal dynamics of a video, suitable for action recognition. Other than ranking functions, we explore different parametric models that could also explain the temporal changes in videos. The proposed functional pooling methods, and rank pooling in particular, is easy to interpret and implement, fast to compute and effective in recognizing a wide variety of actions. We evaluate our method on various benchmarks for generic action, fine-grained action and gesture recognition. Results show that rank pooling brings an absolute improvement of 7-10 average pooling baseline. At the same time, rank pooling is compatible with and complementary to several appearance and local motion based methods and features, such as improved trajectories and deep learning features.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766
Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories
Lazebnik, S., Schmid, C., & Ponce, J. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.68 · Cited by 7,916
Two-Stream Convolutional Networks for Action Recognition in Videos
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1406.2199 · Cited by 6,756
Large-Scale Video Classification with Convolutional Neural Networks
Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., & Fei-Fei, L. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.223 · Cited by 6,354
Support Vector Regression Machines
Drucker, H., Burges, C. J. C., Kaufman, L., Smola, A. J., & Vapnik, V. N. (1996) · Cited by 4,658
Hmdb: a large video database for human motion recognition
Kuehne, H., Jhuang, H., Garrote, E., Poggio, T., & Serre, T. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126543 · Cited by 3,965
Real-time human pose recognition in parts from single depth images
Shotton, J., Fitzgibbon, A., Cook, M., Sharp, T., Finocchio, M., Moore, R., Kipman, A., & Blake, A. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995316 · Cited by 3,491
Learning realistic human actions from movies
Laptev, I., Marszalek, M., Schmid, C., & Rozenfeld, B. (2008) · 2008 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2008.4587756 · Cited by 3,444
Action Recognition with Improved Trajectories
Wang, H., & Schmid, C. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.441 · Cited by 3,407
Vlfeat: an open and portable library of computer vision algorithms
Vedaldi, A., & Fulkerson, B. (2010) · Proceedings of the 18th ACM international conference on Multimedia · doi:10.1145/1873951.1874249 · Cited by 2,664
Laptev, I. (2005) · International Journal of Computer Vision · doi:10.1007/s11263-005-1838-7 · Cited by 2,629
Beyond short snippets: Deep networks for video classification
Ng, J. Y.-H., Hausknecht, M., Vijayanarasimhan, S., Vinyals, O., Monga, R., & Toderici, G. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299101 · Cited by 2,062
Training linear SVMs in linear time
Joachims, T. (2006) · Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1150402.1150429 · Cited by 1,920
Hierarchical recurrent neural network for skeleton based action recognition
Du, Y., Wang, W., & Wang, L. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298714 · Cited by 1,895
Unsupervised Learning of Video Representations using LSTMs
Srivastava, N., Mansimov, E., & Salakhutdinov, R. (2015) · arXiv · doi:10.48550/arxiv.1502.04681 · Cited by 1,861
Learning to rank for information retrieval
Liu, T.-Y. (2010) · Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/1835449.1835676 · Cited by 1,856
Dense Trajectories and Motion Boundary Descriptors for Action Recognition
Wang, H., Kläser, A., Schmid, C., & Liu, C.-L. (2013) · International Journal of Computer Vision · doi:10.1007/s11263-012-0594-8 · Cited by 1,669
Learning to Rank for Information Retrieval
Liu, T.-Y. (2009) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000016 · Cited by 1,568
Three things everyone should know to improve object retrieval
Arandjelovic, R., & Zisserman, A. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248018 · Cited by 1,383
Unsupervised Visual Domain Adaptation Using Subspace Alignment
Fernando, B., Habrard, A., Sebban, M., & Tuytelaars, T. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.368 · Cited by 1,314
Self-citationLaptev, & Lindeberg (2003) · Proceedings Ninth IEEE International Conference on Computer Vision · doi:10.1109/iccv.2003.1238378 · Cited by 1,300
Recognizing human action in time-sequential images using hidden Markov model
Yamato, J., Ohya, J., & Ishii, K. (2003) · Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.1992.223161 · Cited by 1,273
Large-scale Video Classification with Convolutional Neural Networks
Karpathy, A., Toderici, G. D., Shetty, S., Leung, T., Sukthankar, R., & Fei-Fei, L. (2014) · Cited by 1,266
Marszalek, M., Laptev, I., & Schmid, C. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206557 · Cited by 1,168
Le, Q. V., Zou, W. Y., Yeung, S. Y., & Ng, A. Y. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995496 · Cited by 960
Doretto, G., Chiuso, A., Wu, Y. N., & Soatto, S. (2003) · International Journal of Computer Vision · doi:10.1023/a:1021669406132 · Cited by 885
Ensemble of exemplar-SVMs for object detection and beyond
Malisiewicz, T., Gupta, A., & Efros, A. A. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126229 · Cited by 865
Large-scale image retrieval with compressed Fisher vectors
Perronnin, F., Liu, Y., Sanchez, J., & Poirier, H. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5540009 · Cited by 761
Efficient Additive Kernels via Explicit Feature Maps
Vedaldi, A., & Zisserman, A. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2011.153 · Cited by 688
P-CNN: Pose-Based CNN Features for Action Recognition
Cheron, G., Laptev, I., & Schmid, C. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.368 · Cited by 633
Convolutional Learning of Spatio-temporal Features
Taylor, G. W., Fergus, R., LeCun, Y., & Bregler, C. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15567-3_11 · Cited by 629
Dynamic Image Networks for Action Recognition
Bilen, H., Fernando, B., Gavves, E., Vedaldi, A., & Gould, S. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.331 · Cited by 571
Self-citationModeling video evolution for action recognition
Fernando, B., Gavves, E., Jose Oramas, M., Ghodrati, A., & Tuytelaars, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299176 · Cited by 569
Self-citationGkioxari, G., & Malik, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298676 · Cited by 528
A database for fine grained activity detection of cooking activities
Rohrbach, M., Amin, S., Andriluka, M., & Schiele, B. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247801 · Cited by 521
Function representation in geometric modeling: concepts, implementation and applications
Pasko, A., Adzhiev, V., Sourin, A., & Savchenko, V. (1995) · The Visual Computer · doi:10.1007/bf02464333 · Cited by 514
Bousquet, O., Elisseeff, A., & Ron, D. (2001) · MPG.PuRe (Max Planck Society) · Cited by 493
Real-Time Human Pose Recognition in Parts from Single Depth Images
Shotton, J., Fitzgibbon, A., Cook, M., Sharp, T., Finocchio, M., Moore, R., Kipman, A., & Blake, A. (2013) · Studies in Computational Intelligence · doi:10.1007/978-3-642-28661-2_5 · Cited by 411
Better Exploiting Motion for Better Action Recognition
Jain, M., Jegou, H., & Bouthemy, P. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.330 · Cited by 385
HMDB51: A Large Video Database for Human Motion Recognition
Kuehne, H., Jhuang, H., Stiefelhagen, R., & Serre, T. (2013) · High Performance Computing in Science and Engineering ‘12 · doi:10.1007/978-3-642-33374-3_41 · Cited by 377
Learning latent temporal structure for complex event detection
Tang, K., Fei-Fei, L., & Koller, D. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247808 · Cited by 355
Action Recognition with Stacked Fisher Vectors
Peng, X., Zou, C., Qiao, Y., & Peng, Q. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_38 · Cited by 338
Conditional models for contextual human motion recognition
Sminchisescu, C., Kanaujia, A., Li, Z., & Metaxas, D. (2005) · Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1 · doi:10.1109/iccv.2005.59 · Cited by 282
Action Localization with Tubelets from Motion
Jain, M., Van Gemert, J., Jegou, H., Bouthemy, P., & Snoek, C. G. M. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.100 · Cited by 246
Discovering discriminative action parts from mid-level video representations
Raptis, M., Kokkinos, I., & Soatto, S. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247807 · Cited by 233
Wu, D., & Shao, L. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.98 · Cited by 211
Hidden Part Models for Human Action Recognition: Probabilistic versus Max Margin
Wang, Y., & Mori, G. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.214 · Cited by 190
Script Data for Attribute-Based Recognition of Composite Activities
Rohrbach, M., Regneri, M., Andriluka, M., Amin, S., Pinkal, M., & Schiele, B. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33718-5_11 · Cited by 185
What do 15,000 object categories tell us about classifying and localizing actions?
Jain, M., van Gemert, J. C., & Snoek, C. G. M. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298599 · Cited by 185
Conditional models for contextual human motion recognition
Sminchisescu, C., Kanaujia, A., & Metaxas, D. (2006) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2006.07.014 · Cited by 178
Multi-modal gesture recognition challenge 2013: dataset and results
Escalera, S., Gonzàlez, J., Baró, X., Reyes, M., Lopes, O., Guyon, I., Athitsos, V., & Escalante, H. (2013) · Proceedings of the 15th ACM on International conference on multimodal interaction · doi:10.1145/2522848.2532595 · Cited by 177
Actom sequence models for efficient action detection
Gaidon, A., Harchaoui, Z., & Schmid, C. (2011) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2011.5995646 · Cited by 145
Event Retrieval in Large Video Collections with Circulant Temporal Encoding
Revaud, J., Douze, M., Schmid, C., & Jegou, H. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.318 · Cited by 133
Discriminative Hierarchical Rank Pooling for Activity Recognition
Fernando, B., Anderson, P., Hütter, M., & Gould, S. J. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.212 · Cited by 108
Self-citationDomain-Adaptive Discriminative One-Shot Learning of Gestures
Pfister, T., Charles, J., & Zisserman, A. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10599-4_52 · Cited by 107
Action Recognition by Hierarchical Sequence Summarization
Song, Y., Morency, L., & Davis, R. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.457 · Cited by 97
Recognizing Complex Events Using Large Margin Joint Low-Level Event Model
Izadinia, H., & Shah, M. A. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33765-9_31 · Cited by 94
Activity recognition using dynamic subspace angles
Li, B., Ayazoğlu, M., Mao, T. Y., Camps, O., & Sznaier, M. (2011) · doi:10.1109/cvpr.2011.5995672 · Cited by 82
Local Alignments for Fine-Grained Categorization
Gavves, E., Fernando, B., Snoek, C. G. M., Smeulders, A. W. M., & Tuytelaars, T. (2014) · International Journal of Computer Vision · doi:10.1007/s11263-014-0741-5 · Cited by 81
Self-citationImproving Human Action Recognition Using Score Distribution and Ranking
Hoai, M., & Zisserman, A. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-16814-2_1 · Cited by 81
ACTIVE: Activity Concept Transitions in Video Event Classification
Sun, C., & Nevatia, R. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.453 · Cited by 73
Towards Good Practices for Action Video Encoding
Wu, J., Zhang, Y., & Lin, W. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.330 · Cited by 67
Recognizing activities with cluster-trees of tracklets
Gaidon, A., Harchaoui, Z., & Schmid, C. (2012) · British Machine Vision Conference (BMVC) · doi:10.5244/c.26.30 · Cited by 67
Learning end-to-end video classification with rank-pooling
Fernando, B., & Gould, S. J. (2016) · International Conference on Machine Learning · Cited by 66
Self-citationFusing multi-modal features for gesture recognition
Wu, J., Cheng, J., Zhao, C., & Lu, H. (2013) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/2522848.2532589 · Cited by 63
Conceptlets: Selective Semantics for Classifying Video Events
Mazloom, M., Gavves, E., & Snoek, C. G. M. (2014) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2014.2359771 · Cited by 38
Self-citationGesture Recognition Portfolios for Personalization
Yao, A., Gool, L. V., & Kohli, P. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.247 · Cited by 34
Exemplar SVMs as visual feature encoders
Zepeda, J., & Pérez, P. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298924 · Cited by 29
Pipelining Localized Semantic Features for Fine-Grained Action Recognition
Zhou, Y., Ni, B., Yan, S., Moulin, P., & Tian, Q. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10593-2_32 · Cited by 20
Learning to Rank Based on Subsequences
Fernando, B., Gavves, E., Muselet, D., & Tuytelaars, T. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.319 · Cited by 14
Self-citationDense interest features for video processing
Geest, R. D., & Tuytelaars, T. (2014) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2014.7026167 · Cited by 11
Self-citationThe Best of BothWorlds: Combining Data-Independent and Data-Driven Approaches for Action Recognition
Lan, Z., Yu, S.-I., Yao, D., Lin, M., Raj, B., & Hauptmann, A. G. (2016) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2016.152 · Cited by 9
Convex reduction of high-dimensional kernels for visual classification
Gavves, E., Snoek, C. G. M., & Smeulders, A. W. M. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6248106 · Cited by 4
Self-citation
Cited by
Human Action Recognition From Various Data Modalities: A Review
Sun, Z., Ke, Q., Rahmani, H., Bennamoun, M., Wang, G., & Liu, J. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3183112 · Cited by 601
Dynamic Image Networks for Action Recognition
Bilen, H., Fernando, B., Gavves, E., Vedaldi, A., & Gould, S. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.331 · Cited by 571
Self-citationSkeleton-Based Action Recognition Using Spatio-Temporal LSTM Network with Trust Gates
Liu, J., Shahroudy, A., Xu, D., Kot, A. C., & Wang, G. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2771306 · Cited by 530
Self-Supervised Video Representation Learning with Odd-One-Out Networks
Fernando, B., Bilen, H., Gavves, E., & Gould, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.607 · Cited by 421
Self-citationSelf-Supervised Spatiotemporal Learning via Video Clip Order Prediction
Xu, D., Xiao, J., Zhao, Z., Shao, J., Xie, D., & Zhuang, Y. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01058 · Cited by 415
Metadata
- Authors
- Fernando, B., Gavves, E., Oramas M., J., Ghodrati, A., & Tuytelaars, T.
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2017-04-01
- Volume
- 39
- Issue
- 4
- Pages
- 773-787
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- 1939-3539
- OpenAlex
- W2193384753
- Cited by
- 304
- References
- 76
- FWCI
- 33.09
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.