The DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.
Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset
Open accessAbstract
The paucity of videos in current action classification datasets (UCF-101 and HMDB-51) has made it difficult to identify good video architectures, as most methods obtain similar performance on existing small-scale benchmarks. This paper re-evaluates state-of-the-art architectures in light of the new Kinetics Human Action Video dataset. Kinetics has two orders of magnitude more data, with 400 human action classes and over 400 clips per class, and is collected from realistic, challenging YouTube videos. We provide an analysis on how current architectures fare on the task of action classification on this dataset and how much performance improves on the smaller benchmark datasets after pre-training on Kinetics. We also introduce a new Two-Stream Inflated 3D ConvNet (I3D) that is based on 2D ConvNet inflation: filters and pooling kernels of very deep image classification ConvNets are expanded into 3D, making it possible to learn seamless spatio-temporal feature extractors from video while leveraging successful ImageNet architecture designs and even their parameters. We show that, after pre-training on Kinetics, I3D models considerably improve upon the state-of-the-art in action classification, reaching 80.2% on HMDB-51 and 97.9% on UCF-101.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Although I3D [4] and R3D [19] achieve better performance than vanilla ResNet50 [20] and EfficientNet [50], the computation costs are huge, such as R3D50 with 296G FLOPs.”
From TALL: Thumbnail Layout for Deepfake Video Detection (2023) · cites this paper as [4]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“(e) Non-local vs . 3D Conv : A 5-block non-local C2D vs . inflated 3D ConvNet (I3D) [7]. All entries are with ResNet-101. The numbers of parameters and FLOPs are relative to the C2D baseline (43.2M and 34.2B).”
From Non-local Neural Networks (2018) · cites this paper as [7]
The cited source says what the claim attributes to it.
In the citing paper
“With a few non-local blocks, our architecures called non-local neural networks are more accurate for video classification than 2D and 3D convolutional networks [48] (including the inflated variant [7]).”
From Non-local Neural Networks (2018) · cites this paper as [7]
In the citing paper
“many of the deep-learning architectures mentioned in this review have similarities to ones used in analysis of digital media that try to combine multiple modalities to improve predictive performance of video content (Carreira and Zisserman, 2017; Afouras et al., 2019).”
From Missing data in multi-omics integration: Recent advances through artificial intelligence (2023) · cites this paper as Carreira and Zisserman, 2017
In the citing paper
“Compared to RNN-based architectures, Transformers [137] have significant advantages in long-term dependency and parallel computation.”
From Deep learning for visual speech analysis: A survey (2024) · cites this paper as [137]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
Self-citationFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate ShiftPreprint
Ioffe, S. (2024) · arXiv (Cornell University) · doi:10.57702/o9raffed · Cited by 15,273
Learning Spatiotemporal Features with 3D Convolutional Networks
Tran, D., Bourdev, L., Fergus, R., Torresani, L., & Paluri, M. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.510 · Cited by 9,593
TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., et al. (2016) · arXiv · doi:10.48550/arxiv.1603.04467 · Cited by 9,408
Two-Stream Convolutional Networks for Action Recognition in Videos
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1406.2199 · Cited by 6,756
Self-citationLarge-Scale Video Classification with Convolutional Neural Networks
Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., & Fei-Fei, L. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.223 · Cited by 6,354
3D Convolutional Neural Networks for Human Action Recognition
Ji, S., Xu, W., Yang, M., & Yu, K. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.59 · Cited by 6,313
Long-term recurrent convolutional networks for visual recognition and description
Donahue, J., Hendricks, L. A., Guadarrama, S., Rohrbach, M., Venugopalan, S., Darrell, T., & Saenko, K. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298878 · Cited by 5,228
The Kinetics Human Action Video DatasetPreprint
Kay, W., Carreira, J., Simonyan, K., Zhang, B., Hillier, C., Vijayanarasimhan, S., Viola, F., Green, T., Back, T., Natsev, P., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.06950 · Cited by 5,004
Self-citationUcf101: A dataset of 101 human actions classes from videos in the wild
Soomro, K., Zamir, A. R., & Shah, M. (2012) · arXiv · doi:10.48550/arxiv.1212.0402 · Cited by 4,575
Temporal Segment Networks: Towards Good Practices for Deep Action Recognition
Wang, L., Xiong, Y., Wang, Z., Qiao, Y., Lin, D., Tang, X., & Gool, L. V. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46484-8_2 · Cited by 4,232
Hmdb: a large video database for human motion recognition
Kuehne, H., Jhuang, H., Garrote, E., Poggio, T., & Serre, T. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126543 · Cited by 3,965
Learning realistic human actions from movies
Ivan Laptev, A., Marszałek, M., Schmid, C., & Rozenfeld, B. (2008) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2008.4587756 · Cited by 3,481
Action Recognition with Improved Trajectories
Wang, H., & Schmid, C. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.441 · Cited by 3,407
Learning and Transferring Mid-level Image Representations Using Convolutional Neural Networks
Oquab, M., Bottou, L., Laptev, I., & Sivic, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.222 · Cited by 3,150
Convolutional Two-Stream Network Fusion for Video Action Recognition
Feichtenhofer, C., Pinz, A. J., & Zisserman, A. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.213 · Cited by 2,842
Self-citationThe recognition of human movement using temporal templates
Bobick, A., & Davis, J. W. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.910878 · Cited by 2,837
Beyond short snippets: Deep networks for video classification
Ng, J. Y.-H., Hausknecht, M., Vijayanarasimhan, S., Vinyals, O., Monga, R., & Toderici, G. D. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7299101 · Cited by 2,152
A Duality Based Approach for Realtime TV-L 1 Optical Flow
Zach, C., Pock, T., & Bischof, H. (2007) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-74936-3_22 · Cited by 1,658
Unsupervised Learning of Human Action Categories Using Spatial-Temporal Words
Niebles, J. C., Wang, H., & Fei-Fei, L. (2008) · International Journal of Computer Vision · doi:10.1007/s11263-007-0122-4 · Cited by 1,475
Large-scale Video Classification with Convolutional Neural Networks
Karpathy, A., Toderici, G. D., Shetty, S., Leung, T., Sukthankar, R., & Fei-Fei, L. (2014) · Cited by 1,266
Action recognition with trajectory-pooled deep-convolutional descriptors
Wang, L., Qiao, Y., & Tang, X. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299059 · Cited by 1,194
Convolutional Learning of Spatio-temporal Features
Taylor, G. W., Fergus, R., LeCun, Y., & Bregler, C. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15567-3_11 · Cited by 652
Spatiotemporal Residual Networks for Video Action RecognitionPreprint
Feichtenhofer, C., Pinz, A. J., & Wildes, R. P. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.02155 · Cited by 646
Dynamic Image Networks for Action Recognition
Bilen, H., Fernando, B., Gavves, E., Vedaldi, A., & Gould, S. J. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.331 · Cited by 579
Modeling video evolution for action recognition
Fernando, B., Gavves, E., Jose Oramas, M., Ghodrati, A., & Tuytelaars, T. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7299176 · Cited by 573
Gkioxari, G., & Malik, J. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298676 · Cited by 552
VideoLSTM convolves, attends and flows for action recognition
Li, Z., Gavrilyuk, K., Gavves, E., Jain, M., & Snoek, C. G. M. (2017) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2017.10.011 · Cited by 492
Action recognition by learning mid-level motion features
Fathi, A., & Mori, G. (2008) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2008.4587735 · Cited by 465
Convolutional Two-Stream Network Fusion for Video Action RecognitionPreprint
Feichtenhofer, C., Pinz, A. J., & Zisserman, A. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1604.06573 · Cited by 384
Self-citationMulti-region Two-Stream R-CNN for Action Detection
Peng, X., & Schmid, C. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46493-0_45 · Cited by 375
Recurrent Batch NormalizationPreprint
Cooijmans, T., Ballas, N., Laurent, C., Gülçehre, Ç., & Courville, A. C. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.09025 · Cited by 342
Deep Learning for Detecting Multiple Space-Time Action Tubes in Videos
Saha, S., Singh, G., Sapienza, M., Torr, P. H. S., & Fabio Cuzzolin, A. (2016) · British Machine Vision Conference (BMVC) · doi:10.5244/c.30.58 · Cited by 197
Human Focused Action Localization in Video
Kläser, A., Marszałek, M., Schmid, C., & Zisserman, A. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-35749-7_17 · Cited by 137
Self-citationVideoLSTM Convolves, Attends and Flows for Action RecognitionPreprint
Li, Z., Gavves, E., Jain, M., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1607.01794 · Cited by 63
Initialization Strategies of Spatio-Temporal Convolutional Neural NetworksPreprint
Mansimov, E., Srivastava, N., & Salakhutdinov, R. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1503.07274 · Cited by 28
Actions ~ TransformationsPreprint
Wang, X., Farhadi, A., & Gupta, A. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1512.00795 · Cited by 15
Flagged first, then by citations
Cited by
Sorted by citations
Wang, X., Girshick, R., Gupta, A., & He, K. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00813 · Cited by 11,596
DeepLabCut: markerless pose estimation of user-defined body parts with deep learning
Mathis, A., Mamidanna, P., Cury, K. M., Abe, T., Murthy, V. N., Mathis, M. W., & Bethge, M. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0209-y · Cited by 5,187
Han (2023) · doi:10.1109/tpami.2022.3152247 · Cited by 4,682
SlowFast Networks for Video Recognition
Feichtenhofer, C., Fan, H., Malik, J., & He, K. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00630 · Cited by 4,007
A Closer Look at Spatiotemporal Convolutions for Action Recognition
Tran, D., Wang, H., Torresani, L., Ray, J., LeCun, Y., & Paluri, M. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00675 · Cited by 3,672
Metadata
- Authors
- Carreira, J., & Zisserman, A.
- Affiliations
- University of Oxford (GB)
- ORCID
- Zisserman, Andrew
- Venue
- 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2017-07
- Pages
- 4724-4733
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2619082050
- Cited by
- 9,570
- References
- 38
- Citations by year
- 2026: 702 · 2025: 1227 · 2024: 1428 · 2023: 1500 · 2022: 1315 · 2021: 1414 · 2020: 994 · 2019: 697 · 2018: 274 · 2017: 16
- FWCI
- 158.44
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Physical Sciences › Computer Science › Human-Computer Interaction › Hand Gesture Recognition Systems
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- human action recognition, I3D, dual-stream architecture, spatiotemporal feature extraction, pre-training, video classification, UCF101, HMDB51, YouTube videos
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
