Cited by 9,570Human Pose and Action RecognitionHand Gesture Recognition SystemsAdvanced Neural Network Applications

Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset

Open access
Carreira, Joao, Zisserman, Andrew·2017·2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

Abstract

The paucity of videos in current action classification datasets (UCF-101 and HMDB-51) has made it difficult to identify good video architectures, as most methods obtain similar performance on existing small-scale benchmarks. This paper re-evaluates state-of-the-art architectures in light of the new Kinetics Human Action Video dataset. Kinetics has two orders of magnitude more data, with 400 human action classes and over 400 clips per class, and is collected from realistic, challenging YouTube videos. We provide an analysis on how current architectures fare on the task of action classification on this dataset and how much performance improves on the smaller benchmark datasets after pre-training on Kinetics. We also introduce a new Two-Stream Inflated 3D ConvNet (I3D) that is based on 2D ConvNet inflation: filters and pooling kernels of very deep image classification ConvNets are expanded into 3D, making it possible to learn seamless spatio-temporal feature extractors from video while leveraging successful ImageNet architecture designs and even their parameters. We show that, after pre-training on Kinetics, I3D models considerably improve upon the state-of-the-art in action classification, reaching 80.2% on HMDB-51 and 97.9% on UCF-101.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/cvpr.2017.502

Cites

41 of 41 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • I3D and R3D achieve better performance than vanilla ResNet50 and EfficientNet, but computation costs are huge.

    “Although I3D [4] and R3D [19] achieve better performance than vanilla ResNet50 [20] and EfficientNet [50], the computation costs are huge, such as R3D50 with 296G FLOPs.”

    From TALL: Thumbnail Layout for Deepfake Video Detection (2023) · cites this paper as [4]

    Section 5.1 reports that Kinetics-pretrained I3D outperformed earlier methods on UCF-101 and HMDB-51, including a method using ResNet-50. It does not compare I3D with vanilla ResNet50 or EfficientNet, or establish the claimed computation costs; the R3D50 FLOPs figure is attached to a different citation in the citing sentence. (LLM, conf=0.86)

    Experimental — accuracy not yet measured
  • A 5-block non-local C2D with ResNet-101 outperforms inflated 3D ConvNet (I3D) on Kinetics.

    “(e) Non-local vs . 3D Conv : A 5-block non-local C2D vs . inflated 3D ConvNet (I3D) [7]. All entries are with ResNet-101. The numbers of parameters and FLOPs are relative to the C2D baseline (43.2M and 34.2B).”

    From Non-local Neural Networks (2018) · cites this paper as [7]

  • With a few non-local blocks, non-local neural networks are more accurate for video classification than 2D and 3D convolutional networks (including the inflated variant).

    “With a few non-local blocks, our architecures called non-local neural networks are more accurate for video classification than 2D and 3D convolutional networks [48] (including the inflated variant [7]).”

    From Non-local Neural Networks (2018) · cites this paper as [7]

  • Many of the deep-learning architectures mentioned in this review have similarities to ones used in analysis of digital media that try to combine multiple modalities to improve predictive performance of video content.

    “many of the deep-learning architectures mentioned in this review have similarities to ones used in analysis of digital media that try to combine multiple modalities to improve predictive performance of video content (Carreira and Zisserman, 2017; Afouras et al., 2019).”

    From Missing data in multi-omics integration: Recent advances through artificial intelligence (2023) · cites this paper as Carreira and Zisserman, 2017

  • Compared to RNN-based architectures, Transformers have significant advantages in long-term dependency and parallel computation.

    “Compared to RNN-based architectures, Transformers [137] have significant advantages in long-term dependency and parallel computation.”

    From Deep learning for visual speech analysis: A survey (2024) · cites this paper as [137]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 1

  1. [32]32. G. Varol, I. Laptev, and C. Schmid. Long-term temporal convolutions for action recognition. IEEE transactions on pattern analysis and machine intelligence, 2017.

Cited by

Sorted by citations

Authors
Carreira, J., & Zisserman, A.
Affiliations
University of Oxford (GB)
Venue
2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
Publisher
IEEE
Published
2017-07
Pages
4724-4733
ISSN
1063-6919 (ISSN-L)
OpenAlex
W2619082050
Cited by
9,570
References
38
Citations by year
2026: 702 · 2025: 1227 · 2024: 1428 · 2023: 1500 · 2022: 1315 · 2021: 1414 · 2020: 994 · 2019: 697 · 2018: 274 · 2017: 16 · 2016: 1 · 2012: 2
FWCI
158.44
Type
conference paper
Language
en
Open access
Locations
  • IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
  • Physical Sciences › Computer Science › Human-Computer Interaction › Hand Gesture Recognition Systems
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
Keywords
human action recognition, I3D, dual-stream architecture, spatiotemporal feature extraction, pre-training, video classification, UCF101, HMDB51, YouTube videos

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review