The DOI matches its Crossref record; 44 of the 44 references it cites were found in a registry or an index.
Few-Shot Video Classification via Temporal Alignment
Open accessAbstract
Difficulty in collecting and annotating large-scale video data raises a growing interest in learning models which can recognize novel classes with only a few training examples. In this paper, we propose the Ordered Temporal Alignment Module (OTAM), a novel few-shot learning framework that can learn to classify a previously unseen video. While most previous work neglects long-term temporal ordering information, our proposed model explicitly leverages the temporal ordering information in video data through ordered temporal alignment. This leads to strong data-efficiency for few-shot learning. In concrete, our proposed pipeline learns a deep distance measurement of the query video with respect to novel class proxies over its alignment path. We adopt an episode-based training scheme and directly optimize the few-shot learning objective. We evaluate OTAM on two challenging real-world datasets, Kinetics and Something-Something-V2, and show that our model leads to significant improvement of few-shot video classification over a wide range of competitive baselines and outperforms state-of-the-art benchmarks by a large margin.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Gapped BLAST and PSI-BLAST: a new generation of protein database search programs
Altschul, S. (1997) · Nucleic Acids Research · doi:10.1093/nar/25.17.3389 · Cited by 73,302
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Neural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213
(2017) · Cited by 11,625
Wang, X., Girshick, R., Gupta, A., & He, K. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00813 · Cited by 11,596
Learning Spatiotemporal Features with 3D Convolutional Networks
Tran, D., Bourdev, L., Fergus, R., Torresani, L., & Paluri, M. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.510 · Cited by 9,593
Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset
Carreira, J., & Zisserman, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.502 · Cited by 9,570
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
Finn, C., Abbeel, P., & Levine, S. (2017) · arXiv · doi:10.48550/arxiv.1703.03400 · Cited by 8,512
Large-Scale Video Classification with Convolutional Neural Networks
Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., & Fei-Fei, L. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.223 · Cited by 6,354
Large-Scale Machine Learning with Stochastic Gradient Descent
Bottou, L. (2010) · Proceedings of COMPSTAT'2010 · doi:10.1007/978-3-7908-2604-3_16 · Cited by 5,795
Ucf101: A dataset of 101 human actions classes from videos in the wild
Soomro, K., Zamir, A. R., & Shah, M. (2012) · arXiv · doi:10.48550/arxiv.1212.0402 · Cited by 4,575
Temporal Segment Networks: Towards Good Practices for Deep Action Recognition
Wang, L., Xiong, Y., Wang, Z., Qiao, Y., Lin, D., Tang, X., & Van Gool, L. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46484-8_2 · Cited by 3,958
A Closer Look at Spatiotemporal Convolutions for Action Recognition
Tran, D., Wang, H., Torresani, L., Ray, J., LeCun, Y., & Paluri, M. (2017) · arXiv · doi:10.48550/arxiv.1711.11248 · Cited by 3,664
Action Recognition with Improved Trajectories
Wang, H., & Schmid, C. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.441 · Cited by 3,407
The Kinetics Human Action Video Dataset
Kay, W., Carreira, J., Simonyan, K., Zhang, B., Hillier, C., Vijayanarasimhan, S., Viola, F., Green, T., Back, T., Natsev, P., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.06950 · Cited by 2,593
TSM: Temporal Shift Module for Efficient Video Understanding
Lin, J., Gan, C., & Han, S. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00718 · Cited by 2,055
A Spatio-Temporal Descriptor Based on 3D-Gradients
Klaeser, A., Marszalek, M., & Schmid, C. (2008) · Procedings of the British Machine Vision Conference 2008 · doi:10.5244/c.22.99 · Cited by 1,846
A 3-dimensional sift descriptor and its application to action recognition
Scovanner, P., Ali, S., & Shah, M. (2007) · Proceedings of the 15th ACM international conference on Multimedia · doi:10.1145/1291233.1291311 · Cited by 1,596
The “Something Something” Video Database for Learning and Evaluating Visual Common Sense
Goyal, R., Kahou, S. E., Michalski, V., Materzynska, J., Westphal, S., Kim, H., Haenel, V., Fruend, I., Yianilos, P., Mueller-Freitag, M., et al. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.622 · Cited by 1,471
Rethinking Spatiotemporal Feature Learning: Speed-Accuracy Trade-offs in Video Classification
Xie, S., Sun, C., Huang, J., Tu, Z., & Murphy, K. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01267-0_19 · Cited by 1,415
(2007) · Information Retrieval for Music and Motion · doi:10.1007/978-3-540-74048-3_4 · Cited by 1,359
Dynamic Few-Shot Visual Learning Without Forgetting
Gidaris, S., & Komodakis, N. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00459 · Cited by 1,128
Temporal Relational Reasoning in Videos
Zhou, B., Andonian, A., Oliva, A., & Torralba, A. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01246-5_49 · Cited by 1,067
Hollywood in Homes: Crowdsourcing Data Collection for Activity Understanding
Sigurdsson, G. A., Varol, G., Wang, X., Farhadi, A., Laptev, I., & Gupta, A. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46448-0_31 · Cited by 1,013
YouTube-8M: A Large-Scale Video Classification Benchmark
Abu-El-Haija, S., Kothari, N., Lee, J., Natsev, P., Toderici, G. D., Varadarajan, B., & Vijayanarasimhan, S. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1609.08675 · Cited by 848
Low-Shot Visual Recognition by Shrinking and Hallucinating Features
Hariharan, B., & Girshick, R. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.328 · Cited by 792
Videos as Space-Time Region Graphs
Wang, X., & Gupta, A. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01228-1_25 · Cited by 765
Low-Shot Learning from Imaginary Data
Wang, Y.-X., Girshick, R., Hebert, M., & Hariharan, B. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00760 · Cited by 744
Reptile: a Scalable Metalearning AlgorithmPreprint
Nichol, A., & Schulman, J. (2018) · arXiv (Cornell University) · Cited by 648
Low-Shot Learning with Imprinted Weights
Qi, H., Brown, M., & Lowe, D. G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00610 · Cited by 539
Few-Shot Learning with Graph Neural Networks
García, V., & Bruna, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.04043 · Cited by 364
Compound Memory Networks for Few-Shot Video Classification
Zhu, L., & Yang, Y. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_46 · Cited by 239
(2017) · Interval Analysis · doi:10.1515/9783110499469-009 · Cited by 235
Learning to Remember Rare Events
Kaiser, Ł., Nachum, O., Roy, A., & Bengio, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.03129 · Cited by 218
Chang, C., Huang, D.-A., Sui, Y.-N., Fei-Fei, L., & Niebles, J. C. (2019) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2019.00366 · Cited by 186
Self-citationTARN: Temporal Attentive Relation Network for Few-Shot and Zero-Shot Action RecognitionPreprint
Bishay, M., Zoumpourlis, G., & Patras, I. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.09021 · Cited by 183
A Generative Approach to Zero-Shot and Few-Shot Action Recognition
Mishra, A. K., Verma, V. K., Reddy, M. S. K., Arulkumar, S., Rai, P., & Mittal, A. (2018) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2018.00047 · Cited by 164
ProtoGAN: Towards Few Shot Learning for Action Recognition
Dwivedi, S. K., Gupta, V., Mitra, R., Ahmed, S., & Jain, A. (2019) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2019.00166 · Cited by 135
Differentiable Dynamic Programming for Structured Prediction and AttentionPreprint
Mensch, A., & Blondel, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.03676 · Cited by 112
Learning Temporal Action Proposals With Fewer Labels
Ji, J., Cao, K., & Niebles, J. C. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00717 · Cited by 54
Self-citationOne Shot Similarity Metric Learning for Action Recognition
Kliper-Gross, O., Hassner, T., & Wolf, L. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-24471-1_3 · Cited by 38
Embodied One-Shot Video Recognition: Learning from Actions of a Virtual Embodied Agent
Fu, Y., Wang, C., Fu, Y., Wang, Y.-X., Bai, C., Xue, X., & Jiang, Y. (2019) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3343031.3351015 · Cited by 36
Metric-Based Few-Shot Learning for Video Action RecognitionPreprint
Careaga, C., Hutchinson, B., Hodas, N. O., & Phillips, L. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.09602 · Cited by 33
Cited by
Human Action Recognition and Prediction: A Survey
Kong, Y., & Fu, Y. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-022-01594-9 · Cited by 780
Prompting Visual-Language Models for Efficient Video Understanding
Ju, C., Han, T., Zheng, K., Zhang, Y., & Xie, W. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-19833-5_7 · Cited by 391
Metadata
- Authors
- Cao, K., Ji, J., Cao, Z., Chang, C.-Y., & Niebles, J. C.
- Venue
- 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2020-06
- Pages
- 10615-10624
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W3035374961
- Cited by
- 268
- References
- 44
- FWCI
- 14.31
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.