Multiview Transformers for Video Recognition
Open accessAbstract
Video understanding requires reasoning at multiple spatiotemporal resolutions – from short fine-grained motions to events taking place over longer durations. Although transformer architectures have recently advanced the state-of-the-art, they have not explicitly modelled different spatiotemporal resolutions. To this end, we present Multiview Transformers for Video Recognition (MTV). Our model consists of separate encoders to represent different views of the input video with lateral connections to fuse information across views. We present thorough ablation studies of our model and show that MTV consistently performs better than single-view counterparts in terms of accuracy and computational cost across a range of model sizes. Furthermore, we achieve state-of-the-art results on six standard datasets, and improve even further with large-scale pretraining. Code and checkpoints are available at: https://github.com/google-research/scenic.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
One solution is using knowledge distillation, e.g., distilling from multimodal to uni-modal attention in Transformers, distilling from multiple uni-modal Transformer teachers to a shared Transformer encoder.
“One solution is using knowledge distillation, e.g., distilling from multimodal to uni-modal attention in Transformers [275], distilling from multiple uni-modal Transformer teachers to a shared Transformer encoder [276].”
From Multimodal learning with transformers: A survey (2023) · cites this paper as [276]
Distilling multimodal knowledge into small models that are adaptable to uni-modal data and tasks is a successful practice.
“Distilling multimodal knowledge into small models that are adaptable to uni-modal data and tasks is a successful practice [275], [276].”
From Multimodal learning with transformers: A survey (2023) · cites this paper as [276]
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
Jakubův, J., Chvalovský, K., Goertzel, Z., et al. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,648 · see citing works
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 74,248 · see citing works
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · see citing works
Distinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589 · see citing works
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534 · see citing works
Histograms of Oriented Gradients for Human Detection
Dalal, N., & Triggs, B. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.177 · Cited by 33,584 · see citing works
Rethinking the Inception Architecture for Computer Vision
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.308 · Cited by 32,138 · see citing works
(2016) · Cited by 31,590 · see citing works
Feature Pyramid Networks for Object Detection
Lin, T.-Y., Dollar, P., Girshick, R., He, K., Hariharan, B., & Belongie, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.106 · Cited by 30,939 · see citing works
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · see citing works
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 21,793 · see citing works
Xception: Deep Learning with Depthwise Separable Convolutions
Chollet, F. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.195 · Cited by 20,380 · see citing works
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
Tan, M., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1905.11946 · Cited by 17,072 · see citing works
Zhao, H., Shi, J., Qi, X., Wang, X., & Jia, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.660 · Cited by 16,438 · see citing works
Layer NormalizationPreprint
Ba, J., Kiros, J., & Hinton, G. E. (2016) · arXiv.org · doi:10.48550/arxiv.1607.06450 · Cited by 13,762 · see citing works
Kaugeranna, R., Kaugeranna, E., & Aion, (. S. 4. 6. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.giscience.2023.43 · Cited by 13,695 · see citing works
(1981) · Cited by 11,963 · see citing works
Backpropagation Applied to Handwritten Zip Code Recognition
LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., & Jackel, L. D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.4.541 · Cited by 11,891 · see citing works
Wang, X., Girshick, R., Gupta, A., & He, K. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00813 · Cited by 11,596 · see citing works
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · see citing works
Learning Spatiotemporal Features with 3D Convolutional Networks
Tran, D., Bourdev, L., Fergus, R., Torresani, L., & Paluri, M. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.510 · Cited by 9,593 · see citing works
Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset
Carreira, J., & Zisserman, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.502 · Cited by 9,570 · see citing works
Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories
Lazebnik, S., Schmid, C., & Ponce, J. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.68 · Cited by 7,916 · see citing works
Self-citationTwo-Stream Convolutional Networks for Action Recognition in Videos
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1406.2199 · Cited by 6,756 · see citing works
Large-Scale Video Classification with Convolutional Neural Networks
Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., & Fei-Fei, L. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.223 · Cited by 6,354 · cited 1× · see citing works
“At the same time, CNNs and RNNs have quickly become the de-facto backbones for video understanding tasks [32, 50, 57].”
Backgroundmixup: Beyond Empirical Risk Minimization
Zhang, H., Cisse, M., Dauphin, Y. N., & Lopez-Paz, D. (2017) · arXiv · doi:10.48550/arxiv.1710.09412 · Cited by 5,831 · see citing works
Neural Architecture Search with Reinforcement LearningPreprint
Zoph, B., & Le, Q. V. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.01578 · Cited by 5,704 · see citing works
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions
Wang, W., Xie, E., Li, X., Fan, D.-P., Song, K., Liang, D., Lu, T., Luo, P., & Shao, L. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00061 · Cited by 5,259 · cited 3× · see citing works
“In practice, due to the limited amount of training data, applying similar multiscale processing in transformers [11, 21, 44, 74] to reduce complexity of the model has proven to be effective.”
BackgroundThe Kinetics Human Action Video DatasetPreprint
Kay, W., Carreira, J., Simonyan, K., Zhang, B., Hillier, C., Vijayanarasimhan, S., Viola, F., Green, T., Back, T., Natsev, P., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.06950 · Cited by 4,773 · see citing works
SlowFast Networks for Video Recognition
Feichtenhofer, C., Fan, H., Malik, J., & He, K. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00630 · Cited by 4,007 · see citing works
A Closer Look at Spatiotemporal Convolutions for Action Recognition
Tran, D., Wang, H., Torresani, L., Ray, J., LeCun, Y., & Paluri, M. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00675 · Cited by 3,672 · see citing works
Gaussian Error Linear Units (GELUs)
Hendrycks, D., & Gimpel, K. (2016) · arXiv · doi:10.48550/arxiv.1606.08415 · Cited by 3,537 · see citing works
Randaugment: Practical automated data augmentation with a reduced search space
Cubuk, E. D., Zoph, B., Shlens, J., & Le, Q. V. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00359 · Cited by 3,223 · see citing works
ViViT: A Video Vision Transformer
Arnab, A., Dehghani, M., Heigold, G., Sun, C., Lucic, M., & Schmid, C. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00676 · Cited by 2,767 · see citing works
Self-citationRevisiting Unreasonable Effectiveness of Data in Deep Learning Era
Sun, C., Shrivastava, A., Singh, S., & Gupta, A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.97 · Cited by 2,484 · see citing works
Self-citationCrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification
Chen, C.-F. R., Fan, Q., & Panda, R. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00041 · Cited by 2,326 · see citing works
TSM: Temporal Shift Module for Efficient Video Understanding
Lin, J., Gan, C., & Han, S. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00718 · Cited by 2,152 · see citing works
Beyond short snippets: Deep networks for video classification
Ng, J. Y.-H., Hausknecht, M., Vijayanarasimhan, S., Vinyals, O., Monga, R., & Toderici, G. D. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7299101 · Cited by 2,121 · see citing works
Deep Networks with Stochastic Depth
Huang, G., Sun, Y., Liu, Z., Sedra, D., & Weinberger, K. Q. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_39 · Cited by 2,104 · see citing works
A Spatio-Temporal Descriptor Based on 3D-Gradients
Klaeser, A., Marszałek, M., & Schmid, C. (2008) · British Machine Vision Conference (BMVC) · doi:10.5244/c.22.99 · Cited by 1,842 · see citing works
Self-citationDense Trajectories and Motion Boundary Descriptors for Action Recognition
Wang, H., Kläser, A., Schmid, C., & Liu, C.-L. (2013) · International Journal of Computer Vision · doi:10.1007/s11263-012-0594-8 · Cited by 1,669 · see citing works
Self-citationIs Space-Time Attention All You Need for Video Understanding?
Bertasius, G., Wang, H., & Torresani, L. (2021) · arXiv · doi:10.48550/arxiv.2102.05095 · Cited by 1,663 · see citing works
The kinetics human action video dataset
Kay, W. (2024) · TIB Data Manager · doi:10.57702/jtuxnt5m · Cited by 1,646 · see citing works
Rethinking Spatiotemporal Feature Learning: Speed-Accuracy Trade-offs in Video Classification
Xie, S., Sun, C., Huang, J., Tu, Z., & Murphy, K. P. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01267-0_19 · Cited by 1,552 · see citing works
Self-citationThe “Something Something” Video Database for Learning and Evaluating Visual Common Sense
Goyal, R., Kahou, S. E., Michalski, V., Materzynska, J., Westphal, S., Kim, H., Haenel, V., Fruend, I., Yianilos, P., Mueller-Freitag, M., et al. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.622 · Cited by 1,471 · see citing works
Multiscale vision transformers
Fan, H., Xiong, B., Mangalam, K., Li, Y., Yan, Z., Malik, J., & Feichtenhofer, C. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00675 · Cited by 1,407 · see citing works
Large-Scale Evolution of Image ClassifiersPreprint
Real, E., Moore, S., Selle, A. P., Saxena, S., Suematsu, Y. I. L., Tan, J., Le, Q. V., & Kurakin, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.01041 · Cited by 1,372 · see citing works
X3D: Expanding Architectures for Efficient Video Recognition
Feichtenhofer, C. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00028 · Cited by 1,327 · see citing works
Laptev, & Tony Lindeberg, C. (2003) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2003.1238378 · Cited by 1,301 · see citing works
HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips
Miech, A., Zhukov, D., Alayrac, J.-B., Tapaswi, M., Laptev, I., & Sivic, J. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00272 · Cited by 986 · see citing works
Zhai, X., Kolesnikov, A., Houlsby, N., & Beyer, L. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01179 · Cited by 891 · see citing works
Statistics of natural image categories
Torralba, A. B., & Oliva, A. (2003) · Network Computation in Neural Systems · doi:10.1088/0954-898x_14_3_302 · Cited by 845 · see citing works
Spatiotemporal Residual Networks for Video Action RecognitionPreprint
Feichtenhofer, C., Pinz, A. J., & Wildes, R. P. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.02155 · Cited by 646 · see citing works
Video Classification With Channel-Separated Convolutional Networks
Tran, D., Wang, H., Feiszli, M., & Torresani, L. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00565 · Cited by 642 · see citing works
TEA: Temporal Excitation and Aggregation for Action Recognition
Li, Y., Ji, B., Shi, X., Zhang, J., Kang, B., & Wang, L. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00099 · Cited by 600 · see citing works
Human Action Recognition Using Factorized Spatio-Temporal Convolutional Networks
Sun, L., Jia, K., Yeung, D., & Shi, B. E. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.522 · Cited by 580 · see citing works
The Laplacian Pyramid as a Compact Image Code
Burt, P. J., & Adelson, E. H. (1987) · Elsevier eBooks · doi:10.1016/b978-0-08-051581-6.50065-9 · Cited by 564 · see citing works
Moments in Time Dataset: One Million Videos for Event Understanding
Monfort, M., Vondrick, C., Oliva, A., Andonian, A., Zhou, B., Ramakrishnan, K., Bargal, S. A., Yan, T., Brown, L., Fan, Q., et al. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2901464 · Cited by 537 · see citing works
Statistics of natural images and models
Huang, J., & Mumford, D. (2003) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.1999.786990 · Cited by 503 · see citing works
Long-Term Feature Banks for Detailed Video Understanding
Wu, C.-Y., Feichtenhofer, C., Fan, H., He, K., Krähenbühl, P., & Girshick, R. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00037 · Cited by 454 · see citing works
VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and TextPreprint
Akbari, H., Yuan, L., Qian, R., Chuang, W.-H., Chang, S., Cui, Y., & Gong, B. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.11178 · Cited by 433 · see citing works
Attention Bottlenecks for Multimodal Fusion
Nagrani, A., Yang, S., Arnab, A., Jansen, A., Schmid, C., & Sun, C. (2021) · arXiv · doi:10.48550/arxiv.2107.00135 · Cited by 400 · see citing works
Self-citationRescaling egocentric vision: Collection, pipeline and challenges for epic-kitchens-100
Damen, D., Doughty, H., Farinella, G. M., Furnari, A., Kazakos, E., Ma, J., Moltisanti, D., Munro, J., Perrett, T., Price, W., et al. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-021-01531-2 · Cited by 400 · see citing works
Florence: A New Foundation Model for Computer Vision
Yuan, L., Chen, D., Chen, Y.-L., Codella, N., Dai, X., Gao, J., Hu, H., Huang, X., Li, B., Li, C., et al. (2021) · arXiv · doi:10.48550/arxiv.2111.11432 · Cited by 389 · see citing works
Perceiver IO: A General Architecture for Structured Inputs & OutputsPreprint
Jaegle, A., Borgeaud, S., Alayrac, J.-B., Doersch, C., Ionescu, C., Ding, X. D., Koppula, S., Zoran, D., Brock, A., Shelhamer, E., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.14795 · Cited by 357 · see citing works
How to train your ViT? Data, Augmentation, and Regularization in Vision Transformers
Steiner, A., Kolesnikov, A., Zhai, X., Wightman, R., Uszkoreit, J., & Beyer, L. (2021) · arXiv · doi:10.48550/arxiv.2106.10270 · Cited by 292 · see citing works
Large-Scale Weakly-Supervised Pre-Training for Video Action Recognition
Ghadiyaram, D., Tran, D., & Mahajan, D. K. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01232 · Cited by 286 · see citing works
MoViNets: Mobile Video Networks for Efficient Video Recognition
Kondratyuk, D., Yuan, L., Li, Y., Zhang, L., Tan, M., Brown, M., & Gong, B. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01576 · Cited by 286 · see citing works
VidTr: Video Transformer Without Convolutions
Zhang, Y., Li, X., Liu, C., Shuai, B., Zhu, Y., Brattoli, B., Chen, H., Marsic, I., & Tighe, J. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01332 · Cited by 227 · see citing works
Learning Spatio-Temporal Representation With Local and Global Diffusion
Qiu, Z., Yao, T., Ngo, C., Tian, X., & Mei, T. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01233 · Cited by 209 · see citing works
Training data-efficient image transformers & distillation through attention
Touvron, H., Cord, M., Douze, M., Massa, F., Sablayrolles, A., & Jégou, H. (2020) · arXiv · doi:10.48550/arxiv.2012.12877 · Cited by 182 · see citing works
Keeping Your Eye on the Ball: Trajectory Attention in Video TransformersPreprint
Patrick, M., Campbell, D., Asano, Y. M., Misra, I., Metze, F., Feichtenhofer, C., Vedaldi, A., & Henriques, J. F. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.05392 · Cited by 181 · see citing works
TokenLearner: What Can 8 Learned Tokens Do for Images and Videos?Preprint
Ryoo, M. S., Piergiovanni, A., Arnab, A., Dehghani, M., & Angelova, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.11297 · Cited by 152 · see citing works
Self-citationOmni-Sourced Webly-Supervised Learning for Video Recognition
Duan, H., Zhao, Y., Xiong, Y., Liu, W., & Lin, D. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58555-6_40 · Cited by 97 · see citing works
AssembleNet: Searching for Multi-Stream Neural Connectivity in Video ArchitecturesPreprint
Ryoo, M. S., Piergiovanni, A., Tan, M., & Angelova, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.13209 · Cited by 90 · see citing works
A Multigrid Method for Efficiently Training Video Models
Wu, C.-Y., Girshick, R., He, K., Feichtenhofer, C., & Krähenbühl, P. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00023 · Cited by 88 · see citing works
AttentionNAS: Spatiotemporal Attention Cell Search for Video Classification
Wang, X., Xiong, X., Neumann, M., Piergiovanni, A., Ryoo, M. S., Angelova, A., Kitani, K., & Hua, W. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58598-3_27 · Cited by 45 · see citing works
Self-citationUnified Graph Structured Models for Video Understanding
Arnab, A., Sun, C., & Schmid, C. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00801 · Cited by 42 · see citing works
Self-citationCo-training Transformer with Videos and Images Improves Action RecognitionPreprint
Zhang, B., Yu, J., Fifty, C., Han, W., Dai, A. M., Pang, R., & Sha, F. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2112.07175 · Cited by 30 · see citing works
Learning Video Representations from Textual Web SupervisionPreprint
Stroud, J., Lu, Z., Sun, C., Deng, J., Sukthankar, R., Schmid, C., & Ross, D. A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.14937 · Cited by 28 · see citing works
Self-citationRevisiting 3D ResNets for Video RecognitionPreprint
Du, X., Li, Y., Cui, Y., Qian, R., Li, J., & Bello, I. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.01696 · Cited by 16 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805
Flagged first, then by citations
5 of the 92 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
A review of convolutional neural networks in computer vision
Zhao, X., Wang, L., Zhang, Y., Han, X., Deveci, M., & Parmar, M. D. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10721-6 · Cited by 1,326 · see citing works
Multimodal learning with transformers: A survey
Xu, P., Zhu, X., & Clifton, D. A. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3275156 · Cited by 1,072 · see citing works
ImageBind One Embedding Space to Bind Them All
Girdhar, R., El-Nouby, A., Liu, Z., Singh, M., Alwala, K. V., Joulin, A., & Misra, I. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01457 · Cited by 863 · see citing works
Flamingo: a visual language model for few-shot learning
Alayrac, J.-B., Donahue, J., Luc, P., Miech, A., Barr, I., Hasson, Y., Lenc, K., Mensch, A., Millican, K., Reynolds, M., et al. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1723 · Cited by 966 · see citing works
“[134] Shen Yan, Xuehan Xiong, Anurag Arnab, Zhichao Lu, Mi Zhang, Chen Sun, and Cordelia Schmid.”
BackgroundHuman Action Recognition From Various Data Modalities: A Review
Sun, Z., Ke, Q., Rahmani, H., Bennamoun, M., Wang, G., & Liu, J. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3183112 · Cited by 719 · see citing works
VideoMAE V2: Scaling Video Masked Autoencoders with Dual Masking
Wang, L., Huang, B., Zhao, Z., Tong, Z., He, Y., Wang, Y., Wang, Y., & Qiao, Y. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01398 · Cited by 536 · see citing works
Expanding Language-Image Pretrained Models for General Video Recognition
Ni, B., Peng, H., Chen, M., Zhang, S., Meng, G., Fu, J., Xiang, S., & Ling, H. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19772-7_1 · Cited by 401 · see citing works
PoseFormerV2: Exploring Frequency Domain for Efficient and Robust 3D Human Pose Estimation
Zhao, Q., Zheng, C., Liu, M., Wang, P., & Chen, C. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00857 · Cited by 272 · see citing works
Frozen CLIP Models are Efficient Video Learners
Lin, Z., Geng, S., Zhang, R., Gao, P., de Melo, G., Wang, X., Dai, J., Qiao, Y., & Li, H. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-19833-5_23 · Cited by 213 · see citing works
Fine-tuned CLIP Models are Efficient Video Learners
Rasheed, H., Khattak, M. U., Maaz, M., Khan, S., & Khan, F. S. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00633 · Cited by 194 · see citing works
Selva, J., Johansen, A. S., Escalera, S., Nasrollahi, K., Moeslund, T. B., & Clapés, A. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3243465 · Cited by 192 · see citing works
MERLOT RESERVE: Neural Script Knowledge through Vision and Language and Sound
Zellers, R., Lu, J., Lu, X., Yu, Y., Zhao, Y., Salehi, M., Kusupati, A., Hessel, J., Farhadi, A., & Choi, Y. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01589 · Cited by 163 · see citing works
Learning Video Representations from Large Language Models
Zhao, Y., Misra, I., Krähenbühl, P., & Girdhar, R. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00637 · Cited by 164 · see citing works
Human Action Recognition: A Taxonomy-Based Survey, Updates, and Opportunities
Morshed, M. G., Sultana, T., Alam, A., & Lee, Y.-K. (2023) · Sensors · doi:10.3390/s23042182 · Cited by 167 · see citing works
Unmasked Teacher: Towards Training-Efficient Video Foundation Models
Li, K., Wang, Y., Li, Y., Wang, Y., He, Y., Wang, L., & Qiao, Y. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01826 · Cited by 143 · see citing works
Wu, W., Wang, X., Luo, H., Wang, J., Yang, Y., & Ouyang, W. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00640 · Cited by 131 · see citing works
Revisiting Classifier: Transferring Vision-Language Models for Video Recognition
Wu, W., Sun, Z., & Ouyang, W. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i3.25386 · Cited by 112 · see citing works
Moutik, O., Sekkat, H., Tigani, S., Chehri, A., Saadane, R., Tchakoucht, T. A., & Paul, A. (2023) · Sensors · doi:10.3390/s23020734 · Cited by 114 · see citing works
Yang, M., Chengke, W., Guo, Y., Jiang, R., Zhou, F., Zhang, J., & Yang, Z. (2022) · Automation in Construction · doi:10.1016/j.autcon.2022.104703 · Cited by 103 · see citing works
UniFormerV2: Unlocking the Potential of Image ViTs for Video Understanding
Li, K., Wang, Y., He, Y., Li, Y., Wang, Y., Wang, L., & Qiao, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00157 · Cited by 91 · see citing works
Incomplete Multi-view Clustering via Prototype-based Imputation
Li, H., Li, Y., Yang, M., Hu, P., Peng, D., & Peng, X. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/435 · Cited by 78 · see citing works
Multi-View Action Recognition using Contrastive Learning
Shah, K., Shah, A., Lau, C. P., de Melo, C. M., & Chellapp, R. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00338 · Cited by 73 · see citing works
Rethinking Video ViTs: Sparse Video Tubes for Joint Image and Video Learning
Piergiovanni, A., Kuo, W., & Angelova, A. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00220 · Cited by 72 · see citing works
LAVENDER: Unifying Video-Language Understanding as Masked Language Modeling
Li, L., Gan, Z., Lin, K., Lin, C.-C., Liu, Z., Liu, C., & Wang, L. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02214 · Cited by 73 · see citing works
Ma, Y., & Wang, R. (2023) · Pattern Recognition · doi:10.1016/j.patcog.2023.109905 · Cited by 63 · see citing works
Lyu, J., Li, G., Wang, C., Qin, C., Wang, S., Dou, Q., & Qin, J. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.102760 · Cited by 64 · see citing works
Deep learning approaches for seizure video analysis: A review
Ahmedt‐Aristizabal, D., Armin, M. A., Hayder, Z., Garcia‐Cairasco, N., Petersson, L., Fookes, C., Denman, S., & McGonigal, A. (2024) · Epilepsy & Behavior · doi:10.1016/j.yebeh.2024.109735 · Cited by 63 · see citing works
Human Activity Recognition Using Cascaded Dual Attention CNN and Bi-Directional GRU Framework
Ullah, H., & Munir, A. (2023) · Journal of Imaging · doi:10.3390/jimaging9070130 · Cited by 64 · see citing works
Verbs in Action: Improving verb understanding in video-language models
Momeni, L., Caron, M., Nagrani, A., Zisserman, A., & Schmid, C. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01428 · Cited by 58 · see citing works
Self-citationMachine Learning for Human Activity Recognition: State-of-the-Art Techniques and Emerging Trends
Hossen, M. A., & Abas, P. E. (2025) · Journal of Imaging · doi:10.3390/jimaging11030091 · Cited by 70 · see citing works
Dual-Path Adaptation from Image to Video Transformers
Park, J., Lee, J., & Sohn, K. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00219 · Cited by 56 · see citing works
State-wise Safe Reinforcement Learning: A Survey
Zhao, W., He, T., Chen, R., Wei, T., & Liu, C. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/763 · Cited by 55 · see citing works
Streaming Dense Video Captioning
Zhou, X., Arnab, A., Buch, S., Shen, Y., Myers, A., Xiong, X., Nagrani, A., & Schmid, C. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01727 · Cited by 58 · see citing works
Self-citationAn Outlook into the Future of Egocentric Vision
Plizzari, C., Goletto, G., Furnari, A., Bansal, S., Ragusa, F., Farinella, G. M., Damen, D., & Tommasi, T. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-024-02095-7 · Cited by 52 · see citing works
k -NN attention-based video vision transformer for action recognition
Sun, W., Ma, Y., & Wang, R. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.127256 · Cited by 53 · see citing works
AdaMAE: Adaptive Masking for Efficient Spatiotemporal Learning with Masked Autoencoders
Bandara, W. G. C., Patel, N., Gholami, A., Nikkhah, M., Agrawal, M., & Patel, V. M. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01394 · Cited by 53 · see citing works
Video-FocalNets: Spatio-Temporal Focal Modulation for Video Action Recognition
Wasim, S. T., Khattak, M. U., Naseer, M., Khan, S. H., Shah, M. A., & Khan, F. S. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01267 · Cited by 46 · see citing works
Audiovisual Masked Autoencoders
Georgescu, M.-I., Fonseca, E., Ionescu, R. T., Lučić, M., Schmid, C., & Arnab, A. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01479 · Cited by 48 · see citing works
Self-citationMultimodal Transformer for Nursing Activity Recognition
Ijaz, M., Díaz, R., & Chen, C. (2022) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00224 · Cited by 41 · see citing works
Masked Motion Encoding for Self-Supervised Video Representation Learning
Sun, X., Chen, P., Chen, L., Li, C., Li, T. H., Tan, M., & Gan, C. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00222 · Cited by 41 · see citing works
Implicit Temporal Modeling with Learnable Alignment for Video Recognition
Tu, S., Dai, Q., Wu, Z., Cheng, Z.-Q., Hu, H., & Jiang, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01825 · Cited by 40 · see citing works
Disentangling Spatial and Temporal Learning for Efficient Image-to-Video Transfer Learning
Qing, Z., Zhang, S., Huang, Z., Zhang, Y., Gao, C., Zhao, D., & Sang, N. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01281 · Cited by 41 · see citing works
SCOTCH and SODA: A Transformer Video Shadow Detection Framework
Liu, L., Prost, J., Zhu, L., Papadakis, N., Lió, P., Schönlieb, C., & Avilés-Rivero, A. I. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01007 · Cited by 36 · see citing works
Re2TAL: Rewiring Pretrained Video Backbones for Reversible Temporal Action Localization
Zhao, C., Liu, S., Mangalam, K., & Ghanem, B. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01025 · Cited by 36 · see citing works
Sun, J., Dodge, H. H., & Mahoor, M. H. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.121929 · Cited by 33 · see citing works
Learning Correlation Structures for Vision Transformers
Kim, M., Seo, P. H., Schmid, C., & Cho, M. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01792 · Cited by 33 · see citing works
Self-citationSCENIC: A JAX Library for Computer Vision Research and Beyond
Dehghani, M., Gritsenko, A. A., Arnab, A., Minderer, M., & Tay, Y. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.02070 · Cited by 34 · see citing works
Self-citationTransferring Vision-Language Models for Visual Recognition: A Classifier Perspective
Wu, W., Sun, Z., Song, Y., Wang, J., & Ouyang, W. (2023) · International Journal of Computer Vision · doi:10.1007/s11263-023-01876-w · Cited by 33 · see citing works
Deep learning methods for single camera based clinical in-bed movement action recognition
Karácsony, T., Jeni, L. A., la Torre, F. D., & Cunha, J. P. S. (2024) · Image and Vision Computing · doi:10.1016/j.imavis.2024.104928 · Cited by 30 · see citing works
A Comprehensive Survey of Recent Transformers in Image, Video and Diffusion Models
Le, D. P. C., Wang, D., & Le, V.-T. (2024) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2024.050790 · Cited by 36 · see citing works
A Comprehensive Methodological Survey of Human Activity Recognition Across Diverse Data Modalities
Shin, J., Hassan, N., Miah, A. S. M., & Nishimura, S. (2025) · Sensors · doi:10.3390/s25134028 · Cited by 38 · see citing works
Dong, Q., Cai, H.-X., Li, Z., Liu, J., & Hu, B. (2024) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2024.3396457 · Cited by 28 · see citing works
Seeing in Flowing: Adapting CLIP for Action Recognition with Motion Prompts Learning
Wang, Q., Du, J., Yan, K., & Ding, S. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612490 · Cited by 30 · see citing works
Two-path target-aware contrastive regression for action quality assessment
Ke, X., Xu, H.-B., Lin, X., & Guo, W. (2024) · Information Sciences · doi:10.1016/j.ins.2024.120347 · Cited by 36 · see citing works
VicTR: Video-conditioned Text Representations for Activity Recognition
Kahatapitiya, K., Arnab, A., Nagran, A., & Ryoo, M. S. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01755 · Cited by 26 · see citing works
Self-citationTIM: A Time Interval Machine for Audio-Visual Action Recognition
Chalk, J., Huh, J., Kazakos, E., Zisserman, A., & Damen, D. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01719 · Cited by 25 · see citing works
Look for the Change: Learning Object States and State-Modifying Actions from Untrimmed Web Videos
Souček, T., Alayrac, J.-B., Miech, A., Laptev, I., & Šivic, J. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01357 · Cited by 25 · see citing works
A Semantic and Motion-Aware Spatiotemporal Transformer Network for Action Detection
Korban, M., Youngs, P. A., & Acton, S. T. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3377192 · Cited by 25 · see citing works
A novel YOLO LSTM approach for enhanced human action recognition in video sequences
Elnady, M., & Munim, H. E. A. E. (2025) · Scientific Reports · doi:10.1038/s41598-025-01898-z · Cited by 30 · see citing works
RETRACTED: Handcrafted localized phase features for human action recognition
Hejazi, S. M., & Abhayaratne, C. (2022) · Image and Vision Computing · doi:10.1016/j.imavis.2022.104465 · Cited by 24 · see citing works
RetractedEfficient Skeleton-Based Action Recognition via Joint-Mapping strategies
Kang, M., Kang, D., & Kim, H. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00340 · Cited by 24 · see citing works
Wei, G., Zhang, Z., Lan, C., Lü, Y., & Chen, Z. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i3.25376 · Cited by 27 · see citing works
Convolutional Transformer Network for Fine-Grained Action Recognition
Ma, Y., Wang, R., Zong, M., Ji, W., Wang, Y., & Ye, B. (2022) · Neurocomputing · doi:10.1016/j.neucom.2023.127027 · Cited by 28 · see citing works
Human-centric Scene Understanding for 3D Large-scale Scenarios
Xu, Y., Cong, P., Yao, Y., Chen, R., Hou, Y., Zhu, X., He, X., Jingyi Yu, A., & Ma, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01861 · Cited by 25 · see citing works
A Novel Two-Stream Transformer-Based Framework for Multi-Modality Human Action Recognition
Shi, J., Zhang, Y., Wang, W., Xing, B., Hu, D., & Chen, L. (2023) · Applied Sciences · doi:10.3390/app13042058 · Cited by 23 · see citing works
Visualizing and Understanding Contrastive Learning
Sammani, F., Joukovsky, B., & Deligiannis, N. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3346295 · Cited by 23 · see citing works
Frequency Guidance Matters: Skeletal Action Recognition by Frequency-Aware Mixed Transformer
Wu, W., Zheng, C., Yang, Z., Chen, C., Das, S., & Lu, A. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681009 · Cited by 21 · see citing works
A Comprehensive Survey on Multi-View Classification: Methods, Applications, and Challenges
Berahmand, K., Daneshfar, F., Rahmaninia, M., Haghighat, M., & Jalili, M. (2025) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3767728 · Cited by 27 · see citing works
Opportunities and challenges in the application of large artificial intelligence models in radiology
Pan, L., Zhao, Z., Lu, Y., Tang, K., Fu, L., Liang, Q., & Peng, S. (2024) · Meta-Radiology · doi:10.1016/j.metrad.2024.100080 · Cited by 22 · see citing works
Multiview Shapelet Prototypical Network for Few-Shot Fault Incremental Learning
Wan, X., Cen, L., Chen, X., Xie, Y., & Gui, W. (2024) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2024.3413304 · Cited by 20 · see citing works
Deep Convolutional Transformer Network for Stock Movement Prediction
Xie, L., Chen, Z., & Yu, S. (2024) · Electronics · doi:10.3390/electronics13214225 · Cited by 20 · see citing works
Ghimire, A., Kakani, V., & Kim, H. (2023) · IEEE Access · doi:10.1109/access.2023.3278974 · Cited by 23 · see citing works
How can objects help action recognition?
Zhou, X., Arnab, A., Sun, C., & Schmid, C. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00233 · Cited by 20 · see citing works
Self-citationZhao, Y., Luo, C., Tang, C., Chen, D., Codella, N., & Zha, Z.-J. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01403 · Cited by 20 · see citing works
Actor-agnostic Multi-label Action Recognition with Multi-modal Query
Mondal, A., Nag, S., Prada, J. M., Zhu, X., & Dutta, A. (2023) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw60793.2023.00086 · Cited by 21 · see citing works
Mandia, S., Mitharwal, R., & Singh, K. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-17534-9 · Cited by 19 · see citing works
SMART-vision: survey of modern action recognition techniques in vision
AlShami, A. K., Rabinowitz, R., Lâm, K. N., Shleibik, Y., Mersha, M. A., Boult, T. E., & Kalita, J. K. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-024-20484-5 · Cited by 20 · see citing works
What Can Simple Arithmetic Operations Do for Temporal Modeling?
Wu, W., Song, Y., Sun, Z., Wang, J., Xu, C., & Ouyang, W. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01261 · Cited by 19 · see citing works
Ego-Only: Egocentric Action Detection without Exocentric Transferring
Wang, H., Singh, M. K., & Torresani, L. (2023) · IEEE International Conference on Computer Vision · doi:10.1109/iccv51070.2023.00484 · Cited by 50 · see citing works
Exploring view-specific label relationships for multi-view multi-label feature selection
Hao, P., Ding, W., Gao, W., & He, J. (2024) · Information Sciences · doi:10.1016/j.ins.2024.121215 · Cited by 19 · see citing works
A Simple Recipe for Contrastively Pre-Training Video-First Encoders Beyond 16 Frames
Papalampidi, P., Koppula, S., Pathak, S., Chiu, J., Heyward, J., Pătrăucean, V., Shen, J., Miech, A., Zisserman, A., & Nematzdeh, A. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01364 · Cited by 21 · see citing works
Ma, D., Zhang, C., Xu, Q., & Zhou, G. (2025) · Robotics and Computer-Integrated Manufacturing · doi:10.1016/j.rcim.2025.103078 · Cited by 21 · see citing works
Multi-view Classification Using Hybrid Fusion and Mutual Distillation
Black, S., & Souvenir, R. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00034 · Cited by 18 · see citing works
EPK-CLIP: External and Priori Knowledge CLIP for action recognition
Yang, Z., An, G., Zheng, Z., Cao, S., & Wang, F. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.124183 · Cited by 18 · see citing works
ASPnet: Action Segmentation with Shared-Private Representation of Multiple Data Sources
van Amsterdam, B., Kadkhodamohammadi, A., Luengo, I., & Stoyanov, D. V. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00236 · Cited by 20 · see citing works
Schindler, F., Steinhage, V., Calkoen, S. T. S. V. B., & Heurich, M. (2024) · Applied Sciences · doi:10.3390/app14020514 · Cited by 16 · see citing works
Video Action Recognition with Attentive Semantic Units
Chen, Y., Chen, D., Liu, R., Li, H., & Peng, W. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00933 · Cited by 17 · see citing works
A Novel Crop Pest Detection Model Based on YOLOv5
Yang, W., & Qiu, X. (2024) · Agriculture · doi:10.3390/agriculture14020275 · Cited by 17 · see citing works
The Wisdom of Crowds: Temporal Progressive Attention for Early Action Prediction
Stergiou, A., & Damen, D. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01413 · Cited by 17 · see citing works
A Figure Skating Jumping Dataset for Replay-Guided Action Quality Assessment
Liu, Y., Cheng, X., & Ikenaga, T. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3613774 · Cited by 16 · see citing works
Yosry, S., Elrefaei, L. A., ElKamaar, R., & Ziedan, R. R. (2024) · Discover Applied Sciences · doi:10.1007/s42452-024-05774-9 · Cited by 16 · see citing works
Yang, Z., An, G., Zheng, Z., Cao, S., & Ruan, Q. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3390133 · Cited by 16 · see citing works
Chen, Y., Chen, D., Liu, R., Zhou, S., Xue, W., & Peng, W. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01768 · Cited by 16 · see citing works
STAR++: Rethinking spatio-temporal cross attention transformer for video action recognition
Ahn, D., Kim, S., & Ko, B. C. (2023) · Applied Intelligence · doi:10.1007/s10489-023-04978-7 · Cited by 16 · see citing works
Pose pattern mining using transformer for motion classification
Lee, S.-E., Yoo, H., & Chung, K. (2024) · Applied Intelligence · doi:10.1007/s10489-024-05325-0 · Cited by 15 · see citing works
Multi-scale Bottleneck Transformer for Weakly Supervised Multimodal Violence Detection
Sun, S., & Gong, X. (2024) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme57554.2024.10688202 · Cited by 15 · see citing works
A Descriptive Basketball Highlight Dataset for Automatic Commentary Generation
Zhang, B., Gao, J., & Yuan, Y. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681178 · Cited by 16 · see citing works
About Time: Advances, Challenges, and Outlooks of Action Understanding
Stergiou, A., & Poppe, R. W. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02478-4 · Cited by 15 · see citing works
Campus Abnormal Behavior Recognition With Temporal Segment Transformers
Liu, H. L., Chuah, J. H., Khairuddin, A. S. M., Zhao, X. M., & Wang, X. D. (2023) · IEEE Access · doi:10.1109/access.2023.3266440 · Cited by 14 · see citing works
Audio-Visual Contrastive and Consistency Learning for Semi-Supervised Action Recognition
Assefa, M., Jiang, W., Zhan, J., Gedamu, K., Yilma, G., Ayalew, M., & Adhikari, D. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3312856 · Cited by 14 · see citing works
- Authors
- Yan, S., Xiong, X., Arnab, A., Lu, Z., Zhang, M., Sun, C., & Schmid, C.
- Affiliations
- Michigan State University
- Google (United States) (US)
- Michigan State University (US)
- Brown University (US)
- Venue
- 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2022-06
- Pages
- 3323-3333
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W4312658081
- Cited by
- 312
- References
- 92
- Citations by year
- 2026: 26 · 2025: 63 · 2024: 115 · 2023: 91 · 2022: 17
- FWCI
- 25.93
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- video recognition, spatiotemporal resolution, view fusion, lateral connections, video encoder, pre-training, computational efficiency
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
