The DOI matches its DataCite record; 54 of the 58 references it cites were found in a registry or an index.
Is Space-Time Attention All You Need for Video Understanding?
Open accessAbstract
We present a convolution-free approach to video classification built exclusively on self-attention over space and time. Our method, named "TimeSformer," adapts the standard Transformer architecture to video by enabling spatiotemporal feature learning directly from a sequence of frame-level patches. Our experimental study compares different self-attention schemes and suggests that "divided attention," where temporal attention and spatial attention are separately applied within each block, leads to the best video classification accuracy among the design choices considered. Despite the radically new design, TimeSformer achieves state-of-the-art results on several action recognition benchmarks, including the best reported accuracy on Kinetics-400 and Kinetics-600. Finally, compared to 3D convolutional networks, our model is faster to train, it can achieve dramatically higher test efficiency (at a small drop in accuracy), and it can also be applied to much longer video clips (over one minute long). Code and models are available at: https://github.com/facebookresearch/TimeSformer.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Vision Transformer (ViT) [17], and video-specific variants such as TimeSformer [ 18, 19 ] and Video Swin Transformer [20] capture longrange dependencies and global spatiotemporal context that CNNs often miss.”
From Cross-dataset video deepfake detection using Transformer and CNN architectures (2026) · cites this paper as [18]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Similar to previous trials [3, 6], training video transformers from scratch yields unsatisfied results on video datasets.”
From VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training (2022) · cites this paper as [6]
The cited source says what the claim attributes to it.
In the citing paper
“For this reason, due to real-time computation concerns in visual odometry applications, we selected the encoder architecture with the 'divided space-time' self-attention, which is more efficient in terms of processing complexity compared to the 'joint space-time' self-attention.”
From Transformer-Based Model for Monocular Visual Odometry: A Video Understanding Approach (2023) · cites this paper as [14]
The cited source says what the claim attributes to it.
In the citing paper
“We chose the TimeSformer [14] model for this work since it achieved state-of-the-art results in video action recognition tasks compared to previous works based on convolutional neural networks (CNN).”
From Transformer-Based Model for Monocular Visual Odometry: A Video Understanding Approach (2023) · cites this paper as [14]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472
(2008) · Journal of Machine Learning Research · Cited by 33,051
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
End-to-End Object Detection with Transformers
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58452-8_13 · Cited by 18,285
Research works printed but not linked: 5
- Ba, L. J., Kiros, J. R., and Hinton, G. E. Layer normalization. CoRR, 2016.
- Chen, M. X., Firat, O., Bapna, A., Johnson, M., Macherey, W., Foster, G., Jones, L., Schuster, M., Shazeer, N., Par- mar, N., Vaswani, A., Uszkoreit, J., Kaiser, L., Chen, Z., Wu, Y., and Hughes, M. The best of both worlds: Combining recent advances in neural machine translation. In Proceedings of the 56th Annual Meeting of the Asso- ciation for Computational Linguistics. Association for Computational Linguistics, 2018a.
- Chen, Y., Kalantidis, Y., Li, J., Yan, S., and Feng, J. Aˆ2- nets: Double attention networks. In Advances in Neural Information Processing Systems 31, 2018b.
- International Conference on Computer Vision (ICCV), October 2019.
- Cordonnier, J., Loukas, A., and Jaggi, M. On the relation- ship between self-attention and convolutional layers. In 8th International Conference on Learning Representa- tions, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020, 2020.
Cited by
Sorted by citations
Items 101–200 of 1,004
Deep Frequency Filtering for Domain Generalization
Lin, S., Zhang, Z., Huang, Z., Lü, Y., Lan, C., Chu, P., You, Q., Wang, J., Liu, Z., Parulkar, A., et al. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01135 · Cited by 105
Yang, M., Chengke, W., Guo, Y., Jiang, R., Zhou, F., Zhang, J., & Yang, Z. (2022) · Automation in Construction · doi:10.1016/j.autcon.2022.104703 · Cited by 103
VIOLET : End-to-End Video-Language Transformers with Masked Visual-token ModelingPreprint
Fu, T.-J., Li, L., Gan, Z., Lin, K., Wang, W. Y., Wang, L., & Liu, Z. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.12681 · Cited by 212
TubeDETR: Spatio-Temporal Video Grounding with Transformers
Yang, A., Miech, A., Šivic, J., Laptev, I., & Schmid, C. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01595 · Cited by 114
Wang, S., Mei, L., Liu, R., Jiang, W., Yin, Z., Deng, X., & He, T. (2024) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2024.3398004 · Cited by 112
Normalizing Flows for Human Pose Anomaly Detection
Hirschorn, O., & Avidan, S. (2022) · IEEE International Conference on Computer Vision · doi:10.1109/iccv51070.2023.01246 · Cited by 123
Broaden Your Views for Self-Supervised Video Learning
Recasens, A., Luc, P., Alayrac, J.-B., Wang, L., Strub, F., Tallec, C., Malinowski, M., Pătrăucean, V., Altché, F., Valko, M., et al. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00129 · Cited by 98
Dual-AI: Dual-path Actor Interaction Learning for Group Activity Recognition
Han, M., Zhang, D. J., Wang, Y., Yan, R., Yao, L., Chang, X., & Qiao, Y. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00300 · Cited by 91
SimVPv2: Towards Simple Yet Powerful Spatiotemporal Predictive Learning
Tan, C., Gao, Z., Li, S., & Li, S. Z. (2025) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2025.3543051 · Cited by 143
Combining transformers with CNN for multi-focus image fusion
Duan, Z., Luo, X., & Zhang, T. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.121156 · Cited by 90
Align and Tell: Boosting Text-Video Retrieval With Local Alignment and Fine-Grained Supervision
Wang, X., Zhu, L., Zheng, Z., Xu, M., & Yang, Y. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3204444 · Cited by 86
SSformer: A Lightweight Transformer for Semantic Segmentation
Shi, W., Xu, J. L., & Gao, P. (2022) · IEEE International Workshop on Multimedia Signal Processing (MMSP) · doi:10.1109/mmsp55362.2022.9949177 · Cited by 84
Wang, R., Chen, D., Wu, Z., Chen, Y., Dai, X., Liu, M., Yuan, L., & Jiang, Y. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00611 · Cited by 91
Multimodal Motion Conditioned Diffusion Model for Skeleton-based Video Anomaly Detection
Flaborea, A., Collorone, L., di Melendugno, G. M. D., D’Arrigo, S., Prenkaj, B., & Galasso, F. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00947 · Cited by 97
PublicVision: A Secure Smart Surveillance System for Crowd Behavior Recognition
Qaraqe, M., Elzein, A., Başaran, E., Yang, Y., Varghese, E. B., Costandi, W., Rizk, J., & Alam, N. (2024) · IEEE Access · doi:10.1109/access.2024.3366693 · Cited by 96
Zhu, X., Zhu, J., Li, H., Wu, X., Li, H., Wang, X., & Dai, J. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01630 · Cited by 108
Joint Hand Motion and Interaction Hotspots Prediction from Egocentric Videos
Liu, S., Tripathi, S., Majumdar, S., & Wang, X. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00328 · Cited by 82
EgoVLPv2: Egocentric Video-Language Pre-training with Fusion in the Backbone
Pramanick, S., Song, Y., Nag, S., Lin, K. Q., Shah, H., Shou, M. Z., Chellappa, R., & Zhang, P. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00487 · Cited by 106
Han, Y., Yu, K., Batra, R., Boyd, N., Mehta, C., Zhao, T., She, Y., Hutchinson, S., & Zhao, Y. (2024) · IEEE/ASME Transactions on Mechatronics · doi:10.1109/tmech.2024.3400789 · Cited by 90
DirecFormer: A Directed Attention in Transformer Approach to Robust Action Recognition
Truong, T.-D., Bui, Q.-H., Duong, C. N., Seo, H., Phung, S. L., Li, X., & Luu, K. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01940 · Cited by 83
Temporally Efficient Vision Transformer for Video Instance Segmentation
Yang, S., Wang, X., Li, Y., Fang, Y., Fang, J., Liu, W., Zhao, X., & Shan, Y. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00290 · Cited by 83
UniFormerV2: Unlocking the Potential of Image ViTs for Video Understanding
Li, K., Wang, Y., He, Y., Li, Y., Wang, Y., Wang, L., & Qiao, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00157 · Cited by 91
Space-time Mixing Attention for Video TransformerPreprint
Bulat, A., Pérez-Rúa, J.-M., Sudhakaran, S., Martínez, B., & Tzimiropoulos, G. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.05968 · Cited by 85
X2-VLM: All-in-One Pre-Trained Model for Vision-Language Tasks
Zeng, Y., Zhang, X., Li, H., Wang, J., Zhang, J., & Zhou, W. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3339661 · Cited by 116
VideoLLM-online: Online Video Large Language Model for Streaming Video
Chen, J., Lv, Z., Wu, S., Lin, K. Q., Song, C., Gao, D., Liu, J.-W., Gao, Z., Mao, D., & Shou, M. Z. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01742 · Cited by 124
Object-Region Video Transformers
Herzig, R., Avraham, E. B., Mangalam, K., Bar, A., Chechik, G., Rohrbach, A., Darrell, T., & Globerson, A. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00315 · Cited by 84
Stand-Alone Inter-Frame Attention in Video Models
Long, F., Qiu, Z., Pan, Y., Yao, T., Luo, J., & Mei, T. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00319 · Cited by 78
Spatial-Temporal Transformer for Video Snapshot Compressive Imaging
Wang, L., Cao, M., Zhong, Y., & Yuan, X. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3225382 · Cited by 83
OmniMAE: Single Model Masked Pretraining on Images and Videos
Girdhar, R., El-Nouby, A., Singh, M., Alwala, K. V., Joulin, A., & Misra, I. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01003 · Cited by 101
Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrieval
Bain, M., Nagrani, A., Varol, G., & Zisserman, A. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00175 · Cited by 76
Transformer-Based Multi-Scale Feature Integration Network for Video Saliency Prediction
Zhou, X., Wu, S., Shi, R., Zheng, B., Wang, S., Yin, H., Zhang, J., & Yan, C. C. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3278410 · Cited by 76
Revisiting Temporal Modeling for CLIP-Based Image-to-Video Knowledge Transferring
Liu, R., Huang, J., Li, G., Feng, J., Wu, X., & Li, T. H. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00634 · Cited by 78
LoViT: Long Video Transformer for surgical phase recognition
Liu, Y., Boels, M., García-Peraza-Herrera, L. C., Vercauteren, T. K., Prokar Dasgupta, A., Granados, A., & Sebastien Ourselin, A. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103366 · Cited by 99
Revealing Single Frame Bias for Video-and-Language Learning
Lei, J., Berg, T. L., & Bansal, M. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.29 · Cited by 146
Deep Learning Methods for Calibrated Photometric Stereo and Beyond
Ju, Y., Lam, K., Xie, W., Zhou, H., Dong, J., & Shi, B. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3388150 · Cited by 78
VoP: Text-Video Co-Operative Prompt Tuning for Cross-Modal Retrieval
Huang, S., Gong, B., Pan, Y., Jiang, J., Lv, Y., Li, Y., & Wang, D. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00635 · Cited by 78
Rethinking Video ViTs: Sparse Video Tubes for Joint Image and Video Learning
Piergiovanni, A., Kuo, W., & Angelova, A. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00220 · Cited by 72
Self-Distilled Masked Auto-Encoders are Efficient Video Anomaly Detectors
Ristea, N.-C., Croitoru, F.-A., Ionescu, R. T., Popescu, M., Khan, F. S., & Shah, M. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01513 · Cited by 71
Efficient Movie Scene Detection using State-Space Transformers
Islam, M. M., Hasan, M., Athrey, K. S., Braskich, T., & Bertasius, G. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01798 · Cited by 76
Self-citationLAVENDER: Unifying Video-Language Understanding as Masked Language Modeling
Li, L., Gan, Z., Lin, K., Lin, C.-C., Liu, Z., Liu, C., & Wang, L. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02214 · Cited by 90
Exploring Rich Semantics for Open-Set Action Recognition
Hu, Y., Gao, J., Dong, J., Fan, B., & Liu, H. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3333206 · Cited by 71
Class Semantics-based Attention for Action Detection
Sridhar, D., Quader, N., Muralidharan, S., Li, Y., Dai, P., & Lu, J. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01348 · Cited by 69
Object-aware Video-language Pre-training for Retrieval
Wang, A. J., Ge, Y., Cai, G., Yan, R., Lin, X., Shan, Y., Qie, X., & Shou, M. Z. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00331 · Cited by 69
Detector-Free Weakly Supervised Group Activity Recognition
Kim, D.-K., Lee, J., Cho, M., & Kwak, S. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01945 · Cited by 69
HiTeA: Hierarchical Temporal-Aware Video-Language Pre-training
Ye, Q., Xu, G., Yan, M., Xu, H., Qian, Q., Zhang, J., & Huang, F. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01413 · Cited by 87
Yang, D.-K., Huang, S., Xu, Z., Li, Z., Wang, S.-L., Li, M.-C., Wang, Y.-Z., Liu, Y., Yang, K., Chen, Z., et al. (2023) · IEEE International Conference on Computer Vision · doi:10.1109/iccv51070.2023.01871 · Cited by 91
FFT-Based Dynamic Token Mixer for Vision
Tatsunami, Y., & Taki, M. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i14.29457 · Cited by 73
Vision-Language Models Can Identify Distracted Driver Behavior From Naturalistic Videos
Hasan, M. Z., Chen, J., Wang, J., Rahman, M. S., Joshi, A., Velipasalar, S., Hegde, C., Sharma, A., & Sarkar, S. (2024) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2024.3381175 · Cited by 71
Patro, B. N., & Agneeswaran, V. S. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.111279 · Cited by 81
Isolated Sign Language Recognition with Multi-Scale Spatial-Temporal Graph Convolutional Networks
Vázquez-Enríquez, M., Alba‐Castro, J. L., Docío-Fernández, L., & Banga, E. R. (2021) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00385 · Cited by 73
Future Transformer for Long-term Action Anticipation
Gong, D., Lee, J., Kim, M., Ha, S. J., & Cho, M. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00306 · Cited by 67
GateHUB: Gated History Unit with Background Suppression for Online Action Detection
Chen, J., Mittal, G. K., Yu, Y., Kong, Y., & Chen, M. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01930 · Cited by 72
Efficient Semantic Segmentation via Self-Attention and Self-Distillation
An, S., Liao, Q., Lu, Z., & Xue, J. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2021.3139001 · Cited by 66
Enhancing Video-Language Representations With Structural Spatio-Temporal Alignment
Fei, H., Wu, S., Zhang, M., Zhang, M., Chua, T., & Yan, S. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3393452 · Cited by 71
DTCM: Joint Optimization of Dark Enhancement and Action Recognition in Videos
Tu, Z., Liu, Y., Zhang, Y., Mu, Q., & Yuan, J. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3286254 · Cited by 64
Transformer Meets Remote Sensing Video Detection and Tracking: A Comprehensive Survey
Jiao, L., Zhang, X., Liu, X., Liu, F., Yang, S., Ma, W., Li, L., Chen, P., Feng, Z., Guo, Y., et al. (2023) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2023.3289293 · Cited by 68
VindLU: A Recipe for Effective Video-and-Language Pretraining
Cheng, F., Wang, X., Lei, J., Crandall, D., Bansal, M., & Bertasius, G. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01034 · Cited by 76
Self-citationHoloAssist: an Egocentric Human Interaction Dataset for Interactive AI Assistants in the Real World
Wang, X., Kwon, T., Rad, M., Pan, B., Chakraborty, I., Andrist, S., Bohus, D., Feniello, A., Tekin, B., Frujeri, F. V., et al. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01854 · Cited by 64
Transformer in Computer Vision
Bi, J., Zhu, Z., & Meng, Q. (2021) · International Conference on Computer Science, Electronic Information Engineering and Intelligent Control Technology (CEI) · doi:10.1109/cei52496.2021.9574462 · Cited by 69
AGPN: Action Granularity Pyramid Network for Video Action Recognition
Chen, Y., Ge, H., Liu, Y., Cai, X., & Sun, L. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3235522 · Cited by 70
A Cooperative Vehicle-Infrastructure System for Road Hazards Detection With Edge Intelligence
Chen, C., Yao, G., Liu, L., Pei, Q., Song, H., & Dustdar, S. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3241251 · Cited by 68
Memory-and-Anticipation Transformer for Online Action Understanding
Wang, J., Chen, G., Huang, Y., Wang, L., & Lu, T. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01271 · Cited by 62
EAN: Event Adaptive Network for Enhanced Action Recognition
Tian, Y., Yan, Y., Zhai, G., Guo, G., & Gao, Z. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-022-01661-1 · Cited by 64
Learning To Recognize Procedural Activities with Distant Supervision
Lin, X., Petroni, F., Bertasius, G., Rohrbach, M., Chang, S., & Torresani, L. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01348 · Cited by 70
Self-citationLAVIS: A One-stop Library for Language-Vision Intelligence
Li, D., Li, J., Le, H., Wang, G., Savarese, S., & Hoi, S. C. H. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-demo.3 · Cited by 68
TokenLearner: What Can 8 Learned Tokens Do for Images and Videos?Preprint
Ryoo, M. S., Piergiovanni, A., Arnab, A., Dehghani, M., & Angelova, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.11297 · Cited by 152
CAPformer: Pedestrian Crossing Action Prediction Using Transformer
Lorenzo, J., Parra, I., Izquierdo, R., Ballardini, A. L., Hernández, Á., Llorca, D. F., & Sotelo, M. Á. (2021) · Sensors · doi:10.3390/s21175694 · Cited by 60
Deep learning based MRI reconstruction with transformer
Wu, Z., Liao, W., Yan, C., Zhao, M., Liu, G., Ma, N., & Li, X. (2023) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2023.107452 · Cited by 60
Video Saliency Forecasting Transformer
Ma, C., Sun, H., Rao, Y., Zhou, J., & Lu, J. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3172971 · Cited by 61
Styleformer: Transformer based Generative Adversarial Networks with Style Vector
Park, J., & Kim, Y.-G. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00878 · Cited by 60
Lu, Y., Lu, X., Zheng, L., Sun, M., Chen, S., Chen, B., Wang, T., Yang, J., & Lv, C. (2024) · Plants · doi:10.3390/plants13070972 · Cited by 66
Reversible Vision Transformers
Mangalam, K., Fan, H., Li, Y., Wu, C.-Y., Xiong, B., Feichtenhofer, C., & Malik, J. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01056 · Cited by 62
Contrastive Video Question Answering via Video Graph Transformer
Xiao, J., Zhou, P., Yao, A., Li, Y., Hong, R., Yan, S., & Chua, T. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3292266 · Cited by 73
gSDF: Geometry-Driven Signed Distance Functions for 3D Hand-Object Reconstruction
Chen, Z., Chen, S., Schmid, C., & Ivan Laptev, A. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01239 · Cited by 57
Scaling Spike-Driven Transformer With Efficient Spike Firing Approximation Training
Yao, M., Qiu, X., Hu, T., Hu, J., Chou, Y., Tian, K., Liao, J., Leng, L., Xu, B., & Li, G. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3530246 · Cited by 81
Machine Learning for Human Activity Recognition: State-of-the-Art Techniques and Emerging Trends
Hossen, M. A., & Abas, P. E. (2025) · Journal of Imaging · doi:10.3390/jimaging11030091 · Cited by 70
ASFormer: Transformer for Action SegmentationPreprint
Yi, F., Wen, H., & Jiang, T. (2021) · British Machine Vision Conference · doi:10.48550/arxiv.2110.08568 · Cited by 319
MLTR: Multi-Label Classification with Transformer
Cheng, X., Lin, H., Wu, X., Shen, D., Yang, F., Liu, H., & Shi, N. (2022) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme52920.2022.9860016 · Cited by 66
A Comprehensive Review of Recent Deep Learning Techniques for Human Activity Recognition
Le, V.-T., Tran-Trung, K., & Hoang, V. T. (2022) · Computational Intelligence and Neuroscience · doi:10.1155/2022/8323962 · Cited by 58
MAR: M asked Autoencoders for Efficient A ction R ecognition
Qing, Z., Zhang, S., Huang, Z., Wang, X., Wang, Y., Lv, Y., Gao, C., & Sang, N. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3263288 · Cited by 76
Dual-Path Adaptation from Image to Video Transformers
Park, J., Lee, J., & Sohn, K. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00219 · Cited by 64
Exploiting Complementary Dynamic Incoherence for DeepFake Video Detection
Wang, H., Liu, Z., & Wang, S. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3238517 · Cited by 58
Self-citationZheng, W., Yan, L., & Wang, F. (2023) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2023.3282704 · Cited by 58
Shin, H., Kim, H., Kim, S., Jun, Y., Eo, T., & Hwang, D. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00716 · Cited by 55
HARDVS: Revisiting Human Activity Recognition with Dynamic Vision Sensors
Wang, X., Wu, Z., Jiang, B., Bao, Z., Zhu, L. Z., Li, G., Wang, Y., & Tian, Y. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i6.28372 · Cited by 69
Video Pose Distillation for Few-Shot, Fine-Grained Sports Action Recognition
Hong, J. W., Fisher, M., Gharbi, M., & Fatahalian, K. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00912 · Cited by 54
ECSNet: Spatio-Temporal Feature Learning for Event Camera
Chen, Z., Wu, J., Hou, J., Li, L., Dong, W., & Shi, G. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3202659 · Cited by 58
Group Contextualization for Video Recognition
Hao, Y., Zhang, H., Ngo, C., & He, X. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00100 · Cited by 54
Towards Scalable Neural Representation for Diverse Videos
He, B., Yang, X., Wang, H., Wu, Z., Chen, H., Huang, S., Ren, Y., Lim, S.-N., & Shrivastava, A. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00594 · Cited by 54
Self-citationStreaming Dense Video Captioning
Zhou, X., Arnab, A., Buch, S., Shen, Y., Myers, A., Xiong, X., Nagrani, A., & Schmid, C. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01727 · Cited by 58
Is attention to bounding boxes all you need for pedestrian action prediction?
Achaji, L., Moreau, J., Fouqueray, T., Aioun, F., & Charpillet, F. (2022) · IEEE Intelligent Vehicles Symposium · doi:10.1109/iv51971.2022.9827084 · Cited by 58
Likert Scoring with Grade Decoupling for Long-term Action Assessment
Xu, A., Zeng, L.-A., & Zheng, W. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00323 · Cited by 58
A general skeleton-based action and gesture recognition framework for human–robot collaboration
Terreran, M., Barcellona, L., & Ghidoni, S. (2023) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2023.104523 · Cited by 61
How Much Temporal Long-Term Context is Needed for Action Segmentation?
Bahrami, E., Francesca, G., & Gall, J. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00950 · Cited by 57
Automatic Early Diagnosis of Alzheimer’s Disease Using 3D Deep Ensemble Approach
Gamal, A., Elattar, M., & Selim, S. (2022) · IEEE Access · doi:10.1109/access.2022.3218621 · Cited by 56
ECLIPSE: Envisioning CLoud Induced Perturbations in Solar Energy
Paletta, Q., Hu, A., Arbod, G., & Lasenby, J. (2022) · Applied Energy · doi:10.1016/j.apenergy.2022.119924 · Cited by 51
k -NN attention-based video vision transformer for action recognition
Sun, W., Ma, Y., & Wang, R. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.127256 · Cited by 54
Alfasly, S., Chui, C. K., Jiang, Q., Lu, J. J., & Chen, X. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3190367 · Cited by 51
Zhao, L., & Ji, S. (2022) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2022.3219816 · Cited by 60
Liu, T., Zhang, C., Lam, K., & Kong, J. (2022) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2022.3216479 · Cited by 50
Metadata
- Authors
- Bertasius, G., Wang, H., & Torresani, L.
- Affiliations
- University of Pennsylvania (US)
- Meta (United States) (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2021
- Pages
- 813-824
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2102.05095
- OpenAlex
- W3126721948
- Cited by
- 1,663
- References
- 58
- Citations by year
- 2026: 161 · 2025: 390 · 2024: 418 · 2023: 387 · 2022: 199 · 2021: 108
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- International Conference on Machine Learning
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- video classification, self-attention, spatio-temporal attention, TimeSformer, transformer models, divided attention, human action recognition, 3D convolutional networks
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
