The DOI matches its DataCite record; 54 of the 58 references it cites were found in a registry or an index.
Is Space-Time Attention All You Need for Video Understanding?
Open accessAbstract
We present a convolution-free approach to video classification built exclusively on self-attention over space and time. Our method, named "TimeSformer," adapts the standard Transformer architecture to video by enabling spatiotemporal feature learning directly from a sequence of frame-level patches. Our experimental study compares different self-attention schemes and suggests that "divided attention," where temporal attention and spatial attention are separately applied within each block, leads to the best video classification accuracy among the design choices considered. Despite the radically new design, TimeSformer achieves state-of-the-art results on several action recognition benchmarks, including the best reported accuracy on Kinetics-400 and Kinetics-600. Finally, compared to 3D convolutional networks, our model is faster to train, it can achieve dramatically higher test efficiency (at a small drop in accuracy), and it can also be applied to much longer video clips (over one minute long). Code and models are available at: https://github.com/facebookresearch/TimeSformer.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Vision Transformer (ViT) [17], and video-specific variants such as TimeSformer [ 18, 19 ] and Video Swin Transformer [20] capture longrange dependencies and global spatiotemporal context that CNNs often miss.”
From Cross-dataset video deepfake detection using Transformer and CNN architectures (2026) · cites this paper as [18]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Similar to previous trials [3, 6], training video transformers from scratch yields unsatisfied results on video datasets.”
From VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training (2022) · cites this paper as [6]
The cited source says what the claim attributes to it.
In the citing paper
“For this reason, due to real-time computation concerns in visual odometry applications, we selected the encoder architecture with the 'divided space-time' self-attention, which is more efficient in terms of processing complexity compared to the 'joint space-time' self-attention.”
From Transformer-Based Model for Monocular Visual Odometry: A Video Understanding Approach (2023) · cites this paper as [14]
The cited source says what the claim attributes to it.
In the citing paper
“We chose the TimeSformer [14] model for this work since it achieved state-of-the-art results in video action recognition tasks compared to previous works based on convolutional neural networks (CNN).”
From Transformer-Based Model for Monocular Visual Odometry: A Video Understanding Approach (2023) · cites this paper as [14]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472
(2008) · Journal of Machine Learning Research · Cited by 33,051
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
End-to-End Object Detection with Transformers
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58452-8_13 · Cited by 18,285
Research works printed but not linked: 5
- Ba, L. J., Kiros, J. R., and Hinton, G. E. Layer normalization. CoRR, 2016.
- Chen, M. X., Firat, O., Bapna, A., Johnson, M., Macherey, W., Foster, G., Jones, L., Schuster, M., Shazeer, N., Par- mar, N., Vaswani, A., Uszkoreit, J., Kaiser, L., Chen, Z., Wu, Y., and Hughes, M. The best of both worlds: Combining recent advances in neural machine translation. In Proceedings of the 56th Annual Meeting of the Asso- ciation for Computational Linguistics. Association for Computational Linguistics, 2018a.
- Chen, Y., Kalantidis, Y., Li, J., Yan, S., and Feng, J. Aˆ2- nets: Double attention networks. In Advances in Neural Information Processing Systems 31, 2018b.
- International Conference on Computer Vision (ICCV), October 2019.
- Cordonnier, J., Loukas, A., and Jaggi, M. On the relation- ship between self-attention and convolutional layers. In 8th International Conference on Learning Representa- tions, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020, 2020.
Cited by
Sorted by citations
Items 201–300 of 1,004
Towards All-in-One Pre-Training via Maximizing Multi-Modal Mutual Information
Su, W., Zhu, X., Tao, C., Lu, L., Li, B., Huang, G. M., Qiao, Y., Wang, X., Zhou, J., & Dai, J. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01525 · Cited by 58
On the Benefits of 3D Pose and Tracking for Human Action Recognition
Rajasegaran, J., Pavlakos, G., Kanazawa, A., Feichtenhofer, C., & Malik, J. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00069 · Cited by 57
SimDA: Simple Diffusion Adapter for Efficient Video Generation
Xing, Z., Dai, Q., Hu, H., Wu, Z., & Jiang, Y. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00748 · Cited by 92
Panacea: Panoramic and Controllable Video Generation for Autonomous Driving
Wen, Y., Zhao, Y., Liu, Y., Jia, F., Wang, Y., Luo, C., Zhang, C., Wang, T., Sun, X., & Zhang, X. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00659 · Cited by 100
Khan, S. A., & Dang-Nguyen, D.-T. (2024) · IEEE Access · doi:10.1109/access.2023.3348450 · Cited by 53
Chen, C., Gao, R., Calamia, P. T., & Grauman, K. L. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01829 · Cited by 48
Real-Time Weakly Supervised Video Anomaly Detection
Karim, H., Doshi, K., & Yılmaz, Y. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00670 · Cited by 48
Learning Sequential Descriptors for Sequence-Based Visual Place Recognition
Mereu, R., Trivigno, G., Berton, G., Masone, C., & Barbara Caputo, B. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3194310 · Cited by 47
Prompt Learns Prompt: Exploring Knowledge-Aware Generative Prompt Collaboration For Video Captioning
Yan, L., Han, C., Xu, Z., Liu, D., & Wang, Q. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/180 · Cited by 50
DynamicStereo: Consistent Dynamic Depth from Stereo Videos
Karaev, N., Rocco, I., Graham, B., Neverova, N., Vedaldi, A., & Rupprecht, C. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01271 · Cited by 47
Chen, X., Zhang, H., Deng, F., Liang, J. S., & Yang, J. J. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3342040 · Cited by 47
Boosting Few-shot Action Recognition with Graph-guided Hybrid Matching
Xing, J., Wang, M., Ruan, Y., Chen, B., Guo, Y., Mu, B., Dai, G., Wang, J., & Liu, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00167 · Cited by 47
BasicTAD: An astounding RGB-Only baseline for temporal action detection
Yang, M., Chen, G., Zheng, Y.-D., Lü, T., & Wang, L. (2023) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2023.103692 · Cited by 49
Nwoye, C. I., Yu, T., Sharma, S., Murali, A., Alapatt, D., Vardazaryan, A., Yuan, K., Hajek, J., Reiter, W., Yamlahi, A., et al. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.102888 · Cited by 45
AdaMAE: Adaptive Masking for Efficient Spatiotemporal Learning with Masked Autoencoders
Bandara, W. G. C., Patel, N., Gholami, A., Nikkhah, M., Agrawal, M., & Patel, V. M. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01394 · Cited by 53
Video-FocalNets: Spatio-Temporal Focal Modulation for Video Action Recognition
Wasim, S. T., Khattak, M. U., Naseer, M., Khan, S. H., Shah, M. A., & Khan, F. S. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01267 · Cited by 47
Multiscale Vision TransformersPreprint
Fan, H., Xiong, B., Mangalam, K., Li, Y., Yan, Z., Malik, J., & Feichtenhofer, C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.11227 · Cited by 46
A Coding Framework and Benchmark Towards Low-Bitrate Video Understanding
Tian, Y., Lu, G. S., Yan, Y., Zhai, G., Chen, L., & Gao, Z. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3367879 · Cited by 45
Conv-MLP: A Convolution and MLP Mixed Model for Multimodal Face Anti-Spoofing
Wang, W., Wen, F., Zheng, H., Ying, R., & Liu, P. (2022) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2022.3183398 · Cited by 47
Spatial-Temporal Graphs Plus Transformers for Geometry-Guided Facial Expression Recognition
Zhao, R., Liu, T., Huang, Z., Lun, D. P.-K., & Lam, K. (2022) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2022.3181736 · Cited by 45
Wang, J., & Torresani, L. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01366 · Cited by 50
Self-citationMIMO Is All You Need：A Strong Multi-in-Multi-Out Baseline for Video Prediction
Ning, S., Lan, M., Li, Y., Chen, C., Chen, Q., Chen, X., Han, X., & Cui, S. R. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i2.25289 · Cited by 44
Driver Distraction Behavior Recognition for Autonomous Driving: Approaches, Datasets and Challenges
Tan, D., Tian, W., Wang, C., Chen, L., & Xiong, L. (2024) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2024.3405990 · Cited by 48
Rethinking the Self-Attention in Vision Transformers
Kim, K., Wu, B., Dai, X., Zhang, P., Yan, Z., Vajda, P., & Kim, S. J. (2021) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00342 · Cited by 45
Look Outside the Room: Synthesizing A Consistent Long-Term 3D Scene Video from A Single Image
Ren, X., & Wang, X. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00355 · Cited by 48
Ahmadi, N., Tsang, M. Y. C., Gu, A. N., Tsang, T. S. M., & Abolmaesumi, P. (2023) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2023.3305384 · Cited by 43
Making Vision Transformers Efficient from A Token Sparsification View
Chang, S., Wang, P., Lin, M., Wang, F., Zhang, D. J., Jin, R., & Shou, M. Z. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00600 · Cited by 48
Yao, F., Lu, W., Yang, H., Xu, L., Liu, C., Hu, L., Yu, H., Liu, N., Deng, C., Tang, D., et al. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3316166 · Cited by 51
SkeleTR: Towards Skeleton-based Action Recognition in the Wild
Duan, H., Xu, M., Shuai, B., Modolo, D., Tu, Z., Tighe, J., & Bergamo, A. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01254 · Cited by 48
MGMAE: Motion Guided Masking for Video Masked Autoencoding
Huang, B., Zhao, Z., Zhang, G., Qiao, Y., & Wang, L. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01241 · Cited by 52
Wang, H., Liu, F., Jiao, L., Wang, J., Hao, Z., Li, S., Li, L., Chen, P., & Liu, X. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i6.28347 · Cited by 53
Self-citationTowards Generalizable Tumor Synthesis
Chen, Q., Chen, X., Song, H., Xiong, Z., Yuille, A., Wei, C., & Zhou, Z. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01060 · Cited by 54
Huang, Y., Guo, C., Xu, J., Zhang, M., Yang, L., Pei, B., Zhang, H., Lu, D., Wang, Y., Wang, L., et al. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02084 · Cited by 43
Artificial intelligence and machine learning for plasmonic and surface-enhanced sensing
Geddis, A., Williams, H., Bashir, S., Malenfant, J., Dubois, C., Hamlet, L., & Masson, J. (2026) · Chemical Society Reviews · doi:10.1039/d5cs01522g · Cited by 47
Self-Supervised Pretraining via Multimodality Images With Transformer for Change Detection
Zhang, Y., Zhao, Y., Dong, Y., & Du, B. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3271024 · Cited by 44
TarViS: A Unified Approach for Target-Based Video Segmentation
Athar, A., Hermans, A., Luiten, J., Ramanan, D., & Leibe, B. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01797 · Cited by 55
Wang, L., Cao, M., & Yuan, X. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01772 · Cited by 44
Multimodal Fish Feeding Intensity Assessment in Aquaculture
Cui, M., Liu, X., Liu, H., Du, Z., Chen, T., Lian, G., Li, D., & Wang, W. (2024) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2024.3507098 · Cited by 42
CoTracker3: Simpler and Better Point Tracking by Pseudo-Labeling Real Videos
Karaev, N., Makarov, Y., Wang, J.-Y., Neverova, N., Vedaldi, A., & Rupprecht, C. (2025) · IEEE International Conference on Computer Vision · doi:10.1109/iccv51701.2025.00568 · Cited by 310
Masked Motion Encoding for Self-Supervised Video Representation Learning
Sun, X., Chen, P., Chen, L., Li, C., Li, T. H., Tan, M., & Gan, C. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00222 · Cited by 48
Swap Attention in Spatiotemporal Diffusions for Text-to-Video Generation
Wang, W., Yang, H., Tuo, Z., He, H., Zhu, J., Fu, J., & Liu, J. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02349-y · Cited by 159
Xiong, T., He, J., Wang, H., Tang, X., Shi, Z., & Zeng, Q. (2021) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2021.3128522 · Cited by 44
Self-citationGive Me Your Attention: Dot-Product Attention Considered Harmful for Adversarial Patch Robustness
Lovisotto, G., Finnie, N., Muñoz, M., Murnmadi, C. K., & Metzen, J. H. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01480 · Cited by 43
Spatial–temporal graph attention network for video anomaly detection
Chen, H., Mei, X., Ma, Z., Wu, X., & Wei, Y. (2023) · Image and Vision Computing · doi:10.1016/j.imavis.2023.104629 · Cited by 40
Implicit Temporal Modeling with Learnable Alignment for Video Recognition
Tu, S., Dai, Q., Wu, Z., Cheng, Z.-Q., Hu, H., & Jiang, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01825 · Cited by 52
Frame-wise Action Representations for Long Videos via Sequence Contrastive Learning
Chen, M., Wei, F., Li, C., & Cai, D. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01343 · Cited by 46
Dual-Recommendation Disentanglement Network for View Fuzz in Action Recognition
Liu, W., Zhong, X., Zhou, Z., Jiang, K., Wang, Z., & Lin, C. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3273459 · Cited by 38
Tagging before Alignment: Integrating Multi-Modal Tags for Video-Text Retrieval
Chen, Y., Wang, J., Lin, L., Qi, Z., Ma, J., & Shan, Y. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i1.25113 · Cited by 39
Long-Short Temporal Co-Teaching for Weakly Supervised Video Anomaly Detection
Sun, S., & Gong, X. (2023) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme55011.2023.00461 · Cited by 39
MEET: A Multi-Band EEG Transformer for Brain States Decoding
Shi, E., Yu, S., Kang, Y., Wu, J., Zhao, L., Zhu, D., Lv, J., Liu, T., Hu, X., & Zhang, S. (2023) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2023.3339892 · Cited by 43
Dastbaravardeh, E., Askarpour, S., Anari, M. S., & Rezaee, K. (2024) · International Journal of Intelligent Systems · doi:10.1155/2024/1052344 · Cited by 43
Efficient Video Instance Segmentation via Tracklet Query and Proposal
Wu, J., Yarram, S., Liang, H., Lan, T., Yuan, J., Eledath, J., & Medioni, G. G. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00103 · Cited by 39
Transformers for 1D signals in Parkinson’s disease detection from gait
Nguyen, D. M. D., Miah, M., Bilodeau, G.-A., & Bouachir, W. (2022) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr56361.2022.9956330 · Cited by 44
Logo-Former: Local-Global Spatio-Temporal Transformer for Dynamic Facial Expression Recognition
Ma, F., Sun, B., & Shutao Li, A. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10095448 · Cited by 44
STMixer: A One-Stage Sparse Action Detector
Tao, W., Cao, M., Gao, Z., Wu, G., & Wang, L. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01414 · Cited by 38
Disentangling Spatial and Temporal Learning for Efficient Image-to-Video Transfer Learning
Qing, Z., Zhang, S., Huang, Z., Zhang, Y., Gao, C., Zhao, D., & Sang, N. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01281 · Cited by 41
Ibh, M., Graßhof, S., Witzner, D., & Madeleine, P. (2023) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00548 · Cited by 40
Learning Emotion Representations from Verbal and Nonverbal Communication
Zhang, S., Pan, Y., & Wang, J. Z. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01821 · Cited by 42
One Model is Not Enough: Ensembles for Isolated Sign Language Recognition
Hrúz, M., Gruber, I., Kanis, J., Boháček, M., Hlaváč, M., & Krňoul, Z. (2022) · Sensors · doi:10.3390/s22135043 · Cited by 39
LIFT: Learning 4D LiDAR Image Fusion Transformer for 3D Object Detection
Zeng, Y., Zhang, D., Wang, C., Miao, Z., Liu, T., Zhan, X., Hao, D., & Ma, C. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01666 · Cited by 40
Singh, H., Suman, S., Subudhi, B. N., Jakhetiya, V., & Ghosh, A. K. (2022) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2022.3221912 · Cited by 37
CLSA: A Contrastive Learning Framework With Selective Aggregation for Video Rescaling
Tian, Y., Yan, Y., Zhai, G., Chen, L., & Gao, Z. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3242774 · Cited by 37
Procedure-Aware Pretraining for Instructional Video Understanding
Zhou, H., Martín-Martín, R., Kapadia, M., Savarese, S., & Niebles, J. C. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01033 · Cited by 37
HierVL: Learning Hierarchical Video-Language Embeddings
Ashutosh, K., Girdhar, R., Torresani, L., & Grauman, K. L. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02209 · Cited by 50
Self-citationSpikeformer: Training high-performance spiking neural network with transformer
Li, Y., Lei, Y., & Yang, X. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.127279 · Cited by 36
Scalable Video Object Segmentation With Identification Mechanism
Yang, Z., Miao, J., Wei, Y., Wang, W., Wang, X., & Yang, Y. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3383592 · Cited by 49
Pianforini, M., Dazzi, S., Pilzer, A., & Vacondio, R. (2024) · Journal of Hydrology · doi:10.1016/j.jhydrol.2024.131169 · Cited by 36
Multi-Stage Spatio-Temporal Aggregation Transformer for Video Person Re-Identification
Tang, Z., Zhang, R., Peng, Z., Chen, J., & Lin, L. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3231103 · Cited by 34
Gate-Shift-Fuse for Video Action Recognition
Sudhakaran, S., Escalera, S., & Lanz, O. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3268134 · Cited by 35
Re2TAL: Rewiring Pretrained Video Backbones for Reversible Temporal Action Localization
Zhao, C., Liu, S., Mangalam, K., & Ghanem, B. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01025 · Cited by 40
STPrivacy: Spatio-Temporal Privacy-Preserving Action Recognition
Li, M., Xu, X., Fan, H., Zhou, P., Liu, J., Liu, J., Li, J., Keppo, J., Shou, M. Z., & Yan, S. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00471 · Cited by 39
Zhao, Z., Dong, X., Wang, Y., & Hu, C. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3355755 · Cited by 43
Change3D: Revisiting Change Detection and Captioning from A Video Modeling Perspective
Zhu, D., Huang, X., Huang, H., Zhou, H., & Shao, Z. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02236 · Cited by 37
The Animation Transformer: Visual Correspondence via Segment Matching
Casey, E., Perez, V., & Li, Z. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01113 · Cited by 35
Hutchinson, M., & Gadepally, V. N. (2021) · IEEE Access · doi:10.1109/access.2021.3115476 · Cited by 46
Geng, P., Lu, X., Hu, C., Liu, H., & Lyu, L. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3248782 · Cited by 34
Breaking the “Object” in Video Object Segmentation
Tokmakov, P., Li, J., & Gaidon, A. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02187 · Cited by 43
ResFormer: Scaling ViTs with Multi-Resolution Training
Tian, R., Wu, Z., Dai, Q., Hu, H., Qiao, Y., & Jiang, Y. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02176 · Cited by 42
Video ReCap: Recursive Captioning of Hour-Long Videos
Islam, M. M., Ho, N., Yang, X., Nagarajan, T., Torresani, L., & Bertasius, G. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01723 · Cited by 34
Self-citationSporthesia: Augmenting Sports Videos Using Natural Language
Chen, Z., Yang, Q., Xie, X., Beyer, J., Xia, H., Wu, Y., & Pfister, H. (2022) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2022.3209497 · Cited by 51
It's Time for Artistic Correspondence in Music and Video
Surís, D., Vondrick, C., Russell, B. C., & Salamon, J. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01031 · Cited by 35
Alfasly, S., Lu, J., Xu, C., & Zou, Y. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01957 · Cited by 36
SPARTAN: Self-supervised Spatiotemporal Transformers Approach to Group Activity Recognition
Chappa, N. V. S. R., Nguyen, P., Nelson, A., Seo, H., Li, X., Dobbs, P. D., & Luu, K. (2023) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00544 · Cited by 37
Skating-Mixer: Long-Term Sport Audio-Visual Modeling with MLPs
Xia, J., Zhuge, M., Geng, T., Fan, S., Wei, Y., He, Z., & Zheng, F. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i3.25392 · Cited by 43
Hierarchical compositional representations for few-shot action recognition
Li, C., Zhang, J., Wu, S., Jin, X., & Shan, S. (2024) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2023.103911 · Cited by 33
Non-Semantics Suppressed Mask Learning for Unsupervised Video Semantic Compression
Tian, Y., Lu, G. S., Zhai, G., & Gao, Z. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01252 · Cited by 36
A Knowledge-Based Hierarchical Causal Inference Network for Video Action Recognition
Liu, Y., Liu, F., Jiao, L., Bao, Q., Li, L., Guo, Y., & Chen, P. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2024.3386339 · Cited by 33
Learning Correlation Structures for Vision Transformers
Kim, M., Seo, P. H., Schmid, C., & Cho, M. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01792 · Cited by 36
Learning Semantic-Aware Spatial-Temporal Attention for Interpretable Action Recognition
Fu, J., Gao, J., & Xu, C. (2021) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3137023 · Cited by 35
Transferring Image-CLIP to Video-Text Retrieval via Temporal Relations
Fang, H., Xiong, P., Xu, L., & Luo, W. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3227416 · Cited by 33
i-Code: An Integrative and Composable Multimodal Learning Framework
Yang, Z., Fang, Y., Zhu, C., Pryzant, R., Chen, D., Shi, Y., Yi‐chong, X., Qian, Y., Gao, M., Chen, Y., et al. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i9.26290 · Cited by 45
MELTR: Meta Loss Transformer for Learning to Fine-tune Video Foundation Models
Ko, D., Choi, J., Choi, H. K., On, K.-W., Roh, B., & Kim, H. J. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01925 · Cited by 38
Video Test-Time Adaptation for Action Recognition
Lin, W., Mirza, M. J., Koziński, M., Possegger, H., Kuehne, H., & Bischof, H. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02198 · Cited by 41
MetaTransformer: deep metagenomic sequencing read classification using self-attention models
Wichmann, A., Buschong, E., Müller, A., Jünger, D., Hildebrandt, A., Hankeln, T., & Schmidt, B. (2023) · NAR Genomics and Bioinformatics · doi:10.1093/nargab/lqad082 · Cited by 34
Multimodal Distillation for Egocentric Action Recognition
Radevski, G., Grujicic, D., Blaschko, M. B., Moens, M., & Tuytelaars, T. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00481 · Cited by 32
Tao, C., Peng, Y., Zhang, Q., Zhang, Y., Gong, B., Wang, Q., & Wang, W. (2024) · Atmospheric chemistry and physics · doi:10.5194/acp-24-4177-2024 · Cited by 32
Error Detection in Egocentric Procedural Task Videos
Lee, S., Lu, Z., Zhang, Z., Hoai, M., & Elhamifar, E. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01765 · Cited by 33
Multimodal performers for genomic selection and crop yield prediction
Måløy, H., Windju, S. S., Bergersen, S., Alsheikh, M. K., & Downing, K. L. (2021) · Smart Agricultural Technology · doi:10.1016/j.atech.2021.100017 · Cited by 35
Malibari, N., Katib, I. A., & Mehmood, R. (2021) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2021.01212106 · Cited by 32
PolyViT: Co-training Vision Transformers on Images, Videos and AudioPreprint
Likhosherstov, V., Arnab, A., Choromański, K., Lučić, M., Tay, Y., Weller, A., & Dehghani, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.12993 · Cited by 66
Metadata
- Authors
- Bertasius, G., Wang, H., & Torresani, L.
- Affiliations
- University of Pennsylvania (US)
- Meta (United States) (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2021
- Pages
- 813-824
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2102.05095
- OpenAlex
- W3126721948
- Cited by
- 1,663
- References
- 58
- Citations by year
- 2026: 161 · 2025: 390 · 2024: 418 · 2023: 387 · 2022: 199 · 2021: 108
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- International Conference on Machine Learning
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- video classification, self-attention, spatio-temporal attention, TimeSformer, transformer models, divided attention, human action recognition, 3D convolutional networks
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
