The DOI matches its DataCite record; 54 of the 58 references it cites were found in a registry or an index.
Is Space-Time Attention All You Need for Video Understanding?
Open accessAbstract
We present a convolution-free approach to video classification built exclusively on self-attention over space and time. Our method, named "TimeSformer," adapts the standard Transformer architecture to video by enabling spatiotemporal feature learning directly from a sequence of frame-level patches. Our experimental study compares different self-attention schemes and suggests that "divided attention," where temporal attention and spatial attention are separately applied within each block, leads to the best video classification accuracy among the design choices considered. Despite the radically new design, TimeSformer achieves state-of-the-art results on several action recognition benchmarks, including the best reported accuracy on Kinetics-400 and Kinetics-600. Finally, compared to 3D convolutional networks, our model is faster to train, it can achieve dramatically higher test efficiency (at a small drop in accuracy), and it can also be applied to much longer video clips (over one minute long). Code and models are available at: https://github.com/facebookresearch/TimeSformer.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Vision Transformer (ViT) [17], and video-specific variants such as TimeSformer [ 18, 19 ] and Video Swin Transformer [20] capture longrange dependencies and global spatiotemporal context that CNNs often miss.”
From Cross-dataset video deepfake detection using Transformer and CNN architectures (2026) · cites this paper as [18]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Similar to previous trials [3, 6], training video transformers from scratch yields unsatisfied results on video datasets.”
From VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training (2022) · cites this paper as [6]
The cited source says what the claim attributes to it.
In the citing paper
“For this reason, due to real-time computation concerns in visual odometry applications, we selected the encoder architecture with the 'divided space-time' self-attention, which is more efficient in terms of processing complexity compared to the 'joint space-time' self-attention.”
From Transformer-Based Model for Monocular Visual Odometry: A Video Understanding Approach (2023) · cites this paper as [14]
The cited source says what the claim attributes to it.
In the citing paper
“We chose the TimeSformer [14] model for this work since it achieved state-of-the-art results in video action recognition tasks compared to previous works based on convolutional neural networks (CNN).”
From Transformer-Based Model for Monocular Visual Odometry: A Video Understanding Approach (2023) · cites this paper as [14]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472
(2008) · Journal of Machine Learning Research · Cited by 33,051
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
End-to-End Object Detection with Transformers
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58452-8_13 · Cited by 18,285
Research works printed but not linked: 5
- Ba, L. J., Kiros, J. R., and Hinton, G. E. Layer normalization. CoRR, 2016.
- Chen, M. X., Firat, O., Bapna, A., Johnson, M., Macherey, W., Foster, G., Jones, L., Schuster, M., Shazeer, N., Par- mar, N., Vaswani, A., Uszkoreit, J., Kaiser, L., Chen, Z., Wu, Y., and Hughes, M. The best of both worlds: Combining recent advances in neural machine translation. In Proceedings of the 56th Annual Meeting of the Asso- ciation for Computational Linguistics. Association for Computational Linguistics, 2018a.
- Chen, Y., Kalantidis, Y., Li, J., Yan, S., and Feng, J. Aˆ2- nets: Double attention networks. In Advances in Neural Information Processing Systems 31, 2018b.
- International Conference on Computer Vision (ICCV), October 2019.
- Cordonnier, J., Loukas, A., and Jaggi, M. On the relation- ship between self-attention and convolutional layers. In 8th International Conference on Learning Representa- tions, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020, 2020.
Cited by
Sorted by citations
Items 301–400 of 1,004
Drone-HAT: Hybrid Attention Transformer for Complex Action Recognition in Drone Surveillance Videos
Khan, M., Ahmad, J., Saddik, A. E., Gueaieb, W., Masi, G., & Karray, F. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00474 · Cited by 32
Audio-Visual Speech Recognition is Worth $32\times 32\times 8$ Voxels
Serdyuk, D., Braga, O., & Siohan, O. (2021) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru51503.2021.9688191 · Cited by 35
Data Efficient Video Transformer for Violence Detection
Abdali, A. R. (2021) · IEEE International Conference on Communication, Networks and Satellite (COMNETSAT) · doi:10.1109/comnetsat53002.2021.9530829 · Cited by 31
A Large-Scale Robustness Analysis of Video Action Recognition Models
Schiappa, M. C., Biyani, N., Kamtam, P., Vyas, S., Palangi, H., Vineet, V., & Rawat, Y. S. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01412 · Cited by 37
In the Eye of Transformer: Global–Local Correlation for Egocentric Gaze Estimation and Beyond
Lai, B., Liu, M., Ryan, F., & Rehg, J. M. (2023) · International Journal of Computer Vision · doi:10.1007/s11263-023-01879-7 · Cited by 42
AAFormer: A Multi-Modal Transformer Network for Aerial Agricultural Images
Shen, Y., Wang, L., & Jin, Y. (2022) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00177 · Cited by 32
Efficient Video Action Detection with Token Dropout and Context Refinement
Chen, L., Zhan, T., Song, Y., Wu, G., & Wang, L. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00953 · Cited by 30
Wang, Z., & Yan, J.-H. (2024) · Journal of manufacturing systems · doi:10.1016/j.jmsy.2024.04.019 · Cited by 31
A Comprehensive Survey of Recent Transformers in Image, Video and Diffusion Models
Le, D. P. C., Wang, D., & Le, V.-T. (2024) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2024.050790 · Cited by 36
P2AT: Pyramid pooling axial transformer for real-time semantic segmentation
Elhassan, M. A. M., Zhou, C., Benabid, A., & Adam, A. B. M. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.124610 · Cited by 31
MaskFusionNet: A Dual-Stream Fusion Model With Masked Pre-Training Mechanism for rPPG Measurement
Zhang, Y., Shi, J., Wang, J., Zong, Y., Zheng, W., & Zhao, G. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3422849 · Cited by 30
SVFAP: Self-Supervised Video Facial Affect Perceiver
Sun, L., Lian, Z., Wang, K., He, Y., Xu, M., Sun, H., Liu, B., & Tao, J. (2024) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2024.3436913 · Cited by 38
DarkLight Networks for Action Recognition in the Dark
Chen, R., Chen, J., Liang, Z., Gao, H., & Lin, S. (2021) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00094 · Cited by 33
With a Little Help from my Temporal Context: Multimodal Egocentric Action Recognition
Kazakos, E., Huh, J., Nagrani, A., Zisserman, A., & Damen, D. (2021) · British Machine Vision Conference (BMVC) · doi:10.5244/c.35.171 · Cited by 43
A Divided Spatial and Temporal Context Network for Remote Sensing Change Detection
Shi, N., Chen, K., & Zhou, G. (2022) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2022.3176858 · Cited by 32
Citywide Mobile Traffic Forecasting Using Spatial-Temporal Downsampling Transformer Neural Networks
Hu, Y., Zhou, Y., Song, J., Xu, L., & Zhou, X. (2022) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2022.3214483 · Cited by 29
Sparsifiner: Learning Sparse Instance-Dependent Attention for Efficient Vision Transformers
Wei, C., Duke, B., Jiang, R., Aarabi, P., Taylor, G. W., & Shkurti, F. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02172 · Cited by 32
Transformer approach to nowcasting solar energy using geostationary satellite data
Li, R., Wang, D., Wang, Z., Liang, S., Li, Z., Xie, Y., & He, J. (2024) · Applied Energy · doi:10.1016/j.apenergy.2024.124387 · Cited by 29
On the Content Bias in Fréchet Video Distance
Ge, S., Mahapatra, A., Parmar, G., Zhu, J.-Y., & Huang, J.-B. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00695 · Cited by 50
Dyadformer: A Multi-modal Transformer for Long-Range Modeling of Dyadic Interactions
Curto, D., Clapés, A., Selva, J., Smeureanu, S., Jacques, J. C. S., Gallardo‐Pujol, D., Guilera, G., Leiva, D., Moeslund, T. B., Escalera, S., et al. (2021) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw54120.2021.00247 · Cited by 35
Ming, Z., Yu, Z., Ghadi, M., Visani, M., Luqman, M. M., & Burie, J.-C. (2022) · Proceedings - International Conference on Image Processing · doi:10.1109/icip46576.2022.9897560 · Cited by 29
Learning SpatioTemporal and Motion Features in a Unified 2D Network for Action Recognition
Wang, M., Xing, J., Su, J., Chen, J., & Yong, L. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3173658 · Cited by 29
MINTIME: Multi-Identity Size-Invariant Video Deepfake Detection
Coccomini, D. A., Zilos, G. K., Amato, G., Caldelli, R., Falchi, F., Papadopoulos, S., & Gennaro, C. (2024) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2024.3409054 · Cited by 33
Modeling Motion with Multi-Modal Features for Text-Based Video Segmentation
Zhao, W., Wang, K., Chu, X., Xue, F., Wang, X., & You, Y. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01144 · Cited by 29
LoTE-Animal: A Long Time-span Dataset for Endangered Animal Behavior Understanding
Liu, D., Hou, J., Huang, S., Liu, J., He, Y., Zheng, B., Ning, J., & Zhang, J. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01836 · Cited by 30
Learning Procedure-aware Video Representation from Instructional Videos and Their Narrations
Zhong, Y., Yu, L., Bai, Y., Li, S.-W. D., Yan, X., & Li, Y. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01424 · Cited by 37
Zhao, Z., Chen, Z., Li, J., Xie, X., Chen, K., Wang, X., & Shi, G. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.126903 · Cited by 27
One-Shot Action Recognition via Multi-Scale Spatial-Temporal Skeleton Matching
Yang, S., Liu, J., Lu, S., Er, M. H., & Kot, A. C. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3363831 · Cited by 27
G2P-DDM: Generating Sign Pose Sequence from Gloss Sequence with Discrete Diffusion Model
Xie, P., Zhang, Q., Peng, T., Tang, H., Du, Y., & Li, Z. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i6.28441 · Cited by 57
Ramp Rate Metric Suitable for Solar Forecasting
Nouri, B., Fabel, Y., Blum, N. B., Schnaus, D., Zarzalejo, L. F., Kazantzidis, A., & Wilbert, S. (2024) · Solar RRL · doi:10.1002/solr.202400468 · Cited by 28
Paulraj, S., & Vairavasundaram, S. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.109496 · Cited by 27
Foldsformer: Learning Sequential Multi-Step Cloth Manipulation With Space-Time Attention
Mo, K., Xia, C., Wang, X., Deng, Y., Gao, X., & Liang, B. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3229573 · Cited by 26
Concept Parser With Multimodal Graph Learning for Video Captioning
Wu, B., Liu, B., Huang, P., Bao, J., Peng, X., & Yu, J. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3277827 · Cited by 28
Test of Time: Instilling Video-Language Models with a Sense of Time
Bagad, P., Tapaswi, M., & Snoek, C. G. M. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00247 · Cited by 31
VicTR: Video-conditioned Text Representations for Activity Recognition
Kahatapitiya, K., Arnab, A., Nagran, A., & Ryoo, M. S. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01755 · Cited by 26
Scaling Up Your Kernels: Large Kernel Design in ConvNets Toward Universal Representations
Zhang, Y., Ding, X., & Yue, X. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3600702 · Cited by 36
Kowal, M. D., Siam, M., Islam, M. A., Bruce, N. D. B., Wildes, R. P., & Derpanis, K. G. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01361 · Cited by 24
Cross-scale cascade transformer for multimodal human action recognition
Liu, Z., Qin, C., Song, C., & Cheng, J. S. (2023) · Pattern Recognition Letters · doi:10.1016/j.patrec.2023.02.024 · Cited by 27
ITA: An Energy-Efficient Attention and Softmax Accelerator for Quantized Transformers
İslamoğlu, G., Scherer, M., Paulin, G., Fischer, T., Jung, V. J. B., Garofalo, A., & Benini, L. (2023) · International Symposium on Low Power Electronics and Design (ISLPED) · doi:10.1109/islped58423.2023.10244348 · Cited by 33
PIDRo: Parallel Isomeric Attention with Dynamic Routing for Text-Video Retrieval
Guan, P., Pei, R., Shao, B., Liu, J., Li, W., Gu, J., Xu, H., Xu, S., Yan, Y., & Lam, E. Y. M. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01025 · Cited by 25
Shul, Y., & Choi, J. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10447181 · Cited by 25
Interaction Region Visual Transformer for Egocentric Action Anticipation
Roy, D., Rajendiran, R., & Fernando, B. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00660 · Cited by 25
Wu, D., Zhao, Q., Fan, J., Qi, J., Zheng, P., & Hu, J. (2025) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2025.03.016 · Cited by 28
Exploiting Attention-Consistency Loss For Spatial-Temporal Stream Action Recognition
Xu, H., Jin, X., Wang, Q., Hussain, A., & Huang, K. (2022) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3538749 · Cited by 25
Starvqa: Space-Time Attention for Video Quality Assessment
Xing, F., Wang, Y., Wang, H., Li, L., & Zhu, G. (2022) · Proceedings - International Conference on Image Processing · doi:10.1109/icip46576.2022.9897881 · Cited by 27
Self-citationLook for the Change: Learning Object States and State-Modifying Actions from Untrimmed Web Videos
Souček, T., Alayrac, J.-B., Miech, A., Laptev, I., & Šivic, J. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01357 · Cited by 25
MED-VT: Multiscale Encoder-Decoder Video Transformer with Application to Object Segmentation
Karim, R., Zhao, H., Wildes, R. P., & Siam, M. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00612 · Cited by 25
Inherent Redundancy in Spiking Neural Networks
Yao, M., Hu, J., Zhao, G., Wang, Y., Zhang, Z., Xu, B., & Li, G. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01552 · Cited by 28
Kabra, R., Zoran, D., Erdogan, G., Matthey, L., Creswell, A., Botvinick, M., Lerchner, A., & Burgess, C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.03849 · Cited by 41
Early identification of stroke through deep learning with multi-modal human speech and movement data
Ou, Z., Wang, H., Zhang, B., Liang, H., Hu, B., Ren, L., Liu, Y., Zhang, Y., Dai, C., Wu, H., et al. (2024) · Neural Regeneration Research · doi:10.4103/1673-5374.393103 · Cited by 27
Self-citationFedFSLAR: A Federated Learning Framework for Few-shot Action Recognition
Tu, N. A., Abu, A., Aikyn, N., Makhanov, N., Lee, M.-H., Le-Huy, K., & Wong, K. (2024) · IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW) · doi:10.1109/wacvw60836.2024.00035 · Cited by 25
Mehta, N. K., Prasad, S. S., Saurav, S., Saini, R., & Singh, S. K. (2024) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2024.3379075 · Cited by 24
Retrieval-Augmented Egocentric Video Captioning
Xu, J., Huang, Y., Hou, J., Chen, G., Zhang, Y., Feng, R., & Xie, W. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01284 · Cited by 42
Singh, S., & Dhumane, A. (2025) · MethodsX · doi:10.1016/j.mex.2025.103632 · Cited by 24
PreTR: Spatio-Temporal Non-Autoregressive Trajectory Prediction Transformer
Achaji, L., Barry, T., Fouqueray, T., Moreau, J., Aioun, F., & Charpillet, F. (2022) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc55140.2022.9922451 · Cited by 27
FenceNet: Fine-grained Footwork Recognition in Fencing
Zhu, K. X., Wong, A. K. C., & McPhee, J. (2022) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00403 · Cited by 26
TP-VIT: A Two-Pathway Vision Transformer for Video Action Recognition
Jing, Y., & Wang, F. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747276 · Cited by 24
Sequential ISAR Target Classification Based on Hybrid Transformer
Xue, R., Bai, X., Cao, X., & Zhou, F. (2022) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2022.3155246 · Cited by 26
Wei, G., Zhang, Z., Lan, C., Lü, Y., & Chen, Z. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i3.25376 · Cited by 28
Convolutional Transformer Network for Fine-Grained Action Recognition
Ma, Y., Wang, R., Zong, M., Ji, W., Wang, Y., & Ye, B. (2022) · Neurocomputing · doi:10.1016/j.neucom.2023.127027 · Cited by 28
You, H., Zhong, X., Liu, W., Wei, Q., Huang, W., Yu, Z., & Huang, T. (2024) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2024.3375620 · Cited by 25
Semantic-aware frame-event fusion based pattern recognition via large vision–language models
Li, D., Jin, J., Zhang, Y., Zhong, Y., Wu, Y., Chen, L., Wang, X., & Luo, B. (2024) · Pattern Recognition · doi:10.1016/j.patcog.2024.111080 · Cited by 23
Regularity Learning via Explicit Distribution Modeling for Skeletal Video Anomaly Detection
Yu, S., Zhao, Z., Fang, H., Deng, A., Su, H., Wang, D., Gan, W., Lu, C., & Wu, W. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3296118 · Cited by 30
Motion-Guided Global–Local Aggregation Transformer Network for Precipitation Nowcasting
Dong, X., Zhao, Z., Wang, Y., Wang, J., & Hu, C. (2022) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2022.3217639 · Cited by 25
A Novel Two-Stream Transformer-Based Framework for Multi-Modality Human Action Recognition
Shi, J., Zhang, Y., Wang, W., Xing, B., Hu, D., & Chen, L. (2023) · Applied Sciences · doi:10.3390/app13042058 · Cited by 24
Hybrid token transformer for deep face recognition
Su, W., Wang, Y., Li, K., Gao, P., & Qiao, Y. (2023) · Pattern Recognition · doi:10.1016/j.patcog.2023.109443 · Cited by 23
Mahmoudi, S. A., Amel, O., Stassin, S., Liagre, M., Benkedadra, M., & Mancaş, M. (2023) · Electronics · doi:10.3390/electronics12092027 · Cited by 24
Solar Irradiance Anticipative Transformer
Mercier, T. M., Rahman, T., & Sabet, A. (2023) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00200 · Cited by 25
Gao, Z., Wang, Y., Chen, J., Xing, J., Patel, S., Liu, X., & Shi, Y. (2023) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3610872 · Cited by 34
End-to-End Transformer for Compressed Video Quality Enhancement
Yu, L., Chang, W., Wu, S., & Gabbouj, M. (2023) · IEEE Transactions on Broadcasting · doi:10.1109/tbc.2023.3332015 · Cited by 23
Building an Open-Vocabulary Video CLIP Model With Better Architectures, Optimization and Data
Wu, Z., Weng, Z., Peng, W., Yang, X., Li, A., Davis, L. S., & Jiang, Y. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3357503 · Cited by 27
Object-centric Video Representation for Long-term Action Anticipation
Zhang, C., Fu, C., Wang, S., Agarwal, N., Lee, K., Choi, C., & Sun, C. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00661 · Cited by 25
Wu, T., Shang, K., Dai, W., Wang, M., Liu, R., Zhou, J., & Liu, J. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.108574 · Cited by 23
DeepMIM: Deep Supervision for Masked Image Modeling
Ren, S., Wei, F., Zhang, S., & Han, H. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00095 · Cited by 35
Spatiotemporal Transformer for Data Inference and Long Prediction in Sparse Mobile CrowdSensing
Wang, E., Liu, W.-T., Liu, W., Xiang, C., Yang, B. X., & Yang, Y. (2023) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom53939.2023.10228982 · Cited by 23
A transformer-based method for correcting significant wave height numerical forecasting errors
Kang, X., Song, H., Zhang, Z., Yin, X., & Gu, J. (2024) · Frontiers in Marine Science · doi:10.3389/fmars.2024.1374902 · Cited by 22
A Transformer-based method to simulate multi-scale soil moisture
Liu, Y., Xin, Y., & Yin, C. (2025) · Journal of Hydrology · doi:10.1016/j.jhydrol.2025.132900 · Cited by 26
Towards Universal Soccer Video Understanding
Rao, J., Wu, H., Jiang, H., Zhang, Y., Wang, Y., & Xie, W. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00785 · Cited by 22
Beyond Short Clips: End-to-End Video-Level Learning with Collaborative Memories
Yang, X., Fan, H., Torresani, L., Davis, L. S., & Wang, H. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00748 · Cited by 23
Self-citationZhou, Z., Ding, C., Li, J., Mohammadi, E., Liu, G., Yang, Y., & Wu, Q. M. J. (2022) · IEEE Transactions on Image Processing · doi:10.1109/tip.2022.3224877 · Cited by 22
Global Temporal Difference Network for Action Recognition
Xie, Z., Chen, J., Wu, K., Guo, D., & Hong, R. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3224327 · Cited by 22
Dave, I. R., Rizve, M. N., Chen, C., & Shah, M. A. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00232 · Cited by 22
Wang, X., Rong, Y., Wu, Z., Zhu, L., Jiang, B., Tang, J., & Tian, Y. (2025) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2025.3568833 · Cited by 37
Multilevel Semantic Interaction Alignment for Video–Text Cross-Modal Retrieval
Chen, L., Deng, Z., Liu, L., & Yin, S. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3360530 · Cited by 24
Appearance-posture fusion network for distracted driving behavior recognition
Yang, X., Qiao, Y., Han, S., Feng, Z., & Chen, Y. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.124883 · Cited by 21
Adapting Short-Term Transformers for Action Detection in Untrimmed Videos
Yang, M., Gao, H., Guo, P., & Wang, L. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01757 · Cited by 28
Ren, H., Zhou, Y., Zhu, J., Lin, X., Fu, H., Huang, Y., Fang, Y., Ma, F., Yu, H., & Cheng, B. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3556561 · Cited by 41
Delta Sampling R-BERT for limited data and low-light action recognition
Hira, S., Das, R., Modi, A., & Pakhomov, D. (2021) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00095 · Cited by 22
Space-Time Crop & Attend: Improving Cross-modal Video Representation Learning
Patrick, M., Huang, P.-Y., Misra, I., Metze, F., Vedaldi, A., Asano, Y. M., & Henriques, J. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01039 · Cited by 28
SCVRL: Shuffled Contrastive Video Representation Learning
Dorkenwald, M., Xiao, F., Brattoli, B., Tighe, J., & Modolo, D. (2022) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00458 · Cited by 22
Compressed Video Action Recognition With Dual-Stream and Dual-Modal Transformer
Mou, Y., Jiang, X., Xu, K., Sun, T., & Wang, Z. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3319140 · Cited by 21
Limited Data, Unlimited Potential: A Study on ViTs Augmented by Masked Autoencoders
Das, S., Jain, T., Reilly, D., Balaji, P., Karmakar, S., Marjit, S., Li, X., Das, A., & Ryoo, M. S. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00673 · Cited by 23
Bonyani, M., Soleymani, M., & Wang, C. (2024) · Automation in Construction · doi:10.1016/j.autcon.2024.105508 · Cited by 21
Event Voxel Set Transformer for Spatiotemporal Representation Learning on Event Streams
Xie, B., Deng, Y., Shao, Z., Xu, Q., & Li, Y. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3448615 · Cited by 32
BT-Adapter: Video Conversation is Feasible Without Video Instruction Tuning
Liu, R., Li, C., Ge, Y., Li, T. H., Shan, Y., & Li, G. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01296 · Cited by 54
FocusMAE: Gallbladder Cancer Detection from Ultrasound Videos with Focused Masked Autoencoders
Basu, S., Gupta, M., Madan, C., Gupta, P. K., & Arora, C. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01113 · Cited by 22
Mutual Support of Data Modalities in the Task of Sign Language Recognition
Gruber, I., Krňoul, Z., Hrúz, M., Kanis, J., & Boháček, M. (2021) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00381 · Cited by 22
Improved Multiscale Vision Transformers for Classification and DetectionPreprint
Li, Y., Wu, C.-Y., Fan, H., Mangalam, K., Xiong, B., Malik, J., & Feichtenhofer, C. (2021) · arXiv (Cornell University) · Cited by 23
Synchformer: Efficient Synchronization From Sparse Cues
Iashin, V., Xie, W., Rahtu, E., & Zisserman, A. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10448489 · Cited by 43
Komninos, P., Verraest, A. E. C., Eleftheroglou, N., & Zarouchas, D. S. (2024) · Composites Part B Engineering · doi:10.1016/j.compositesb.2024.111863 · Cited by 21
Metadata
- Authors
- Bertasius, G., Wang, H., & Torresani, L.
- Affiliations
- University of Pennsylvania (US)
- Meta (United States) (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2021
- Pages
- 813-824
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2102.05095
- OpenAlex
- W3126721948
- Cited by
- 1,663
- References
- 58
- Citations by year
- 2026: 161 · 2025: 390 · 2024: 418 · 2023: 387 · 2022: 199 · 2021: 108
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- International Conference on Machine Learning
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- video classification, self-attention, spatio-temporal attention, TimeSformer, transformer models, divided attention, human action recognition, 3D convolutional networks
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
