The DOI matches its Crossref record; 106 of the 111 references it cites were found in a registry or an index.
Multiscale Vision Transformers
Open accessAbstract
We present Multiscale Vision Transformers (MViT) for video and image recognition, by connecting the seminal idea of multiscale feature hierarchies with transformer models. Multiscale Transformers have several channel-resolution scale stages. Starting from the input resolution and a small channel dimension, the stages hierarchically expand the channel capacity while reducing the spatial resolution. This creates a multiscale pyramid of features with early layers operating at high spatial resolution to model simple low-level visual information, and deeper layers at spatially coarse, but complex, high-dimensional features. We evaluate this fundamental architectural prior for modeling the dense nature of visual signals for a variety of video recognition tasks where it outperforms concurrent vision transformers that rely on large scale external pre-training and are 5-10× more costly in computation and parameters. We further remove the temporal dimension and apply our model for image classification where it outperforms prior work on vision transformers. Code is available at: https://github.com/facebookresearch/SlowFast.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 2×
“As in original ViT [25], we use residual connections [45] and Layer Normalization (LN) [2] in the pre-normalization configuration that applies LN at the beginning of the residual function, and our MLPs consist of two linear layers with GELU activation [48], where the first layer expands the dimension from D to 4D, and…”
MethodResultVaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 6×
“In contrast to original Multi Head Attention (MHA) operators [98], where the channel dimension and the spatiotemporal resolution remains fixed, MHPA pools the sequence of latent tensors to reduce the sequence length (resolution) of the attended input.”
BackgroundMethodInfluentialVery Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 1×
“Incorporating downsampling, shift invariance, and shared weights, ConvNets are de-facto standard backbones for computer vision tasks for image [65, 62, 88, 90, 46, 12, 15, 34, 93, 81, 41] and video [87, 31, 11, 79, 69, 106, 96, 30, 105, 35, 29, 117, 57].”
BackgroundImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 1×
“Incorporating downsampling, shift invariance, and shared weights, ConvNets are de-facto standard backbones for computer vision tasks for image [70, 67, 94, 96, 51, 15, 18, 39, 99, 87, 46] and video [93, 36, 14, 85, 74, 112, 102, 34, 111, 40, 33, 123, 62].”
BackgroundFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534 · cited 1×
“Finally, compared to the best model of concurrent Swin [79] (which was designed for image recognition), MViT has +0.”
ResultRethinking the Inception Architecture for Computer Vision
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.308 · Cited by 32,138
Research works printed but not linked: 11
- gao, 2019, Res2net: A new multi-scale backbone architecture, IEEE PAMI
- fan, 2020, Pyslowfast
- feichtenhofer, 2019, SlowFast networks for video recognition in ActivityNet challenge 2019
- 10.1109/CVPR.2009.5206848 · DOI · Paper page
- 10.1109/CVPR.2019.00033 · DOI
- hang, 2020, ResNeSt Split-Attention Networks
- 10.1109/ICCV.2019.00718 · DOI
- 10.1109/CVPR.2015.7298594 · DOI · Paper page
- carion, 2020, Endto-end object detection with transformers, Proc ECCV, 213
- loshchilov, 2018, Fixing weight decay regularization in adam
- qiu, 2017, Learning spatiotemporal representation with pseudo-3d residual networks, Proc ICCV
Cited by
Sorted by citations
Items 601–700 of 1,003
SoGAR: Self-Supervised Spatiotemporal Attention-Based Social Group Activity Recognition
Chappa, N. V. S. R., Nguyen, P., Nelson, A., Seo, H., Li, X., Dobbs, P. D., & Luu, K. (2025) · IEEE Access · doi:10.1109/access.2025.3541986 · Cited by 9
“Fan et al. [54] used multiscale feature aggregation to improve the learning performance of features.”
ResMoE: Space-efficient Compression of Mixture of Experts LLMs via Residual Restoration
Ai, M., Wei, T., Chen, Y., Zeng, Z., Zhao, R., Varatkar, G. V., Rouhani, B. D., Tang, X., Tong, H., & He, J. (2025) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3690624.3709196 · Cited by 22
“…architecture in the domain of machine learning [67] is undeniable, for the fields including natural language processing [3, 6, 9, 15, 19, 29, 49, 54– 56, 67, 86, 88, 89], computer vision [18, 43, 70, 72, 75], information retrieval [8, 32, 74, 81, 83], and graph modeling [39, 41, 52, 77, 79].”
YOLO-Act: Unified Spatiotemporal Detection of Human Actions Across Multi-Frame Sequences
Alzahrani, N., Bchir, O., & Ismail, M. M. B. (2025) · Sensors · doi:10.3390/s25103013 · Cited by 11
“Recent advancements in transformer architecture have significantly catalyzed progress in action recognition, with extensive research exploiting transformer-based backbones and self-attention mechanisms to push field boundaries [9,49–52].”
InfluentialPAFusion: A general image fusion network with adversarial representation learning
Liu, X., Hirota, K., Dai, Y., Mersha, B. W., Shao, S., & Wang, J. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113815 · Cited by 8
Abdallah, A., Kasem, M. S., Abdelhalim, I., Alghamdi, N. S., & El-Baz, A. S. (2025) · IEEE Access · doi:10.1109/access.2025.3581582 · Cited by 11
“Self-Supervised Learning (SSL) has emerged as a powerful paradigm in machine learning, enabling models to learn meaningful representations from unlabeled data by formulating auxiliary tasks known as pretext tasks [70], [71], [72], [73].”
Detecting AI-Generated Video via Frame Consistency
Ma, L., Yan, Z., Guo, Q., Liao, Y., Yu, H., & Zhou, P. (2025) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme59968.2025.11210049 · Cited by 28
Self-citationV-JEPA 2.1: Unlocking Dense Features in Video Self-Supervised Learning
Mur-Labadia, L., Muckley, M. J., Bar, A., Assran, M., Sinha, K., Rabbat, M., LeCun, Y., & Ballas, N. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37369-4_36 · Cited by 150
Patch-based Separable Transformer for Visual Recognition
Sun, S., Yue, X., Zhao, H., Torr, P. H. S., & Bai, S. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3231725 · Cited by 8
“Some concurrent works like Swin Transformer [28], PVT [43] and MViT [10] use either local attention or global attention for feature extraction.”
Mixture lightweight transformer for scene understanding
Zhou, Q., Sun, Z., Wang, L., Kang, B., Zhang, S., & Wu, X. (2023) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2023.108698 · Cited by 9
Nguyen, S. M., Le, D. V., & Havinga, P. (2023) · IEEE International Conference on Pervasive Computing and Communications · doi:10.1109/percom56429.2023.10099376 · Cited by 9
“In addition, some endeavors [13], [32]±[35] are made to accommodate ViT to other vision tasks.”
BackgroundRelational Space-Time Query in Long-Form Videos
Yang, X., Chu, F.-J., Feiszli, M., Goyal, R., Torresani, L., & Tran, D. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00619 · Cited by 8
“Meanwhile, current video understanding models mostly focus on processing short video clips [12, 13, 52] and solving basic perception tasks such as action recognition [16, 28, 31, 44, 48] and detection [7,19].”
BackgroundMethodAlleviating Spatial Misalignment and Motion Interference for UAV-based Video Recognition
Shi, G., Fu, X., Cao, C., & Zha, Z.-J. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3611799 · Cited by 8
“When compared with a similar model in GFlops MVIT [12], our method can achieve 15.4% higher accuracy, demonstrating the effectiveness.”
MethodInfluentialTwo-stream Multi-level Dynamic Point Transformer for Two-person Interaction Recognition
Liu, Y., Cui, G., Luo, J., Chang, X., & Yao, L. (2024) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3639470 · Cited by 8
“MViT [6] introduces the concept of multiscale in ViT, constructing the model by gradually expanding the channel capacity and reducing spatio-temporal resolution.”
BackgroundVT-Grapher: Video Tube Graph Network With Self-Distillation for Human Action Recognition
Liu, X., Liu, J., Cheng, X., Li, J., Wan, W., & Sun, J. (2024) · IEEE Sensors Journal · doi:10.1109/jsen.2024.3380321 · Cited by 8
“Therefore, we select some state-of-the art methods [5], [13], [32], [34], [35] and adopt lightweight baseline networks (e.g., tiny, small model) to retrain the methods under the same training epoch.”
MethodMultiscale Global Attention for Abnormal Geological Hazard Segmentation
Cai, W., Huang, W., Tian, L., Huang, C., & Yan, J. (2024) · IEEE Sensors Journal · doi:10.1109/jsen.2024.3383233 · Cited by 8
“Inspired by the works on multiscale attention mechanisms [4], [5], a novel MSGAM is proposed for effectively integrating low-and high-level features.”
MethodTouma, S., Hammou, B. A., Antaki, F., Boucher, M. C., & Duval, R. (2024) · International Journal of Retina and Vitreous · doi:10.1186/s40942-024-00555-3 · Cited by 8
MA-VLAD: a fine-grained local feature aggregation scheme for action recognition
Feng, N., Tang, Y. G., Song, Z., Yu, J., Chen, Y. P., & Yang, W. H. (2024) · Multimedia Systems · doi:10.1007/s00530-024-01341-9 · Cited by 9
IRET: Incremental Resolution Enhancing Transformer
Latibari, B. S., Salehi, S., Homayoun, H., & Sasan, A. (2024) · ACM Great Lakes Symposium on VLSI (GLSVLSI) · doi:10.1145/3649476.3660380 · Cited by 9
React: recognize every action everywhere all at once
Chappa, N. V. S. R., Nguyen, P., Dobbs, P. D., & Luu, K. (2024) · Machine Vision and Applications · doi:10.1007/s00138-024-01561-z · Cited by 8
“In the video domain, transformers [5, 8, 18, 20, 30, 34, 54] have demonstrated their effectiveness in learning video representations by utilizing spatial and temporal self-attention.”
Learning Motion-Guided Multi-Scale Memory Features for Video Shadow Detection
Lin, J., Shen, J., Yang, X. S., Fu, H., Zhang, Q., Li, P., Sheng, B., Wang, L., & Zhu, L. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3424219 · Cited by 8
“Restrictions apply. attention [59] has been developed to refine an input feature map by adding a pooling attention mechanism into a self-attention block.”
M-adapter: Multi-level image-to-video adaptation for video action recognition
Li, R., Xu, T., Wu, X., Li, L., Yang, X. P., Shen, Z., & Kittler, J. (2024) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2024.104150 · Cited by 8
LiFT: A Surprisingly Simple Lightweight Feature Transform for Dense ViT Descriptors
Suri, S., Walmer, M., Gupta, K., & Shrivastava, A. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72667-5_7 · Cited by 9
C2C: Component-to-Composition Learning for Zero-Shot Compositional Action Recognition
Li, R., Feng, Z., Xu, T., Li, L., Wu, X., Awais, M., Atito, S., & Kittler, J. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72920-1_21 · Cited by 11
DNTextSpotter: Arbitrary-Shaped Scene Text Spotting via Improved Denoising Training
Qiao, Q., Xie, Y., Gao, J., Wu, T., Huang, S., Fan, J., Cao, Z., Wang, Z., & Zhang, Y. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3680981 · Cited by 12
“With the impressive success of Transformers [38] in visual tasks [8, 9, 17, 21, 30, 31, 46, 51], also influenced by the DETR family [5,19,25,34,50,54], more recent works explore Transformer-based structures for the Text Spot-ting.”
Optimizing Vision Transformers: Unveiling ‘Focus and Forget’ for Enhanced Computational Efficiency
Latibari, B. S., Homayoun, H., & Sasan, A. (2025) · IEEE Access · doi:10.1109/access.2025.3540399 · Cited by 9
SVFormer: A Direct Training Spiking Transformer for Efficient Video Action Recognition
Yu, L., Huang, L., Zhou, C., Zhang, H., Ma, Z., Zhou, H., & Tian, Y. (2025) · Communications in computer and information science · doi:10.1007/978-981-96-4001-0_11 · Cited by 8
Two-Stream Convolutional Neural Network for Video Action Recognition
Qiao, H., Liu, S., Xu, Q., Liu, S., & Yang, W. (2021) · KSII Transactions on Internet and Information Systems · doi:10.3837/tiis.2021.10.011 · Cited by 12
“In addition, [48] proposed a multiscale vision Transformer (MVIT) for video and image recognition, linking the idea of multiscale feature levels with the Transformer model.”
MethodHierarchical Hourglass Convolutional Network for Efficient Video Classification
Tan, Y., Hao, Y., Zhang, H., Wang, S., & He, X. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3547841 · Cited by 8
“H 2 CN only requires 101.4G FLOPs, which is about 12.5 times cheaper than MViT-B (1365G FLOPs) and 8.5 times lower than Video-Swin-B (963G FLOPs).”
MethodInfluentialWLiT: Windows and Linear Transformer for Video Action Recognition
Sun, R., Zhang, T., Wan, Y., Zhang, F., & Wei, J. (2023) · Sensors · doi:10.3390/s23031616 · Cited by 9
“We use the same training strategy as MViT [46].”
BackgroundMethodInfluentialGSWIN: Gated MLP Vision Model with Hierarchical Structure of Shifted Window
Go, M., & Tachibana, H. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10096453 · Cited by 9
“, 2021), MViT (fine-to-coarse aggregation using multi-head pooling attention) (Fan et al., 2021), PVT (pyramid structure for dense prediction) (Wang et al.”
BackgroundLiu, Y., Liu, Y., Yao, Y., & Zhong, M. (2023) · Sensors · doi:10.3390/s23094477 · Cited by 8
“Compared to other visual algorithms, such as Transformer [51], the YOLOv4 Compared to other visual algorithms, such as Transformer [51], the YOLOv4 algorithm requires fewer computational resources, has faster recognition speed, and is more compatible with hardware environments. algorithm requires fewer computational r…”
InfluentialIntentional Microgesture Recognition for Extended Human-Computer Interaction
Kandoi, C., Jung, C., Mannan, S., VanderHoeven, H., Meisman, Q., Krishnaswamy, N., & Blanchard, N. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-35596-7_32 · Cited by 8
“The action recognition models we evaluated were VideoMAE [22], Multiscale Vision Transformers (MViT) [3], 3D ResNet [5], and C3D [23].”
MethodA Light Touch Approach to Teaching Transformers Multi-view Geometry
Bhalgat, Y. S., Henriques, J. F., & Zisserman, A. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00480 · Cited by 8
“There are many such recent examples, where shedding priors in favour of learning from more data has proven to be a successful strategy, from image classification [1,13,20,29,90], action recognition [7, 23, 27, 50, 58], to text-image matching [36, 45, 62, 71] and 3D recognition [40, 91].”
BackgroundForeground/Background-Masked Interaction Learning for Spatio-temporal Action Detection
Chen, K., Shu, X., Xie, G.-S., Yan, R., & Tang, J. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3611945 · Cited by 8
Semantic Video Transformer for Robust Action Recognition
Doshi, K., & Yılmaz, Y. (2023) · IEEE Conference on Dependable and Secure Computing (DSC) · doi:10.1109/dsc61021.2023.10354225 · Cited by 8
Action knowledge for video captioning with graph neural networks
Hendria, W. F., Velda, V., Putra, B. H. H., Adzaka, F., & Jeong, C. (2023) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2023.03.006 · Cited by 8
DIML: Deep Interpretable Metric Learning via Structural Matching
Zhao, W., Rao, Y., Zhou, J., & Lu, J. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3336668 · Cited by 8
“Recently,anumber offollow-ups[18],[31],[33],[34],[65],[72]extendvision Transformerstohierarchicalarchitecturesthataresuitableto moredownstreamtasksincludingsemanticsegmentationand objectdetection.”
Reconstruction of OFDM Signals Using a Dual Discriminator CGAN with BiLSTM and Transformer
Li, Y., Fan, Y., Hou, S., Niu, Y., Fu, Y., & Li, H. (2024) · Sensors · doi:10.3390/s24144562 · Cited by 7
“These models have demonstrated out-standing performance in tasks such as image classification [20], object detection [21], image generation [22], and others.”
BackgroundSelf-citationAdaptive Multi-Scale Transformer Tracker for Satellite Videos
Zhang, X., Jiao, L., Li, L., Liu, X., Liu, F., Ma, W., & Yang, S. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3441038 · Cited by 7
“MViT [43], [44] performed the pooling operations on the input tokens to construct a pyramid architecture.”
BackgroundMultimodal temporal context network for tracking dynamic changes in emotion
Zhang, X., Zhou, J., & Qi, G. (2024) · The Journal of Supercomputing · doi:10.1007/s11227-024-06484-0 · Cited by 7
“For capturing long-range temporal dependencies in video data, we adopt an enhanced MViT [34] framework, a Vision Transformer derivative, as our pre-training model.”
DEVIAS: Learning Disentangled Video Representations of Action and Scene
Bae, K., Ahn, G., Kim, Y., & Choi, J. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73113-6_25 · Cited by 7
“Key approaches to action recognition include 2D CNNs [22,38,48,57,63,89], two-stream CNNs [27,63], 3D CNNs [9,26,27,37,67, 68,73], 2D and 1D separable CNNs [68,81] or more recent transformer architectures [1,5,25,28,33,51,58,77,83].”
MethodMP-FocalUNet: Multiscale parallel focal self-attention U-Net for medical image segmentation
Wang, C., Jiang, M., Li, Y., Wei, B., Li, Y., Wang, P., & Yang, G. (2024) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2024.108562 · Cited by 9
Self-citationTPRNN: A top-down pyramidal recurrent neural network for time series forecasting
Chen, L., Cui, J., Shang, Z., & Cui, D. (2024) · Information Sciences · doi:10.1016/j.ins.2024.121792 · Cited by 7
Enhancing few-shot object detection through pseudo-label mining
Garcia-Fernandez, P., Cores, D., & Mucientes, M. (2024) · Image and Vision Computing · doi:10.1016/j.imavis.2024.105379 · Cited by 7
Robust Motion-Guided Frame Sampler With Interpretive Evaluation for Video Action Recognition
Bai, J., Zhang, Y., Wang, Y., Xiao, Z. Q., Xiong, Y., & Jiao, L. (2025) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2025.3541580 · Cited by 8
LowFormer: Hardware Efficient Design for Convolutional Transformer Backbones
Nottebaum, M., Dunnhofer, M., & Micheloni, C. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00681 · Cited by 11
“Others [9, 41, 43] downsample the keys K and V before the attention operation, either with convolutions or pooling .”
A comprehensive review of on-board action recognition models in public transportation systems
Meurie, C., & Lézoray, O. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.128311 · Cited by 9
Viewport prediction with cross modal multiscale transformer for 360° video streaming
Tian, Y., Zhong, Y., Han, Y., & Chen, F. (2025) · Scientific Reports · doi:10.1038/s41598-025-16011-7 · Cited by 7
“Multi-head grouping attention is a self-attention operation that enables flexible resolution modeling in transformer blocks 21 .”
How AI-based computer vision algorithms are impacting the sports industry: a survey
Rossi, L. F., Sanna, A., Manuri, F., & Bianco, M. D. (2025) · Discover Artificial Intelligence · doi:10.1007/s44163-025-00586-1 · Cited by 9
Liu, F., Tian, C., Wang, J., Jin, Y., Luxiang, C., & Lee, I. (2023) · Mathematics · doi:10.3390/math11143074 · Cited by 7
BFRNet: Bidimensional Feature Representation Network for Remote Sensing Images Classification
Wang, W., Hu, T., Wang, X., & Li, J. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3313800 · Cited by 7
“In order to reduce the number of tokens used in computing MHSA, MViT employed a single pooling operation [31].”
MethodMake a Long Image Short: Adaptive Token Length for Vision Transformers
Zhou, Q., & Zhu, Y. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-43415-0_5 · Cited by 8
Cross-attention Spatio-temporal Context Transformer for Semantic Segmentation of Historical Maps
Wu, S., Chen, Y., Schindler, K., & Hurni, L. (2023) · ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems (SIGSPATIAL GIS) · doi:10.1145/3589132.3625572 · Cited by 7
“Instead of hard-set positional encoding, we add to the features the learnable positional embeddings as [14] before the attention blocks.”
MethodDeep video representation learning: a survey
Ravanbakhsh, E., Liang, Y., Ramanujam, J., & Li, X. S. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-17815-3 · Cited by 8
MaskConver: Revisiting Pure Convolution Model for Panoptic Segmentation
Rashwan, A. M., Zhang, J., Taalimi, A., Yang, F., Zhou, X., Yan, C., Chen, L.-C., & Li, Y. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00090 · Cited by 7
“Transformers have surpassed ConvNets on several vision problems beyond panoptic segmentation [32, 42] including classification [16, 18, 20, 52, 53, 75, 80], detection [6,21,41,44,84], and segmentation [17,63,73,79,88].”
MethodSelf-citationDifferentially Private Video Activity Recognition
Luo, Z., Zou, Y., Yang, Y., Durante, Z., Huang, D.-A., Yu, Z., Xiao, C., Fei-Fei, L., & Anandkumar, A. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00652 · Cited by 10
“These modules are initialized as identity mappings and gradually trained to stabilize model training.”
BackgroundMethodInfluentialEdiTor: Edge-guided Transformer for Ghost-free High Dynamic Range Imaging
Guan, Y., Xu, R., Yao, M., Huang, J., & Xiong, Z. (2024) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3657293 · Cited by 7
“The great success of single-image transformers also inspires multi-frame transformers [8, 39].”
MLENet: Multi-Level Extraction Network for video action recognition
Wang, F., Li, X., Xiong, H., Mo, H., & Li, Y. (2024) · Pattern Recognition · doi:10.1016/j.patcog.2024.110614 · Cited by 7
Self-citationSkatingVerse: A large‐scale benchmark for comprehensive evaluation on human action understanding
Gan, Z., Jin, L., Cheng, Y., Cheng, Y., Teng, Y., Li, Z., Li, Y., Yang, W., Zhu, Z. D., Xing, J., et al. (2024) · IET Computer Vision · doi:10.1049/cvi2.12287 · Cited by 8
Self-citationContextSeg: Sketch Semantic Segmentation by Querying the Context with Attention
Wang, J., & Li, C. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00353 · Cited by 7
Kung, C.-H., Lu, S.-W., Tsai, Y., & Chen, Y.-T. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01746 · Cited by 9
“We downsample the input image to 224 × 224 for both MViT [13] and VideoMAE [30] on all datasets.”
Peng, D., Feng, H., Wu, J., Wen, Y. F., Han, T., Li, Y., Yang, G., & Qu, L. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3474736 · Cited by 7
Self-citationCross-view action recognition understanding from exocentric to egocentric perspective
Truong, T.-D., & Luu, K. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128731 · Cited by 10
LookupViT: Compressing Visual Information to a Limited Number of Tokens
Koner, R., Jain, G., Jain, P., Volker Tresp, C., & Paul, S. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73016-0_19 · Cited by 13
LI3D-BiLSTM: A Lightweight Inception-3D Networks with BiLSTM for Video Action Recognition
Wang, F., Jin, X., & Yi, S. (2024) · ICCK Transactions on Emerging Topics in Artificial Intelligence · doi:10.62762/tetai.2024.628205 · Cited by 7
“To address this issue, Fan et al. [14] proposed the Multiscale Vision Transformer (MViT), which achieves multiscale modeling through hierarchical pooling, maintaining high performance while improving computational efficiency.”
LMF-Net: A Learnable Multimodal Fusion Network for Semantic Segmentation of Remote Sensing Data
Li, J., Zhang, W., Zhang, W., Zhou, R., Li, C., Tong, B., Sun, X., & Fu, K. (2025) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2025.3527213 · Cited by 9
“Forthisreason,manyscholarshavealso exploredsolutionsrepresentedbypoolingoperatortoreduce thecomputationcomplexityofMLP-liketransformer[60],[61], [62],[63].”
VC-Mamba: Causal Mamba representation consistency for video implicit understanding
Hu, Y., Zhao, J., Qi, C., Qiang, Y., Zhao, J., & Pei, B. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113437 · Cited by 7
LATTE: A Real-time Lightweight Attention-based Traffic Accident Anticipation Engine
Zhang, J., Guan, Y., Wang, C., Liao, H., Zhang, G., & Li, Z. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103173 · Cited by 7
“FLOPs (M) Params (M) UniFormerV2 (Li, Wang, He, Li, Wang, Wang and Qiao, 2022) 3600000.00 115.00 VideoSwin (Liu, Ning, Cao, Wei, Zhang, Lin and Hu, 2022) 282000.00 88.10 MViTv2 (Fan, Xiong, Mangalam, Li, Yan, Malik and Feichtenhofer, 2021) 206000.00 51.00 DSTA (Karim et al., 2022) 8868 4).”
InfluentialHaihong, E., Ding, J., & Yuan, L. (2022) · International Conference on Image Processing, Computer Vision and Machine Learning (ICICML) · doi:10.1109/icicml57342.2022.10009714 · Cited by 8
“With the amazing performance of the transformer model in main tasks of natural language processing [19-23], researchers in the field of computer vision began to think about the application of the self-attention method in classical CV tasks including image classification [29,30,34], target detection [31,32,33].”
MethodSelf-supervised Video-centralised Transformer for Video Face Clustering
Wang, Y., Dong, M., Shen, J., Luo, Y., Lin, Y., Ma, P., Petridis, S., & Pantić, M. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3243812 · Cited by 8
“[58] propose a multi-scale ViT that can efficiently operate on videos.”
BackgroundMAST: Multiscale Audio Spectrogram Transformers
Ghosh, S., Seth, A., Umesh, S., & Manocha, D. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10095023 · Cited by 8
“This setting is similar to the original MViT [14].”
MethodResultInfluentialCo-Salient Object Detection with Semantic-Level Consensus Extraction and Dispersion
Xu, P., & Mu, Y. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612133 · Cited by 7
“Afterward, a large number of ViT variants and related techniques have been proposed, such as [2, 9, 13, 19, 22, 31, 32, 35].”
BackgroundToward Privacy-Supporting Fall Detection via Deep Unsupervised RGB2Depth Adaptation
Xiao, H., Peng, K., Huang, X., Roitberg, A., Li, H., Wang, Z., & Stiefelhagen, R. (2023) · IEEE Sensors Journal · doi:10.1109/jsen.2023.3322479 · Cited by 8
“Action recognition approaches can be categorized into video-based methods [36], [37], [38], [39], [39], [40], [41], [42], [43], [44], [45] and skeleton-based methods [46], [47], [48], [49], [50], [51], [52], where the latter group is more closely related to our work.”
SAKD: Sparse attention knowledge distillation
Guo, Z., Zhang, P.-Z., & Liang, P. (2024) · Image and Vision Computing · doi:10.1016/j.imavis.2024.105020 · Cited by 9
Listen to Look Into the Future: Audio-Visual Egocentric Gaze Anticipation
Lai, B., Ryan, F., Jia, W., Liu, M., & Rehg, J. M. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72673-6_11 · Cited by 7
Mutual modality learning for video action classification
Komkov, S. A., Dzabraev, M., & Petiushko, A. (2023) · Computer Optics · doi:10.18287/2412-6179-co-1277 · Cited by 8
Krishnan, K. S., & Krishnan, K. S. (2022) · IEEE Annual Computing and Communication Workshop and Conference (CCWC) · doi:10.1109/ccwc54503.2022.9720737 · Cited by 7
“We tested out R(2+1)D[6], SlowFast[11], X3D[7], MViT[8], C2D[9], CSN[10] and SlowFast ResNet50[11] pre-trained models and fine-tuned it on the combined dataset.”
BackgroundMethodInfluentialTriplet Temporal-based Video Recognition with Multiview for Temporal Action Localization
Le, H. D., Vu, M. Q., Tran, M. T., & Phúc, N. V. (2023) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00573 · Cited by 7
“Feature Extraction Backbone Models : We employ the X3D-L model and MViTv2-B model, pre-trained on the Ki-netics dataset [9].”
BackgroundMethodInfluentialCategory-Specific Prompts for Animal Action Recognition with Pretrained Vision-Language Models
Jing, Y., Wang, C., Zhang, R., Liang, K., & Ma, Z. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612551 · Cited by 6
Ni, K., Yuan, C., Zheng, Z., Zhang, B., & Wang, P. (2024) · IEEE Transactions on Aerospace and Electronic Systems · doi:10.1109/taes.2024.3382622 · Cited by 6
“…MPT-SFANet with FPN [53] for image segmentation, statistical convolutional neural network (SCNN) [58], wavelet-driven subspace basis learning network (WDSBLN) [59], and other transformer-basednetworks,includingMViT [34],transformer-MLP(T-MLP), transformer-FCN (T-FCN), and transformer-UNet (T-UNet).”
BackgroundGaitGMT: Global feature mapping transformer for gait recognition
Chen, G., Huang, J., Chen, G., Chen, X., Deng, X., Lan, Y., Long, Y., & Tian, Q. C. (2024) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2024.104139 · Cited by 6
Improving AI-assisted video editing: Optimized footage analysis through multi-task learning
Li, Y., Xu, H., Cai, F., & Tian, F. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128485 · Cited by 6
Self-citationUnifying Global and Local Scene Entities Modelling for Precise Action Spotting
Tran, K. H., Do, P. V., Ly, N. Q., & Le, N. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10650009 · Cited by 11
“To efficiently extract the global environment feature in both spatial and temporal dimensions with less computational cost, we utilize a 2D backbone network [1], [12], [36] and a time-shift module [43].”
MethodBackground debiased class incremental learning for video action recognition
Nguyen, L. Q., Choi, J., Dang, L. M., & Moon, H. (2024) · Image and Vision Computing · doi:10.1016/j.imavis.2024.105295 · Cited by 7
Yang, X., Qiao, Y., Si, T., Wang, J., & Xu, T. (2025) · Pattern Recognition · doi:10.1016/j.patcog.2025.111590 · Cited by 6
Mishra, D., Saha, P., Zhao, H., Hernandez-Cruz, N., Patey, O. V., Papageorghiou, A. T., & Noble, J. A. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103611 · Cited by 7
KFF: K-feature fusion token merging for vision transformer
Yang, Y., Zhou, Y., Hu, X., & Duan, S. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.128206 · Cited by 6
Zhang, R., Yang, Z., Dai, T., Sun, D., Ma, S., & Zhao, X. (2025) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2025.3566502 · Cited by 6
Huo, Y., Deng, Z., Zhang, P., Lu, Z., Zhang, F., Li, M., Hou, Y., Xi, P., Huang, W., & Zhang, Y. (2025) · Pattern Recognition · doi:10.1016/j.patcog.2025.111865 · Cited by 6
Towards Student Actions in Classroom Scenes: New Dataset and Baseline
Tan, Z., Gao, C., Qin, A., Chen, R., Song, T., Yang, F., & Meng, D. (2025) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2025.3590899 · Cited by 6
“The existing deep learning-based video action detection methods can be mainly divided into CNN-based [35], [36], [37] and transformer-based methods [38], [39], [40].”
Long, Q., Yao, D., Yang, J., Dong, J., Bin, Z., & Dai, Y. (2025) · Advanced Engineering Informatics · doi:10.1016/j.aei.2025.103933 · Cited by 6
Action recognition: A comprehensive survey of tasks, methods, and challenges
Heo, S., Moon, J., & Jung, S. K. (2025) · ICT Express · doi:10.1016/j.icte.2025.11.015 · Cited by 7
KUŞ, İ., Koçak, C., & Keleş, A. (2025) · Intelligent Systems with Applications · doi:10.1016/j.iswa.2025.200615 · Cited by 11
Gazzoni, M., Salvia, M. L., Torti, E., Secco, G., Perlini, S., & Leporati, F. (2023) · Bioengineering · doi:10.3390/bioengineering10030282 · Cited by 6
“Multiscale Vision Transformer (MViT) [31]: can classify videos joining multiscale feature hierarchies with transformer models.”
Compressed Vision for Efficient Video Understanding
Wiles, O., Carreira, J., Barr, I. R., Zisserman, A., & Malinowski, M. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-26293-7_40 · Cited by 6
Dual attentional transformer for video visual relation prediction
Qu, M., Deng, G., Di, D., Cui, J., & Su, T. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.126372 · Cited by 8
Xu, X., Li, Y., & Lu, C. (2023) · International Journal of Computer Vision · doi:10.1007/s11263-023-01850-6 · Cited by 6
Self-citationDPT‐tracker: Dual pooling transformer for efficient visual tracking
Fang, Y., Xie, B., Khairuddin, U. B., Min, Z., Jiang, B., & Li, W. (2024) · CAAI Transactions on Intelligence Technology · doi:10.1049/cit2.12296 · Cited by 6
Tang, J., Gong, Z., Tao, B., & Yin, Z. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102390 · Cited by 7
P-Age: Pexels Dataset for Robust Spatio-Temporal Apparent Age Classification
Ali, A., Marisetty, A., & Brémond, F. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00841 · Cited by 6
“Precision/Recall F1-Score X3D [15] 77.61% 0.80 / 0.76 0.77 Slow-Fast [16] 78.11 % 0.79 / 0.78 0.78 I3D [7] 81.59% 0.82 / 0.82 0.82 MViT [14] 86.00% 0.86 / 0.87 0.86 TimeSformer [6] 87.56% 0.88 / 0.88 0.88 Proposed 89.55% 0.90 / 0.90 0.90 Table 3.”
Influential
Metadata
- Authors
- Fan, H., Xiong, B., Mangalam, K., Li, Y., Yan, Z., Malik, J., & Feichtenhofer, C.
- Affiliations
- Meta (Israel) (IL)
- Meta (United States) (US)
- Venue
- 2021 IEEE/CVF International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2021-10
- Pages
- 6804-6815
- OpenAlex
- W3202698164
- Cited by
- 1,407
- References
- 111
- Citations by year
- 2026: 147 · 2025: 345 · 2024: 380 · 2023: 349 · 2022: 176 · 2021: 10
- FWCI
- 74.81
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- arXiv (Cornell University)
- TIB Data Manager
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- video recognition, image classification, feature pyramid, spatial resolution reduction, computational efficiency
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
