The DOI matches its Crossref record; 106 of the 111 references it cites were found in a registry or an index.
Multiscale Vision Transformers
Open accessAbstract
We present Multiscale Vision Transformers (MViT) for video and image recognition, by connecting the seminal idea of multiscale feature hierarchies with transformer models. Multiscale Transformers have several channel-resolution scale stages. Starting from the input resolution and a small channel dimension, the stages hierarchically expand the channel capacity while reducing the spatial resolution. This creates a multiscale pyramid of features with early layers operating at high spatial resolution to model simple low-level visual information, and deeper layers at spatially coarse, but complex, high-dimensional features. We evaluate this fundamental architectural prior for modeling the dense nature of visual signals for a variety of video recognition tasks where it outperforms concurrent vision transformers that rely on large scale external pre-training and are 5-10× more costly in computation and parameters. We further remove the temporal dimension and apply our model for image classification where it outperforms prior work on vision transformers. Code is available at: https://github.com/facebookresearch/SlowFast.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 2×
“As in original ViT [25], we use residual connections [45] and Layer Normalization (LN) [2] in the pre-normalization configuration that applies LN at the beginning of the residual function, and our MLPs consist of two linear layers with GELU activation [48], where the first layer expands the dimension from D to 4D, and…”
MethodResultVaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 6×
“In contrast to original Multi Head Attention (MHA) operators [98], where the channel dimension and the spatiotemporal resolution remains fixed, MHPA pools the sequence of latent tensors to reduce the sequence length (resolution) of the attended input.”
BackgroundMethodInfluentialVery Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 1×
“Incorporating downsampling, shift invariance, and shared weights, ConvNets are de-facto standard backbones for computer vision tasks for image [65, 62, 88, 90, 46, 12, 15, 34, 93, 81, 41] and video [87, 31, 11, 79, 69, 106, 96, 30, 105, 35, 29, 117, 57].”
BackgroundImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 1×
“Incorporating downsampling, shift invariance, and shared weights, ConvNets are de-facto standard backbones for computer vision tasks for image [70, 67, 94, 96, 51, 15, 18, 39, 99, 87, 46] and video [93, 36, 14, 85, 74, 112, 102, 34, 111, 40, 33, 123, 62].”
BackgroundFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534 · cited 1×
“Finally, compared to the best model of concurrent Swin [79] (which was designed for image recognition), MViT has +0.”
ResultRethinking the Inception Architecture for Computer Vision
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.308 · Cited by 32,138
Research works printed but not linked: 11
- gao, 2019, Res2net: A new multi-scale backbone architecture, IEEE PAMI
- fan, 2020, Pyslowfast
- feichtenhofer, 2019, SlowFast networks for video recognition in ActivityNet challenge 2019
- 10.1109/CVPR.2009.5206848 · DOI · Paper page
- 10.1109/CVPR.2019.00033 · DOI
- hang, 2020, ResNeSt Split-Attention Networks
- 10.1109/ICCV.2019.00718 · DOI
- 10.1109/CVPR.2015.7298594 · DOI · Paper page
- carion, 2020, Endto-end object detection with transformers, Proc ECCV, 213
- loshchilov, 2018, Fixing weight decay regularization in adam
- qiu, 2017, Learning spatiotemporal representation with pseudo-3d residual networks, Proc ICCV
Cited by
Sorted by citations
Items 901–1,000 of 1,003
Pedestrian Crossing Intent Prediction Using Vision Transformers
Elgazwy, A., Elmoghazy, S., Elgazzar, K., & Khamis, A. M. (2024) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc58415.2024.10919958 · Cited by 2
“Although ViTs are gaining popularity in image classification tasks [20]–[23] and also in object detection tasks [24]–[26], they have not been used for pedestrian intention prediction to date.”
Importance-Guided Interpretability and Pruning for Video Transformers in Driver Action Recognition
Panadero, R., Schörkhuber, D., & Gelautz, M. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00517 · Cited by 2
“To address this, recent research has explored hierarchical structures in video transformers, exemplified by models like MViT [14, 19] and Video Swin Transformer [23], aiming to represent video more efficiently.”
Text-Scene Retrieval for Driving Scenes in Transportation Cyber-Physical Systems
Song, T., Mou, Z., Yang, Z., Chen, X., & Zheng, Y. (2025) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2025.3562058 · Cited by 2
“CUSA [22] urther combines unimodal and cross-modal alignment techniques, introducing additional supervisory signals for local semantic modeling to mitigate the adverse effects of false negatives on retrieval performance.”
Reddy, S., Poddar, N., Das, S., & Das, A. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-87657-8_27 · Cited by 2
Generalized Concordant Vision Transformer With Masked Image Tokens for Object Detection
Quan, Y., Zhang, D., & Tang, J. (2025) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2025.3570504 · Cited by 2
“…(CMPA) that enhances the model’s perception of the omni-feature representations and position embedding information through the built-in cross-attention (CA) block and multi-pooling attention [40] (MLP) block, thereby strengthening the correlation between semantic content and positional information.”
InfluentialMIM4D: Masked Modeling with Multi-View Video for Autonomous Driving Representation Learning
Zou, J., Liao, B., Zhang, Q., Liu, W., & Wang, X. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02464-w · Cited by 7
“A series of works [4,8,11,55], follow the contrastive paradigm.”
Leveraging negative correlation for Full-Range Self-Attention in Vision Transformers
Long, W., Chen, Z., Li, W., Zhang, Y., Yao, H., Peng, J., & Cui, Z. (2025) · Pattern Recognition · doi:10.1016/j.patcog.2025.111899 · Cited by 2
Near-Earth asteroid detection using video transformer networks
Wolfe, S., & Emami, M. R. (2025) · Aerospace Science and Technology · doi:10.1016/j.ast.2025.110495 · Cited by 2
Temporal Attention-based Vision Transformer for Source-Free Video Unsupervised Domain Adaptation
Yosry, S., Elrefaei, L. A., ElKamaar, R., & Ziedan, R. R. (2025) · Discover Applied Sciences · doi:10.1007/s42452-025-06909-2 · Cited by 3
“Over the last decade, large-scale, varied video datasets [6–8] have emerged, and deep networks and deep learning [9, 10] have advanced video representations.”
EAAR: Efficient and Accurate Action Recognition model with enhanced spatio-temporal perception
Chen, S., Wang, X., Sun, Y., Yang, K., & Wei, X. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.107737 · Cited by 2
Computer Vision and Deep Learning for Sports Analytics
Müller-Budack, E., Theiner, J., & Ewerth, R. (2025) · Springer eBooks · doi:10.1007/978-3-662-70155-3_5 · Cited by 2
KMANet: A spatio-temporal enhancement network for micro-action recognition
Zhou, J. Y., Yao, J., Su, N., Lu, J., Yu, Q., Zhang, Y., & Hu, W. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.114139 · Cited by 2
Wang, X., Kong, D., Li, J., Wang, J., & Yin, B. (2025) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2025.3595145 · Cited by 2
“The latest trend in the field is the use of visual transformers for learning spatial and temporal relationships via self-attention mechanisms [33], [34], [35], [36], [37].”
Adaptive feature decoupled network for polyp segmentation
Zou, S., Wang, G., Zhang, Y., Shen, Y., Yuan, M., & Su, Y. (2025) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2025.108327 · Cited by 2
Are you Struggling? Dataset and Baselines for Struggle Determination in Assembly Videos
Feng, S., Wray, M., Sullivan, B., Jang, Y., Ludwig, C. J. H., Gilchrist, I. D., & Mayol-Cuevas, W. W. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02559-4 · Cited by 4
“…such as ViViT (Arnab et al., 2021) and Video Transformer Networks (VTN) (Neimark, Bar, Zohar, & Asselmann, 2021), and the integration of spatial and temporal attention modules such as TimeSformer (Bertasius, Wang, & Torresani, 2021), MViTv1 (Fan et al., 2021), and MViTv2 (Y. Li et al., 2022).”
WMSA–WBS: Efficient Wave Multi-Head Self-Attention with Wavelet Bottleneck
Li, X., Li, Y., Fan, P., & Zhang, X. (2025) · Sensors · doi:10.3390/s25165046 · Cited by 2
“To address the above problems, many researchers have attempted to design more efficient self-attention mechanisms [12–15].”
Self-citationLanPPT: Enhancing landslide crack detection through pyramid pooling transformers
Sreelakshmi, S., & S, V. C. S. (2025) · Applied Soft Computing · doi:10.1016/j.asoc.2025.113765 · Cited by 2
Li, M., Zhang, X., Liu, Y., Wang, H., Sun, Y., Zhang, P., & Wang, Q. (2025) · Journal of Forecasting · doi:10.1002/for.70023 · Cited by 3
A multi-scale adaptive fusion model for multimodal sarcasm detection
Wu, H., & Zang, Y. (2025) · Discover Computing · doi:10.1007/s10791-025-09730-y · Cited by 2
“• MVIT [12]: A Transformer-based video analysis model used for capturing complex dynamic features in videos.”
Chakravarthy, S. R. S., & Rajaguru, H. (2025) · Scientific Reports · doi:10.1038/s41598-025-21360-4 · Cited by 2
Enhanced Multi-scale Hierarchical Network for Micro-expression Recognition
Yip, Y. H., & Hu, J. (2025) · Lecture notes in computer science · doi:10.1007/978-981-95-3729-7_4 · Cited by 2
${\text{CA}^{2}\text{ST}}$: Cross-Attention in Audio, Space, and Time for Holistic Video Recognition
Lee, J., Chang, J., Lee, D., & Choi, J. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3628653 · Cited by 6
“Recently, Transformer-based approaches [3], [36], [37], [38], [39], [40], [41] become popular in the community due to the long-term context modeling capabilities.”
Kang, S., Huo, H., Xu, J., Mei, A., & Zhang, C. (2025) · Sensors · doi:10.3390/s25226849 · Cited by 2
“Multi-scale Vision Transformers (MViTs) [13] designed multi-scale attention to suppress background noise.”
Hou, P., Li, G., Cai, Z., Zhang, J., & Huang, D. (2025) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2025.3638017 · Cited by 2
Diao, R. (2025) · Scientific Reports · doi:10.1038/s41598-025-29586-y · Cited by 3
“Consequently, the Multi-Scale Vision Transformer (MViT) struggles to discern whether the player is positioned just inside or beyond the three-point line leading to inconsistent maneuver classification.”
InfluentialVisual Semantic Knowledge Discovery for Multimodal Intent Recognition
Cheng, Y. (2025) · IEEE Access · doi:10.1109/access.2025.3642324 · Cited by 2
“The introduction of the attention mechanism [17], [41], [42], [43], [44] can effectively model relationships between elements regardless of spatiotemporal distance and reduce computational costs, leading to a shift in backbone architectures from CNNs to Transformers [45].”
InfluentialWang, Z., Peng, Y., Zhang, X., & Sun, H. (2025) · IEEE Internet of Things Journal · doi:10.1109/jiot.2025.3642607 · Cited by 2
“Multiscale ViTs (MViTs) [28] combine the idea of multiscale feature extraction with a Transformer, achieving multiscale feature extraction through multiple channel-resolution scale stages.”
MLAANet: A multi-level attention-aware network for pathological nuclei segmentation
Zhu, S., Li, Y., Lu, L., Mao, T., Yan, Y., & Dai, X. (2025) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2025.108716 · Cited by 2
Self-citationECViT: Efficient Convolutional Vision Transformer with Local-Attention and Multi-scale Stages
Qian, Z. (2025) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn64981.2025.11227612 · Cited by 2
“ViT and its variants have demonstrated state-of-the-art performance for various tasks including image classification, detection, and segmentation [3]–[5], [21], [23]–[29], [31], [34].”
DL-VLM: A Dynamic Lightweight Vision-Language Model for Bridge Health Diagnosis
Liang, S., He, Z., Gui, H., & Liu, F. (2025) · Big Data and Cognitive Computing · doi:10.3390/bdcc10010003 · Cited by 2
“Beyond VLMs, efficiency improvements have been explored through techniques such as dynamic sampling, and multi-scale feature fusion [23–25].”
He, Y., Huang, Y., Guo, C., Lü, L., Pei, B., Xu, J., Lu, T., & Sato, Y. (2026) · International Journal of Computer Vision · doi:10.1007/s11263-025-02608-y · Cited by 9
“Extending egocentric and exocentric collaboration techniques to domains such as medicine and industry could unlock new opportunities in these fields.”
EinsPT: Efficient Instance-Aware Pre-Training of Vision Foundation Models
Wang, Z., Tian, Y., Xie, L., Wang, Y., & Ye, Q. (2026) · IEEE Transactions on Image Processing · doi:10.1109/tip.2026.3652371 · Cited by 2
“For example, several methods incorporate convolutional biases into transformer designs to capture local context more e ff ectively [50], [51], while others adopt hybrid structures [52], [53], hierarchical feature pyramids [3], [54], or windowed / local self-attentions [3], [17], [55] to address…”
Ma, D., Zhang, C., Zhou, G., & Ma, C. (2026) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2026.3650942 · Cited by 2
“To address the above issues, leveraging the benefits from multiscale data [10], [11], and also inspired by a famous philosophy, “Think big, start small,” this article proposes a novel ensemble encoder-enabled proactive HAIR approach to enhance the e ffi cient and secure HRC assembly, shown in Fig.”
Kumar, S. (2025) · IEEE Annual Information Technology, Electronics and Mobile Communication Conference (IEMCON) · doi:10.1109/iemcon67450.2025.11381265 · Cited by 2
“Recent advances such as TinyBERT, MobileViT, DistilBERT [22], [23], [24], and EfficientFormer aim to reduce model size through pruning, quantization, and architecture redesign.”
Tarashima, S., Wang, Y., & Tagawa, N. (2025) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw69036.2025.00007 · Cited by 2
“VFMs can be broadly categorized into non-hierarchical models [2, 6, 15, 19, 76, 84, 86, 95, 104, 105] and hierarchical models [18, 24, 44, 55, 56, 80, 89, 90, 112, 115].”
MVPC-CLIP: Multi-Granularity Visual Prompt Co-Operative for Aerial Video Recognition
Zhan, Y., & Yuan, Y. (2026) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2026.3668495 · Cited by 5
“Fan et al. [31] combined multi-scale feature hierarchies with Transformer and proposed multi-scale vision transformer (MViT) for video recognition.”
SAM2-RoadNet: Topology-Aware Multi-Scale Road Extraction from High-Resolution Remote Sensing Images
Feng, R., Guo, Z., Du, X., & Wu, T. (2026) · Remote Sensing · doi:10.3390/rs18060913 · Cited by 2
Snakes and Ladders: Two Steps Up for VideoMamba
Lü, H., Salah, A. A., & Poppe, R. W. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.02246 · Cited by 2
Ahmed, S., Li, J., Zhuang, W., Chen, C., & Lyu, L. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.01968 · Cited by 2
“Building on this progress, numerous vision trans-formers have emerged [9, 33], delivering remarkable results across various benchmarks.”
Learning Unified Spatio-temporal Representations for Efficient Compressed Video Understanding
Biswas, S. D., Soufleri, E., Roy, A., & Roy, K. (2026) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61042.2026.00462 · Cited by 2
“Fully transformer-based architectures followed after [22, 36, 52–54], achieving state-of-the-4756 art performance across benchmarks.”
TM-Adapter: Temporal Merge Adapter for Efficient Global Temporal Modeling
Hahm, W. J., Jang, S., Kim, H. T., Lee, D., & Kim, K. (2026) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61042.2026.00592 · Cited by 2
“More recently, the emergence of Vision Transformers (ViTs) [14, 51], known for their superior generalization and transferability, has led to their widespread adoption in video understanding tasks [1, 2, 6, 15, 31–33, 37, 41, 44, 45, 55].”
Jeong, D., Soh, M.-S., Jeon, J., Jeong, H., Cho, J., Kim, J., Lee, J., Lee, S.-A., Shin, J.-H., Jang, Y., et al. (2026) · Journal of the American Society of Echocardiography · doi:10.1016/j.echo.2026.05.050 · Cited by 2
Improving Video Model Transfer with Dynamic Representation Learning
Li, Y., & Vasconcelos, N. M. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01868 · Cited by 2
“These include pooling convolutional features with self-attention [16, 85], or in more recent works [1, 5, 23], replacing all convolutional layers with Transformer blocks [21, 83].”
Self-citationPrecise Gymnastic Scoring from TV Playback
Li, Q., Cui, Z., Kitahara, I., & Sagawa, R. (2022) · IEEE Global Conference on Consumer Electronics (GCCE) · doi:10.1109/gcce56475.2022.10014208 · Cited by 2
“The Transformer-based regressor [21] has 16 blocks, each containing eight headers.”
MethodSpatio-temporal Analysis of Dashboard Camera Videos for Time-To-Accident Forecasting
Anjum, T., Kumar, D., & Narayan, A. (2023) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn54540.2023.10191967 · Cited by 2
“Multiscale vision transformers (MViTs) introduced by [22], combine the idea of multiscale feature hierarchies with transformer models.”
BackgroundZhang, H., & Chen, H. (2023) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi53787.2023.10230683 · Cited by 2
MFT: Multi-scale Fusion Transformer for Infrared and Visible Image Fusion
Zhang, C.-M., Yuan, C., Luo, Y., & Zhou, X. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-44223-0_39 · Cited by 2
Action recognition based on adaptive region perception
Lu, T., Yang, Q., Min, F., & Zhang, Y. (2023) · Neural Computing and Applications · doi:10.1007/s00521-023-09069-9 · Cited by 2
Mitigating Context Bias in Action Recognition via Skeleton-Dominated Two-Stream Network
Li, Q., Huang, X., Luo, Y., Hu, X., Sun, X., & Wang, Z. (2023) · doi:10.1145/3606042.3616458 · Cited by 2
“The concatenation results x cat and squeeze result x squ are defined as where H ( · ) represents the concatenation, ω ∈ [2 , 5] and ω with an inital value of 2.”
BackgroundMethodMulti-Modal Human Action Segmentation Using Skeletal Video Ensembles
Dickens, J., & Payeur, P. (2023) · International Electronic Conference on Sensors and Applications (ECSA) · doi:10.3390/ecsa-10-16257 · Cited by 2
A novel dual-granularity lightweight transformer for vision tasks
Zhang, J., Yu, M., Lu, W., Dai, Y., Shi, H., & You, R. (2024) · Intelligent Data Analysis · doi:10.3233/ida-230799 · Cited by 2
“Some studies (such as MViT [41], MViTv2 [42] and CrossVit [43]) have shown that integrating multi-scale feature information with the Transformer model can effectively improve its performance.”
Li, X., He, Z., Huang, K., Yang, Z., & Zhang, G. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10448135 · Cited by 2
“Furthermore, these GNN-based models leverage attention mechanisms to capture and assess information across nodes within a graph at either the individual point level or the overarching series level, making it less practical for large-scale datasets[9].”
BackgroundToken singularity understanding and removal for transformers
Wang, D., Jiao, L., Zhang, R., Yang, S., & Liu, F. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.111718 · Cited by 2
Embedding Task Structure for Action Detection
Peven, M., & Hager, G. D. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00647 · Cited by 3
“Recent transformer based methods [2,17,23,51] are able to model these dynamics but require enormous computational cost.”
MethodTewolde, T. T., Manjotho, A. A., & Niu, Z. (2024) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2024.3387344 · Cited by 2
“In contrast to ViT [53], which is based on single-scale blocks, MViTv1 [54] introduces four-scale stages for low-and high-levelvisualfeatures,similartoResNet[14].”
InfluentialMulti-granularity hypergraph-guided transformer learning framework for visual classification
Jiang, J., Chen, Z., Lei, F., Xu, L., Huang, J., & Yuan, X. (2024) · The Visual Computer · doi:10.1007/s00371-024-03541-w · Cited by 2
“The swim transformer models low-level visual information in the early stages and learns high-level complex features in deeper layers [43].”
DTA: Deformable Temporal Attention for Video Recognition
Lei, X., Yang, M., Xi, G., Liu, Y., Li, J., Zhang, L., & Tian, Y. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10650436 · Cited by 2
“To address the multi-scale modeling demands of dense prediction tasks, MViT [10] utilizes pooling self-attention to downsample the tensors.”
MethodCounterfactual Gradients-based Quantification of Prediction Trust in Neural Networks
Prabhushankar, M., & AlRegib, G. (2024) · IEEE International Conference on Multimedia Information Processing and Retrieval (MIPR) · doi:10.1109/mipr62202.2024.00091 · Cited by 2
“95 for AlexNet and > 0 .”
Pixel-REfocused Navigated Tri-margin for Semi-supervised Action Detection
Liu, W., Zhao, S., Han, X., Yi, A., Jiang, K., Wang, Z., & Zhong, X. (2024) · International Workshop on Human-centric Multimedia Analysis (HCMA) · doi:10.1145/3688865.3689478 · Cited by 2
“The former relies on manual detectors to annotate action regions on keyframes [5–7, 24, 30], as seen in works like Slow-Fast [7] and X3D [6], which involve two-stage training and significant human and computational resources.”
Transformers Fusing Wavelet with High-Frequency Enhancement for SAR Ship Detection
Wang, Z., Yang, J.-L., & Liu, J. (2024) · Lecture notes in computer science · doi:10.1007/978-981-96-0128-8_11 · Cited by 2
He, H., Liu, W., & Xing, W. (2024) · Lecture notes in computer science · doi:10.1007/978-981-96-0908-6_15 · Cited by 2
“Transformer-based methods [1, 3, 9, 28, 59] leverage the capabilities of self-attention mechanism to capture long-range temporal dependencies.”
Ha, M., Bae, W.-G., Bae, G., & Lee, J. T. (2024) · Lecture notes in computer science · doi:10.1007/978-981-96-0960-4_2 · Cited by 2
Qiu, M., Li, Y., Huang, W., Zhang, H., Zheng, W., Fan, X., & Lei, W. (2024) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc53108.2024.10781710 · Cited by 2
“To address the challenges of high-resolution inputs in Transformer, MViT [18] utilized pooling attention techniques to aggregate local features.”
Self-citationA hierarchical Transformer network for smoke video recognition
Cheng, G., Xian, B., Liu, Y., Chen, X., Hu, L., & Song, Z. (2024) · Digital Signal Processing · doi:10.1016/j.dsp.2024.104959 · Cited by 2
Low, P. J., Ng, B. Y., Mahzan, N. I., Tian, J., & Leung, C.-C. (2025) · Sensors · doi:10.3390/s25010255 · Cited by 2
A Static-Dynamic Composition Framework for Efficient Action Recognition
Chen, X., Han, Y., Li, C., Chang, X., Sun, Y., & Yang, Y. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3525187 · Cited by 4
“MviT [32] introduced the famous multiscale feature hierarchies to transformer models.”
Camera Absolute Pose Estimation Using Hierarchical Attention in Multi-Scene
Lu, X., Miao, J., Xue, Q., Wan, H., & Zhang, H. (2025) · IEEE Access · doi:10.1109/access.2025.3531896 · Cited by 2
“…of step (c.) in Figure 2 is shown in Figure 3, the HA mechanism, in training the camera pose estimation model for multiple scenes, achieves the modeling of global dependencies and the preservation of local details by combining both downsampling [24] and window-based [25] transformers.”
Ahmed, S., Yoshiura, N., Hassan, S., & Khan, A. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-81010-7_13 · Cited by 2
ViTHAR: Vision Transformers for Human Action Recognition in Videos
Divya Rani, R., & Prabhakar, C. J. (2024) · International Conference on Multimedia Processing, Communication & Information Technology (MPCIT) · doi:10.1109/mpcit62449.2024.10892707 · Cited by 2
“Fan et al. [15] presented a multiscale approach to video action recognition, where the Transformer captures features at varying resolutions.”
Li, X., Cheng, P., Liu, X., & Huang, Y. J. (2025) · Signal Image and Video Processing · doi:10.1007/s11760-025-03910-5 · Cited by 2
Hierarchical Attention Network for Alzheimer's Disease Diagnosis Using sMRI Imaging Data
Khatri, U., Kim, J., & Kwon, G. (2025) · International Conference on Consumer Electronics · doi:10.1109/icce63647.2025.10929875 · Cited by 2
“…which maintains the same feature resolution without downsampling, it is generally more effective to learn feature representations in a multi-scale approach for improved performance, especially in applications like detection and segmentation, as suggested by Fan et al. [14] and Wang et al [15].”
Wang, G., Hou, M., Qin, M., Wu, X., Gao, Z., Chao, G., & Zhang, X. (2025) · Journal of Marine Science and Engineering · doi:10.3390/jmse13050966 · Cited by 5
Dual cross transformer based on multi-scale fusion for fine-grained action recognition
Di, J., Hu, Z., Zhang, H., Zhang, Q., & Sun, Z. (2025) · Applied Intelligence · doi:10.1007/s10489-025-06742-5 · Cited by 2
“Prominent models such as TimeSformer [13],Uniformer [14], ViViT [15], VTN [ 16 ], Mformer [ 17 ], and MViT [18] stand as exemplary representatives of this transformative paradigm.”
Efficient High-Order Spatial Interactions for Visual Perception
Liu, Z., Rao, Y., Zhao, W., Zhou, J., & Lu, J. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3603181 · Cited by 2
“…successful designs in CNN architectures and the new self-attention mechanism, vision Transformers have shown leading performance on various vision tasks such as image classifi-cation [12], [42], [54], object detection [41], [80], semantic segmentation [7], [8] and video understanding [18], [73].”
Wang, H., Ahn, E., Joseph, A., Pathan, F. K., Negishi, K., & Kim, J. (2025) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2025.108886 · Cited by 2
Multiscale Attention-Based Model for Image Enhancement and Classification
guo, M., Editorial Office, & Takubo, T. (2025) · Journal of Advanced Computational Intelligence and Intelligent Informatics · doi:10.20965/jaciii.2025.p1484 · Cited by 2
Du, Y., Tong, Z., Wan, J., Zhang, B., & Zhao, Y. (2022) · IEEE International Conference on Multimedia and Expo workshops · doi:10.1109/icmew56448.2022.9859481 · Cited by 2
“MViT [11] pretrained on Kinetics400 [27] is applied as the baseline classifier.”
MethodInfluentialAmeri, K., Hempel, M., Sharif, H., Lopez, J., & Perumalla, K. S. (2022) · Journal of Cybersecurity and Privacy · doi:10.3390/jcp2020022 · Cited by 2
Delving into Details: Synopsis-to-Detail Networks for Video Recognition
Liang, S., Shen, X., Huang, J., & Hua, X. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19772-7_16 · Cited by 2
“However, note that our method can also apply with recent Vision Transformers [1, 3, 7, 8, 39].”
Deep Saliency Map Generators for Multispectral Video Classification
Bayer, J., Münch, D., & Arens, M. (2022) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr56361.2022.9955639 · Cited by 2
In Defense of Image Pre-Training for Spatiotemporal Recognition
Li, X., Wang, H., Wei, C., Mei, J., Yuille, A., Zhou, Y., & Xie, C. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19806-9_39 · Cited by 2
Transformer Video Classification algorithm based on video token-to-token
Gong, X., & Li, Z. (2022) · International Conference on Computer and Communications Management (ICCCM) · doi:10.1145/3556223.3556241 · Cited by 2
“Method name Top1 Top5 I3D [22] 71.9 90.1 AttentionNAS [37] 79.8 94.4 MoViNet-A5 [33] 82.7 - LGD-3D-101 [34] 81.5 95.6 SlowFast-R101-NL [35] 81.8 95.1 TimeSFormer-L [12] 82.2 95.6 VATT-Large [17] 83.6 96.6 ViViT-L/16*2-FE [11] 82.9 94.6 ViViT-L/16*2-FE(JFT) [11] 84.3 94.9 TVC-T2T-AV 84.6 95.1 0.3 respectively.”
BackgroundInfluentialWeakly-Guided Self-Supervised Pretraining for Temporal Activity Detection
Kahatapitiya, K., Ren, Z., Li, H., Wu, Z., Ryoo, M. S., & Hua, G. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i1.25189 · Cited by 6
“…Schmid 2017; Piergiovanni and Ryoo 2018), graphical modeling (Zhao, Thabet, and Ghanem 2021; Mavroudi, Haro, and Vidal 2020), object detections (Baradel et al. 2018; Zhou et al. 2019) or attention mechanisms (Nawhal and Mori 2021; Chang et al. 2021; Fan et al. 2021) to improve video understanding.”
BackgroundEnhancing Next Active Object-Based Egocentric Action Anticipation with Guided Attention
Thakur, S. K., Beyan, C., Morerio, P., Murino, V., & Bue, A. D. (2023) · Proceedings - International Conference on Image Processing · doi:10.1109/icip49359.2023.10222445 · Cited by 2
“In this paper, we utilize the Multi-Scale Vision Transformer (MViT) network [12, 13] which introduced multiscale vision feature hierarchy for long-term video modeling, as the foundation of our architecture to extract and contextualize motion information from a given video clip.”
MethodTSNet: Token Sparsification for Efficient Video Transformer
Wang, H., Zhang, W., & Liu, G. (2023) · Applied Sciences · doi:10.3390/app131910633 · Cited by 4
“TSNet(UniFormer-S)/0.3 outperforms MViT-B16 [13] while saving over 80% in computation.”
BackgroundInfluentialSelective Depth Attention Networks for Adaptive Multiscale Feature Representation
Guo, Q., Wu, X., Xu, T., Si, T., Hu, C., & Tian, J. (2024) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2024.3401652 · Cited by 2
“Recently, the transformer architecture has received more attention for CV tasks, such as vision transformer (ViT) [37], multiscale vision transformer (MViT) [38], convolutional vision transformer (CvT) [39], and pyramid vision transformer (PVT) [40], [41].”
BackgroundBalaji, P., Alqahtani, O., Babu, S., Chaurasia, M. A., & Prakasam, S. (2024) · Computer Modeling in Engineering & Sciences · doi:10.32604/cmes.2024.053158 · Cited by 2
PrefAce: Face-Centric Pretraining with Self-Structure Aware Distillation
Hu, S., Wang, Z., Hu, P., Peng, X., Wu, J., Zhu, H., & Ong, Y.-S. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i11.29147 · Cited by 2
Elucidating the Hierarchical Nature of Behavior with Masked AutoencodersPreprint
Stoffl, L., Bonnetto, A., d’Ascoli, S., & Mathis, A. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.08.06.606796 · Cited by 2
“The recent Hiera model (80), which is based on hierarchical Vision Trans-former (ViT) (81, 82), is designed for sparse MAE pre-training in a spatially hierarchical way and exhibits strong performance.”
MethodDiverse Perceptual Representations Across Visual Pathways Emerge from A Single ObjectivePreprint
Tang, Y., Gökce, A., Jedoui, K., Yamins, D. L. K., & Schrimpf, M. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.07.22.664908 · Cited by 10
“…action recognition models trained to recognise human actions, including convolution-based [128, 137, 17, 32, 31, 8] and transformer-based models [29, 79, 83, 76, 75]; 5. the mask autoencoding models trained to reconstruct missing spatiotemporal patches in videos in the pixel or latent…”
Sparse Dense Transformer Network for Video Action Recognition
Qu, X., Zhang, Z., Xiao, W., Ran, J., Wang, G., & Zhang, Z. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-10986-7_4 · Cited by 1
Hyperspectral Image Classification Via 3D-CNN MHSA Fusion Transformer
Xie, W., Zhang, Y., Sun, H., & Wu, Q. (2023) · IEEE International Geoscience and Remote Sensing Symposium (IGARSS) · doi:10.1109/igarss52108.2023.10283141 · Cited by 1
“…through more regularization and stronger data enhancement [8]; Multi-scale visual transformers by fusing transformers of different sizes [10]; Enhance the depth of feature information learning by increasing the number of Transformer blocks [11]; Transform-ers that optimize the image…”
BackgroundPraveenkumar, S. M., Patil, P. R., & Hiremath, P. S. (2024) · International Journal of Pattern Recognition and Artificial Intelligence · doi:10.1142/s0218001424500058 · Cited by 1
Towards Lightweight Transformer Architecture: an Analysis on Semantic Segmentation
Іванюк-Скульський, Б., Shvai, N., Llanza, A., & Nakib, A. (2024) · International Conference on Artificial Intelligence, Computer, Data Sciences and Applications (ACDSA) · doi:10.1109/acdsa59508.2024.10467926 · Cited by 1
“3) Pool-Unpool self-attention: Taking into account the pooling modification proposed in [24], we applied pooling to the calculation of self-attention in Equations (2).”
MethodFew-shot Action Recognition with Video Transformer
Aikyn, N., Abu, A., Zhaksylyk, T., & Tu, N. A. (2023) · International Conference on Signal-Image Technology & Internet-Based Systems (SITIS) · doi:10.1109/sitis61268.2023.00027 · Cited by 2
“Multiscale feature hierarchies have been combined with Transformer models to expand feature complexity [25].”
MethodRETRACTED: Video-MLP: Convolution-free, attention-free architecture for video classification
Zhang, Q., Bai, E., Shao, M., & Hong, L. (2024) · Journal of Intelligent & Fuzzy Systems · doi:10.3233/jifs-240310 · Cited by 1
“Multiscale Vision Transformers (MViT) [28] builds a multi-scale feature pyramid based on ViT, first modeling low-level semantic information at high resolution and then capturing high-level features at lower resolutions.”
RetractedLee, I. Y., Nguyen-Duc, T., Ueno, R., Smith, J., & Chan, P. (2024) · IEEE Sensors Letters · doi:10.1109/lsens.2024.3408320 · Cited by 1
“After preprocessing, we are left with 458 videos from eight classes (see Table 1).”
Method3D Swin Transformer for Partial Medical Auto Segmentation
Rangnekar, A., Jiang, J., & Veeraraghavan, H. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-58776-4_18 · Cited by 1
“Choice of Transformer: Hierarchical Vision Transformers [21,8] are pyramid-shaped architectures that rely on gradual down-sampling, similar to convolutional neural networks, while maintaining a global look-out with their multi-scale designs.”
BackgroundBird Action Recognition in Wetlands using Deep Learning
Rodriguez-Juan, J., Berenguer-Agullo, A., Benavent-Lledó, M., Mulero-Pérez, D., García‐Rodríguez, J., & Sebastián‐González, E. (2024) · International Conference on Information Technology for Social Good (GoodIT) · doi:10.1145/3677525.3678681 · Cited by 1
“The models compared for the video-level approach are: Video MViT [10], ResNet3D [29], R2Plus1 [29], Video Swin Transformer [19] and S3D [33].”
InfluentialBootstrapping SparseFormers from Vision Foundation Models
Gao, Z., Tong, Z., Lin, K. Q., Chen, J., & Shou, M. Z. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01677 · Cited by 1
“It is always an appealing topic to build efficient vision transformers, e.g ., with efficient attention mechanisms [10, 30, 66], or using compact transformer architectures [17, 25, 40, 67].”
Background
Metadata
- Authors
- Fan, H., Xiong, B., Mangalam, K., Li, Y., Yan, Z., Malik, J., & Feichtenhofer, C.
- Affiliations
- Meta (Israel) (IL)
- Meta (United States) (US)
- Venue
- 2021 IEEE/CVF International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2021-10
- Pages
- 6804-6815
- OpenAlex
- W3202698164
- Cited by
- 1,407
- References
- 111
- Citations by year
- 2026: 147 · 2025: 345 · 2024: 380 · 2023: 349 · 2022: 176 · 2021: 10
- FWCI
- 74.81
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- arXiv (Cornell University)
- TIB Data Manager
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- video recognition, image classification, feature pyramid, spatial resolution reduction, computational efficiency
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
