The DOI matches its Crossref record; 106 of the 111 references it cites were found in a registry or an index.
Multiscale Vision Transformers
Open accessAbstract
We present Multiscale Vision Transformers (MViT) for video and image recognition, by connecting the seminal idea of multiscale feature hierarchies with transformer models. Multiscale Transformers have several channel-resolution scale stages. Starting from the input resolution and a small channel dimension, the stages hierarchically expand the channel capacity while reducing the spatial resolution. This creates a multiscale pyramid of features with early layers operating at high spatial resolution to model simple low-level visual information, and deeper layers at spatially coarse, but complex, high-dimensional features. We evaluate this fundamental architectural prior for modeling the dense nature of visual signals for a variety of video recognition tasks where it outperforms concurrent vision transformers that rely on large scale external pre-training and are 5-10× more costly in computation and parameters. We further remove the temporal dimension and apply our model for image classification where it outperforms prior work on vision transformers. Code is available at: https://github.com/facebookresearch/SlowFast.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 2×
“As in original ViT [25], we use residual connections [45] and Layer Normalization (LN) [2] in the pre-normalization configuration that applies LN at the beginning of the residual function, and our MLPs consist of two linear layers with GELU activation [48], where the first layer expands the dimension from D to 4D, and…”
MethodResultVaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 6×
“In contrast to original Multi Head Attention (MHA) operators [98], where the channel dimension and the spatiotemporal resolution remains fixed, MHPA pools the sequence of latent tensors to reduce the sequence length (resolution) of the attended input.”
BackgroundMethodInfluentialVery Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 1×
“Incorporating downsampling, shift invariance, and shared weights, ConvNets are de-facto standard backbones for computer vision tasks for image [65, 62, 88, 90, 46, 12, 15, 34, 93, 81, 41] and video [87, 31, 11, 79, 69, 106, 96, 30, 105, 35, 29, 117, 57].”
BackgroundImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 1×
“Incorporating downsampling, shift invariance, and shared weights, ConvNets are de-facto standard backbones for computer vision tasks for image [70, 67, 94, 96, 51, 15, 18, 39, 99, 87, 46] and video [93, 36, 14, 85, 74, 112, 102, 34, 111, 40, 33, 123, 62].”
BackgroundFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534 · cited 1×
“Finally, compared to the best model of concurrent Swin [79] (which was designed for image recognition), MViT has +0.”
ResultRethinking the Inception Architecture for Computer Vision
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.308 · Cited by 32,138
Research works printed but not linked: 11
- gao, 2019, Res2net: A new multi-scale backbone architecture, IEEE PAMI
- fan, 2020, Pyslowfast
- feichtenhofer, 2019, SlowFast networks for video recognition in ActivityNet challenge 2019
- 10.1109/CVPR.2009.5206848 · DOI · Paper page
- 10.1109/CVPR.2019.00033 · DOI
- hang, 2020, ResNeSt Split-Attention Networks
- 10.1109/ICCV.2019.00718 · DOI
- 10.1109/CVPR.2015.7298594 · DOI · Paper page
- carion, 2020, Endto-end object detection with transformers, Proc ECCV, 213
- loshchilov, 2018, Fixing weight decay regularization in adam
- qiu, 2017, Learning spatiotemporal representation with pseudo-3d residual networks, Proc ICCV
Cited by
Sorted by citations
Items 801–900 of 1,003
Research on faults diagnosis of dynamic balance in scroll compressor based on CWT-MViTV2
Xingshen, D., Lin, L., Li, H., Jun, B., Yinghua, C., Zhang, X., & Wenyu, M. (2024) · Measurement Science and Technology · doi:10.1088/1361-6501/ad86dd · Cited by 4
Prompting Continual Person Search
Zhang, P., Yu, X., Xiao, B., Zheng, J. G., & Ning, X. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681240 · Cited by 4
“Different from the image classification [12, 33] or person re-identification tasks [64, 67] that directly extract object features from object-centered images, the person search task requires to extract features of instances in the scene images.”
BackgroundTelling Stories for Common Sense Zero-Shot Action Recognition
Gowda, S. N., & Sevilla-Lara, L. (2024) · Lecture notes in computer science · doi:10.1007/978-981-96-0908-6_26 · Cited by 5
Zhao, Y., Li, Y., Zhang, Y., Yan, X., Yin, G., & Liu, L. (2025) · IEEE Access · doi:10.1109/access.2025.3531705 · Cited by 4
Self-citationLi, M., Lan, J., & Wang, L. (2025) · Measurement · doi:10.1016/j.measurement.2025.116971 · Cited by 4
From ensemble to knowledge distillation: Improving large-scale food recognition
Nong, L., Peng, G., Xu, T., & Zhu, J. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.110727 · Cited by 4
Multimodal understanding of human values in videos: A benchmark dataset and PLM-based method
Wang, Y., Tao, Z., He, C., Huang, N., Jin, L., & Luo, X. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.130170 · Cited by 4
Optimal transport-based fusion of two-stream convolutional networks for action recognition
Yenduri, S., Gudavalli, M., & Gayathri, C. (2025) · Applied Intelligence · doi:10.1007/s10489-025-06518-x · Cited by 4
Bootstrap Masked Visual Modeling via Hard Patch Mining
Wang, H., Fan, J., Wang, Y., Song, K., Wang, T., Zhang, X., & Zhang, Z. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3557001 · Cited by 9
BUS-M2AE: Multi-scale Masked Autoencoder for Breast Ultrasound Image Analysis
Yu, L., Gou, B., Xia, X., Yang, Y., Zhang, Y., Min, X., & He, T. (2025) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2025.110159 · Cited by 6
PEAR: phrase-based hand-object interaction anticipation
Zhang, Z., Luo, H., Zhai, W., Cao, Y. H., & Kang, Y. (2025) · Science China Information Sciences · doi:10.1007/s11432-024-4405-4 · Cited by 14
Towards an AI-Powered Video Assistant Referee System (VARS) for Association Football
Held, J., Cioppa, A., Giancola, S., Hamdi, A., Devue, C., Ghanem, B., & Droogenbroeck, M. V. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-95911-0_21 · Cited by 4
A Large-Scale Analysis on Contextual Self-Supervised Video Representation Learning
Kumar, A. S., Kumar, A., Vineet, V., & Rawat, Y. S. (2025) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw67362.2025.00072 · Cited by 4
“Big Capacity networks: Comparing across four trans-former architectures, ViViT [5] Timesformer [8], VideoSwin [47] and MViT [18], we selected VideoSwin, because it outperforms others on Kinetics 400 dataset.”
Rani, S. S., Balasubramanian, V., Rajinikanth, V., & Selvarajan, R. (2025) · International Journal of Systems Assurance Engineering and Management · doi:10.1007/s13198-025-02944-9 · Cited by 4
M2-Conformer: Multi-modal CNN- Transformer for Driving Behavior Detection
Gao, J., Yi, J., & Murphey, Y. L. (2022) · International Symposium on Autonomous Systems (ISAS) · doi:10.1109/isas55863.2022.9757336 · Cited by 4
High-order Correlation Network for Video Recognition
Dong, W., Wang, Z., Zhang, B., Zhang, J., & Zhang, Q. (2022) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn55064.2022.9892897 · Cited by 4
“By using these pre-trained transformer architectures, several welldesigned transformer-based video recognition architectures have emerged [1], [2], [9], [34], [36].”
MethodCMC_v2: Towards More Accurate COVID-19 Detection with Discriminative Video Priors
Hou, J., Xu, J., Zhang, N., Wang, Y., Zhang, Y., Zhang, X., & Feng, R. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-25082-8_32 · Cited by 4
Nan, Z., & Xiang, T. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3286442 · Cited by 4
“Wit-nessing the success of transformer in solving NLP problems, Dosovitskiy et al. [64] attempted to apply transformer to solve the image classification problem in 2020, after which many visual transformer models [65], [66], [67], [68], [69], [70] are proposed.”
BackgroundLGANet: Local and global attention are both you need for action recognition
Wang, H., Zhao, B., Zhang, W., & Liu, G. (2023) · IET Image Processing · doi:10.1049/ipr2.12876 · Cited by 4
Explainable Driver Activity Recognition Using Video Transformer in Highly Automated Vehicle
Sonth, A., Sarkar, A., Bhagat, H., & Abbott, L. (2023) · IEEE Intelligent Vehicles Symposium · doi:10.1109/iv55152.2023.10186584 · Cited by 4
“Recent studies ([20], [21], [22]) have shown that Video Transformers can achieve state-of-the-art performance on several benchmark datasets for video classification [23] and action recognition tasks [24], [25].”
BackgroundConCAP: Contrastive Context-Aware Prompt for Resource-hungry Action Recognition
Zhang, H., Zeng, Z., Zhao, Q., & Zhai, Z. (2023) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme55011.2023.00137 · Cited by 4
“However, the lowlevel visual features in shallow layers differ from high-level ones in deep layers drastically [31], and the extensibility of naive migration is poor due to a lack of capability to model such variation.”
BackgroundMask Again: Masked Knowledge Distillation for Masked Video Modeling
Li, X., He, S., Wu, J., Yu, Y., Nie, L., & Zhang, M. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612129 · Cited by 5
“Various learning approaches have been explored, including supervised methods [1, 11, 14, 32, 36, 58], semi-supervised approaches [50, 63] and self-supervised learning (SSL) methods [9, 13, 16, 19, 40, 56].”
DNAT: Multi-scale Transformer with Dilated Neighborhood Attention for Image Classification
Jiang, C., Zhou, Q., Mo, Z., Wang, J., Fan, Y., Wu, X., Zhang, S., & Kang, B. (2023) · International Conference on Wireless Communications and Signal Processing (WCSP) · doi:10.1109/wcsp58612.2023.10404854 · Cited by 4
“The multi-scale interactions have gained great attraction in recent ViTs [9], [10], [22], [23], which can be roughly divided into three categories: window-based ViTs [9], [13], token aggregation [22] and partition [6], and feature pyramid [10], [11], [24].”
BackgroundMethodReal-time human action recognition from aerial videos using autozoom and synthetic data
Xian, R., Vogel, B. I., de Melo, C. M., Harrison, A., & Manocha, D. (2024) · Synthetic Data for Artificial Intelligence and Machine Learning: Tools, Techniques, and Applications · doi:10.1117/12.3013547 · Cited by 4
An Audiovisual Correlation Matching Method Based on Fine-Grained Emotion and Feature Fusion
Su, Z., Feng, Y., Liu, J., Peng, J., Jiang, W., & Liu, J. (2024) · Sensors · doi:10.3390/s24175681 · Cited by 4
“After the aforementioned stages of audio and video feature extraction, a small fraction of the data may still have missing values.”
BackgroundInfluentialChen, Y.-H., Bui, T.-A., Lee, P., & Hsu, C.-H. (2024) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2024.3455891 · Cited by 4
“The multiscale visual transformer proposed in this article differs from the method presented in MViT [16], which enhances multiscale capabilities by utilizing features at different resolutions.”
MethodSTCovidNet: Automatic Detection Model of Novel Coronavirus Pneumonia Based on Swin TransformerPreprint
Wang, B., Zhang, D., & Tian, Z. (2022) · Research Square · doi:10.21203/rs.3.rs-1401026/v1 · Cited by 4
“…we used the following SOTA model: ViT [8] was implemented in computer vision applications by employing multi-head self-attention [8], [10], [42], [43] as an image extraction approach, following the recent breakthrough of Transformers [27] in handling natural language processing tasks [44].”
MethodChen, N., Xu, T., Sun, M., Yao, C., & Yang, D. (2025) · Intelligent Computing · doi:10.34133/icomputing.0143 · Cited by 5
AdaFPP: Adapt-Focused Bi-Propagating Prototype Learning for Panoramic Activity Recognition
Cao, M., Yan, R., Shu, X., Dai, G., Yao, Y., & Xie, G.-S. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3680755 · Cited by 3
“Recent research on the STAD task can be mainly categorized into two classes, including two-stage STAD [11, 12, 35, 39, 41] and single-stage STAD [29, 37, 43, 53].”
Semi-supervised action recognition using logit aligned consistency and adaptive negative learning
Zuo, F., Xu, Y., & Wang, M. (2025) · Scientific Reports · doi:10.1038/s41598-025-01922-2 · Cited by 3
E-harnet: an efficient hybrid transformer network for human activity recognition
Iqbal, A., Rauf, M. A., Salim, Mahamud, M. D. S., Khalil, M. M. Y., & Qin, Z. (2025) · The Journal of Supercomputing · doi:10.1007/s11227-025-07618-8 · Cited by 3
“In the transformer landscape, continued innovation is evident in models like MViT [30] and the Video Swin Transformer [31], which introduce more effective mechanisms for integrating spatial and temporal features.”
Lian, J., Ji, J., Niu, Y., & Jiao, W. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.111795 · Cited by 3
Liao, H.-C., Rao, B., Sun, H., Wang, C.-Y., Chang, Q., Li, S., Xu, C.-Z., & Li, Z.-N. (2025) · IEEE transactions on intelligent transportation systems (Print) · doi:10.1109/tits.2025.3597411 · Cited by 5
Yang, N., Tian, C., Gu, X., Zhang, Y., Li, X., & Zhang, F. (2025) · Sensors · doi:10.3390/s25175531 · Cited by 3
“Fan et al. [47] proposed the Multi-scale Vision Transformer (MViT), which hierarchically increases the channel capacity while progressively reducing the spatial resolution across stages.”
NExNet Seg: Neuron Expansion Network for Medical Image Segmentation
Reyes-Angulo, A. A., & Paheding, S. (2025) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw67362.2025.00005 · Cited by 3
“While state-of-the-art architectures, e.g. efficientNet V2 [32], ViT [7], and MviT [8] are popular for image classification, their ultimate performance is proportional to the computational cost required Recently, the Retentive Network [30] has introduced the Manhattan Self-Attention (MaSA)…”
Edge computing and server-based high-precision flood level classification system
Lu, A., Cao, R., Wang, Y., Hu, W., Gao, Y., Hu, Z., & Zang, Y. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.112442 · Cited by 3
VTLG: A vision-tactile-language grasp generation method oriented towards task
Li, T., Yu, C., Yan, Y., Song, D., Shuai, Y., Wang, Y., & Chen, G. (2025) · Robotics and Computer-Integrated Manufacturing · doi:10.1016/j.rcim.2025.103152 · Cited by 4
Automated neural network generation for industrial datasets, application to laser powder bed fusion
Avval, S. S. P., Abadie, T., & Yaghoubi, V. (2025) · Journal of Intelligent Manufacturing · doi:10.1007/s10845-025-02707-0 · Cited by 3
“These are the 3D-CNN-based methods 3D ResNet (Blanc et al., 2023), SlowFast (Feichtenhofer et al., 2019), the transformers MViT (Fan et al., 2021) and Swin3D (Yang et al., 2025). al., 2022 ) or aging of the actuator (Creac’h et al., 2015).”
Castilla-Arquillo, R., Pérez-del-Pulgar, C. J., Gerdes, L., Garcia-Cerezo, A. J., & Olivares-Méndez, M. A. (2025) · International Conference on Space Robotics (iSpaRo) · doi:10.1109/isparo66239.2025.11436158 · Cited by 3
“Transformer-based architectures with shifted windows (Swin) [10] and multi-scale attention mechanisms [11] have recently gained attention for their ability to handle such complexity.”
Spatiotemporal Deformable Transformer for Lung Cancer Risk Prediction Using Low-Dose CT Images
Chang, Y.-H., Lin, Y., Meng, H.-C., Wu, S., Lin, Y.-Y., & Huang, C.-R. (2026) · IEEE Transactions on Radiation and Plasma Medical Sciences · doi:10.1109/trpms.2026.3652592 · Cited by 3
“A naive idea to discover the temporal interimage correlations between features of I 1 and I 2 is to apply the multihead self-attention scheme to patches of corresponding spatial positions across different time points which is commonly applied in video prediction methods [32], [33].”
Ayoubi, M., & Arashpour, M. (2026) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2026.104681 · Cited by 3
A Stacked Hybrid Ensemble of Swin-V2 and MViT-V2 for High-Performance Skin Disease Classification
Singh, D. P., Patel, V. M., Doshi, D., Mahajan, S., & Kadry, S. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-19865-5_21 · Cited by 3
PromptLearner-CLIP: Contrastive Multi-Modal Action Representation Learning with Context Optimization
Zheng, Z., An, G., Cao, S., Yang, Z., & Ruan, Q. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-26316-3_33 · Cited by 3
Rotated and Masked Image Modeling: A Superior Self-Supervised Method for Classification
Yan, D., Gong, X., & Zhang, Z. (2023) · IEEE Signal Processing Letters · doi:10.1109/lsp.2023.3324245 · Cited by 3
Sun, Y., Xu, W., Yu, X., & Gao, J. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-17788-3 · Cited by 3
Ming, Y., Xiong, L., Jia, X., Zheng, Q., & Zhou, J. (2023) · IEEE Smart World Congress (SWC) · doi:10.1109/swc57546.2023.10448953 · Cited by 3
“MViT [7] achieves a hierarchical structure and reduces the computational cost.”
BackgroundDeep Learning-based Student Learning Behavior Understanding Framework in Real Classroom Scene
Yang, Y., Ren, Z., Lenart, C., Corsello, A., Kosko, K. W., Su, S. M., & Guan, Q. (2023) · International Conference on Machine Learning and Applications · doi:10.1109/icmla58977.2023.00067 · Cited by 4
“We will now delve into the technical details of the student action recognition model, which is based on the Multi-Head Pooling Attention (MHPA) in Multiscale Vision Trans-former (MViT) [21].”
MethodSampleformer: An efficient conformer-based Neural Network for Automatic Speech Recognition
Fan, Z., Zhang, X., Huang, M., & Bu, Z. (2024) · Intelligent Data Analysis · doi:10.3233/ida-230612 · Cited by 3
“Similar to spatial downsampling in computer vision models, this temporal downsampling technique is commonly utilized to economize computational resources and generate hierarchical-level features [41,42,43,44].”
Panev, S., Kim, E., Namburu, S. A. S., Nikolova, D., de Melo, C., la Torre, F. D., & Hodgins, J. K. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00453 · Cited by 3
“We used three deep video activity recognition models in our experiments: SlowFast [12], MViT [10], and X3D [11].”
BackgroundMethodInfluentialSEMA: Semantic Attention for Capturing Long-Range Dependencies in Egocentric Lifelogs
Nagar, P., Shastry, K. N. A., Chaudhari, J., & Arora, C. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00687 · Cited by 3
“The long sequence problem becomes even more critical in vision problems, where researchers have used spatial [41], temporal [1], and hierarchical [17, 21] cues to scale the attention for large image and video data.”
BackgroundGeneric-to-Specific Distillation of Masked Autoencoders
Huang, W., Peng, Z., Dong, L., Wei, F., Ye, Q., & Jiao, J. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3393474 · Cited by 3
Adaptive Prototype Learning for Weakly-Supervised Temporal Action Localization
Luo, W., Ren, H., Zhang, T., Yang, W., & Zhang, Y. (2024) · IEEE Transactions on Image Processing · doi:10.1109/tip.2024.3431915 · Cited by 4
“More recently, there are some attempts at introducing Trans-former architecture to video representation learning [62], [63], [64], [65], [66], [67], [68], [69], [70].”
Multimodal Forgery Detection in Videos Using a Tri-Network Model
Chandran, P. P., Sriram, S., & Babu, C. (2024) · International Conference on Advanced Computing and Communication Systems (ICACCS) · doi:10.1109/icaccs60874.2024.10717044 · Cited by 3
“Video Vision Transformer (ViViT) [16] and Multiscale Vision Transformer (MViT) [17] employ global self-attention mechanisms, enabled by spatio-temporal factorization, to comprehensively capture information across both spatial and temporal dimensions.”
BackgroundAggregating multi-scale flow-enhanced information in transformer for video inpainting
Li, G., Zhang, K., Su, Y., & Wang, J. (2024) · Multimedia Systems · doi:10.1007/s00530-024-01625-0 · Cited by 3
“Furthermore, existing methods lack multi-scale representation for objects of varying sizes, which is crucial for effective visual processing [21], limiting their ability to aggregate large-scale structural and small-scale texture information simultaneously.”
Wang, J., Liu, Y., & Yang, Z.-X. (2025) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2025.3529511 · Cited by 3
“MViT [35] first uses a single pooling operation to reduce the number of tokens required to calculate multi-head self-attention.”
Hierarchical Task-aware Temporal Modeling and Matching for few-shot action recognition
Zhan, Y., Pan, Y., Wu, S., Zhang, Y., & Sun, X. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.129467 · Cited by 4
ETDformer: an effective transformer block for segmentation of intracranial hemorrhage
Gong, W., Luo, Y., Yang, F., Zhou, H., Lin, Z., Cai, C., Lin, Y., & Chen, J. (2025) · Medical & Biological Engineering & Computing · doi:10.1007/s11517-025-03333-x · Cited by 3
“The final network architecture model is conceived in accordance with downstream tasks, which encompass image classification tasks [29, 30], instance segmentation [31, 32], and object detection [33, 34].”
Cheng, G., Yang, L., Yu, Z., Li, X., & Fu, G. (2025) · Fire · doi:10.3390/fire8050197 · Cited by 3
“We compare our CSFResNet with widely used deep networks for smoke recognition, including both CNN-based models (AlexNet [54], ResNet [13], DenseNet [67], Ef-ficientNet [68], RepVGG [69]) and Transformer-based models (ViT [11], DeiT [70], Swin Transformer [71], MViT [72]).”
Improving risk stratification and detection of early HCC using ultrasound-based deep learning models
Dana, J., Meyer, A., Paisant, A., Rode, A., Sartoris, R., Séror, O., Cassinotto, C., Milot, L. M., Grégory, J., Cœur, J., et al. (2025) · JHEP Reports · doi:10.1016/j.jhepr.2025.101510 · Cited by 3
Gated Region-Refine pose transformer for human pose estimation
Wang, T., & Zhang, X. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.01.090 · Cited by 3
Less is More: Decoupled High-Semantic Encoding for Action Recognition
Zhang, C., Ren, K., Bian, Q., & Shi, Y. (2023) · International Conference on Multimedia Retrieval · doi:10.1145/3591106.3592233 · Cited by 3
An Investigation on Hardware-Aware Vision Transformer Scaling
Li, C., Kim, K., Wu, B., Zhang, P., Zhang, H., Dai, X., Vajda, P., & Lin, Y. (2023) · ACM Transactions on Embedded Computing Systems · doi:10.1145/3611387 · Cited by 3
“In order to build more efficient ViT models, [7] leverages multiple branches to extract and fuse features at different scales; [14, 20, 25, 36, 53] apply a pyramid-like architecture commonly used in CNNs to ViT and [20, 36, 56] propose more efficient attention mechanisms or feature projection…”
MethodVisual perception of fluid viscosity: Toward realistic fluid simulation
Zhang, Y., Dong, Z., & Du, F. (2023) · Computers & Graphics · doi:10.1016/j.cag.2023.10.004 · Cited by 3
Slowfast Diversity-aware Prototype Learning for Egocentric Action Recognition
Dai, G., Shu, X., Yan, R., Huang, P. F., & Tang, J. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612144 · Cited by 3
“So far, there have emerged several impressive Transformer models, such as TimeS-former [2], ViViT [1], MFormer [44], MViT [15], XViT [3], etc.”
MethodXiang, T., Lv, P., Sun, L., Yang, Y., & Hao, J. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.111202 · Cited by 3
DAAPS: A Deformable-Attention-Based Anchor-Free Person Search Model
Xin, X., Han, D., & Cui, M. (2023) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2023.042308 · Cited by 3
BusEye: A multi-stream approach for driver behavior analysis on public bus driver cameras
Bilge, Y. C., Mutlu, B., & Esin, Y. E. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.123148 · Cited by 3
Explainable activity recognition for the elderly
Gaya-Morey, F. X., Manresa-Yee, C., & Rubio, J. M. B. (2023) · International Conference on Human Computer Interaction (Interacción) · doi:10.1145/3612783.3612790 · Cited by 3
“However, there have been recent attempts to adapt the standard Transformer architecture to video, achieving convolution-free networks [4, 12].”
BackgroundCrime Detection from Pre-crime Video Analysis with Augmented Pose and Emotion Information
Kilic, S., & Tüceryan, M. (2024) · IEEE Southwest Symposium on Image Analysis and Interpretation (SSIAI) · doi:10.1109/ssiai59505.2024.10508694 · Cited by 3
Posture Detection of Heart Disease Using Multi-Head Attention Vision Hybrid (MHAVH) Model
Naz, H., Zhang, Z., Al‐Habib, M. I., Awwad, F. A. A., Ismail, E. A. A., & Khan, Z. A. (2024) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2024.049186 · Cited by 3
“The 3D CNN [24,25] is employed to extract or retrieve the spatial feature map from the global feature modeling carefully transformed by the input transformer.”
Li, N., Huang, K., Wu, Q., & Zhao, Y. (2024) · Applied Intelligence · doi:10.1007/s10489-024-05661-1 · Cited by 3
“Several works [37, 40] propose to introduce CNN-like inductive bias into the transformer structure to remedy the drawback.”
Multi-Distillation Underwater Image Super-Resolution via Wavelet Transform
Dharejo, F. A., Alawode, B., Ganapathi, I. I., Wang, L., Mian, A. S., Timofte, R., & Javed, S. (2024) · IEEE Access · doi:10.1109/access.2024.3449136 · Cited by 3
“With Multi-scale Vision Transformers (MViT) [22], a multi-scale feature hierarchy can be acquired by gradually increasing channel capacity while simultaneously reducing spatial resolution.”
BackgroundMethodInfluentialEnhanced Industrial Action Recognition Through Self-Supervised Visual Transformers
Xiao, Y., Xiang, H., Wang, T., & Wang, Y. (2024) · IEEE Access · doi:10.1109/access.2024.3455749 · Cited by 3
“Models such as Mvit [28] aim to perceive dynamism across multiple scales by incorporating hierarchical structures and attention pooling, albeit with potential pitfalls in discerning intricate sequences.”
BackgroundStrizhkova, V., Ferrari, L. M., Kachmar, H., Dantcheva, A., & Brémond, F. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00472 · Cited by 3
“The encoder and decoder are vanilla Vision Transformers (ViT) [7] with joint space-time attention [2, 10, 21, 22] so all pair tokens could interact with each other in the multi-head self-attention layer.”
MethodMultimodal Attack Detection for Action Recognition Models
Mumcu, F., & Yilmaz, Y. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00302 · Cited by 3
“It is seen that, except for the attack on MVIT [4], Shuffle [9] does not perform well against strong attacks.”
BackgroundMethodInfluentialEnhancing Skeletal Pose Estimation from mmWave Point Clouds Through Uncertainty Reduction
Chiang, H.-C., Li, G.-H., Wang, F., Shirmohammadi, S., & Hsu, C.-H. (2024) · International Workshop on Human-centric Multimedia Analysis (HCMA) · doi:10.1145/3688865.3689479 · Cited by 3
“Secondly, we replace the CNN with the Multi-scale Vision Transformer (MViT) [6], which is a video-specialized version of the original Vision Transformer [5] and can effectively leverage temporal information compared to the CNN. Thirdly, we introduce an additional head for uncertainty estimation and…”
Kim, H.-W., & Choi, Y. S. (2024) · Sensors · doi:10.3390/s24216842 · Cited by 3
Elucidating the Hierarchical Nature of Behavior with Masked Autoencoders
Stoffl, L., Bonnetto, A., d’Ascoli, S., & Mathis, A. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73039-9_7 · Cited by 3
Faster-slow network fused with enhanced fine-grained features for action recognition
Wu, X., Zhu, J., & Liu, Y. (2024) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2024.104328 · Cited by 3
SCP: Soft Conditional Prompt Learning for Aerial Video Action Recognition
Wang, X., Xian, R., Guan, T., Liu, F., & Manocha, D. (2024) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros58592.2024.10802074 · Cited by 5
MARINE: A Computer Vision Model for Detecting Rare Predator-Prey Interactions in Animal Videos
Katona, Z., Ziabari, S. S. M., & Nejadasl, F. K. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-81821-9_11 · Cited by 3
Wang, C., Liu, F., Chen, Y., Kwok, C. F., Elliott, M. S., Peña‐Solórzano, C. A., McCarthy, D. J., Frazer, H. M. L., & Carneiro, G. (2025) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2025.3558508 · Cited by 3
“…as input the feature map F ∈ F ⊂ R ¯ H × ¯ W × ¯ D ( ¯ H ≤ H , ¯ W ≤ W ) extracted from an image x by a model backbone (e.g., ResNet [39] and ViTs [40]) (we denote the backbone as f θ f : X → F ), and learns N × C class-specificprototypes P = { p n } N × C n =1 ,with N denotingthenumber of…”
A simple monocular depth estimation network for balancing complexity and accuracy
Liu, X., Tang, S., Feng, M., Guo, X., Zhang, Y., & Wang, Y. (2025) · Scientific Reports · doi:10.1038/s41598-025-97568-1 · Cited by 3
“To delve deeper into the impact of the number of bins ( N ) on the model’s performance, we systematically trained models using AbsRel as the evaluation metric for different N values (32, 64, 128, 256, 384, 512).”
An improved u-net for image segmentation of iron roughnecks on offshore oil drilling platforms
Zhang, Q., Zhu, H., Lv, Y., Jiang, D., Li, Z., Chauhan, S., & Vashishtha, G. (2025) · Measurement · doi:10.1016/j.measurement.2025.117689 · Cited by 3
MG-SSAF: An advanced vision Transformer
Yang, S., Hu, C., Xie, L., Lee, F., & Chen, Q. (2025) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2025.104578 · Cited by 3
Combatting Data Imbalance and Noise in Micro-Action Recognition
Wang, C., Chen, W., Cui, X., Zhao, Y., Qi, Z., Huang, P., Liu, X., & Zhang, W. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3762088 · Cited by 3
“MViT integrated a feature pyramid structure into a Transformer [10], while Video Swin Transformer [30] and Swin Transformer V2 [29] used shifted windows for efficient local-global modeling.”
Selective Volume Mixup for Video Action Recognition
Tan, Y., Qiu, Z., Hao, Y., Yao, T., & Mei, T. (2026) · International Journal of Computer Vision · doi:10.1007/s11263-025-02605-1 · Cited by 3
“MViT (Fan et al., 2021) and Video Swin (Liu, Ning, et al., 2022) follow the philosophy of CNNs, where the channel dimen-sion increases while the spatial resolution shrinks with the layer going deeper, to reduce the computational cost.”
Zhang, B., Wan, J., Zhao, Y., Tong, Z., & Du, Y. (2022) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2022.105055 · Cited by 3
DOAD: Decoupled One Stage Action Detection Network
Chang, S., Wang, P., Wang, F., Feng, J., & Shou, M. Z. (2023) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00314 · Cited by 3
“Benefit from the self-attention mechanism which broadens wider receptive field with fewer parameters and lower computation costs, those methods [3, 1, 25, 9, 52, 30] present state-of-the-art performance and impressive potential.”
BackgroundMethodGraphVid: It only Takes a Few Nodes to Understand a Video
Kosman, E., & Castro, D. D. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19833-5_12 · Cited by 3
Learning Transferable Spatiotemporal Representations from Natural Script Knowledge
Zeng, Z., Ge, Y., Liu, X., Chen, B., Luo, P., Xia, S., & Ge, Y. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02210 · Cited by 3
“, CVRL [42], MViT [16], SCVRL [14], SVT [44], and VideoMAE [49].”
BackgroundEvaluating Transformers for Lightweight Action RecognitionPreprint
Koot, R. E., Hennerbichler, M., & Lu, H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.09641 · Cited by 8
“Recent transformer-based models [3, 4, 11, 20, 25, 27, 29, 39] show much promise in dealing with the complexity of GPU 1 GPU 2 Mobile Device 1”
BackgroundMethodInfluentialPR-YOLO: Improved YOLO for fast protozoa classification and segmentationPreprint
Yang, W., CHEN, S., Chen, G., & SHI, Q. (2023) · Research Square · doi:10.21203/rs.3.rs-3199595/v1 · Cited by 3
Lin, Z., Wu, Y., Chen, J., & Wang, S. (2024) · Guidance Navigation and Control · doi:10.1142/s2737480724410073 · Cited by 2
Xu, Z., Qin, J., Li, C., Bu, D., & Zhao, Y. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72104-5_47 · Cited by 2
Boosting Depth Estimation for Self-Driving in a Self-Supervised Framework via Improved Pose Network
Dayoub, Y., Savchenko, A. V., & Makarov, I. (2024) · IEEE Open Journal of the Computer Society · doi:10.1109/ojcs.2024.3505876 · Cited by 2
“Mean-while, Zhao et al. [59] focused on model performance and utilized the MViT transformer [9] to combine local and global features.”
Towards Hierarchical Temporal Excitation for Video Violence Recognition
Mao, A., Wu, W., Yan, W., Li, Y., & Wang, H. (2025) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2024.3522201 · Cited by 2
Self-citationST-HViT: spatial-temporal hierarchical vision transformer for action recognition
Xia, L., & Fu, W. (2025) · Pattern Analysis and Applications · doi:10.1007/s10044-024-01407-4 · Cited by 2
SSIM: self-supervised learning method based on spatially selected shifts and irregular image masking
Shao, Y., Wang, Z., & Wang, L. (2025) · The Journal of Supercomputing · doi:10.1007/s11227-025-07013-3 · Cited by 2
“DINO utilizes the self-attention mechanism in Vision Transformer to enhance the expressiveness and robustness of the features [1, 24].”
Metadata
- Authors
- Fan, H., Xiong, B., Mangalam, K., Li, Y., Yan, Z., Malik, J., & Feichtenhofer, C.
- Affiliations
- Meta (Israel) (IL)
- Meta (United States) (US)
- Venue
- 2021 IEEE/CVF International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2021-10
- Pages
- 6804-6815
- OpenAlex
- W3202698164
- Cited by
- 1,407
- References
- 111
- Citations by year
- 2026: 147 · 2025: 345 · 2024: 380 · 2023: 349 · 2022: 176 · 2021: 10
- FWCI
- 74.81
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- arXiv (Cornell University)
- TIB Data Manager
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- video recognition, image classification, feature pyramid, spatial resolution reduction, computational efficiency
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
