The DOI matches its DataCite record, with a small difference in title or year; 56 of the 56 references it cites were found in a registry or an index.
Transformer in Transformer
Open accessAbstract
Transformer is a new kind of neural architecture which encodes the input data as powerful features via the attention mechanism. Basically, the visual transformers first divide the input images into several local patches and then calculate both representations and their relationship. Since natural images are of high complexity with abundant detail and color information, the granularity of the patch dividing is not fine enough for excavating features of objects in different scales and locations. In this paper, we point out that the attention inside these local patches are also essential for building visual transformers with high performance and we explore a new architecture, namely, Transformer iN Transformer (TNT). Specifically, we regard the local patches (e.g., 16$\times$16) as "visual sentences" and present to further divide them into smaller patches (e.g., 4$\times$4) as "visual words". The attention of each word will be calculated with other words in the given visual sentence with negligible computational costs. Features of both words and sentences will be aggregated to enhance the representation ability. Experiments on several benchmarks demonstrate the effectiveness of the proposed TNT architecture, e.g., we achieve an 81.5% top-1 accuracy on the ImageNet, which is about 1.7% higher than that of the state-of-the-art visual transformer with similar computational cost. The PyTorch code is available at https://github.com/huawei-noah/CV-Backbones, and the MindSpore code is available at https://gitee.com/mindspore/models/tree/master/research/cv/TNT.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“For example, in multi-tenant architectures, system prompts are shared across multiple requests and frequently contain detailed instructions or examples to guide the model [19].”
From HotPrefix: Hotness-Aware KV Cache Scheduling for Efficient Prefix Sharing in LLM Inference Systems (2025) · cites this paper as [19]
In the citing paper
“PromptCache [19] proposes modular reuse beyond prefixes but sacrifices accuracy, with reported performance drops of up to 43% [72], making it unsuitable for tasks requiring high precision.”
From HotPrefix: Hotness-Aware KV Cache Scheduling for Efficient Prefix Sharing in LLM Inference Systems (2025) · cites this paper as [19]
In the citing paper
“related experiments results further demonstrate ViT also has remarkable representation capacity when dealing with images and videos [19].”
From Deepfake Video Detection with Spatiotemporal Dropout Transformer (2022) · cites this paper as [19]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 6×
“Compared to CNNs, TNT can outperform the widely-used ResNet and RegNet.”
BackgroundMethodInfluentialVaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 3×
“The effect of #heads in standard transformer has been investigated in multiple works [21, 35] and a head width of 64 is recommended for visual tasks [9, 31].”
BackgroundMethodImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 1×
“In the past decade, the mainstream deep neural architectures used in the computer vision (CV) are mainly established on convolutional neural networks (CNNs) [17, 12, 11].”
BackgroundFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472 · cited 1×
“As a general backbone network, TNT can also be applied with multi-scale vision models like Faster RCNN [29].”
MethodBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472 · cited 7×
“The recent transformerbased models like ViT [9] and DeiT [31] are compared.”
MethodInfluentialImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · cited 3×
“ImageNet ILSVRC 2012 [26] is an image classification benchmark consisting of 1.”
BackgroundInfluentialSqueeze-and-Excitation NetworksPreprint
Hu, J., Shen, L., & Sun, G. (2017) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.48550/arxiv.1709.01507 · Cited by 37,845 · cited 1×
“Inspired by squeeze-and-excitation (SE) network for CNNs [16], we propose to explore channel-wise attention for transformers.”
MethodPyTorch: An Imperative Style, High-Performance Deep Learning Library
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al. (2019) · arXiv · doi:10.48550/arxiv.1912.01703 · Cited by 31,473
Feature Pyramid Networks for Object Detection
Lin, T.-Y., Dollar, P., Girshick, R., He, K., Hariharan, B., & Belongie, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.106 · Cited by 30,939 · cited 2×
“We evaluate TNT-S and DeiT-S on Faster RCNN with FPN [21].”
BackgroundMethod
Research works printed but not linked: 5
- [6][6] Mark Chen, Alec Radford, Rewon Child, Jeffrey Wu, Heewoo Jun, David Luan, and Ilya Sutskever. Generative pretraining from pixels. In ICML , 2020. -
- [11][11] Kai Han, Yunhe Wang, Hanting Chen, Xinghao Chen, Jianyuan Guo, Zhenhua Liu, Yehui Tang, An Xiao, Chunjing Xu, Yixing Xu, et al. A survey on vision transformer. arXiv preprint arXiv:2012.12556 , 2020. - · arXiv
- [23][23] Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101 , 2017. - · arXiv
- [26][26] Omkar M Parkhi, Andrea Vedaldi, Andrew Zisserman, and CV Jawahar. Cats and dogs. In CVPR , pages 3498-3505. IEEE, 2012. -
- [37][37] Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research , 9(11), 2008. -
Cited by
Sorted by citations
Items 501–600 of 1,009
Medical Image Classification with a Hybrid SSM Model Based on CNN and Transformer
Hu, C., Cao, N., Zhou, H., & Guo, B. (2024) · Electronics · doi:10.3390/electronics13153094 · Cited by 22
“Han et al. [35] introduced the Transformer model, which has demonstrated significant success in NLP. Inspired by this, an increasing number of studies have applied the Trans-former model to visual tasks, including image classification, semantic segmentation, and object detection.”
MethodFedsNet: the real-time network for pedestrian detection based on RT-DETR
Peng, H., & Chen, S. (2024) · Journal of Real-Time Image Processing · doi:10.1007/s11554-024-01523-8 · Cited by 24
“Subsequently, an increasing number of scholars have explored the application of this new framework to various computer vision tasks, including Image GPT [20] for image classification tasks and TNT [21] for image recognition tasks.”
Hybrid Proposal Refiner: Revisiting DETR Series from the Faster R-CNN Perspective
Zhao, J., Wei, F., & Xu, C. X. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01649 · Cited by 22
“…debut in 2017, the Transformer [46] has revolutionized a wide range of NLP tasks and has swiftly expanded its influence into the realm of computer vision, proving instrumental in tasks such as image recognition [5, 10, 12, 17, 34, 45] and object detection [4, 22, 31, 32, 37, 49, 55, 57, 59, 60].”
BackgroundSelf-citationZhang, T., Lian, C., Xu, B., Su, Y., & Zeng, Z. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.112175 · Cited by 24
Aden, I., Child, C., & Reyes-Aldasoro, C. (2024) · Big Data and Cognitive Computing · doi:10.3390/bdcc8050047 · Cited by 22
“Although the TNT model is primarily designed for visual tasks, its methodological innovations provide useful parallels for text-based applications like ICD prediction [7].”
MethodNavigating Cultural Chasms: Exploring and Unlocking the Cultural POV of Text-To-Image Models
Ventura, M., Ben-David, E., Korhonen, A., & Reichart, R. (2023) · Transactions of the Association for Computational Linguistics · doi:10.48550/arxiv.2310.01929 · Cited by 22
“The introduction of foundational architectures of Transformer (Vaswani et al., 2017) and ViT (Han et al., 2021) served as powerful cata-lysts for developing versatile multilingual text encoders, including mBERT, mT5, and XLM models (Devlin, 2018; Xue et al., 2020; Goyal et al., 2021; Scao et al., 2022).”
InfluentialSMPConv: Self-Moving Point Representations for Continuous Convolution
Kim, S., & Park, E. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00992 · Cited by 27
Robust transformer with locality inductive bias and feature normalization
Manzari, O. N., Kashiani, H., Dehkordi, H. A., & Shokouhi, S. B. (2023) · Engineering Science and Technology an International Journal · doi:10.1016/j.jestch.2022.101320 · Cited by 19
“Visualizing Attention Maps To present qualitative result comparisons to TNT as baseline for our improved model, we compare the attention maps generated by the TNT [29] and the LNL using CAM [58].”
MethodViT-based Terrain Recognition System for wearable soft exosuit
Yang, F., Chen, C., Wang, Z., Chen, H., Liu, Y. Z., Li, G., & Wu, X. (2023) · Biomimetic Intelligence and Robotics · doi:10.1016/j.birob.2023.100087 · Cited by 20
CPSAA: Accelerating Sparse Attention Using Crossbar-Based Processing-In-Memory Architecture
Li, H., Jin, H., Zheng, L., Liao, X., Huang, Y., Liu, C., Xu, J., Duan, Z., Chen, D., & Gui, C. (2023) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2023.3344524 · Cited by 21
“Attention is also widely used in computer vision tasks [17] including image classification [6, 18], object detection [4, 30], image recognition [12, 46], etc.”
BackgroundDynamic Group Transformer: A General Vision Transformer Backbone with Dynamic Group Attention
Liu, K., Wu, T., Liu, C., & Guo, G.-D. (2022) · International Joint Conference on Artificial Intelligence · doi:10.48550/arxiv.2203.03937 · Cited by 22
PyramidTNT: Improved Transformer-in-Transformer Baselines with Pyramid ArchitecturePreprint
Han, K., Guo, J., Tang, Y., & Wang, Y. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2201.00978 · Cited by 29
“We utilize the same training strategy as in DeiT [31] and TNT [11], as described in Table 2.”
BackgroundMethodInfluentialSelf-citationSimViT: Exploring a Simple Vision Transformer with Sliding Windows
Li, G., Xu, D., Xing, C., Si, L., & Zheng, C. (2022) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme52920.2022.9859907 · Cited by 30
A Survey of Generative Techniques for Spatial-Temporal Data Mining
Zhang, Q., Wang, H., Wen, H., Long, C., Su, L., He, X., Wu, T., Jensen, C. S., Yiu, S., & Yin, H. (2026) · Data Science and Engineering · doi:10.1007/s41019-026-00346-w · Cited by 25
“Based on existing studies [ 15, 16, 48–66 ], we provide illus - trations of Large Language Models as ST Generators as follows: ST dependency mechanisms.”
Transformer-based EEG Decoding: A Survey
Zhang, H.-D., & Li, H. (2025) · IEEE Transactions on Human-Machine Systems · doi:10.48550/arxiv.2507.02320 · Cited by 21
“2) TNT & TNT-like Model Application : Theoretically, TNT [133] extends the ViT with a dual-layer structure that captures both micro and macro image features.”
Neri, F., Yang, M., & Xue, Y. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.114052 · Cited by 22
Redhaei, A. A., Fraihat, S., & Al‐Betar, M. A. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11286-8 · Cited by 19
“The following lines demonstrate the mathematical modeling used to calculate the total number of FLOPs in transformer-based models as stated in (Han et al. 2021).”
Gaussian Processes in Power Systems: Techniques, Applications, and Future Works
Tan, B.-D., Su, T., Weng, Y.-T., Ye, K.-T., Pareek, P., Vorobev, P., Nguyen, H., Zhao, J.-B., & Deka, D. (2025) · Applied Energy · doi:10.48550/arxiv.2505.15950 · Cited by 21
“A variety of machine learning (ML) techniques have been introduced for power system analysis, including shallow learning methods such as support vector machines (SVM) [3], as well as newer deep learning models like Transformer [4].”
Chen, X., Liu, C., Xia, H., & Chi, Z. (2024) · Control Engineering Practice · doi:10.1016/j.conengprac.2024.106165 · Cited by 24
Correlation-Aware Cross-Modal Attention Network for Fashion Compatibility Modeling in UGC Systems
Cui, K., Liu, S.-H., Feng, W., Deng, X.-J., Gao, L., Cheng, M., Lu, H., & Yang, L. T. (2024) · ACM Trans. Multim. Comput. Commun. Appl. · doi:10.1145/3698772 · Cited by 21
ConVision Benchmark: A Contemporary Framework to Benchmark CNN and ViT Models
Vijayakumar, S. B., Chitty-Venkata, K. T., Arya, K., & Somani, A. K. (2024) · AI · doi:10.3390/ai5030056 · Cited by 23
“…[41] CvT [42] DeepViT [43] EdgeNeXt [44] EfficientFormer [45] FocalTransformer [46] GC-ViT [47] LeViT [48] LVT [49] Max-ViT [50] MLP-Mixer [51] MobileFormer [52] PiT [53] PoolFormer [54] PVT [55] Region-ViT [56] SepViT [57] Swin [15] T2T-ViT [14] TNT [58] Twins [16] VAN [59] Vision Transformer [13]”
GM-DETR: Research on a Defect Detection Method Based on Improved DETR
Liu, X., Yang, X., Shao, L., Wang, X., Gao, Q., & Shi, H. (2024) · Sensors · doi:10.3390/s24113610 · Cited by 29
“The model adopts a transformer [28] encoder – decoder architecture, where the self-attention mechanism explicitly models all pairwise elements in the sequence.”
BackgroundMethodInfluentialGuo, J.-J., Su, R., & Ye, J. (2024) · Neural Networks · doi:10.1016/j.neunet.2024.106403 · Cited by 21
Self-citationParZC: Parametric Zero-Cost Proxies for Efficient NAS
Dong, P.-J., Li, L., Pan, X., Wei, Z., Liu, X., Wang, Q., & Chu, X.-W. (2024) · AAAI Conference on Artificial Intelligence · doi:10.48550/arxiv.2402.02105 · Cited by 21
Lu, W., & Chen, X. (2024) · Frontiers in Energy Research · doi:10.3389/fenrg.2024.1355222 · Cited by 21
PATrans: Pixel-Adaptive Transformer for edge segmentation of cervical nuclei on small-scale datasets
Hu, H.-X., Zhang, J., Yang, T., Hu, Q., Yu, Y., & Huang, Q. (2023) · Comput. Biol. Medicine · doi:10.1016/j.compbiomed.2023.107823 · Cited by 21
FDViT: Improve the Hierarchical Architecture of Vision Transformer
Xu, Y., Li, C., Li, D., Sheng, X., Jiang, F., Tian, L., & Sirasao, A. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00547 · Cited by 17
“…computational cost caused by global self-attention operation, many methods constrain the range of attention within a local region [34, 14, 29, 48] or cooperate with local attention [20, 61, 9, 31, 60] to improve the efficiency of vision transformer while keeping the performance of the model.”
A Swin-transformer-based model for efficient compression of turbulent flow data
Zhang, M., Yousif, M. Z., Yu, L., & Lim, H.-C. (2023) · Physics of Fluids · doi:10.1063/5.0160755 · Cited by 20
TransDetector: A Transformer-Based Detector for Underwater Acoustic Differential OFDM Communications
Li, Y., Wang, S., Liu, D., Zhou, C., & Gui, Z. (2023) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2024.3367179 · Cited by 21
“In addition, it has been shown that the Transformer performs well in various kinds of fields, such as target detection and computer vision [21], [22], [23].”
BackgroundAre Large Kernels Better Teachers than Transformers for ConvNets?
Huang, T., Yin, L., Zhang, Z. (., Shen, L.-J., Fang, M., Pechenizkiy, M., Wang, Z.-Y., & Liu, S.-W. (2023) · International Conference on Machine Learning · doi:10.48550/arxiv.2305.19412 · Cited by 21
“3 TNT-S (Han et al., 2021) Supervised 300 224×224 24M 5.”
Ultra-High Resolution SVBRDF Recovery from a Single Image
Guo, J., Lai, S.-C., Tu, Q.-H., Tao, C., Zou, C., & Guo, Y.-W. (2023) · ACM Transactions on Graphics · doi:10.1145/3593798 · Cited by 21
“Transformer-in-transformer (TNT) [Han et al. 2021] models both patch-level and pixel-level representation by utilizing an outer transformer to process the patch embeddings and an inner transformer to model the relation between pixel embeddings.”
MethodSelf-citationMulti-Scale Human-Object Interaction Detector
Cheng, Y.-M., Wang, Z., Zhan, W.-H., & Duan, H. (2023) · IEEE transactions on circuits and systems for video technology (Print) · doi:10.1109/tcsvt.2022.3216663 · Cited by 21
“Built upon their success, many efforts have been devoted to designing better transformer based architectures for prevalent vision tasks, including image classification [19], [20], [21], [22], object detection [23], [24] and semantic segmentation [25], [26].”
BackgroundConvolution-Enhanced Vision Transformer Network for Smoke Recognition
Cheng, G., Zhou, Y., Gao, S., Li, Y., & Yu, H. (2023) · Fire Technology · doi:10.1007/s10694-023-01378-8 · Cited by 25
What Makes for Good Tokenizers in Vision Transformer?
Qian, S., Zhu, Y., Li, W., Li, M., & Jia, J. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3231442 · Cited by 15
“Recent progress that exploited convolutional stem [8], [12], multi-scale embeddings [21], [46], [47], or an additional transformer that models sub-token [50] can be classified in this category.”
BackgroundSelected Topics in Time Series Forecasting: Statistical Models vs. Machine Learning
Tjøstheim, D. (2025) · Entropy · doi:10.3390/e27030279 · Cited by 20
“One example of further development of an aspect of transformer methodology is [57].”
Human-Activity AGV Quality Assessment: A Benchmark Dataset and an Objective Evaluation Metric
Zhang, Z., Sun, W., Li, X., Li, Y., Ge, Q., Jia, J., Zhang, Z., Ji, Z., Sun, F., Jui, S., et al. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3754731 · Cited by 14
“For F inner ,i and F cross ,i , we apply a CNN followed by a Transformer encoder [23] to further refine the inner-body and cross-body features: where Transformer denotes the Transformer encoder, and the Conv represents a two-layer CNN. Finally, we concate-nate them and then apply a CNN layer…”
Liu, Z., Duan, P., Xue, X., Zhang, C., Yue, W., & Zhang, B. (2024) · Applied Soft Computing · doi:10.1016/j.asoc.2024.112524 · Cited by 21
Rate-Distortion-Cognition Controllable Versatile Neural Image Compression
Liu, J.-M., Feng, R., Qi, Y.-P., Chen, Q.-Y., Chen, Z.-B., Zeng, W.-J., & Jin, X. (2024) · European Conference on Computer Vision · doi:10.48550/arxiv.2407.11700 · Cited by 20
LBRT: Local-Information-Refined Transformer for Image Copy–Move Forgery Detection
Liang, P., Li, Z., Tu, H., & Zhao, H. (2024) · Italian National Conference on Sensors · doi:10.3390/s24134143 · Cited by 20
“…Transformer can achieve out-standing performance in other CV tasks, such as image classification (DeiT [27], Swin Trans-former [28], PVT [29], TNT [30]), object detection (DETR [31], and Deformable DETR [32]) and image segmentation (SETR [33] and TransUNet [34]), effectively helping networks to…”
BackgroundMethodDual-stage temporal perception network for continuous sign language recognition
Huang, Z., Xue, W., Zhou, Y., Sun, J., Wu, Y., Yuan, T., & Chen, S. (2024) · The Visual Computer · doi:10.1007/s00371-024-03516-x · Cited by 20
“…a temporal modeling module (connecting 1D-CNN with BiLSTM in series [6, 14–16, 23], connecting 1D-CNN with BiGRU in parallel [13] , or BiLSTM [3, 11]) to model Transformer has recently been achieving success in various fields, such as natural language processing [25] and computer vision [26, 27].”
SWAT: An Efficient Swin Transformer Accelerator Based on FPGA
Dong, Q.-W., Xie, X., & Wang, Z.-F. (2024) · Asia and South Pacific Design Automation Conference · doi:10.1109/asp-dac58780.2024.10473931 · Cited by 20
“Recently, Transformer has achieved impressive performance in computer vision tasks [1]–[3].”
BackgroundFAFuse: A Four-Axis Fusion framework of CNN and Transformer for medical image segmentation
Xu, S.-K., Xiao, D., Yuan, B., Liu, Y., Wang, X., Li, N., Shi, L., Chen, J., Zhang, J.-X., Wang, Y., et al. (2023) · Comput. Biol. Medicine · doi:10.1016/j.compbiomed.2023.107567 · Cited by 20
Self-citationLearning Local to Global Feature Aggregation for Speech Emotion Recognition
Lu, C., Lian, H.-L., Zheng, W.-M., Zong, Y., Zhao, Y., & Li, S.-N. (2023) · Interspeech · doi:10.48550/arxiv.2306.01491 · Cited by 20
“Besides, in Equation (3) and (4), MSA(·), MLP(·) and LN(·) are the operations of Multihead Self-Attention (MSA), MultiLayer Perceptron (MLP), and Layer Normalization (LN), respectively, according to [14], [18].”
BackgroundMethodInfluentialLocal Reversible Transformer for semantic segmentation of grape leaf diseases
Zhang, X., Li, F., Jin, H., & Mu, W. (2023) · Applied Soft Computing · doi:10.1016/j.asoc.2023.110392 · Cited by 20
Edge Enhanced Image Style Transfer via Transformers
Zhang, C., Dai, Z., Cao, P., & Yang, J. (2023) · International Conference on Multimedia Retrieval · doi:10.1145/3591106.3592257 · Cited by 22
“Inherited the ability to capture the longrange dependencies from transformers in natural language processing (NLP), vision transformers have been developed in a wide variety of vision tasks, including image classification [12, 18, 21, 26, 28, 41, 48, 49, 59, 60, 66, 72, 74], object detection [5, 7, 13, 42, 79], semant…”
BackgroundLiteVL: Efficient Video-Language Learning with Enhanced Spatial-Temporal Modeling
Chen, D., Tao, C., Hou, L., Shang, L.-F., Jiang, X., & Liu, Q. (2022) · Conference on Empirical Methods in Natural Language Processing · doi:10.48550/arxiv.2210.11929 · Cited by 20
“Transformers (Vaswani et al., 2017), originally designed for natural language tasks, have recently been applied to the computer vision domain to model images and videos (Dosovitskiy et al., 2021; Liu et al., 2021; Touvron et al., 2021; Wang et al., 2021; Han et al., 2021).”
MethodSun, J., Zhang, J., Gao, X., Wang, M.-T., Ou, D.-H., Wu, X., & Zhang, D.-J. (2022) · Remote Sensing · doi:10.3390/rs14091968 · Cited by 20
Daily insider threat detection with hybrid TCN transformer architecture
Ye, X. L., Cui, H., Luo, F., Wang, J., Xiong, X., Zhang, W., Yu, J., & Zhao, W. (2025) · Scientific Reports · doi:10.1038/s41598-025-12063-x · Cited by 26
“Through the self-attention mechanism, Transformers can capture relationships between any two-time steps in the input sequence, enhancing the richness and expressiveness of feature representations 14,15 .”
Sudden Fall Detection of Human Body Using Transformer Model
Kibet, D., So, M.-S., Kang, H., Han, Y., & Shin, J.-H. (2024) · Italian National Conference on Sensors · doi:10.3390/s24248051 · Cited by 19
Deep multi-similarity hashing via label-guided network for cross-modal retrieval
Wu, L., Qin, Q., Hou, J., Dai, J., Huang, L., & Zhang, W. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128830 · Cited by 18
Add-Vit: CNN-Transformer Hybrid Architecture for Small Data Paradigm Processing
Chen, J., Wu, P., Zhang, X., Xu, R., & Liang, J. (2024) · Neural Processing Letters · doi:10.1007/s11063-024-11643-8 · Cited by 24
“[13,17,24–26].”
Hierarchical Attention Network for Open-Set Fine-Grained Image Recognition
Sun, J., Wang, H., & Dong, Q. (2024) · IEEE transactions on circuits and systems for video technology (Print) · doi:10.1109/tcsvt.2023.3325001 · Cited by 19
“Vision transformers [4], [5], [6], [7], [8] have shown their effectiveness in object recognition, mainly due to their multiple spatial self-attention operations.”
BackgroundMethodInfluentialMultimodal knowledge graph construction for risk identification in water diversion projects
Wang, L., Liu, X., Liu, Y., Li, H., Liu, J., & Yang, L. (2024) · Journal of Hydrology · doi:10.1016/j.jhydrol.2024.131155 · Cited by 21
Learning Nighttime Semantic Segmentation the Hard Way
Liu, W., Cai, J., Li, Q., Liao, C., Cao, J., He, S., & Yu, Y. (2024) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3650032 · Cited by 10
Swin transformer and ResNet based deep networks for low-light image enhancement
Xu, L., Hu, C., Zhang, B., Wu, F., & Cai, Z. (2023) · Multimedia tools and applications · doi:10.1007/s11042-023-16650-w · Cited by 19
“…performance of the proposed approach under various low-light conditions, images in NTM are divided into two test subsets based on the image Mean Pixel Intensity(MPI) value, named as NTM1 and NTM2, whose MPI values are in [0, 30) and [30, 50) respectively, and 582 and 82 images are in NTM1 and NTM2.”
Liu, Z., & Shen, L. (2023) · Comput. Biol. Medicine · doi:10.48550/arxiv.2302.02314 · Cited by 19
“Additionally, Han et al. [21] developed a TNT approach, in which local patches are regarded as “visual sentences.””
BackgroundAli, L., Jassmi, H. A., Khan, W., & Alnajjar, F. S. K. (2022) · Buildings · doi:10.3390/buildings13010055 · Cited by 15
Zhang, H.-Q., Chen, H., Qin, J., Wang, B.-N., Ma, G.-L., Wang, P., Zhong, D.-R., & Liu, J. (2022) · Frontiers in Oncology · doi:10.3389/fonc.2022.925903 · Cited by 19
“After that, transformer in transformer (TNT) (15) combines the patch-level and pixel-level transformer blocks; thus, this architecture can effectively represent the feature relations between and within regions.”
BackgroundMethodInfluentialHierarchical Interdisciplinary Topic Detection Model for Research Proposal Classification
Xiao, M., Qiao, Z., Fu, Y., Dong, H., Du, Y., Wang, P., Xiong, H., & Zhou, Y. (2023) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2023.3248608 · Cited by 21
“Inspired by ViT [17] and TNT [18], we set the operation as: (1) a Vectorization Operation on W i .”
MethodLearning Viewpoint-Agnostic Visual Representations by Recovering Tokens in 3D Space
Shang, J., Das, S., & Ryoo, M. (2022) · Neural Information Processing Systems · doi:10.48550/arxiv.2206.11895 · Cited by 19
“We use Swin [36] (Tiny) and TnT [24] (Small).”
BackgroundMethodSP-ViT: Learning 2D Spatial Priors for Vision TransformersPreprint
Zhou, Y., Xiang, W., Li, C., Wang, B., Wei, X., Zhang, L., Keuper, M., & Hua, X. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2206.07662 · Cited by 12
“1, including DeiT [20], TNT [11], T2T [27], Swin Trans-former [15], DeepViT [29], LV-ViT [13], Swin Transformer [15], CaiT [21], etc..”
BackgroundInfluentialA Novel Principal Component Analysis-Informer Model for Fault Prediction of Nuclear Valves
An, Z., Cheng, L., Guo, Y., Ren, M., Feng, W., Sun, B. C., Ling, J., Chen, H., Chen, W., Luo, Y., et al. (2022) · Machines · doi:10.3390/machines10040240 · Cited by 25
Multi-Tailed Vision Transformer for Efficient Inference
Wang, Y.-K., Du, B., Wang, W.-Y., & Xu, C. (2022) · Neural Networks · doi:10.48550/arxiv.2203.01587 · Cited by 19
“CrossViT [4] exploits multi-scale feature of the image in vision Transformer and TNT [15] focuses on investigating the attention inside a single patch and propose the divide of a single patch into multiple smaller patches.”
BackgroundInfluentialSelf-citationBrown, E. E. J., Svoboda, F., Meredith, N. P., Lane, N. D., & Horne, R. B. (2022) · Space Weather · doi:10.1029/2021sw002976 · Cited by 33
“The Vision Transformer was the first transformer architecture successfully used in image recognition (Dosovits-kiy et al., 2020).”
BackgroundInfluentialSearching for Efficient Multi-Stage Vision TransformersPreprint
Liao, Y., Karaman, S., & Sze, V. (2021) · arXiv.org · doi:10.48550/arxiv.2109.00642 · Cited by 19
“Subsequent works improve the performance of ViT on ImageNet through either better training [11, 23] or architectures [9, 14, 17, 26, 27, 42, 48, 52, 58, 61].”
BackgroundMethodFan, Z.-G., Wu, Y.-T., Ba, D., Zhang, M., Liu, Y., & Du, J. (2025) · Aerospace Science and Technology · doi:10.1016/j.ast.2025.110209 · Cited by 18
SD-Mamba: A lightweight synthetic-decompression network for cross-modal flood change detection
Shen, Y.-A., Yao, S., Qiang, Z.-K., & Pei, G. (2025) · International Journal of Applied Earth Observation and Geoinformation · doi:10.1016/j.jag.2025.104409 · Cited by 18
LS-VIT: Vision Transformer for action recognition based on long and short-term temporal difference
Chen, D., Wu, P., Chen, M., Wu, M., Zhang, T., & Li, C. (2024) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2024.1457843 · Cited by 21
“Recently, the adaptation of Transformers for 2D vision tasks (Dosovitskiy et al., 2021; Liu et al., 2021; Yuan et al., 2021; Han et al., 2021; Touvron et al., 2020; Chen et al., 2021) has witness a significant surge, especially in video action recognition (Bertasius et al., 2021; Arnab et al., 2021;…”
Chen, Y.-F., Zhang, X.-Q., He, Y., Peng, L., Pu, L., & Sun, F. (2024) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2024.106513 · Cited by 18
Cap4Video++: Enhancing Video Understanding With Auxiliary Captions
Wu, W., Wang, X., Luo, H., Wang, J., Yang, Y., & Ouyang, W. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3410329 · Cited by 9
“In recent years, there has been a trend towards adopting transformer-based architectures for video recognition, movingawayfromCNNsandtowardsVisionTransformers[65], [66],[67],[68].”
SecureNet: A Comparative Study of DeBERTa and Large Language Models for Phishing Detection
Mahendru, S., & Pandit, T. (2024) · IEEE International Conference on Big Data and Artificial Intelligence (BDAI) · doi:10.1109/bdai62182.2024.10692765 · Cited by 34
“Attention-based architectures [14] enhance interpretability by capturing linguistic patterns and contextual relationships in email data.”
BackgroundSPTR: Structure-Preserving Transformer for Unsupervised Indoor Depth Completion
Zhao, L., Zheng, W., Duan, Y., Zhou, J., & Lu, J. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3312664 · Cited by 15
RGB Road Scene Material Segmentation
Cai, S., Wakaki, R., Nobuhara, S., & Nishino, K. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-26284-5_16 · Cited by 10
“Related models LeViT [13] and TNT [14] also improved window-MSA by infusing extra local details.”
MethodEfficient Vision Transformer for Human-Centric AIoT Applications Through Token Tracking Assignment
Li, C., Peng, Y., Liu, G., Li, Y., Yang, X., & Chen, C. (2023) · IEEE Transactions on Consumer Electronics · doi:10.1109/tce.2023.3323373 · Cited by 20
“By breaking the input data into multiple chunks, a hierarchical structure is created that is akin to clauses and paragraphs in natural language [46], [47].”
BackgroundComposite Foundation Settlement Prediction Based on LSTM–Transformer Model for CFG
Li, Z., Peng, Y.-P., Li, J., & Tang, Z. (2024) · Applied Sciences · doi:10.3390/app14020732 · Cited by 18
The YOLO model that still excels in document layout analysis
Deng, Q., Ibrayim, M., Hamdulla, A., & Zhang, C. (2023) · Signal, Image and Video Processing · doi:10.1007/s11760-023-02838-y · Cited by 18
“The MCD module is inspired by the structural block of the ConvNeXt [11] network and the convolutional layer design is shown in Fig.”
Multi-Task Learning and Temporal-Fusion-Transformer-Based Forecasting of Building Power Consumption
Ji, W., Cao, Z., & Li, X. (2023) · Electronics · doi:10.3390/electronics12224656 · Cited by 19
Towards Transferable Adversarial Attacks on Image and Video Transformers
Wei, Z., Chen, J., Goldblum, M., Wu, Z., Goldstein, T., Jiang, Y., & Davis, L. S. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3331582 · Cited by 15
“TNT [29] utilizes an outer transformer block and an inner transformer block to learn relationships between patches and within patches separately.”
MethodMRFT: Multiscale Recurrent Fusion Transformer Based Prior Knowledge for Bit-Depth Enhancement
Wen, X. L., Nie, W., Liu, J., & Su, Y. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3260274 · Cited by 18
“Following this paradigm, a series of variants of ViT [24], [25], [26] have been proposed to improve the performance on vision tasks.”
Method2D Semantic Segmentation: Recent Developments and Future Directions
Guo, Y., Nie, G., Gao, W., & Liao, M. (2023) · Future Internet · doi:10.3390/fi15060205 · Cited by 24
Deep Factorized Metric Learning
Wang, C., Zheng, W., Li, J., Zhou, J., & Lu, J. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00741 · Cited by 12
FIT: Far-reaching Interleaved TransformersPreprint
Chen, T., & Li, L.-L. (2023) · arXiv.org · doi:10.48550/arxiv.2305.12689 · Cited by 18
“More recent approaches [33, 19, 20, 2, 1, 30, 29, 28, 22, 5, 21, 42, 18] explore the use of latent tokens that dynamically attend to data tokens and perform additional computations.”
BackgroundMethodSAT: Size-Aware Transformer for 3D Point Cloud Semantic Segmentation
Zhou, J., Xiong, Y., Chiu, C., Liu, F.-Y., & Gong, X. (2023) · arXiv.org · doi:10.48550/arxiv.2301.06869 · Cited by 18
“Recently, the pioneering work ViT [6] and its follow-ups [4, 13, 43, 52] apply a pure attention-based network to image classification and achieve promising performance.”
BackgroundDense-TNT: Efficient Vehicle Type Classification Neural Network Using Satellite Imagery
Luo, R.-K., Song, Y., Zhao, H., Zhang, Y., Zhang, Y., Zhao, N.-B., Huang, L.-P., & Su, R. (2022) · Italian National Conference on Sensors · doi:10.3390/s24237662 · Cited by 18
“Similarly, after careful comparison from literature, TNT [32] is selected as the variant in our proposed hybrid model.”
Multiple Instance Neuroimage Transformer
Singla, A., Zhao, Q., Daniel, K., Zhou, Y., Pohl, K. M., & Adeli, E. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-16919-9_4 · Cited by 13
“MiGNiT: Multiple Instance Global NiT In this architecture, we compute global attention [16] by additionally computing self-attention on the output class embeddings produced by each block using an NiT model block.”
MethodTraining Object Detectors from Scratch: An Empirical Study in the Era of Vision Transformer
Hong, W., Ren, W., Lao, J., Xie, L., Zhong, L., Wang, J., Chen, J., Liu, H., & Chu, W. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-024-01988-x · Cited by 4
PC-SwinMorph: Patch Representation for Unsupervised Medical Image Registration and Segmentation
Liu, L., Huang, Z., Liò, P., Schönlieb, C., & Avilés-Rivero, A. (2022) · arXiv.org · doi:10.48550/arxiv.2203.05684 · Cited by 18
Reconstructing 4D Spatial Intelligence: A Survey
Cao, Y., Lu, J., Huang, Z., Shen, Z., Zhao, C., Hong, F., Chen, Z., Li, X., Wang, W., Liu, Y., et al. (2026) · International Journal of Computer Vision · doi:10.1007/s11263-026-03072-y · Cited by 17
“Building on transformer-based advances [49], [50], [51], [52], [53], VGGT [54] further introduces an end-to-end framework that rapidly estimates these low-level 3D cues within seconds.”
ELAFormer: Early Local Attention in multi-scale vision transFormers
Zhou, X., Ren, Z., Zhang, Y., Jiang, Z., Yu, T., Luo, H., & Zhou, S. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113851 · Cited by 14
Zhang, Y., Feng, L., Shan, H., & Zhu, Z. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3439600 · Cited by 16
“…and projecting each patch through a fully connected layer (FCL), the application field of Transformer models [1] extends from NLP [2], [3] to various computer vision fields such as image classification [4], [5], [6], [7], object detection [8], [9], [10], [11], and multi-modal tasks [12], [13].”
QSIM: A Quantum-inspired hierarchical semantic interaction model for text classification
Hui, G., Zhang, P., Zhang, J., & Yang, C. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128658 · Cited by 13
DA-GAN: Dual-attention generative adversarial networks for real-world exquisite makeup transfer
Jiao, Q., Xu, Z., Wu, S., & Wong, H. (2024) · Pattern Recognition · doi:10.1016/j.patcog.2024.111049 · Cited by 18
Latent Semantic and Disentangled Attention
Chien, J., & Huang, Y. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3432631 · Cited by 20
“On the other hand, the effect on FLOPs by using different methods is comparable with that on parameter size [48].”
MethodBbouzidi, S., Hcini, G., Jdey, I., & Drira, F. (2024) · arXiv.org · doi:10.48550/arxiv.2406.03478 · Cited by 17
Accelerating ViT Inference on FPGA through Static and Dynamic Pruning
Parikh, D., Li, S., Zhang, B., Kannan, R., Busart, C., & Prasanna, V. K. (2024) · IEEE International Symposium on Field-Programmable Custom Computing Machines (FCCM) · doi:10.1109/fccm60383.2024.00018 · Cited by 18
“Vision Transformers (ViTs) [1] have demonstrated superior performance in comparison to Convolutional Neural Networks (CNNs) in various vision tasks [2], [3], [4], [5], [6], [7], [8].”
BackgroundTransformer-Based Fused Attention Combined with CNNs for Image Classification
Jiang, J., Xu, H., Xu, X.-L., Cui, Y., & Wu, J.-T. (2023) · Neural Processing Letters · doi:10.1007/s11063-023-11402-1 · Cited by 17
“Kai et al. [34] further refined the image structure by breaking it down into smaller tokens and proposed a transformer network that uses both internal and external connections.”
STTracker: Spatio-Temporal Tracker for 3D Single Object Tracking
Cui, Y., Li, Z.-H., & Fang, Z. (2023) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2023.3290524 · Cited by 17
“Therefore, following previous vision transformer works [24], [25], [26], [27], we also divide per-frame feature map into many non-overlapping local patch grids.”
BackgroundSegT: Separated edge-guidance transformer network for polyp segmentation.
Chen, F., Ma, H., & Zhang, W. (2023) · Mathematical biosciences and engineering : MBE · doi:10.3934/mbe.2023791 · Cited by 17
“TNT [34] adopts a transformer suitable for fine-grained image tasks to segment the original image patch and perform self-attention mechanism calculations in small units.”
MethodGupta, K., Singh, A., Yeduri, S. R., Srinivas, M. B., & Cenkeramaddi, L. R. (2022) · Journal of Ambient Intelligence and Humanized Computing · doi:10.1007/s12652-022-04506-4 · Cited by 17
“Feature transformation and non-linearity is obtained by applying MLP between self-attention layers which can be defined as (Han et al. 2021) where, Here, W is the weight and b is the bias term of fully-connected layer.”
BackgroundMethodInfluentialA CNN-transformer fusion network for COVID-19 CXR image classification
Cao, K., Deng, T., Zhang, C.-L., Lu, L.-M., & Li, L. (2022) · PLoS ONE · doi:10.1371/journal.pone.0276758 · Cited by 17
Metadata
- Authors
- Han, K., Xiao, A., Wu, E., Guo, J., Xu, C., & Wang, Y.
- Affiliations
- Huawei Technologies (China) (CN)
- University of Macau (MO)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2021
- Volume
- 34
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2103.00112
- OpenAlex
- W3133696297
- Cited by
- 1,394
- References
- 56
- Citations by year
- 2026: 14 · 2025: 276 · 2024: 457 · 2023: 387 · 2022: 186 · 2021: 74
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Natural Science Foundation of China (0015/2019/AKP, 62072449, 61632003, 0018/2019/AKP)
- Fundo para o Desenvolvimento das Ciências e da Tecnologia (0018/2019/AKP, 0015/2019/AKP)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Keywords
- vision transformer, attention mechanism, local patches, visual words, image classification, feature aggregation, computational efficiency
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
