Transformers for Vision: A Survey on Innovative Methods for Computer Vision
Open accessThe DOI matches its Crossref record; 120 of the 120 references it cites were found in a registry or an index.
Abstract
Transformers have emerged as a groundbreaking architecture in the field of computer vision, offering a compelling alternative to traditional convolutional neural networks (CNNs) by enabling the modeling of long-range dependencies and global context through self-attention mechanisms. Originally developed for natural language processing, transformers have now been successfully adapted for a wide range of vision tasks, leading to significant improvements in performance and generalization. This survey provides a comprehensive overview of the fundamental principles of transformer architectures, highlighting the core mechanisms such as self-attention, multi-head attention, and positional encoding that distinguish them from CNNs. We delve into the theoretical adaptations required to apply transformers to visual data, including image tokenization and the integration of positional embeddings. A detailed analysis of key transformer-based vision architectures such as ViT, DeiT, Swin Transformer, PVT, Twins, and CrossViT are presented, alongside their practical applications in image classification, object detection, video understanding, medical imaging, and cross-modal tasks. The paper further compares the performance of vision transformers with CNNs, examining their respective strengths, limitations, and the emergence of hybrid models. Finally, current challenges in deploying ViTs, such as computational cost, data efficiency, and interpretability, and explore recent advancements and future research directions including efficient architectures, self-supervised learning, and multimodal integration are discussed.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Transformer-based architectures have demonstrated remarkable performance in both vision and video tasks, thanks to their ability to model long-range dependencies and capture spatiotemporal correlations [7, 8].”
From Cross-dataset video deepfake detection using Transformer and CNN architectures (2026) · cites this paper as [7]
Citations
Cites
You Only Look Once: Unified, Real-Time Object Detection
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.91 · Cited by 53,803
Squeeze-and-Excitation Networks
Hu, J., Shen, L., & Sun, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00745 · Cited by 35,667
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534
SSD: Single Shot MultiBox Detector
Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.-Y., & Berg, A. C. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46448-0_2 · Cited by 22,128 · cited 1×
“YOLO [54], and SSD [55] are notable examples.”
Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., & Adam, H. (2018) · arXiv · doi:10.48550/arxiv.1802.02611 · Cited by 20,063
Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., & Adam, H. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_49 · Cited by 20,054
End-to-End Object Detection with Transformers
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58452-8_13 · Cited by 18,285
Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising
Zhang, K., Zuo, W., Chen, Y., Meng, D., & Zhang, L. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2662206 · Cited by 8,983
Masked Autoencoders Are Scalable Vision Learners
He, K., Chen, X., Xie, S., Li, Y., Dollar, P., & Girshick, R. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01553 · Cited by 8,489 · cited 1×
“…recent innovations, including incorporating hierarchical attention in Swin Transformers [6] and masked autoencoding in Masked AutoEncoder (MAE) [7], have substantially bridged the gaps in data efficiency and computational performance, ushering in a novel era in visual information processing.”
Analyzing and Improving the Image Quality of StyleGAN
Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J., & Aila, T. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00813 · Cited by 6,332
Research works printed but not linked: 9
- Keles, On the computational complexity of self-attention, 597
- Touvron, Training data-efficient image transformers & distillation through attention, 10347
- Saumya, Learn to pay attention, 1209
- Giuste, 2020, CIFAR-10 image classification using feature ensembles, arXiv:2002.03846 · arXiv
- Ge, 2021, YOLOX: Exceeding YOLO series in 2021, arXiv:2107.08430 · arXiv
- Dosovitskiy, 2020, An image is worth 16×16 words: Transformers for image recognition at scale, arXiv:2010.11929 · arXiv
- Chen, 2023, 3D TransUNet: Advancing medical image segmentation through vision transformers, arXiv:2310.07781 · arXiv
- Bianco, 2023, Improving image captioning descriptiveness by ranking and LLM-based fusion, arXiv:2306.11593 · arXiv
- Liu, 2024, Efficient partitioning vision transformer on edge devices for distributed inference, arXiv:2410.11650 · arXiv
Cited by
Kumru, E., Ugurlu, G., Sevindik, М., Ekinci, F., Güzel, M. S., Açıcı, K., & Akata, İ. (2025) · Biology · doi:10.3390/biology14070816 · Cited by 15
“In contrast, transformer-based architectures like ViT and MaxViT demonstrated diminished performance, likely due to their greater dependence on large datasets and inherently weaker inductive biases [17,18].”
An Overview Study of Deep Learning in Geophysics: Cross-Cutting Research to Advance Geoscience
Wenxue, Z., Shikun, D., Hongjun, T., Dexiang, Z., Ying, Z., & Jiang, F. (2025) · IEEE Access · doi:10.1109/access.2025.3586693 · Cited by 15
Kumar, R., Kim, Y., & Byun, Y.-C. (2025) · IEEE Access · doi:10.1109/access.2025.3600622 · Cited by 17
“Vision Transformer (ViT) heads represent a breakthrough in computer vision by adapting transformer architectures— originally designed for natural language processing—to image classification tasks [57].”
CDU-Net: An adaptive conditional denoising model for microseismic signals in complex environments
Li, J., Tang, B., Zhang, J., Zhang, S., Tang, S., Huang, X., Yi, G., & Song, C. S. (2026) · Underground Space · doi:10.1016/j.undsp.2026.02.002 · Cited by 8
Cross-dataset video deepfake detection using Transformer and CNN architectures
Petmezas, G. (2026) · Machine Vision and Applications · doi:10.1007/s00138-026-01809-w · Cited by 6
Redefining Deep Learning Across NLP, Image Classification and Beyond - Vision Transformers
Mathews, A., Moharana, S. K., Singh, P. N., & Pattnaik, S. (2025) · IEEE International Conference on ICT in Business Industry & Government (ICTBIG) · doi:10.1109/ictbig68706.2025.11323670 · Cited by 5
“The authors of [12] conducted an in-depth analysis of prominent transformer-based vision architectures, including ViT, DeiT, Swin Transformer, PVT, Twins, and CrossViT, highlighting their implementation across domains with medical imaging, and cross-modal tasks.”
Selvaraj, J., Almutairi, F., Aslam, S. M., & Umapathy, S. (2025) · Life · doi:10.3390/life15071124 · Cited by 7
“Also, Vision Transformers (ViTs) have gained prominence as an effective alternative, utilizing self-attention mechanisms to model long-range interactions and spatial relationships within an image [14,15].”
Enhanced fault diagnosis of rolling bearings using a VMD-CWT-ViT integrated framework
Khan, M. J., Sun, B., Songhua, X., & Junqi, H. (2026) · Measurement Science and Technology · doi:10.1088/1361-6501/ae4cbe · Cited by 4
“Hassija et al demonstrated that ViT outperformed conventional CNNs on multiple image recognition benchmarks, leveraging global attention to process entire images as a single entity [17].”
A lightweight transformer framework for open set anomaly segmentation in smart city applications
Prabha, M. M., & Suganthi, K. (2025) · Scientific Reports · doi:10.1038/s41598-025-28708-w · Cited by 3
Nofrizal, A. Y., Sonobe, R., Yamashita, H., Morita, A., & Ikka, T. (2026) · Journal of Food Composition and Analysis · doi:10.1016/j.jfca.2026.108919 · Cited by 4
Russo, M., Santoro, C., Santoro, F. F., & Tudisco, A. (2026) · Actuators · doi:10.3390/act15020091 · Cited by 3
“Vision Transformers (ViTs) [32] represent an emerging approach that leverages self-attention mechanisms to capture long-range dependencies in high-resolution plant images [33,34], showing promising improvements over CNNs for complex disease classification tasks.”
LIU, Y. J., Feng, S., & Liu, M. (2026) · International Scientific Technical and Economic Research · doi:10.71451/istaer2615 · Cited by 3
SwinT-PGK: a lightweight and accurate framework for tea leaf age grading
Xie, M., Wang, T., Hua, X. G., Gao, X., & Zhang, X. (2025) · Engineering Research Express · doi:10.1088/2631-8695/adf798 · Cited by 3
Marcos, L., Babyn, P., & Alirezaie, J. (2025) · International Journal of Imaging Systems and Technology · doi:10.1002/ima.70220 · Cited by 2
“This turns image patches into patch tokens, which serves as the transformer's input sequence, analogous to word tokens in Natural Language Processing (NLP) [32].”
Hamza, A., & Damaševičius, R. (2025) · Annals of Computer Science and Information Systems · doi:10.15439/2025f0901 · Cited by 2
“Regardless, both approaches address limitations [10].”
Darmatasia, & H, M. H. (2025) · JURNAL INSTEK (INFORMATIKA SAINS DAN TEKNOLOGI) · doi:10.24252/instek.v10i2.61891 · Cited by 2
HiGATE: hierarchical graph attention for multi-scale tissue encoder in computational pathology
Dad, I., He, J., & Shen, T. (2026) · Frontiers in Oncology · doi:10.3389/fonc.2026.1798040 · Cited by 2
“The fi eld has progressed from handcrafted features to DL (6, 35), ViTs (36), and GNNs (37), with current efforts focusing on hierarchical, multi-scale modeling (38).”
InfluentialSim, K. S., & Islam, M. T. (2026) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2026.1848963 · Cited by 5
“In addition, some models is able to learn multiple representations and is used in audiovisual speech recognition and cross-modal retrieval task (Palanisamy et al., 2025).”
Transformer-based learned dimensional collapse for silicon photonic inverse design
Arslan, A., Dasdemir, A. O., Sarikamis, F. A., Kıraz, B., Kıraz, A., Mağden, A., & Magden, E. S. (2026) · Physics and simulation of optoelectronic devices. Conference · doi:10.1117/12.3080973 · Cited by 1
“Local convolutional operators struggle to represent non-local electromagnetic coupling, 3, 12 while conventional dimensionality reduction techniques discard information essential for reliable inverse design.”
Hu, N., Li, S., & Tan, J. (2026) · Sensors · doi:10.3390/s26020627 · Cited by 1
Garg, P., Gupta, B., Garg, S., Jindal, N., Mandia, S., & SINGH, K. (2025) · IEEE International Conference on Intelligent Signal Processing and Effective Communication Technologies (INSPECT) · doi:10.1109/inspect67393.2025.11350330 · Cited by 1
“Combined structures that incorporate CNNs with attention systems or Vision Transformers (ViTs) have demonstrated potential in enhancing invisibility, data capacity, and strength all at once [9], [10].”
Islam, M. R., Emran, M. A., & Md. Wajed, A. L. I. (2026) · Signal Image and Video Processing · doi:10.1007/s11760-026-05162-3 · Cited by 1
“Subsequent work increased feature representation and efficiency through attention processes and dense connections, enabling detection of more complicated postures and object interactions [13].”
VLM-inspired contrastive learning framework RxVLM for robust handwritten medicine name understanding
Niu, D., Yi, J. J., & Sun, H. (2026) · The Visual Computer · doi:10.1007/s00371-026-04420-2 · Cited by 1
Wu, C., Yu, H., Liu, Y., & Gong, C. (2026) · Machines · doi:10.3390/machines14040371 · Cited by 3
Yang, Z., Khairuddin, A. S. M., Wong, W. R., Chuah, J. H., Noman, H. M. F., & Putri, T. W. O. (2026) · Neurocomputing · doi:10.1016/j.neucom.2026.133858 · Cited by 1
Liu, Z., Zhu, Y., Zhang, Z., & Zhang, T. (2026) · Sensors · doi:10.3390/s26154917 · Cited by 1
“Hassija et al. reported that ViT outperforms conventional CNNs on multiple recognition benchmarks [27].”
Multimodal Deep Learning and Foundation Models for Early Detection and Forecasting of Plant Diseases
Manda, T., Huang, T., Ding, Y., Dai, S., Yang, L., & Dai, T. (2026) · Plants · doi:10.3390/plants15172564 · Cited by 1
“These patches are linearly embedded and processed as sequential tokens through multi-head self-attention layers [27].”
Deteksi Deepfake Real-Time pada Perangkat Mobile Menggunakan Arsitektur MobileViT-CBAM Teroptimasi
Hutagalung, C. A., Akhmad, D. M., Resita, E., & Rahmi, T. (2025) · RIGGS Journal of Artificial Intelligence and Digital Business · doi:10.31004/riggs.v4i4.3634 · Cited by 1
“MobileViT menggabungkan konvolusi lokal dan transformer global, mengatasi kompleksitas ViT standar (O(n²)) (Lee et al., 2024; Palanisamy et al., 2025).”
Distilled vision transformers with CNN fusion for robust cashew apple maturity prediction
Lingamgunta, S., Jeevaratnam, M., Vincent, P. M. D. R., & M., F. S. A. (2026) · Frontiers in Plant Science · doi:10.3389/fpls.2026.1787609 · Cited by 1
“The selected dataset is also smaller, which additionally poses a signi fi cant challenge (Alves de Moraes et al., 2024; Charukeshi et al., 2024; Wakchaure et al., 2024; Feng et al., 2025; Hassija et al., 2025).”
Çifçi, M. A., Karataş, K., Yildirim, F., & DOĞAN, A. (2026) · Diagnostics · doi:10.3390/diagnostics16091361 · Cited by 1
“State space vision models argue that attention may not be the end of the story for efficient global context modeling [39], while robustness studies warn against treating any architecture family as automatically secure or reliable [40,41].”
Κούκιου, Γ. (2026) · Electronics · doi:10.3390/electronics15112402 · Cited by 1
Öztaş, D., Salehi, H., DEMİR, Ö., Açıcı, K., Güzel, M. S., Eraslan, E. C., Sevindik, М., Akata, I., & Ekinci, F. (2026) · Plant Systematics and Evolution · doi:10.1007/s00606-026-02005-z · Cited by 2
“Vision Transformers, in contrast, employ self-attention mechanisms to model long-range dependencies across entire images, offering enhanced global representation capacity at the cost of increased computational complexity (Khan et al. 2022; Hassija et al. 2025).”
InfluentialA Hybrid Attention-Enhanced Deep Learning Model for Robust Nighttime Traffic Sign Detection
Akbar, W., Li, S., Hussain, A., Shah, T. A., Soomro, A., & Haq, M. I. U. (2026) · International Conference on Robotics, Automation and Intelligent Transportation Systems (RAITS) · doi:10.1109/raits68656.2026.11580172 · Cited by 1
“Thedetection of traffic signs has been extensively studied using traditional image processing methods, handcrafted feature extraction, and modern deep learning approaches such as convolutional neural networks (CNNs) and transformer-based architectures [5].”
Hybrid CNN with angular margin supervision for robust face identification and verification
Nemavhola, A., Chibaya, C., & Viriri, S. (2026) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2026.1816796 · Cited by 1
“More recently, transformer-based architectures have demonstrated strong global contextual modeling capability through self-attention mechanisms (Palanisamy et al., 2025).”
Transformers for Internet of Things Security and Communications: A Comprehensive Survey
Salman, M., Saeed, N., & Khurshid, K. (2026) · IEEE Internet of Things Journal · doi:10.1109/jiot.2026.3717847 · Cited by 2
“For automated control with safety implications, maximum accuracy may justify reduced interpretability—but raises questions about certification for systems without transparent decision processes [123].”
Liang, L., Wan, Y., Duan, P., Ding, Y., Yi, Z., Li, J., & Plaza, A. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.133863 · Cited by 1
LENS: A mammography-specific hybrid CNN-Transformer with lesion-aware evidence modeling
Quy, H. D., Cao, V. K., Nguyen, T. N., & Sơn, N. N. (2026) · PLoS ONE · doi:10.1371/journal.pone.0350720 · Cited by 1
Analysis of model-based ensemble learning for accurate diabetic retinopathy grading
Shih, H.-C., Cheng, Y., Chen, C.-S., Huang, T., & Wang, S. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.114581
FiT: Feature Integration Transformer with Universal Language Interface for Multi-Task VisionPreprint
Cheema, S., Gilanie, G., Alsahfi, T., Alesawi, S., Alsini, R. A. I., & Daud, A. Y. (2025) · Research Square · doi:10.21203/rs.3.rs-7602995/v1
Reading Assistance System for Visually Impaired Using Raspberry Pi
Samuel, K., Shanmitha, S., Tarukesh, T. J., & Cameron, A. K. (2025) · International Conference on Information Technology, Civil Innovation, Science, and Management (ICITSM) · doi:10.4108/eai.28-4-2025.2358084
Beyond deceptive flatness: Dual-order solution for strengthening adversarial transferability
Zhang, Z., Wang, P., Zheng, X., Qing, L., & Liu, Q. (2025) · Pattern Recognition · doi:10.1016/j.patcog.2025.112710
Tilahun, T. T., Degu, T. D., Temesgen, T., Demessie, M. B., & Alambo, H. F. (2025) · International Conference on Information and Communication Technology for Development for Africa (ICT4DA) · doi:10.1109/ict4da67218.2025.11282532
“It ac-complishes this by giving the most significant elements in a sequence more weight after comparing each element to the entire sequence [31].”
Tsuchiya, Y., Hirono, Y., & Sonobe, R. (2026) · AgriEngineering · doi:10.3390/agriengineering8010036
“This may improve estimation accuracy when a broader spectral context is important [40,41].”
Ma, L., Qi, L., Lu, X., Zhang, L., & Wang, J. (2026) · International Conference on Electronic Information Engineering and Artificial Intelligence (EIEAI) · doi:10.1117/12.3100785
Wang, Z., Zhang, X., Yang, Y., Wang, Y., Zhang, M., & Wang, N. (2025) · International Conference on Artificial Intelligence and Computer Engineering (ICAICE) · doi:10.1109/icaice68195.2025.11382453
Evaluating Vision Transformers for Robust Feature Extraction in Engineering Applications
Shi, O., Xie, H. S., Bhattacharya, M., & Zore, R. R. (2025) · International Symposium on AI-Driven Image Processing and Computer Vision Technology (AIPCVT) · doi:10.1109/aipcvt67868.2025.11405530
“Systematic reviews confirm that hybrids often outperform pure Vision Transformers in segmentation and detection tasks, particularly under resource constraints [9-11].”
InfluentialHybrid Vision Graph Network Learning for Accurate Hyperspectral Crop Disease Detection
Sharif, M. M., Gui, J., Alam, M. M., & Anjum, A. M. T. (2026) · International Journal of Advanced Engineering and Management Research · doi:10.51505/ijaemr.2026.11110
ViT-HAR: Vision Transformer-Based Human Activity Recognition in Cluttered Environments
Mewada, A., Ahmad, S., & Ansari, M. A. (2026) · Sakarya University Journal of Computer and Information Sciences · doi:10.35377/saucis...1745614
Borovyk, D., Alhumaid, K. F., Marichev, D., Leoshchenko, S., Oliinyk, A., & Alsayaydeh, J. A. J. (2026) · Journal of King Saud University - Computer and Information Sciences · doi:10.1007/s44443-026-00689-4
Bearing fault diagnosis based on multi-branch enhanced GhostNet with adaptive focal loss
Zhang, L., Jiang, T., Liu, T., Chen, J., & Guo, Y. (2026) · Scientific Reports · doi:10.1038/s41598-026-49801-8
BiKAN-ViT: Enhancing vision transformers via spline-based patch embedding and nonlinear token mixing
Yılmaz, E. K., Adem, K., & Zontul, M. (2026) · Applied Soft Computing · doi:10.1016/j.asoc.2026.115297
Kim, K.-M., Han, K.-U., Yun, S.-J., & Hong, H. (2026) · American Journal of Neural Networks and Applications · doi:10.11648/j.ajnna.20261201.13
MULTI-LEVEL INTRUTION DETECTION USING DEEP LEARNING FOR FACE RECOGNITION SYTEM
Salihu, Y. A., YAKUBU, Y., & MUHAMMAD, S. (2026) · International Journal of African Innovation and Multidisciplinary Research · doi:10.70382/mejaimr.v12i2.0109
Wang, S., Jia, C., Liu, H., Cheng, X., Shi, F., & Chen, S. (2026) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2026.3693340
“Despite their strong representation power, CNN-based methods inherently rely on local operations with limited receptive fields, making it di ffi cult to capture long-range contextual cues and e ff ectively suppress large-scale maritime clutter [28].”
Chen, M., Gu, Y., Wang, M., Yang, L., Zhang, B., Li, J., Liu, X., Hao, J., Ma, H., Zhang, W., et al. (2026) · International Journal of Imaging Systems and Technology · doi:10.1002/ima.70367
“On the other hand, Transformer-based models, such as the ViT, excel at capturing global semantic relationships but suffer from quadratic increases in computational complexity [32].”
Automated Vision Transformer Framework For Intelligent Crime-Scene Evidence Detection
Nithishwar, T., N, R. S., & Joseph, M. (2026) · International Conference on Intelligent Computing (ICONIC) · doi:10.1109/iconic67661.2026.11517757
“…surveillance footage, and visual evidence in a performant manner that outweighs the traditional image-processing pipelines, and which provides key insights into spatial-level indicators, overall environmental conditions, along with object-level features applicable to crime reconstruction [1].”
Hierarchical Sparse Vision Transformers for Real-Time Drone-Based Object Detection
Vivekanandam, B. (2026) · Journal of Artificial Intelligence and Capsule Networks · doi:10.36548/jaicn.2026.2.006
“Wider scans on transformers in vision focus on the lack of scalability, memory overheads, and quadratic attention complexity as fundamental issues [13], [14], [18].”
Devihosur, N., & Kumar, M. G. R. (2026) · SN Computer Science · doi:10.1007/s42979-026-05082-6
“Vision Transformers (ViTs) and their variants introduced a paradigm shift by leveraging self-attention mechanisms capable of modeling both local and global contextual information [16].”
Explainable Hybrid Encoder–Decoder for Robust Polyp Segmentation in Colonoscopy
Islam, M. J., Alam, M. T., Imtiaz, F., & Mafuj, M. M. I. (2026) · IEEE International Conference on Quantum Photonics, Artificial Intelligence & Networking (QPAIN) · doi:10.1109/qpain69676.2026.11546333
“The advent of Vision Trans-formers (ViT), allowing distant feature interactions and global receptive fields [5], has led to a remarkable paradigm shift.”
García, R., & Vega, H. V. R. (2026) · AI · doi:10.3390/ai7070233
“Originally designed for large-scale language modeling, compact Vision-style Transformers (ViT) have been adapted for time-series classification, capturing long-range dependencies across multi-channel sensor signals via multi-head self-attention without convolutional inductive biases [21].”
Vision Transformer-Based Temporal Attention Recalibrated Network for Efficient Crop Yield Prediction
Kumar, S. V., Ratan., J., Sunkara, R., Vishalakshi, N. B., & Kumar, K. S. S. (2026) · International Conference on Intelligent Systems and Computational Networks (ICISCN) · doi:10.1109/iciscn67954.2026.11566211
“From all the integrations, Vision Transformer (ViT) has become more impactful due to its capability for capturing global contextual relationships from the given input temporal and image data [5].”
Feng, Y., Peng, Y., & Yu, D. Z. (2026) · Sensors · doi:10.3390/s26134298
“Recently, the success of the Transformer architecture in computer vision has led to its application in DDD [14].”
Vision Transformer-Based Dog Breed Classification with a Hybrid Detection-Classification Framework
Benarkah, N., Siswantoro, J., & Porayouw, B. (2026) · Teknika · doi:10.34148/teknika.v15i2.1484
“Transformer-based architectures have recently emerged as powerful alternatives in computer vision due to their ability to model long-range dependencies through self-attention mechanisms [14], [15].”
Adaptive MoE Routing and KAN-Transformer with Distillation for Robust Cell Image Classification
Ding, Y., Wang, Z., Cao, Y., & Chen, Y. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3485-1_28
Sun, D., Chen, Q., Yu, G., Han, X., Li, C. Z., Zhou, C., & Ye, H. (2026) · Sensors · doi:10.3390/s26144570
“By leveraging the global self-attention mechanism of a vision transformer, the model achieved a paradigm shift from localized feature extraction constrained by limited receptive fields to extensive contextual semantic correlations [18].”
Alghamdi, A. A. (2026) · Journal of Imaging · doi:10.3390/jimaging12070330
“ViT handles an image as a sequence of linearly embedded fixed size patches and processes the resulting token via a stack of multi-head self-attention with learned positional encodings of feed-forward layers [23,24].”
Dehkordi, H. T., Bahramy, A., Julian, D. R., Cohen, J. A., Neal, M., Bumgardner, C., Nelson, P. T., Pearce, T. M., & Kofler, J. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.07.16.739032
“Self-attention Self-attention mechanisms remain more resource-intensive than convolutional operations, even when constrained through window-based and shifted-window attention as implemented in Swin Transformers (Liu et al., 2021; Palanisamy et al., 2025).”
Shao, Z., & Liu, S. (2026) · International Journal of Data Mining and Bioinformatics · doi:10.1504/ijdmb.2026.10080104
Jendersie, R., Margenberg, N., Lessig, C., & Richter, T. (2026) · Computer Methods in Applied Mechanics and Engineering · doi:10.1016/j.cma.2026.119254
“Their weaker inductive bias makes them well-suited also for spatial data [67].”
Rathore, U., Jatrurapitpornchai, R., Shakya, A., Kumar, R., & Sharma, L. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7231438
Dogga, A., Sivasubramanian, R., & Shanthi, S. (2026) · Artificial Intelligence and Applications · doi:10.47852/bonviewaia620210397
Emanuilov, S., Dimov, A., & Makaveev, I. (2026) · International Conference on Computer Systems and Technologies (CompSysTech) · doi:10.1109/compsystech69798.2026.11683100
“The model is pre-trained on a large-scale image dataset (in this case ImageNet-21k) and later fine-tuned for the target task - NSFW image classification [16].”
Kavya, N., Reeba Rose, L., & T, A. R. J. (2026) · International Conference on Inventive Systems and Control (ICISC) · doi:10.1109/icisc69558.2026.11681441
“Task-aware feature sharing has recently become the focus of many research groups working on collaborative learning frameworks in the field of medical image computing[18].”
An Object Detection Method Based on Frequency-Band Enhancement and Multi-Scale Fusion
Dai, Z., Qiu, Y., & Lu, Y. (2026) · Journal of Imaging · doi:10.3390/jimaging12090447
Vision Transformer-based Plant Disease Classification for Precision Agriculture
Appavu, N. (2026) · International Conference on Electronics and Sustainable Communication Systems (ICESC) · doi:10.1109/icesc70182.2026.11687142
“ViTs split images into patches and use transformer encoder layers and multi-head self-attention [10] to sequentially process the patches as tokens.”
Miorelli, R., Roué, D., & Robert, S. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7493853
LRMNet: Learnable ROI and Mamba Dual-Stream Network for Facial StO2 Stress Recognition
Yang, L., Zhang, A., Wang, H., & Chen, T. (2026) · International Conference on Computational Intelligence and Machine Learning (CIML) · doi:10.1109/ciml69155.2026.11692803
“Similarly, Lin et al. introduced MLNet to capture local features via a Vision Transformer (ViT) [8], but this approach remained limited by a predefined ROI configuration.”
Next-Generation Plastic Waste Recognition through Heterogeneous Deep Meta-Learning
Choudhary, R. K., Borade, P., Bhoyar, A., Zunke, S., Patil, V., Samundre, P., & Dulewale, A. D. (2026) · IEEE World Conference on Applied Intelligence and Computing (AIC) · doi:10.1109/aic70896.2026.11717304
Mikalauskas, T., Remeika, D., & Grigon, O. (2026) · Applied Manufacturing Intelligence Journal · doi:10.68406/amij.2026.vol5iss3np5:65-78
“SwinV2 introduces mechanisms for stable scaling and resolution transfer that provide a foundation for adapting visual representations to changing deployment conditions [4].”
Diaconu, C., Bompani, L., Dequino, A., Nadalini, D., & Conti, F. (2026) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn69854.2026.11729795
“In recent years, Transformer models [38] have revolutionized the field of Artificial Intelligence (AI), consistently outperforming Convolutional Neural Networks (CNNs) in vision tasks [13] and Recurrent Neural Networks (RNNs) in natural language processing [5].”
Chen, X., Yang, R., Zhang, H., Wu, B., Zhu, S., & Biancardo, S. A. (2026) · Ocean Engineering · doi:10.1016/j.oceaneng.2026.128842
Guidelines for the responsible use of artificial intelligence (AI) in scientific research
Alsobaie, S. F. (2026) · Elsevier eBooks · doi:10.1016/b978-0-443-44253-7.00013-8
A Survey of Sparse Attention in Vision Transformers: Taxonomy, Methods and Analysis
Yin, W., Zhang, Z., Han, Y., Xi, X., Wang, C., Chen, W., Wang, C., Zhu, L., Tan, S., Li, L., et al. (2026) · Neural Networks · doi:10.1016/j.neunet.2026.109837
Metadata
- Authors
- Palanisamy, B., Hassija, V., Chatterjee, A., Mandal, A., Chakraborty, D., Pandey, A., Chalapathi, G. S. S., & Kumar, D.
- Affiliations
- KIIT University (IN)
- Bennett University (IN)
- Birla Institute of Technology and Science, Pilani (IN)
- ORCID
- Venue
- IEEE Access
- Published
- 2025
- Volume
- 13
- Pages
- 95496-95523
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W4410536721
- Semantic Scholar
- ac9cc0c28838a037e77f4e19433de170f47b3de9
- Cited by
- 83
- References
- 118
- Citations by year
- 2026: 64 · 2025: 19
- FWCI
- 25.24
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- IEEE Access (publishedVersion, cc-by-nc-nd)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Birla Institute of Technology and Science, Pilani, India
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Keywords
- vision transformer, self-attention, positional encoding, image tokenization, ViT, Swin Transformer, DeiT, PVT, twins, image classification, object detection, video understanding, diagnostic imaging, CNN-Transformer hybrid, self-supervised learning, interpretability, data efficiency
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.