A survey of transformers

Deep Residual Learning for Image Recognition

He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474

Attention Is All You Need

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325

An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472

Swin Transformer: Hierarchical Vision Transformer using Shifted Windows

Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · arXiv · Cited by 35,608

Swin Transformer: Hierarchical Vision Transformer using Shifted Windows

Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818

Language Models are Few-Shot Learners

Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372

End-to-End Object Detection with Transformers

Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58452-8_13 · Cited by 18,285

Transformers for Graph-based Recommender Systems: A Survey

Zangari, L., Raso, S., & Tagarelli, A. (2026) · Machine Intelligence Research · doi:10.1007/s11633-025-1607-8 · Cited by 5

Natural language generation in healthcare: A review of methods and applications

Lyu, M., Li, X., Chen, Z., Pan, J., Peng, C., Talankar, S., & Wu, Y. (2026) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2026.104997 · Cited by 14

Exploring potential of large language models for automated essay scoring in education

Mughal, N., Imran, A. S., Daudpota, S. M., Kastrati, Z., & Noor, W. (2026) · Discover Artificial Intelligence · doi:10.1007/s44163-026-01002-y · Cited by 6

XNFT: Explainable network flow transformer for transparent detection of DDoS attacks in real-world networks

Ajayan, A., Kirubavathi, G., & Sarker, I. H. (2026) · Computer Networks · doi:10.1016/j.comnet.2026.112224 · Cited by 6

Wind power forecasting with a VMD-LSTM-informer hybrid deep learning model

Zheng, Q., Liu, Y., & Gu, T. (2026) · Energy and AI · doi:10.1016/j.egyai.2026.100765 · Cited by 4

Iformer: Irregular Spatial–Spectral Transformer for Hyperspectral Image Classification

Ahmad, M. I., & Mazzara, M. (2026) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2026.3710374 · Cited by 4

Identifying Major Depressive Disorder From Clinical Notes Using Neural Language Models with Distant Supervision

Miller, K., Kshatriya, B. S. A., Núñez, N. A., Gardea‐Resendez, M., Ryu, E., Coombes, B. J., Fu, S., Frye, M. A., Biernacka, J. M., Huang, M. S., et al. (2023) · IEEE International Conference on Healthcare Informatics (ICHI) · doi:10.1109/ichi57859.2023.00034 · Cited by 5

Application of Artificial Intelligence in Support of NAFLD Diagnosis

Płudowski, J., & MULAWKA, J. J. (2024) · Applied Sciences · doi:10.3390/app142210237 · Cited by 4

TriSwinUNETR lobe segmentation model for computing DIR-free CT-ventilation

Nomura, G. R. O., Luong, A. T., Prakash, A., Alemand, A., Bhowmick, T., Ali, A., Ren, J., Rehani, B., Nair, G., Castillo, R., et al. (2025) · Frontiers in Oncology · doi:10.3389/fonc.2025.1475133 · Cited by 4

Large Model Era: Deep Learning in Osteoporosis Drug Discovery

Xu, J., Wen, X., Sun, L. J., Xing, K., Xue, L., Zhou, S., Hu, J., Ai, Z., Kong, Q., Wen, Z., et al. (2025) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.4c02264 · Cited by 5

Deep Neural Network-Based Modeling of Multimodal Human–Computer Interaction in Aircraft Cockpits

Wang, L., Zhang, H., & Wang, C. (2025) · Future Internet · doi:10.3390/fi17030127 · Cited by 4

An Effective Summary Preprocessing Method for Time Series Forecasting With Multiple Temporal Granularities

Lee, H., Kim, Y., & Jung, S. (2025) · IEEE Access · doi:10.1109/access.2025.3553154 · Cited by 5

Memory-MambaNav: Enhancing object-goal navigation through integration of spatial–temporal scanning with state space models

Sun, L., & Yoshiyasu, Y. (2025) · Image and Vision Computing · doi:10.1016/j.imavis.2025.105522 · Cited by 5

A Transformer-Based Model for the Automatic Detection of Administrative Burdens in Transposed Legislative Documents

Costa, V., Castelli, M., & Coelho, P. S. (2025) · Technologies · doi:10.3390/technologies13040134 · Cited by 4

Remaining useful life prediction of rolling bearings based on multi-wavelet-time convolution transformer

Lu, Y., Cheng, S., Zhu, D., Zhao, D., & Gao, Q. (2025) · Measurement Science and Technology · doi:10.1088/1361-6501/adc9d3 · Cited by 4

Prompt-to-Slate: Diffusion Models for Prompt-Conditioned Slate Generation

Tomasi, F., Fabbri, F., Carter, J., Kalomiris, E., Lalmas, M., & Dai, Z. (2025) · ACM Conference on Recommender Systems (RecSys) · doi:10.1145/3705328.3748072 · Cited by 10

Transformer-GAN architecture for anomaly detection in multivariate time series

蔡, 美., 汪, 家., 刘, 金., 唐, 朝., & 谢, 永. (2022) · Scientia Sinica Informationis · doi:10.1360/ssi-2022-0133 · Cited by 4

Flexible ConvNext Block Based Multi-task Learning Framework for Liver MRI Images Analysis

Sui, D., Liu, W., Guo, M., Li, Y., Zhang, L., Luo, G., & Wang, K. (2022) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm55620.2022.9994956 · Cited by 4

Survey on Transformer for image classification

Shi, Z., Li, C., Zhou, L., Zhang, Z., Wu, C., You, Z., & Ren, W. (2023) · Journal of Image and Graphics · doi:10.11834/jig.220799 · Cited by 4

A Position-Free Signal Transformer via Multiband Inner Relationship Extraction for Understanding Information Flow of Machinery Diagnosis

Lv, H., Chen, J., He, S., & Zhou, Z. (2023) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2023.3309399 · Cited by 4

Local spectral attention for full-band speech enhancement

Hou, Z., Hu, Q., Chen, K., Cao, Z., & Lu, J. Z. (2023) · JASA Express Letters · doi:10.1121/10.0022268 · Cited by 4

Stiefel-Attentive Transformer for Refrigerator Odor Detection

Sun, J., Sun, Z., Zheng, H., & Zhao, X. (2023) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2023.3346823 · Cited by 4

Forecasting agricultures security indices: Evidence from transformers method

Ammouri, B. (2024) · Journal of Forecasting · doi:10.1002/for.3113 · Cited by 4

Pruning Attention in Transformers for Nonlinear Channel Compensation in Optical Systems

Behinaein, B., Najafi, H., Bakhshali, A., & Zhang, Z. (2024) · Optical Fiber Communication Conference · doi:10.1364/ofc.2024.m4k.2 · Cited by 4

A Prior-Guided Dual Branch Multi-Feature Fusion Network for Building Segmentation in Remote Sensing Images

Wu, Y., Zhao, P., Wang, F., Zhou, M., Geng, S., & Zhang, D. (2024) · Buildings · doi:10.3390/buildings14072006 · Cited by 4

A Cross-Modal Transformer Based Model for Box-office Revenue Prediction

Madongo, C. T., Tang, Z., & Hassan, J. (2024) · Journal of Advances in Information Technology · doi:10.12720/jait.15.7.822-837 · Cited by 5

An Efficient Two-Stage Pipelined Compute-in-Memory Macro for Accelerating Transformer Feed-Forward Networks

Zhang, H., Yin, W., He, S., Du, Y., & Du, L. (2024) · IEEE Transactions on Very Large Scale Integration (VLSI) Systems · doi:10.1109/tvlsi.2024.3432403 · Cited by 5

Defect-scanner: a comparative empirical study on language model and deep learning approach for software vulnerability detection

Pham, V.-H., Hien, D. T. T., Hoang, H. D., & Duy, P. T. (2024) · International Journal of Information Security · doi:10.1007/s10207-024-00901-4 · Cited by 4

Enhanced Local Feature Learning With Simple Offset Attention for Semantic Segmentation of Large-Scale Point Clouds

Chen, D., Wang, Y., Zhang, L., & Kang, Z. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3453966 · Cited by 4

GHA: A Gated Hierarchical Attention Mechanism for the Detection of Abusive Language in Social Media

Jarquín-Vásquez, H., Escalante, H. J., Montes-y-Gómez, M., & Gonzalez, F. A. (2024) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2024.3483010 · Cited by 5

A Novel Transformer-Based Self-Supervised Learning Method to Enhance Photoplethysmogram Signal Artifact Detection

Le, T.-D., Macabiau, C., Albert, K., Jouvet, P. A., & Noumeir, R. (2024) · IEEE Access · doi:10.1109/access.2024.3488595 · Cited by 7

Generative Models for Source Code: Fine-Tuning Techniques for Structured Pattern Learning

Franzoni, V., Tagliente, S., & Milani, A. (2024) · Technologies · doi:10.3390/technologies12110219 · Cited by 7

LONG EXPOSURE: Accelerating Parameter-Efficient Fine-Tuning for LLMs under Shadowy Sparsity

Wang, T., Li, K., Hao, Z., Bai, D., Ren, J., Zhang, Y., Cao, T., & Yang, M. (2024) · International Conference for High Performance Computing, Networking, Storage and Analysis (SC) · doi:10.1109/sc41406.2024.00081 · Cited by 6

Application of Transformers to Chemical Synthesis

Jin, D. Z., Liang, Y., Xiong, Z., Yang, X., Wang, H., Zeng, J., & Gu, S. (2025) · Molecules · doi:10.3390/molecules30030493 · Cited by 7

Temporal Relation Modeling and Multimodal Adversarial Alignment Network for Pilot Workload Evaluation

Li, X.-H., Li, A., Fu, W., Song, X., Li, F., Ma, Q., Peng, Y., & Lv, Z. (2025) · IEEE Journal of Translational Engineering in Health and Medicine · doi:10.1109/jtehm.2025.3542408 · Cited by 3

Research on Structural Mechanics Stress and Strain Prediction Models Combining Multi-Sensor Image Fusion and Deep Learning

Shan, Y., Zhen, M., & Fill, H. (2025) · Applied Sciences · doi:10.3390/app15074067 · Cited by 4

CAT: A simple heterogeneous ensemble learning framework for network intrusion detection

Zhang, Z., Das, A. K., Huang, G., & Baskiyar, S. (2025) · Peer-to-Peer Networking and Applications · doi:10.1007/s12083-025-02000-0 · Cited by 7

Deep Sparse Conformer for Speech Recognition

Wu, X. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-10384 · Cited by 4

Joint Attention Networks with Inherent and Contextual Preference-Awareness for Successive POI Recommendation

Zhong, H., He, W., Cui, L., Liu, L., Yan, Z., & Zhao, K. (2022) · Data Science and Engineering · doi:10.1007/s41019-022-00199-z · Cited by 4

Decoupling Source and Semantic Encoding: An Implementation Study

Feng, Y., Jin, X., Liang, C., Yu, G., Hu, L., & Yuan, T. (2023) · Electronics · doi:10.3390/electronics12132755 · Cited by 5

Vision Transformers for Breast Cancer Histology Image Classification

Baroni, G. L., Rasotto, L., Roitero, K., Siraj, A. H., & Mea, V. D. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-51026-7_2 · Cited by 4

Water Quality Index Forecasting via Transformers: A Comparative Experimental Study

Zhang, L., Shi, Y. K., Jin, X., Xu, S., Wang, C., & Liu, F. (2023) · China Automation Congress (CAC) · doi:10.1109/cac59555.2023.10450547 · Cited by 4

The classification of SSVEP-BCI based on ear-EEG via RandOm Convolutional KErnel Transform with Morlet wavelet

Li, X., Haba, T., Cui, G., Kinoshita, F., & Touyama, H. (2024) · Discover Applied Sciences · doi:10.1007/s42452-024-05816-2 · Cited by 5

Large Language Model Inference Acceleration Based on Hybrid Model Branch Prediction

Duan, G., Chen, J., Zhou, Y., Zheng, X., & Zhu, Y. (2024) · Electronics · doi:10.3390/electronics13071376 · Cited by 4

Grounding Conversational Robots on Vision Through Dense Captioning and Large Language Models

Grassi, L., Hong, Z., Recchiuto, C. T., & Sgorbissa, A. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10611232 · Cited by 6

Evaluating Off-the-Shelf Machine Listening and Natural Language Models for Automated Audio CaptioningPreprint

Weck, B., Favory, X., Drossos, K., & Serra, X. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.07410 · Cited by 5

Building Representation Learning Models for Antibody ComprehensionPreprint

Barton, J., Gaspariunas, A., Galson, J. D., & Leem, J. (2023) · Cold Spring Harbor Perspectives in Biology · doi:10.1101/cshperspect.a041462 · Cited by 4

Spiking-Physformer: Camera-Based Remote Photoplethysmography with Parallel Spike-Driven TransformerPreprint

Liu, M., Tang, J., Li, H., Qi, J., Li, S., Wang, K., Wang, Y., & Chen, H. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4729635 · Cited by 4

Dataset of tensorial optical and transport properties of materials from the Wannier function method

Fang, Z., Hsu, T.-W., & Yan, Q. (2025) · Scientific Data · doi:10.1038/s41597-025-05396-9 · Cited by 4

Malicious URL Detection Using Transformers’ NLP Models and Machine Learning

Niyaoui, O., & Reda, O. M. (2024) · Lecture Notes in Networks and Systems · doi:10.1007/978-3-031-54318-0_35 · Cited by 3

Predicting gene expression using millions of yeast promoters reveals cis -regulatory logic

Dash, T., & Bornelöv, S. (2024) · Bioinformatics Advances · doi:10.1093/bioadv/vbaf130 · Cited by 4

Dementia risk prediction using decision-focused content selection from medical notes

Li, S., Dexter, P. R., Miled, Z. B., & Boustani, M. A. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.109144 · Cited by 3

A Brief Review on Question Answering System: Techniques, Challenges and Future Directions

Kumar, R., Singh, G. B., Shailendra, P., & Sharma, N. (2024) · International Conference on Advanced Computing and Communication Systems (ICACCS) · doi:10.1109/icaccs60874.2024.10717204 · Cited by 3

Gamma Log Inversion of Seismic Data Based on Transformer With Stratigraphic Position Encoding

Zhou, Y., Qi, H., Zhang, W., & Shan, X. (2025) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2025.3535723 · Cited by 3

Intelligent Micro-Kick Detection Using a Multi-Head Self-Attention Network

Zhang, D., Sun, W., Dai, Y., Wang, D., Guo, Y., & Gong, C. (2025) · Processes · doi:10.3390/pr13020465 · Cited by 3

Advancing Plant Disease Prediction Accuracy with ViT Based Deep Learning Model

M., F. S. A., & Kavitha, B. R. (2024) · International Conference on Computing, Semiconductor, Mechatronics, Intelligent Systems and Communications (COSMIC) · doi:10.1109/cosmic63293.2024.10871372 · Cited by 3

Inferring genotype–phenotype maps using attention models

Rijal, K., Holmes, C. M., Petti, S., Reddy, G., Desai, M. M., & Mehta, P. (2026) · PNAS Nexus · doi:10.1101/2025.04.11.648465 · Cited by 5

Self-attention transformer architectures for cyberattack detection and secure state reconstruction of integrated chemical process systems

Bagheri, A., Ebrahimi, A., & Pourkargar, D. B. (2025) · Process Safety and Environmental Protection · doi:10.1016/j.cherd.2025.06.038 · Cited by 3

DAFNet: A novel image restoration model with mixed SimpleGate

Peng, C., Liao, J., Jiang, L., Liang, W., Espósito, A., Li, K., & Yuan, J. (2025) · Digital Signal Processing · doi:10.1016/j.dsp.2025.105492 · Cited by 3

Deep insights into cognitive decline: A survey of leveraging non-intrusive modalities with deep learning techniques

Ortiz-Pérez, D., Benavent-Lledó, M., Garcia-Rodriguez, J., Tomás, D., & Vizcaya-Moreno, M. F. (2025) · Applied Soft Computing · doi:10.1016/j.asoc.2025.113787 · Cited by 6

Deep learning approaches for image-based snoring sound analysis in the diagnosis of obstructive sleep apnea-hypopnea syndrome: A systematic review

Ding, L., Peng, J., & Song, Y. (2025) · World Journal of Radiology · doi:10.4329/wjr.v17.i9.109116 · Cited by 4

Deep long-range spatiotemporal dependency synthetic minority oversampling technique for imbalance-bearing fault diagnosis

Dai, H.-L., Lin, D.-J., He, J.-P., Huang, S.-T., & Fang, X.-Y. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.114944 · Cited by 3

MoEcho: Exploiting Side-Channel Attacks to Compromise User Privacy in Mixture-of-Experts LLMs

Ding, R., Xu, T., Shen, X., Ding, A. A., & Fei, Y. (2025) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3719027.3765174 · Cited by 4

Enhancing clustering stability, compactness, and separation in multimodal data environments

Silva, F. D. S., dos Reis, J. C., & Reis, M. S. (2025) · Data & Knowledge Engineering · doi:10.1016/j.datak.2025.102536 · Cited by 3

Physics-informed transformers for electronic quantum states

Sobral, J. A., Perle, M., & Scheurer, M. S. (2025) · Nature Communications · doi:10.1038/s41467-025-66844-z · Cited by 10

TSNet: a foundation model for wireless network status prediction in digital twins

Song, S., Sun, G., Chang, Y., Zhao, N., & Yu, Y. (2025) · Frontiers of Information Technology & Electronic Engineering · doi:10.1631/fitee.2400295 · Cited by 4

Wavelet-decoupled GRU with adaptive attention for multi-step carbon dioxide concentration prediction in intelligent glass greenhouse

Tian, P., Sun, J., Cheng, J., Zhao, J., Wang, X., Zhang, X., Guo, R., & Qian, Y. (2025) · Smart Agricultural Technology · doi:10.1016/j.atech.2025.101653 · Cited by 3

Emotion recognition based on spatio-temporal connectivity of prefrontal EEG signals

Wu, F., Wang, E., Xue, B., & Liu, Y. (2025) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2025.109369 · Cited by 3

Informer deep learning forecasting for high-speed aeroelastics

Drikakis, D., Kokkinakis, I. W., Fung, D., Spottswood, S. M., Brouwer, K. R., Riley, Z. B., Daub, D., & Gülhan, A. (2025) · Aerospace Science and Technology · doi:10.1016/j.ast.2025.111547 · Cited by 3

Bidirectional Translation of ASL and English Using Machine Vision and CNN and Transformer Networks

Amiruzzaman, S., Amiruzzaman, M., Batchu, R. M., Dracup, J., Pham, A., Crocker, B., Ngo, L. B., & Dewan, M. A. A. (2026) · Computers · doi:10.3390/computers15010020 · Cited by 3

SGMT-IDS: A Dual-Branch Semi-Supervised Intrusion Detection Model Based on Graphs and Transformers

Wu, Y., & Wan, L. (2026) · Electronics · doi:10.3390/electronics15020348 · Cited by 3

Transformer-based artificial intelligence for forecasting energy demand in irrigation districts

Bernabé, M. A. I., Perea, R. G., Díaz, J. A. R., & Morillo, J. G. (2026) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2026.111611 · Cited by 3

Predictive Network Slicing Resource Orchestration: A VNF Approach

Cardenas, A. J. A., Sigcha, L., & Mosahebfard, M. (2026) · Future Internet · doi:10.3390/fi18030149 · Cited by 4

A fusion-based multitask Transformer model for structural damage detection

Jiang, S., Xiao, S., Sun, Y., Fan, G., Cheng, M., & Zhang, J. (2026) · Structural Health Monitoring · doi:10.1177/14759217261435183 · Cited by 3

Unboxing the Black Box: A Survey on Mechanistic Interpretability for Algorithmic Understanding of Neural Networks

Kowalska, B., & Kwaśnicka, H. (2026) · Machine Learning · doi:10.1007/s10994-026-07129-4 · Cited by 11

Speformer: An Efficient Hardware-Software Cooperative Solution for Sparse Spectral Transformer

Sun, Y., Hu, W., Liu, F., Jiang, M., Huang, F., & Xu, D. G. (2022) · IEEE International Conference on Cyber Security and Cloud Computing / IEEE International Conference on Edge Computing and Scalable Cloud (CSCloud/EdgeCom) · doi:10.1109/cscloud-edgecom54986.2022.00039 · Cited by 3

Improving the hERG model fitting using a deep learning-based method

Song, J. C., Kim, Y.-J., & Leem, C. H. (2023) · Frontiers in Physiology · doi:10.3389/fphys.2023.1111967 · Cited by 4

Diffusing Gaussian Mixtures for Generating Categorical Data

Regol, F., & Coates, M. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i8.26145 · Cited by 3

Optimization of binding affinities in chemical space with generative pre-trained transformer and deep reinforcement learning

Xu, X., Zhou, J., Zhu, C., Zhan, Q., Li, Z., Zhang, R., Wang, Y., Liao, X., & Gao, X. (2023) · F1000Research · doi:10.12688/f1000research.130936.1 · Cited by 5

Identifying and matching 12‐level multistained glomeruli via deep learning for diagnosis of glomerular diseases

He, Q., Zeng, S., Ge, S., Wang, Y., Ye, J., He, Y., Guan, T., Wang, Z., & Li, J. (2024) · International Journal of Imaging Systems and Technology · doi:10.1002/ima.23032 · Cited by 3

Long-Term, Multi-Variate Production Forecasting Using Non-Stationary Transformer

Kumar, A. (2024) · International Petroleum Technology Conference · doi:10.2523/iptc-24308-ms · Cited by 3

RefreshNet: learning multiscale dynamics through hierarchical refreshing

Farooq, J., Rafiq, D., Vlachas, P. R., & Bazaz, M. A. (2024) · Nonlinear Dynamics · doi:10.1007/s11071-024-09813-3 · Cited by 3

Recognition of cross-regional loess water content based on machine vision: Case study of three regions

Zhang, Y., Zhang, H., Lan, H., Li, Y., Sun, D., Wang, E., & Dong, Z. (2024) · CATENA · doi:10.1016/j.catena.2024.108263 · Cited by 3

A Comprehensive Examination of ChatGPT's Contribution to the Healthcare Sector and Hepatology

Kumari, K., Pahuja, S. K., & Kumar, K. S. (2024) · Digestive Diseases and Sciences · doi:10.1007/s10620-024-08659-4 · Cited by 3

GFIDF:gradual fusion intent detection framework

Yang, Q., Liu, Y., Lu, L., & Liu, L. (2024) · The Journal of Supercomputing · doi:10.1007/s11227-024-06708-3 · Cited by 4

Predicting CRISPR Cas9 Off-Target Activities With a Two-Stage Deep Learning Framework

Zhang, T., Jiang, B., Yang, G., Bao, Z., & Peng, J. (2025) · IEEE Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbbio.2024.3521405 · Cited by 4

Federated learning for oncology: Breast Cancer subtyping Case Study Using Gene Expression

Paygambar, K., Mziou-Sallami, M., Baligand, F., & Meyer, V. (2024) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm62325.2024.10822609 · Cited by 4

A Systematic Method Combining Rotated Convolution and State Space Augmented Transformer for Digitizing and Classifying Paper ECGs

Wang, X., & Yang, J. (2025) · Symmetry · doi:10.3390/sym17010120 · Cited by 3

UMamba Adjustment: Advancing GTV Segmentation for Head and Neck Cancer in MRI-Guided RT with UMamba and NnU-Net ResEnc Planner

Ren, J., Hochreuter, K., Kallehauge, J. F., & Korreman, S. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-83274-1_9 · Cited by 3

SGNetPose+: Stepwise Goal-Driven Networks with Pose Information for Trajectory Prediction in Autonomous Driving

Ghiya, A., AlShami, A. K., & Kalita, J. K. (2025) · IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW) · doi:10.1109/wacvw65960.2025.00077 · Cited by 3

The Drosophila Connectome as a Computational Reservoir for Time-Series Prediction

Costi, L., Hadjiivanov, A., Dold, D., Hale, Z. F., & Izzo, D. (2025) · Biomimetics · doi:10.3390/biomimetics10050341 · Cited by 5

Fast training of a transformer for global multi-horizon time series forecasting on tensor processing units

García-Nava, J.-L., Flores, J. J., Tellez, V. M., & Calderón, F. (2022) · The Journal of Supercomputing · doi:10.1007/s11227-022-05009-x · Cited by 3

Papez: Resource-Efficient Speech Separation with Auditory Working Memory

Oh, H., Yi, J., & Lee, Y. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10095136 · Cited by 3

Content-based comparison of communities in social networks: Ex-Yugoslavian reactions to the Russian invasion of Ukraine

Evkoski, B., Novak, P. K., & Ljubešić, N. (2023) · Applied Network Science · doi:10.1007/s41109-023-00561-8 · Cited by 3

Automatic ICD-10 Code Association: A Challenging Task on French Clinical Texts

Tchouka, Y., Couchot, J.-F., Laiymani, D., Selles, P., & Rahmani, A. (2023) · Proceedings - IEEE Symposium on Computer-Based Medical Systems · doi:10.1109/cbms58004.2023.00198 · Cited by 5

A multi-Task Water Level Prediction Method Based on Attention Mechanism and LSTM

Huang, F., Yi, P., Chen, J., & Yang, J. (2023) · International Conference on Artificial Intelligence and Big Data (ICAIBD) · doi:10.1109/icaibd57115.2023.10206108 · Cited by 3

Multiscale Contextual Learning for Speech Emotion Recognition in Emergency Call Center Conversations

Deschamps-Berger, T., Lamel, L. F., & Devillers, L. Y. (2023) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3610661.3616189 · Cited by 3

Classification of integers based on residue classes via modern deep learning algorithms

Wu, D., Yang, J., Ahsan, M. U., & Wang, K. (2023) · Patterns · doi:10.1016/j.patter.2023.100860 · Cited by 5

Passing Heatmap Prediction Based on Transformer Model Using Tracking Data for Football Analytics

Pei, Y., Silva, D., & Caine, M. P. (2023) · Communications in computer and information science · doi:10.1007/978-3-031-46335-8_13 · Cited by 3

HyperRouter: Towards Efficient Training and Inference of Sparse Mixture of Experts

Do, T., Khiem, L. H., Pham, Q., Nguyen, T., Doan, T.-N., Nguyen, B., Liu, C., Ramasamy, S., Li, X., & Hoi, S. C. H. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.351 · Cited by 3

Efficient transformers: A survey

Tay, Y., Dehghani, M., Bahri, D., & Metzler, D. (2023) · ACM Computing Surveys · doi:10.1145/3530811 · Cited by 1,267

Survey: Transformer-based Models in Data Modality ConversionPreprint

Rashno, E., Eskandari, A., Anand, A., & Zulkernine, F. H. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2408.04723 · Cited by 8

A comprehensive survey on applications of transformers for deep learning tasks

Islam, S., Elmekki, H., Elsebai, A., Bentahar, J., Drawel, N., Rjoub, G., & Witold Pedrycz, C. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.122666 · Cited by 590

Transformer Models and their Cross-Domain Applications: A Systematic Review

Pathania, K., Singh, S., & Sharma, A. (2026) · Archives of Computational Methods in Engineering · doi:10.1007/s11831-026-10742-0

A Review on Transformer Models: Applications, Taxonomies, Open Issues and Challenges

Passi, N., Raj, M., & Shelke, N. A. (2024) · Asian Conference on Innovation in Technology (ASIANCON) · doi:10.1109/asiancon62057.2024.10838047 · Cited by 8