A survey of transformers

Deep Residual Learning for Image Recognition

He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474

Attention Is All You Need

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325

An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472

Swin Transformer: Hierarchical Vision Transformer using Shifted Windows

Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · arXiv · Cited by 35,608

Swin Transformer: Hierarchical Vision Transformer using Shifted Windows

Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818

Language Models are Few-Shot Learners

Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372

End-to-End Object Detection with Transformers

Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58452-8_13 · Cited by 18,285

A transformer-based deep learning method for automatic pixel-level crack detection and feature quantification

Ji, A., Xue, X., Zhang, L., Luo, X., & Man, Q. (2023) · Engineering Construction & Architectural Management · doi:10.1108/ecam-06-2023-0613 · Cited by 13

Tree ring segmentation using UNEt TRansformer neural network on stained microsections for quantitative wood anatomy

García‐Hidalgo, M., Garcia-Pedrero, A. M., Rozas, V., Sangüesa‐Barreda, G., García‐Cervigón, A. I., Resente, G., Wilmking, M., & Olano, J. M. (2024) · Frontiers in Plant Science · doi:10.3389/fpls.2023.1327163 · Cited by 16

DeepLION2: deep multi-instance contrastive learning framework enhancing the prediction of cancer-associated T cell receptors by attention strategy on motifs

Qian, X., Yang, G., Li, F., Zhang, X., Zhu, X., Lai, X., Xiao, X., Wang, T., & Wang, J. (2024) · Frontiers in Immunology · doi:10.3389/fimmu.2024.1345586 · Cited by 14

Switching Self-Attention Text Classification Model with Innovative Reverse Positional Encoding for Right-to-Left Languages: A Focus on Arabic Dialects

Baniata, L. H., & Kang, S. (2024) · Mathematics · doi:10.3390/math12060865 · Cited by 13

Never‐Ending Learning for Explainable Brain Computing

Kuai, H., Chen, J., Tao, X., Cai, L., Imamura, K., Matsumoto, H., Liang, P., & Zhong, N. (2024) · Advanced Science · doi:10.1002/advs.202307647 · Cited by 13

HBCVTr: an end-to-end transformer with a deep neural network hybrid model for anti-HBV and HCV activity predictor from SMILES

Meewan, I., Panmanee, J., Petchyam, N., & Lertvilai, P. (2024) · Scientific Reports · doi:10.1038/s41598-024-59933-4 · Cited by 14

Mining anti-hypertensive peptides in animal food through deep learning: a case study of gastrointestinal digestive products of royal jelly

Pan, F., Liu, D., Tuersuntuoheti, T., Xing, H.-D., Zhu, Z., Fang, Y., Zhao, L., Zhao, L., Li, X.-X., Le, Y., et al. (2024) · Food Science of Animal Products · doi:10.26599/fsap.2024.9240053 · Cited by 11

Parameter-efficient fine-tuning of pre-trained code models for just-in-time defect prediction

Talib, M. W. A., Nassif, A. B., Azzeh, M., Alesh, Y., & Afadar, Y. (2024) · Neural Computing and Applications · doi:10.1007/s00521-024-09930-5 · Cited by 14

Optic Disc Segmentation in Human Retina Images Using a Meta Heuristic Optimization Method and Disease Diagnosis with Deep Learning

Almeshrky, H. A., & Karacı, A. (2024) · Applied Sciences · doi:10.3390/app14125103 · Cited by 13

Frailty Modeling Using Machine Learning Methodologies: A Systematic Review With Discussions on Outstanding Questions

Yang, H., Chang, J., He, W., Wee, C. F., Soong, J. T. Y., & Feng, M. (2024) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2024.3430226 · Cited by 14

Deep Learning and Hybrid Fusion-Based LOS/NLOS Identification in Substation Scenarios for Power Internet of Things

Zhou, T. Y., Wang, Y., Lin, Y., Ai, B., & Liu, L. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3432798 · Cited by 14

Reducing Carbon Footprint in AI: A Framework for Sustainable Training of Large Language Models

Iftikhar, S., & Davy, S. (2024) · Lecture notes in networks and systems · doi:10.1007/978-3-031-73110-5_22 · Cited by 14

Enhancing dysarthric speech recognition through SepFormer and hierarchical attention network models with multistage transfer learning

Vinotha, R., Hepsiba, D. E., Anand, L. D. V., Andrew, J., & Jennifer Eunice, R. (2024) · Scientific Reports · doi:10.1038/s41598-024-80764-w · Cited by 15

Variational Multiple-Instance Learning With Embedding Correlation Modeling for Hyperspectral Target Detection

Yang, B., Jiao, C., Wu, J., & Li, L. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3510385 · Cited by 13

Improving mispronunciation detection and diagnosis for non- native learners of the Arabic language

Alrashoudi, N., Al-Khalifa, H. S., & Alotaibi, Y. A. (2025) · Discover Computing · doi:10.1007/s10791-024-09489-8 · Cited by 22

MultiTec: A Data-Driven Multimodal Short Video Detection Framework for Healthcare Misinformation on TikTok

Shang, L., Zhang, Y., Deng, Y., & Wang, D. (2025) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2025.3533919 · Cited by 13

SugarViT—Multi-objective regression of UAV images with Vision Transformers and Deep Label Distribution Learning demonstrated on disease severity prediction in sugar beet

Günder, M., Yamati, F. R. I., Barreto, A., Mahlein, A., Sifa, R., & Bauckhage, C. (2025) · PLoS ONE · doi:10.1371/journal.pone.0318097 · Cited by 13

Retrieving false claims on Twitter during the Russia-Ukraine conflict

Gatta, V. L., Wei, C., Luceri, L., Pierri, F., & Ferrara, E. (2023) · International World Wide Web Conference (WWW) · doi:10.1145/3543873.3587571 · Cited by 22

animal2vec and MeerKAT: A self‐supervised transformer for rare‐event raw audio input and a large‐scale reference dataset for bioacoustics

Zimmermann, J., Demartsev, V., Averly, B., Dhanjal‐Adams, K. L., Duteil, M., Gall, G. E. C., Faiß, M., Johnson-Ulrich, L., Stowell, D., Manser, M. B., et al. (2025) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.70218 · Cited by 27

Hybrid deep learning framework for environmental microplastic classification: Integrating CNN-based spectral feature extraction and transformer models

Li, F., Tian, J., Guo, X., & Xu, L. (2025) · Environmental Pollution · doi:10.1016/j.envpol.2025.126989 · Cited by 12

Automating Life Cycle Assessments through Artificial Intelligence Agents and Integrated Assessment Models

Preuss, N., & You, F. (2025) · Environmental Science & Technology · doi:10.1021/acs.est.5c14493 · Cited by 16

An Uncertainty-Aware and Sex-Prior Guided Biological Age Estimation From Orthopantomogram Images

Zhang, D., Yang, J., Du, S., Bu, W.-Q., & Guo, Y.-C. (2023) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2023.3297610 · Cited by 13

LSAB: User Behavioral Pattern Modeling in Sequential Recommendation by Learning Self-Attention Bias

Han, D., Huang, Y., Liu, J., Liao, K., & Lin, K. (2023) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3632625 · Cited by 12

Remaining useful life estimation of bearing using spatio-temporal convolutional transformer

Zhu, D., Lyu, J., Gao, Q., Lu, Y., & Zhao, D. (2024) · Measurement Science and Technology · doi:10.1088/1361-6501/ad214d · Cited by 12

Optimizing the Structures of Transformer Neural Networks Using Parallel Simulated Annealing

Trzciński, M., Łukasik, S., & Gandomi, A. H. (2024) · Journal of Artificial Intelligence and Soft Computing Research · doi:10.2478/jaiscr-2024-0015 · Cited by 12

KITE-DDI: A Knowledge Graph Integrated Transformer Model for Accurately Predicting Drug-Drug Interaction Events From Drug SMILES and Biomedical Knowledge Graph

Tamir, A., & Yuan, J. S. (2025) · IEEE Access · doi:10.1109/access.2025.3547594 · Cited by 14

Modeling the spread of forest fires through cellular automata by leveraging deep learning to derive transition rules

Zhou, Z., Xu, Q., Yi, J., Li, Y., Zhang, S., & Li, W. (2025) · Ecological Informatics · doi:10.1016/j.ecoinf.2025.103150 · Cited by 15

Machine learning innovations in CPR: a comprehensive survey on enhanced resuscitation techniques

Islam, S., Rjoub, G., Elmekki, H., Bentahar, J., Witold Pedrycz, C., & Cohen, R. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11214-w · Cited by 14

SoK: Leveraging Transformers for Malware Analysis

Kunwar, P., Aryal, K., Gupta, M., Abdelsalam, M., & Bertino, E. (2025) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2025.3576708 · Cited by 21

Improving accuracy and generalization in single kernel oil characteristics prediction in maize using NIR-HSI and a knowledge-injected spectral tabtransformer

Song, A., Guo, X., Wen, W., Wang, C., Gu, S., Chen, X., Wang, J., & Zhao, C. (2025) · Artificial Intelligence in Agriculture · doi:10.1016/j.aiia.2025.05.007 · Cited by 12

Enhancing procedure quality: Advanced language tools for identifying ambiguity and high-potential violation triggers

Johnson, K., de Morais, C. P. M., & Patelli, E. (2025) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2025.111308 · Cited by 15

Engineering artificial intelligence: framework, challenges, and future direction

Lee, J., Su, H., Ji, D.-Y., & Minami, T. (2025) · Machine Learning Engineering · doi:10.1088/3049-4761/adce0d · Cited by 28

Threat detection in the 6G enabled Industrial IoT Networks using Deep Learning: A review on the state-of-the-art solutions, challenges and future research directions

Guo, G., Qamar, F., Kazmi, S. H. A., & Rehman, M. H. U. (2025) · Internet of Things · doi:10.1016/j.iot.2025.101686 · Cited by 12

Lightweight individual cow identification based on Ghost combined with attention mechanism

Fu, L., Li, S., Kong, S., Ni, R., Pang, H., Sun, Y., Hu, T., Mu, Y., Guo, Y., & Gong, H. (2022) · PLoS ONE · doi:10.1371/journal.pone.0275435 · Cited by 12

Towards Efficient NLP: A Standard Evaluation and A Strong Baseline

Liu, X., Sun, T., He, J., Wu, J., Wu, L., Zhang, X., Jiang, H., Cao, Z., Huang, X., & Qiu, X. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.240 · Cited by 18

Deep Learning for Heart Sound Analysis: A Literature ReviewPreprint

Zhao, Q.-H., Geng, S., Wang, B., Sun, Y., Nie, W., Bai, B., Yu, C., Zhang, F., Tang, G., Zhang, D.-Y., et al. (2023) · medRxiv · doi:10.1101/2023.09.16.23295653 · Cited by 7

Deep learning in intracranial EEG for seizure detection: advances, challenges, and clinical applications

Qamar, W. U. R., Lee, M.-H., & Abibullaev, B. (2025) · Frontiers in Neuroscience · doi:10.3389/fnins.2025.1677898 · Cited by 11

Deep Domain Adaptation Enhances Amplification Curve Analysis for Single-Channel Multiplexing in Real-Time PCR

Mao, Y., Xu, K., Miglietta, L., Kreitmann, L., Moser, N., Georgiou, P., Holmes, A. H., & Rodríguez-Manzano, J. (2023) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2023.3257727 · Cited by 11

Sentiment Analysis on Algerian Dialect with Transformers

Benmounah, Z., Boulesnane, A., Fadheli, A., & Khial, M. (2023) · Applied Sciences · doi:10.3390/app132011157 · Cited by 11

A Survey on Congestion Control and Scheduling for Multipath TCP: Machine Learning vs Classical Approaches

Maliha, M., Habibi, G., & Atiquzzaman, M. (2023) · Annals of Computer Science and Information Systems · doi:10.15439/2023f9832 · Cited by 11

Online Transformers with Spiking Neurons for Fast Prosthetic Hand Control

Leroux, N., Finkbeiner, J., & Neftci, E. O. (2023) · IEEE Biomedical Circuits and Systems Conference (BioCAS) · doi:10.1109/biocas58349.2023.10388996 · Cited by 11

Addressing Challenges in Hate Speech Detection Using BERT-Based Models: A Review

Aljawazeri, J. A., & Jasim, M. N. (2024) · Iraqi Journal for Computer Science and Mathematics · doi:10.52866/ijcsm.2024.05.02.001 · Cited by 13

Multi-step Transfer Learning in Natural Language Processing for the Health Domain

Manaka, T., van Zyl, T. L., Kar, D., & Wade, A. N. (2024) · Neural Processing Letters · doi:10.1007/s11063-024-11526-y · Cited by 11

A novel intelligent indoor fire and combustibles detection method based on multi-channel transfer learning strategy with acoustic signals

Wang, Z., Zhang, Z., Wang, Z., Lu, J., & An, W. (2024) · Process Safety and Environmental Protection · doi:10.1016/j.psep.2024.06.020 · Cited by 12

mBGT: Encoding Brain Signals With Multimodal Brain Graph Transformer

Peng, C., Guo, T., Xie, C., Bai, X., Zhou, J., Zhao, X., He, E., & Xia, F. (2024) · IEEE Transactions on Consumer Electronics · doi:10.1109/tce.2024.3425478 · Cited by 11

BRYT: Automated keyword extraction for open datasets

Ahmed, U. Z., Alexopoulos, C., Piangerelli, M., & Polini, A. (2024) · Intelligent Systems with Applications · doi:10.1016/j.iswa.2024.200421 · Cited by 14

Cognitive Computing Advancements: Improving Precision Crop Protection through UAV Imagery for Targeted Weed Monitoring

Mesías-Ruiz, G. A., Peña, J. M., de Castro, A. I., Borra‐Serrano, I., & Dorado, J. (2024) · Remote Sensing · doi:10.3390/rs16163026 · Cited by 14

Crossing the Trust Gap in Medical AI: Building an Abductive Bridge for xAI

Gouveia, S. S., & Malík, J. (2024) · Philosophy & Technology · doi:10.1007/s13347-024-00790-4 · Cited by 14

Improving Anomalous Sound Detection Via Low-Rank Adaptation Fine-Tuning of Pre-Trained Audio Models

Zheng, X., Jiang, A., Han, B., Qian, Y., Fan, P., Liu, J., & Zhang, W.-Q. (2024) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt61566.2024.10832335 · Cited by 11

Toward Large Energy Models: A comparative study of Transformers’ efficacy for energy forecasting

Gu, Y., Jazizadeh, F., & Wang, X. (2025) · Applied Energy · doi:10.1016/j.apenergy.2025.125358 · Cited by 14

Code Clone Detection Techniques Based on Large Language Models

Almatrafi, A. A., Eassa, F. E., & Sharaf, S. A. (2025) · IEEE Access · doi:10.1109/access.2025.3549780 · Cited by 13

A Mamba-based foundation model for materials

Soares, E. A., Brazil, E. V., Shirasuna, V. Y., Zubarev, D. Y., Cerqueira, R., & Schmidt, K. (2025) · npj Artificial Intelligence · doi:10.1038/s44387-025-00009-7 · Cited by 11

OccMamba: Semantic Occupancy Prediction with State Space Models

Li, H., Hou, Y., Xing, X., Ma, Y., Sun, X., & Zhang, Y. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01116 · Cited by 14

A Deep Learning Approach to Teeth Segmentation and Orientation from Panoramic X-Rays

Deb, M., Deb, M., & Dhar, M. K. (2025) · Signals · doi:10.3390/signals6030040 · Cited by 11

A multimodal fusion model for real-time environment emotion recognition using audio-visual-textual features

Gupta, C., Gill, N. S., Gulia, P., Kumar, A., Karamti, H., Moges, D. M., & Safra, I. (2025) · Journal Of Big Data · doi:10.1186/s40537-025-01300-9 · Cited by 11

Multi-temporal dependency handling in video smoke recognition: A holistic approach spanning spatial, short-term, and long-term perspectives

Yang, F., Xue, Q., Cao, Y., Li, X., Zhang, W., & Li, G. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.123081 · Cited by 11

Bi-LSTM-Based Neural Source Code Summarization

Aljumah, S., & Berriche, L. (2022) · Applied Sciences · doi:10.3390/app122412587 · Cited by 10

A Robust Frequency-Domain-Based Graph Adaptive Network for Parkinson's Disease Detection From Gait Data

Zhong, C., & Ng, W. W. Y. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3217392 · Cited by 13

Transform Drug Discovery and Development With Generative Artificial Intelligence

Lavecchia, A. (2025) · Wiley eBooks · doi:10.1002/9781394280735.ch25 · Cited by 14

AIoT for human emotion recognition: Potentials, challenges, and healthcare applications

Yaqoob, S., Ullah, F., Abuali, N. A., Anwar, H., Saeed, N., & Hayajneh, M. (2025) · Computer Science Review · doi:10.1016/j.cosrev.2025.100859 · Cited by 16

A comprehensive survey on pose estimation and tracking in sports: methodologies, datasets, challenges, and future directions

Uçar, M. H. B., Solak, S. B., Jimale, A. O., Ünal, H., & Eken, S. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11344-1 · Cited by 10

Integrating Relational Knowledge With Text Sequences for Script Event Prediction

Wang, Z., Li, L., & Zeng, D. D. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3233371 · Cited by 10

A Transformer Based Method with Wide Attention Range for Enhanced Short-term Load Forecasting

Jiang, B., Liu, Y., Geng, H., Zeng, H., & Ding, J. (2022) · International Conference on Smart Power & Internet Energy Systems (SPIES) · doi:10.1109/spies55999.2022.10082249 · Cited by 12

Attention and self-attention in random forests

Utkin, L. V., Konstantinov, A. V., & Kirpichenko, S. R. (2023) · Progress in Artificial Intelligence · doi:10.1007/s13748-023-00301-0 · Cited by 14

Crossing language identification: Multilingual ASR framework based on semantic dataset creation & Wav2Vec 2.0

Anidjar, O. H., Yozevitch, R., Bigon, N., Abdalla, N. A., Myara, B., & Marbel, R. (2023) · Machine Learning with Applications · doi:10.1016/j.mlwa.2023.100489 · Cited by 11

The evolution of transformer models from unidirectional to bidirectional in Natural Language Processing

Sun, Y. (2024) · Applied and Computational Engineering · doi:10.54254/2755-2721/42/20230794 · Cited by 10

ABC-Learning: Attention-Boosted Contrastive Learning for unsupervised person re-identification

Zhang, S., Wang, C., & Peng, J. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.108344 · Cited by 10

Precision forecasting of grinding wheel Wear: A TransBiGRU model for advanced industrial predictive maintenance

Si, Z., Si, S., & Mu, D. (2024) · Measurement · doi:10.1016/j.measurement.2024.114859 · Cited by 11

The Synergy between Artificial Intelligence, Remote Sensing, and Archaeological Fieldwork Validation

Canedo, D., Hipólito, J., Fonte, J., Dias, R., do Pereiro, T., Georgieva, P., Gonçalves‐Seco, L., Vázquez, M., Pires, N., Fábrega‐Álvarez, P., et al. (2024) · Remote Sensing · doi:10.3390/rs16111933 · Cited by 10

MI-MBFT: Superior Motor Imagery Decoding of Raw EEG Data Based on a Multibranch and Fusion Transformer Framework

Luo, J., Cheng, Q., Wang, H., Du, Q., Wang, Y., & Li, Y. (2024) · IEEE Sensors Journal · doi:10.1109/jsen.2024.3445971 · Cited by 10

Supply Chain Network Extraction and Entity Classification Leveraging Large Language Models

Liu, T., & Meidani, H. (2024) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata62323.2024.10826017 · Cited by 10

Atleus: Accelerating Transformers on the Edge Enabled by 3D Heterogeneous Manycore Architectures

Dhingra, P., Doppa, J. R., & Pande, P. P. (2025) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2025.3531255 · Cited by 12

Large Language Models (LLMs) for Smart Manufacturing and Industry X.0

Baptista, M. L., Yue, N., Islam, M. M. M., & Prendinger, H. (2025) · Springer series in advanced manufacturing · doi:10.1007/978-3-031-80154-9_5 · Cited by 10

How to Write Effective Prompts for Screening Biomedical Literature Using Large Language Models

Colangelo, M. T., Guizzardi, S., Meleti, M., Calciolari, E., & Galli, C. (2025) · BioMedInformatics · doi:10.3390/biomedinformatics5010015 · Cited by 12

SwinVision: Detecting Small Objects in Low-Light Environments

Dai, T., Wang, Q., Shen, Y., & Gao, S. (2025) · IEEE Access · doi:10.1109/access.2025.3548151 · Cited by 10

Improving Transformer Performance for French Clinical Notes Classification Using Mixture of Experts on a Limited Dataset

Le, T.-D., Jouvet, P. A., & Noumeir, R. (2025) · IEEE Journal of Translational Engineering in Health and Medicine · doi:10.1109/jtehm.2025.3576570 · Cited by 17

LSTN:Long Short-Term Traffic Flow Forecasting with Transformer Networks

Yang, L., Ren, Q., Hu, J., & Han, M. (2022) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr56361.2022.9956632 · Cited by 10

Reconstruction of the flame nonlinear response using deep learning algorithms

Wu, J., Nan, J., Yang, L., & Li, J. (2022) · Physics of Fluids · doi:10.1063/5.0131928 · Cited by 10

CTS-Net: A Segmentation Network for Glaucoma Optical Coherence Tomography Retinal Layer Images

Xue, S., Wang, H., Guo, X., Sun, M., Song, K., Shao, Y., Zhang, H., & Zhang, T. (2023) · Bioengineering · doi:10.3390/bioengineering10020230 · Cited by 10

An N400 identification method based on the combination of Soft-DTW and transformer

Ma, Y., Tang, Y., Zeng, Y., Ding, T., & Liu, Y. (2023) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2023.1120566 · Cited by 10

PBFormer: Point and Bi-Spatiotemporal Transformer for Pointwise Change Detection of 3D Urban Point Clouds

Han, M., Sha, J., Wang, Y., & Wang, X. (2023) · Remote Sensing · doi:10.3390/rs15092314 · Cited by 11

Detection of Wastewater Pollution Through Natural Language Generation With a Low-Cost Sensing Platform

Roitero, K., Portelli, B., Serra, G., Mea, V. D., Mizzaro, S., Cerro, G., Vitelli, M., & Molinara, M. (2023) · IEEE Access · doi:10.1109/access.2023.3277535 · Cited by 12

Efficient Hardware Acceleration of Emerging Neural Networks for Embedded Machine Learning: An Industry Perspective

Raha, A., Sung, R., Ghosh, S. K., Gupta, P. K., Mathaikutty, D. A., Cheema, U. I., Hyland, K., Brick, C., & Raghunathan, V. (2023) · Springer eBooks · doi:10.1007/978-3-031-19568-6_5 · Cited by 10

A Comparative Study and a New Industrial Platform for Decentralized Anomaly Detection Using Machine Learning Algorithms

Gerz, F., Basturk, T. R., Kirchhoff, J., Denker, J., Al-Shrouf, L., & Jelali, M. (2022) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn55064.2022.9892939 · Cited by 13

Slice-Based Code Change Representation Learning

Zhang, F., Chen, B., Zhao, Y., & Peng, X. (2023) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner56733.2023.00038 · Cited by 9

Touchless Heart Rate Monitoring from an Unmanned Aerial Vehicle Using Videoplethysmography

Pająk, A., Przybyło, J., & Augustyniak, P. (2023) · Sensors · doi:10.3390/s23167297 · Cited by 9

Explainable Transformer-Based Anomaly Detection for Internet of Things Security

Saghir, A., Beniwal, H., Tran, K. D., Raza, A., Koehl, L., Zeng, X., & Tran, K. P. (2024) · EAI/Springer Innovations in Communication and Computing · doi:10.1007/978-3-031-53028-9_6 · Cited by 10

Recovering Physiological Signals From Facial Videos: Recent Advances and Applications in Intelligent Vehicles

Xiang, G., Ou, Y., Li, J., Wang, L., Hu, Y., Wang, X., Wu, X., & Peng, Y. L. (2024) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2024.3386859 · Cited by 9

Dual Structural Consistent Partial Domain Adaptation Network for Intelligent Machinery Fault Diagnosis

Yu, K., Wang, X., Cheng, Y., Feng, K., Zhang, Y., & Xing, B. (2024) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2024.3396831 · Cited by 10

Local Climate Zone Classification via Semi-Supervised Multimodal Multiscale Transformer

Lin, H., Wang, H., Yin, J., & Yang, J. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3399048 · Cited by 9

Detecting abnormal logins by discovering anomalous links via graph transformers

Gonçalves, L., & Zanchettin, C. (2024) · Computers & Security · doi:10.1016/j.cose.2024.103944 · Cited by 9

Learning-based distortion correction enables proximal-scanning endoscopic OCT elastography

Zhang, H., Gu, C., Lan, Q., Zhang, W., Liu, C., & Yang, J. (2024) · Biomedical Optics Express · doi:10.1364/boe.528522 · Cited by 9

Integer-only Quantized Transformers for Embedded FPGA-based Time-series Forecasting in AIoT

Ling, T., Qian, C., & Schiele, G. (2024) · IEEE Annual Congress on Artificial Intelligence of Things (AIoT) · doi:10.1109/aiot63253.2024.00017 · Cited by 9

TPE-Optimized DNN with Attention Mechanism for Prediction of Tower Crane Payload Moving Conditions

Akber, M. Z., Chan, W. K., Lee, H. H., & Anwar, G. A. (2024) · Mathematics · doi:10.3390/math12193006 · Cited by 9

Exploration of an intrinsically explainable self-attention based model for prototype generation on single-channel EEG sleep stage classification

Adey, B., Habib, A., & Karmakar, C. K. (2024) · Scientific Reports · doi:10.1038/s41598-024-79139-y · Cited by 9

Remote Sensing Image Change Detection Based on Lightweight Transformer and Multiscale Feature Fusion

Li, J., Zheng, P., & Wang, L. (2025) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2025.3529529 · Cited by 10

Self-Distillation Attention for Efficient and Accurate Motion Prediction in Autonomous Driving

Lu, Z., Cai, Y., Sun, X., Wang, H., Lian, Y., & Chen, L. (2025) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2025.3535772 · Cited by 9

Geodesign in the era of artificial intelligence

Ye, X., Huang, T., Song, Y. H., Li, X. S., Newman, G., Lin, Z., & Wu, D. J. (2025) · Frontiers of Urban and Rural Planning · doi:10.1007/s44243-025-00054-5 · Cited by 10

Application of LLMs/Transformer-Based Models for Metabolite Annotation in Metabolomics

Liu, Y., Zhang, F., Ge, Y., Liu, Q., He, S., & Shen, X. (2025) · Health and Metabolism · doi:10.53941/hm.2025.100014 · Cited by 9

Speech translation for multilingual medical education leveraged by large language models

Iranzo-Sánchez, J., Santamaría-Jordà, J., Mas-Mollà, G., Díaz-Munío, G. V. G., Iranzo-Sánchez, J., Jorge, J., Silvestre-Cerdà, J. A., Giménez, A., Civera, J., Sanchís, A., et al. (2025) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2025.103147 · Cited by 12

Efficient transformers: A survey

Tay, Y., Dehghani, M., Bahri, D., & Metzler, D. (2023) · ACM Computing Surveys · doi:10.1145/3530811 · Cited by 1,267

Survey: Transformer-based Models in Data Modality ConversionPreprint

Rashno, E., Eskandari, A., Anand, A., & Zulkernine, F. H. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2408.04723 · Cited by 8

A comprehensive survey on applications of transformers for deep learning tasks

Islam, S., Elmekki, H., Elsebai, A., Bentahar, J., Drawel, N., Rjoub, G., & Witold Pedrycz, C. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.122666 · Cited by 590

Transformer Models and their Cross-Domain Applications: A Systematic Review

Pathania, K., Singh, S., & Sharma, A. (2026) · Archives of Computational Methods in Engineering · doi:10.1007/s11831-026-10742-0

A Review on Transformer Models: Applications, Taxonomies, Open Issues and Challenges

Passi, N., Raj, M., & Shelke, N. A. (2024) · Asian Conference on Innovation in Technology (ASIANCON) · doi:10.1109/asiancon62057.2024.10838047 · Cited by 8