A survey of transformers

Deep Residual Learning for Image Recognition

He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474

Attention Is All You Need

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325

An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472

Swin Transformer: Hierarchical Vision Transformer using Shifted Windows

Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · arXiv · Cited by 35,608

Swin Transformer: Hierarchical Vision Transformer using Shifted Windows

Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818

Language Models are Few-Shot Learners

Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372

End-to-End Object Detection with Transformers

Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58452-8_13 · Cited by 18,285

The Helicobacter pylori AI-clinician harnesses artificial intelligence to personalise H. pylori treatment recommendations

Higgins, K., Nyssen, O. P., Southern, J., Laponogov, I., Miralles-Marco, A., Cabeza-Segura, M., Jiménez-Martí, E., Castillo, J., Leja, M., Poļaka, I., et al. (2025) · Nature Communications · doi:10.1038/s41467-025-61329-5 · Cited by 24

Sentiment Analysis and Classification of Hotel Opinions in Twitter With the Transformer Architecture.

Arroni, S., Galán, Y., Guzmán-Guzmán, X., Núñez‐Valdéz, E. R., & Gómez, A. (2023) · International Journal of Interactive Multimedia and Artificial Intelligence · doi:10.9781/ijimai.2023.02.005 · Cited by 23

Extrapolation of affective norms using transformer-based neural networks and its application to experimental stimuli selection

Plisiecki, H., & Sobieszek, A. (2023) · Behavior Research Methods · doi:10.3758/s13428-023-02212-3 · Cited by 24

Network intrusion detection system by learning jointly from tabular and text‐based features

Düzgün, B., Çayır, A., Ünal, U., & Dağ, H. (2023) · Expert Systems · doi:10.1111/exsy.13518 · Cited by 25

A review on machine learning-guided design of energy materials

Kim, S., Xu, J., Shang, W., Xu, Z., Lee, E., & Luo, T. (2024) · Progress in Energy · doi:10.1088/2516-1083/ad7220 · Cited by 19

Fcg-Former: Identification of Functional Groups in FTIR Spectra Using Enhanced Transformer-Based Model

Doan, V. H. M., Ly, C. D., Mondal, S., Truong, T. T., Nguyen, T. D., Choi, J., Lee, B., & Oh, J. (2024) · Analytical Chemistry · doi:10.1021/acs.analchem.4c01622 · Cited by 19

Spiers Memorial Lecture: How to do impactful research in artificial intelligence for chemistry and materials science

Cheng, A., Ser, C. T., Skreta, M., Guzmán-Cordero, A., Thiede, L., Burger, A., Aldossary, A., Leong, S. X., Pablo‐García, S., Strieth‐Kalthoff, F., et al. (2024) · Faraday Discussions · doi:10.1039/d4fd00153b · Cited by 22

Impact of EU non-financial reporting regulation on Spanish companies’ environmental disclosure: a cutting-edge natural language processing approach

Villacampa-Porta, J., Vaca, M. C., & Garrido‐Merchán, E. C. (2025) · Environmental Sciences Europe · doi:10.1186/s12302-025-01067-z · Cited by 29

Integrating biological and machine intelligence: Attention mechanisms in brain–computer interfaces

Wang, J., Ye, W., He, J., Zhang, L., Huang, G., Yu, Z., & Liang, Z. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103417 · Cited by 24

Pathway-Aware Multimodal Transformer (PAMT): Integrating Pathological Image and Gene Expression for Interpretable Cancer Survival Analysis

Yan, R., Zhang, X., Jiang, Z., Wang, B., Bian, X., Ren, F., & Zhou, S. K. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3611531 · Cited by 20

Building Blocks for a Complex-Valued Transformer Architecture

Eilers, F., & Jiang, X. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10095349 · Cited by 19

Atmospheric visibility estimation: a review of deep learning approach

Ouadil, K. A., Idbraim, S., Bouhsine, T., Bouaynaya, N. C., Alfergani, H., & Johnson, C. C. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-16855-z · Cited by 19

Graph gating-mixer for sequential recommendation

Wu, B., Su, X., Liang, J., Sun, Z., Zhong, L., & Ye, Y. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.122060 · Cited by 19

Exploring deep learning approaches for video captioning: A comprehensive review

Yousif, A. J., & Al-Jammas, M. H. (2023) · e-Prime - Advances in Electrical Engineering Electronics and Energy · doi:10.1016/j.prime.2023.100372 · Cited by 25

Multivariate time series anomaly detection via separation, decomposition, and dual transformer-based autoencoder

Fu, S., Gao, X., Li, B., Zhai, F., Lu, J., Xue, B., Yu, J., & Xiao, C. (2024) · Applied Soft Computing · doi:10.1016/j.asoc.2024.111671 · Cited by 22

Context, Composition, Automation, and Communication: The C 2 AC Roadmap for Modeling and Simulation

Uhrmacher, A. M., Frazier, P. I., Hähnle, R., Klügl, F., Lorig, F., Ludäscher, B., Nenzi, L., Ruiz-Martín, C., Rumpe⋆, B., Szabo, C., et al. (2024) · ACM Transactions on Modeling and Computer Simulation · doi:10.1145/3673226 · Cited by 21

HSMix: Hard and soft mixing data augmentation for medical image segmentation

Sun, D.-Y., Dornaika, F., & Barrena, N. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102741 · Cited by 20

A novel method for intelligent operation and maintenance of transformers using deep visual large model DETR + X and digital twin

Zhang, X., Sun, W., Chen, K., & Song, S. (2025) · Scientific Reports · doi:10.1038/s41598-024-83561-7 · Cited by 19

Using Temporal Deep Learning Models to Estimate Daily Snow Water Equivalent Over the Rocky Mountains

Duan, S., Ullrich, P. A., Risser, M. D., & Rhoades, A. M. (2024) · Water Resources Research · doi:10.1029/2023wr035009 · Cited by 24

Synthetic Aperture Radar Imaging Meets Deep Unfolded Learning: A comprehensive review

Wang, M., Hu, Y., Wei, S., Shi, J., Cui, G., Kong, L., & Guo, Y. (2024) · IEEE Geoscience and Remote Sensing Magazine · doi:10.1109/mgrs.2024.3494754 · Cited by 19

A systematic review of Machine Learning and Deep Learning approaches in Mexico: challenges and opportunities

Castillo, J. L. U., Celestino, A. E. M., Cruz, D. A. M., Tuxpan-Vargas, J., Ramos‐Leal, J. A., & Morán‐Ramírez, J. (2025) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1479855 · Cited by 24

Vehicle-to-everything mode of operation technologies: A state-of-art systematic review

Safari, A., & Rahimi, A. (2025) · Applied Energy · doi:10.1016/j.apenergy.2025.126382 · Cited by 28

A survey of transformer architectures for autonomous driving

Chu, F., Li, H., Xie, L., & Zhao, J. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.130338 · Cited by 20

Explainable deep learning based techniques for ECG-Based heart disease classification: A systematic literature review and future direction

Manimaran, G., Peimankar, A., Puthusserypady, S. K., Momeni, M., Asyari, R. A. I., Jahan, M. S., Moll, J., Wiil, U. K., & Ebrahimi, A. (2025) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2025.111324 · Cited by 23

Paradigm Shift Toward Distributed Learning in IoT Intelligence: A Comprehensive Survey of Opportunities and Challenges

AbdelRaouf, H., Hasnaine, Q. R., Fouda, M. M., Fadlullah, Z. M., & Ibrahem, M. I. (2026) · IEEE Internet of Things Journal · doi:10.1109/jiot.2026.3657733 · Cited by 18

Opioid death projections with AI-based forecasts using social media language

Matero, M., Giorgi, S., Curtis, B. L., Ungar, L., & Schwartz, H. A. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00776-0 · Cited by 19

Early and Accurate Detection of Tomato Leaf Diseases Using TomFormer

Khan, A. A., Nawaz, U., Kshetrimayum, L., Seneviratne, L. D., & Hussain, I. (2023) · International Conference on Advanced Robotics (ICAR) · doi:10.1109/icar58858.2023.10436499 · Cited by 18

Assessing the Predictive Power of Transformers, ARIMA, and LSTM in Forecasting Stock Prices of Moroccan Credit Companies

Lahboub, K., & Benali, M. (2024) · Journal of risk and financial management · doi:10.3390/jrfm17070293 · Cited by 18

AI Computing Systems for Large Language Models Training

Zhang, Z., Wen, Y., Lyu, H.-Q., Liu, C., Zhang, R., Li, X., Wang, C., Du, Z., Guo, Q., Li, L., et al. (2025) · Journal of Computer Science and Technology · doi:10.1007/s11390-024-4178-1 · Cited by 22

Survey on legal information extraction: current status and open challenges

Premasiri, D., Ranasinghe, T., Mitkov, R., El‐Haj, M., & Frommholz, I. (2025) · Knowledge and Information Systems · doi:10.1007/s10115-025-02600-5 · Cited by 25

SPRNet: Sitting Posture Recognition Using improved Vision Transformer

Fang, Y., Shi, S., Fang, J., & Yin, W. (2022) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn55064.2022.9892021 · Cited by 19

Prompting and Tuning: A Two-Stage Unsupervised Domain Adaptive Person Re-identification Method on Vision Transformer Backbone

Yu, S., Dou, Z., & Wang, S. (2023) · Tsinghua Science & Technology · doi:10.26599/tst.2022.9010044 · Cited by 17

Predicting Respiratory Rate from Electrocardiogram and Photoplethysmogram Using a Transformer-Based Model

Zhao, Q., Liu, F., Song, Y., Fan, X., Wang, Y., Yao, Y., Mao, Q., & Zhao, Z. (2023) · Bioengineering · doi:10.3390/bioengineering10091024 · Cited by 17

Asymmetric cross-modal attention network with multimodal augmented mixup for medical visual question answering

Li, Y., Yang, Q., Wang, F. L., Lee, L., Qu, Y., & Hao, T. (2023) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2023.102667 · Cited by 17

Comprehensive assessment of protein loop modeling programs on large-scale datasets: prediction accuracy and efficiency

Wang, T., Wang, L., Zhang, X., Shen, C., Zhang, O., Wang, J., Wu, J., Jin, R., Zhou, D., Chen, S., et al. (2023) · Briefings in Bioinformatics · doi:10.1093/bib/bbad486 · Cited by 19

Multi-Year Time Series Transfer Learning: Application of Early Crop Classification

Račič, M., Oštir, K., Zupanc, A., & Zajc, L. Č. (2024) · Remote Sensing · doi:10.3390/rs16020270 · Cited by 17

tFUSFormer: Physics-Guided Super-Resolution Transformer for Simulation of Transcranial Focused Ultrasound Propagation in Brain Stimulation

Shin, M., Seo, M., Yoo, S., & Yoon, K. (2024) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2024.3389708 · Cited by 17

ACTNet: Attention based CNN and Transformer network for respiratory rate estimation

Chen, H., Zhang, X., Guo, Z., Ying, N., Yang, M., & Guo, C. (2024) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2024.106497 · Cited by 20

Advanced techniques for automated emotion recognition in dogs from video data through deep learning

Franzoni, V., Biondi, G., & Milani, A. (2024) · Neural Computing and Applications · doi:10.1007/s00521-024-10042-3 · Cited by 18

Using Large Language Models to Understand Telecom Standards

Karapantelakis, A., Thakur, M., Nikou, A., Moradi, F., Olrog, C., Gaim, F., Holm, H., Nimara, D. D., & Huang, V. (2024) · IEEE International Conference on Machine Learning for Communication and Networking (ICMLCN) · doi:10.1109/icmlcn59089.2024.10624786 · Cited by 31

Transformer-based approaches for neuroimaging: an in-depth review of their role in classification and regression tasks

Zhu, X., Sun, S., Lin, L., Wu, Y., & Ma, X. (2024) · Reviews in the Neurosciences · doi:10.1515/revneuro-2024-0088 · Cited by 18

Short-term residential load forecasting via transfer learning and multi-attention fusion for EVs’ coordinated charging

Gao, S., Liu, Y., Wang, J., Wang, Z., Xu, W., Yue, R., Cui, R., Liu, Y., & Fan, X. (2024) · International Journal of Electrical Power & Energy Systems · doi:10.1016/j.ijepes.2024.110349 · Cited by 19

CNN-Transformer network for student learning effect prediction using EEG signals based on spatio-temporal feature fusion

Xie, H. M., Dong, Z., Yang, H., Luo, Y., Ren, S., Zhang, P., He, J., Jia, C., Yang, Y., Jiang, M., et al. (2024) · Applied Soft Computing · doi:10.1016/j.asoc.2024.112631 · Cited by 20

Integration of Mamba and Transformer - MAT for Long-Short Range Time Series Forecasting with Application to Weather Dynamics

Zhang, W., Huang, J., Wang, R., Wei, C., Huang, W., & Qiao, Y. (2024) · International Conference on Electrical, Communication, and Computer Engineering (ICECCE) · doi:10.1109/icecce63537.2024.10823516 · Cited by 17

State-space modeling in long sequence processing: a survey on recurrence in the transformer era

Tiezzi, M., Casoni, M., Betti, A., Gori, M., & Melacci, S. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.108039 · Cited by 29

AI-powered language models for alloy design and laser-based manufacturing: A review of NLP applications in materials science

Nabavi, S. F., Garmestani, H., & Fekri, F. (2025) · Journal of Manufacturing Processes · doi:10.1016/j.jmapro.2025.11.035 · Cited by 19

Applying BERT for Early-Stage Recognition of Persistence in Chat-Based Social Engineering Attacks

Tsinganos, N., Fouliras, P., & Mavridis, I. K. (2022) · Applied Sciences · doi:10.3390/app122312353 · Cited by 18

MITNet: a fusion transformer and convolutional neural network architecture approach for T-cell epitope prediction

Darmawan, J. T., Leu, J., Avian, C., & Ratnasari, N. R. P. (2023) · Briefings in Bioinformatics · doi:10.1093/bib/bbad202 · Cited by 17

Model-based learning on state-based potential games for distributed self-optimization of manufacturing systems

Yuwono, S., & Schwung, A. (2023) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2023.10.003 · Cited by 17

Identifying Winter Wheat Using Landsat Data Based on Deep Learning Algorithms in the North China Plain

Zhang, Q., Wang, G., Wang, G., Song, W., Wei, X., & Hu, Y. (2023) · Remote Sensing · doi:10.3390/rs15215121 · Cited by 16

Performance Evaluation of Swin Vision Transformer Model Using Gradient Accumulation Optimization Technique

Aburass, S., & Dorgham, O. M. (2023) · Lecture notes in networks and systems · doi:10.1007/978-3-031-47448-4_5 · Cited by 16

Multi-model deep learning approach for segmentation of teeth and periapical lesions on pantomographs

Adnan, N., Umer, F., Malik, S., & Hussain, O. A. (2023) · Oral Surgery Oral Medicine Oral Pathology and Oral Radiology · doi:10.1016/j.oooo.2023.11.006 · Cited by 16

A novel parallel feature extraction-based multibatch process quality prediction method with application to a hot rolling mill process

Zhang, K., Zhang, X., & Peng, K. (2024) · Journal of Process Control · doi:10.1016/j.jprocont.2024.103166 · Cited by 16

A Systematic Review on Semantic Role Labeling for Information Extraction in Low-Resource Data

Ariyanto, A. D. P., Purwitasari, D., & Fatichah, C. (2024) · IEEE Access · doi:10.1109/access.2024.3392370 · Cited by 20

Exposing the Achilles’ heel of textual hate speech classifiers using indistinguishable adversarial examples

Aggarwal, S., & Vishwakarma, D. K. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.124278 · Cited by 17

Exploring the Landscape of Ubiquitous In-home Health Monitoring: A Comprehensive Survey

Pourpanah, F., & Etemad, A. (2024) · ACM Transactions on Computing for Healthcare · doi:10.1145/3670854 · Cited by 19

Continuous blood pressure monitoring based on transformer encoders and stacked attention gated recurrent units

Huang, Z., Shao, J., Zhou, P., Liu, B., Zhu, J., & Fang, D. (2024) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2024.106860 · Cited by 16

A survey on cell nuclei instance segmentation and classification: Leveraging context and attention

Nunes, J. D., Montezuma, D., Oliveira, D., Pereira, T., & Cardoso, J. S. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103360 · Cited by 19

Application of deep learning in cloud cover prediction using geostationary satellite images

Lee, Y., Min, S., Yoon, J., Ha, J., Jeong, S., Ryu, S., & Ahn, M. (2024) · GIScience & Remote Sensing · doi:10.1080/15481603.2024.2440506 · Cited by 16

Interpretability and performance assessment of advanced machine learning models for α-factor prediction in wastewater treatment plants

Tenneti, S., Divya, P. D., Tejaswini, E. S. S., Randhi, R., & Ambati, S. R. (2025) · Journal of Water Process Engineering · doi:10.1016/j.jwpe.2025.107637 · Cited by 15

Transformer and Large Language Models for Automatic Multiple-Choice Question Generation: A Systematic Literature Review

Awalurahman, H. W., Aji, R. F., & Budi, I. D. (2025) · IEEE Access · doi:10.1109/access.2025.3590423 · Cited by 21

Emotion Analysis Using Multilayered Networks for Graphical Representation of Tweets

Nguyen, A., Longa, A., Luca, M., Kaul, J., & Lopez, G. (2022) · IEEE Access · doi:10.1109/access.2022.3207161 · Cited by 15

Sparse and Hierarchical Transformer for Survival Analysis on Whole Slide Images

Yan, R., Lv, Z., Yang, Z., Lin, S., Zheng, C.-H., & Zhang, F. (2023) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2023.3307584 · Cited by 16

DFECF-DET: All-Weather Detector Based on Differential Feature Enhancement and Cross-Modal Fusion With Visible and Infrared Sensors

Wang, C., Sun, D., Yang, J., Li, Z., & Gao, Q. (2023) · IEEE Sensors Journal · doi:10.1109/jsen.2023.3324451 · Cited by 15

Switch-Transformer Sentiment Analysis Model for Arabic Dialects That Utilizes a Mixture of Experts Mechanism

Baniata, L. H., & Kang, S. (2024) · Mathematics · doi:10.3390/math12020242 · Cited by 15

Exploring the potential of representation and transfer learning for anatomical neuroimaging: Application to psychiatry

Dufumier, B., Gori, P., Petiton, S., Louiset, R., Mangin, J., Grigis, A., & Duchesnay, É. (2024) · NeuroImage · doi:10.1016/j.neuroimage.2024.120665 · Cited by 18

Rethinking Position Embedding Methods in the Transformer Architecture

Zhou, X., Ren, Z., Zhou, S., Jiang, Z., Yu, T., & Luo, H. (2024) · Neural Processing Letters · doi:10.1007/s11063-024-11539-7 · Cited by 18

Slimmable transformer with hybrid axial-attention for medical image segmentation

Hu, Y., Mu, N., Liu, L., Zhang, L., Jiang, J., & Li, X. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.108370 · Cited by 17

Knowledge-Enhanced Conversational Recommendation via Transformer-Based Sequential Modeling

Zou, J., Sun, A., Long, C., & Kanoulas, E. (2024) · ACM Transactions on Information Systems · doi:10.1145/3677376 · Cited by 16

A survey on moral foundation theory and pre-trained language models: current advances and challenges

Zangari, L., Greco, C. M., Picca, D., & Tagarelli, A. (2025) · AI & Society · doi:10.1007/s00146-025-02225-w · Cited by 21

Textual Pre-Trained Models for Gender Identification Across Community Question-Answering Members

Schwarzenberg, P., & Figueroa, A. (2023) · IEEE Access · doi:10.1109/access.2023.3235735 · Cited by 17

NMRMind: A Transformer-Based Model Enabling the Elucidation from Multidimensional NMR to Structures

Xue, X., Sun, H., Sun, J., Patiny, L., Liu, X., Chen, K., Yan, J., Liangning, L., Liu, X., Xu, S., et al. (2025) · Analytical Chemistry · doi:10.1021/acs.analchem.5c03783 · Cited by 20

A survey on molecular-scale learning systems with relevance to DNA computing

Nagipogu, R. T., Fu, D., & Reif, J. H. (2023) · Nanoscale · doi:10.1039/d2nr06202j · Cited by 15

NDT Method for Weld Defects Based on FMPVit Transformer Model

Liu, Y., Yuan, K., Li, T., & Li, S. (2023) · IEEE Access · doi:10.1109/access.2023.3283589 · Cited by 14

TransforLearn: Interactive Visual Tutorial for the Transformer Model

Gao, L., Shao, Z., Luo, Z., Hu, H., Turkay, Ç., & Chen, S. (2023) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2023.3327353 · Cited by 15

ULSeq-TA: Ultra-Long Sequence Attention Fusion Transformer Accelerator Supporting Grouped Sparse Softmax and Dual-Path Sparse LayerNorm

Wang, J., Zhang, L., Li, X., Yang, H., & Liu, Y. (2023) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2023.3329039 · Cited by 15

Cross-Parallel Transformer: Parallel ViT for Medical Image Segmentation

Dong, W., Wang, Z., Chen, L., Xiao, H., & Yang, B. (2023) · Sensors · doi:10.3390/s23239488 · Cited by 16

A hybrid convolutional neural network-transformer method for received signal strength indicator fingerprinting localization in Long Range Wide Area Network

Lutakamale, A. S., Myburgh, H. C., & Freitas, A. D. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.108349 · Cited by 14

DePNR : A DeBERTa ‐based deep learning model with complete position embedding for place name recognition from geographical literature

Weirong, L., Sun, K., Wang, S., Zhu, Y., Dai, X., & Hu, L. (2024) · Transactions in GIS · doi:10.1111/tgis.13170 · Cited by 13

Learning-Based Model Reduction and Predictive Control of an Ammonia Synthesis Process

Cabral, T. O., Bagheri, A., & Pourkargar, D. B. (2024) · Industrial & Engineering Chemistry Research · doi:10.1021/acs.iecr.4c00340 · Cited by 14

FRAME: A Modular Framework for Autonomous Map Merging: Advancements in the Field

Stathoulopoulos, N., Lindqvist, B., Коваль, А. В., Agha–mohammadi, A., & Nikolakopoulos, G. (2024) · IEEE transactions on field robotics. · doi:10.1109/tfr.2024.3419439 · Cited by 14

ResNeTS: A ResNet for Time Series Analysis of Sentinel-2 Data Applied to Grassland Plant-Biodiversity Prediction

Dieste, Á. G., Argüello, F., Heras, D. B., Magdon, P., Linstädter, A., Dubovyk, O., & Muro, J. (2024) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2024.3454271 · Cited by 16

Distributed digital twins on the open-source OpenTwins framework

Infante, S., Robles, J., Martín, C., Rubio, B., & Díáz, M. (2024) · Advanced Engineering Informatics · doi:10.1016/j.aei.2024.102970 · Cited by 14

Visual Prompt Flexible-Modal Face Anti-Spoofing

Yu, Z., Cai, R., Cui, Y., Liu, A., & Chen, C. (2023) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2024.3520534 · Cited by 10

Chaotic recurrent neural networks for brain modelling: A review

Mattera, A., Alfieri, V., Granato, G., & Baldassarre, G. (2024) · Neural Networks · doi:10.1016/j.neunet.2024.107079 · Cited by 19

Deadline-Aware Online Job Scheduling for Distributed Training in Heterogeneous Clusters

Zhang, Y., Luo, L., Sun, G., Yu, H., & Li, B. (2025) · IEEE Transactions on Cloud Computing · doi:10.1109/tcc.2025.3548604 · Cited by 14

Ultra-short-term wind speed forecasting based on secondary decomposition and Transformer-MLR combined model

Yue, Y., Zheng, W., Wu, A., Jin, X., Huang, Z., & Zhang, H. (2025) · Electric Power Systems Research · doi:10.1016/j.epsr.2025.111702 · Cited by 14

Time-Frequency Domain Lightweight Dual-Branch MSCFormer for PMSM ITSC Fault Diagnosis

Yu, H., Li, S., Xu, Y., Zhang, Z., Mu, H., & Han, W. (2025) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2025.3579108 · Cited by 16

DropDim: A Regularization Method for Transformer Networks

Zhang, H., Qu, D., Shao, K., & Yang, X. (2022) · IEEE Signal Processing Letters · doi:10.1109/lsp.2022.3140693 · Cited by 14

SensorFormer: Efficient Many-to-Many Sensor Calibration With Learnable Input Subsampling

Cheng, Y., Saukh, O., & Thiele, L. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2022.3177948 · Cited by 14

Automatic generation of scientific papers for data augmentation in document layout analysis

Pisaneschi, L., Gemelli, A., & Marinai, S. (2023) · Pattern Recognition Letters · doi:10.1016/j.patrec.2023.01.018 · Cited by 14

Comprehensive application of denoising diffusion probabilistic models towards the automation of analog integrated circuit sizing

de Azevedo, F. P., Lourenço, N. C. C., & Martins, R. M. F. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.128414 · Cited by 15

CAP: The creativity assessment platform for online testing and automated scoring

Patterson, J. D., Pronchick, J., Panchanadikar, R., Fuge, M., van Hell, J. G., Miller, S. R., Johnson, D. R., & Beaty, R. E. (2025) · Behavior Research Methods · doi:10.3758/s13428-025-02761-9 · Cited by 13

A Review on Sound Source Localization in Robotics: Focusing on Deep Learning Methods

Jalayer, R., Jalayer, M., & Baniasadi, A. (2025) · Applied Sciences · doi:10.3390/app15179354 · Cited by 21

Multi-source heterogeneous data fusion and intelligent prediction modeling for chemical engineering construction projects based on improved transformer architecture

Xin, D. (2025) · Scientific Reports · doi:10.1038/s41598-025-22752-2 · Cited by 16

A Systematic Review of Diffusion Models for Medical Image-Based Diagnosis: Methods, Taxonomies, Clinical Integration, Explainability, and Future Directions

Azad, M., Fahad, N. M., Raiaan, M. A. K., Anik, T. R., Khan, M. F. K., Toyé, H. M. K., & Muhammad, G. (2026) · Diagnostics · doi:10.3390/diagnostics16020211 · Cited by 13

Large language model agent as a mechanical designer

Jadhav, Y., & Farimani, A. B. (2026) · Journal of Engineering Design · doi:10.1080/09544828.2026.2624356 · Cited by 18

DPED: Bio-inspired dual-pathway network for edge detection

Chen, Y., Lin, C., & Qiao, Y. (2022) · Frontiers in Bioengineering and Biotechnology · doi:10.3389/fbioe.2022.1008140 · Cited by 13

Detecting anomalous electricity consumption with transformer and synthesized anomalies

Mu, T., Yu, Y., Feng, G., Luo, H., & Yang, H. (2023) · PeerJ Computer Science · doi:10.7717/peerj-cs.1721 · Cited by 13

Improving Efficiency and Generalisability of Motion Predictions With Deep Multi-Agent Learning and Multi-Head Attention

Benrachou, D. E., Glaser, S., Elhenawy, M. M. Z., & Rakotonirainy, A. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3339640 · Cited by 13

Efficient transformers: A survey

Tay, Y., Dehghani, M., Bahri, D., & Metzler, D. (2023) · ACM Computing Surveys · doi:10.1145/3530811 · Cited by 1,267

Survey: Transformer-based Models in Data Modality ConversionPreprint

Rashno, E., Eskandari, A., Anand, A., & Zulkernine, F. H. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2408.04723 · Cited by 8

A comprehensive survey on applications of transformers for deep learning tasks

Islam, S., Elmekki, H., Elsebai, A., Bentahar, J., Drawel, N., Rjoub, G., & Witold Pedrycz, C. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.122666 · Cited by 590

Transformer Models and their Cross-Domain Applications: A Systematic Review

Pathania, K., Singh, S., & Sharma, A. (2026) · Archives of Computational Methods in Engineering · doi:10.1007/s11831-026-10742-0

A Review on Transformer Models: Applications, Taxonomies, Open Issues and Challenges

Passi, N., Raj, M., & Shelke, N. A. (2024) · Asian Conference on Innovation in Technology (ASIANCON) · doi:10.1109/asiancon62057.2024.10838047 · Cited by 8