The DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
Convolutional Transformer: An Enhanced Attention Mechanism Architecture for Remaining Useful Life Estimation of Bearings
Abstract
Nowadays, deep learning (DL) methods for prognostic and health management (PHM) have vastly broadened the scope of applications in this field. Numerous approaches based on deep neural networks have been presented and applied to the remaining useful life (RUL) estimation of bearings. However, few of these methods are yet fully competent for the task of extracting degradation-related information from raw signals both locally and globally. To fill this research gap, we proposed a novel convolutional Transformer (CoT) which combines the global context capturing of attention mechanism with the local dependencies modeling of convolutional operation. Specifically, we designed a multi-scale convolutional (MSC) module with Swish activation for Transformer architecture to embed local feature learning into global sequence modeling. Our CoT fuses intra-token convolution and inter-token self-attention operations to enable simultaneous extraction of local dependencies and global interactions from the raw temporal signal into a trainable class token. Then, an end-to-end RUL estimation framework based on CoT is presented, which provides a mapping from raw vibration signals to estimated RULs. Finally, comprehensive case studies, including comparative studies and ablation experiments, fully validate the effectiveness and advancements of our CoT-based RUL estimation method.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Recent advancements have focused on developing sophisticated loss functions and regularization techniques, aimed at enhancing the performance and generalization capabilities of supervised learning models, ensuring they remain robust and effective across a wide range of tasks and data types [166], [167], [168].”
From From Google Gemini to OpenAI Q* (Q-Star): A Survey on Reshaping the Generative Artificial Intelligence (AI) Research Landscape (2025) · cites this paper as [166]
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Transformer-XL: Attentive language models beyond a fixed-length context
Dai, Z., Yang, Z., Yang, Y., Carbonell, J., Le, Q., & Salakhutdinov, R. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1285 · Cited by 3,169
Conformer: Convolution-augmented Transformer for Speech Recognition
Gulati, A., Qin, J., Chiu, C.-C., Parmar, N., Zhang, Y., Yu, J., Han, W., Wang, S., Zhang, Z., Wu, Y., et al. (2020) · Interspeech 2020 · doi:10.21437/interspeech.2020-3015 · Cited by 3,150
Searching for activation functions
Ramachandran, P., Zoph, B., & Le, Q. V. (2017) · arXiv · doi:10.48550/arxiv.1710.05941 · Cited by 2,335
Attention Augmented Convolutional Networks
Bello, I., Zoph, B., Le, Q. V., Vaswani, A., & Shlens, J. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00338 · Cited by 1,347
Efficient transformers: A survey
Tay, Y., Dehghani, M., Bahri, D., & Metzler, D. (2023) · ACM Computing Surveys · doi:10.1145/3530811 · Cited by 1,267
PRONOSTIA : An experimental platform for bearings accelerated degradation tests.
Nectoux, P., Gouriveau, R., Medjaher, K., Ramasso, E., Chebel‐Morello, B., Noureddine Zerhouni, A., & Varnier, C. (2012) · HAL · Cited by 928
Machine Health Monitoring Using Local Feature-Based Gated Recurrent Unit Networks
Zhao, R., Wang, D., Yan, R., Mao, K., Shen, F., & Wang, J. (2017) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2017.2733438 · Cited by 899
Remaining useful life estimation of engineered systems using vanilla LSTM neural networks
Wu, Y., Yuan, M., Dong, S., Lin, L., & Liu, Y. (2018) · Neurocomputing · doi:10.1016/j.neucom.2017.05.063 · Cited by 789
Estimation of Bearing Remaining Useful Life Based on Multiscale Convolutional Neural Network
Zhu, J., Chen, N., & Peng, W. (2018) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2018.2844856 · Cited by 723
Visual Transformers: Token-based Image Representation and Processing for Computer VisionPreprint
Wu, B., Xu, C., Dai, X., Wan, A., Zhang, P., Yan, Z., Tomizuka, M., Gonzalez, J., Keutzer, K., & Vajda, P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.03677 · Cited by 649
Pay Less Attention with Lightweight and Dynamic ConvolutionsPreprint
Wu, F., Fan, A., Baevski, A., Dauphin, Y., & Auli, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.10430 · Cited by 585
Ding, Y., Jia, M., Miao, Q., & Cao, Y. (2022) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2021.108616 · Cited by 559
Chen, J., Jing, H., Chang, Y., & Liu, Q. (2019) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2019.01.006 · Cited by 506
Deep separable convolutional network for remaining useful life prediction of machinery
Wang, B., Lei, Y., Li, N., & Yan, T. (2019) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2019.106330 · Cited by 424
Chen, Y., Peng, G., Zhu, Z., & Li, S. (2019) · Applied Soft Computing · doi:10.1016/j.asoc.2019.105919 · Cited by 400
Zhao, M., Kang, M., Tang, B., & Pecht, M. (2018) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2017.2762639 · Cited by 380
Lite Transformer with Long-Short Range AttentionPreprint
Wu, Z., Liu, Z., Lin, J., Lin, Y., & Han, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.11886 · Cited by 317
Prediction of Bearing Remaining Useful Life With Deep Convolution Neural Network
Ren, L., Sun, Y., Wang, H. H., & Li, Z. (2018) · IEEE Access · doi:10.1109/access.2018.2804930 · Cited by 292
Liao, L., Jin, W., & Pavel, R. (2016) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2016.2586442 · Cited by 285
A Deep Learning-Based Remaining Useful Life Prediction Approach for Bearings
Cheng, C., Ma, G., Zhang, Y., Sun, M., Teng, F., Ding, H., & Yuan, Y. (2020) · IEEE/ASME Transactions on Mechatronics · doi:10.1109/tmech.2020.2971503 · Cited by 251
A BiGRU method for remaining useful life prediction of machinery
She, D., & Jia, M. (2020) · Measurement · doi:10.1016/j.measurement.2020.108277 · Cited by 248
Feature Extraction for Data-Driven Remaining Useful Life Prediction of Rolling Bearings
Zhao, H., Liu, H., Jin, Y. F., Dang, X., & Deng, W. (2021) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2021.3059500 · Cited by 197
He, Z., Shao, H., Ding, Z., Jiang, H., & Cheng, J. (2021) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2021.3050382 · Cited by 168
Weighted Transformer Network for Machine TranslationPreprint
Ahmed, K., Keskar, N. S., & Socher, R. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.02132 · Cited by 152
Li, X., Cheng, J., Shao, H., Liu, K., & Cai, B. (2021) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2021.3125385 · Cited by 135
Chen, Z., Li, Y., Xia, T., & Pan, E. (2017) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2017.09.002 · Cited by 131
Token shift transformer for video classification
ZHANG Hao, Hao, Y., & Ngo, C. (2021) · Singapore Management University Institutional Knowledge (InK) (Singapore Management University) · Cited by 121
Chao, Q., Gao, H., Tao, J., Wang, Y., Zhou, J. H., & Liu, C. (2021) · Science China Technological Sciences · doi:10.1007/s11431-021-1904-7 · Cited by 103
Remaining Useful Life Estimation Under Multiple Operating Conditions via Deep Subdomain Adaptation
Ding, Y., Jia, M., & Cao, Y. (2021) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2021.3076567 · Cited by 91
Tang, G., Wang, Y., Huang, Y., Liu, N., & He, J. (2020) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2020.2965634 · Cited by 78
Jin, C., & Chen, X. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.114570 · Cited by 70
Shao, H., Li, W., Xia, M., Zhang, Y., Shen, C., Williams, D., Kennedy, A. R., & de Silva, C. W. (2021) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2021.3111977 · Cited by 64
Alexakos, C. T., Karnavas, Y. L., Drakaki, M., & Tziafettas, I. A. (2021) · Machine Learning and Knowledge Extraction · doi:10.3390/make3010011 · Cited by 62
A Convolutional Transformer Architecture for Remaining Useful Life Estimation
Ding, Y., & Jia, M. (2021) · Global Reliability and Prognostics and Health Management Conference (PHM) · doi:10.1109/phm-nanjing52125.2021.9612814 · Cited by 21
A Transformer Model-Based Approach to Bearing Fault Diagnosis
Bao, Z., Du, J., Zhang, W., Wang, J., Qiu, T., & Cao, Y. (2021) · Communications in computer and information science · doi:10.1007/978-981-16-5940-9_5 · Cited by 19
Research works printed but not linked: 3
- Dosovitskiy, An image is worth $16\times16$ words: Transformers for image recognition at scale
- 10.1145/3474085.3475272 · DOI
- Liang, EViT: Expediting vision transformers via token reorganizations
Cited by
Sorted by citations
Yan, S., Shao, H., Wang, J., Zheng, X., & Liu, B. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.121338 · Cited by 409
Qiu, S., Cui, X., Ping, Z., Shan, N., Li, Z., Bao, X., & Xu, X. (2023) · Sensors · doi:10.3390/s23031305 · Cited by 268
Han, S.-Y., Shao, H., Cheng, J., Yang, X., & Cai, B. (2022) · IEEE/ASME Transactions on Mechatronics · doi:10.1109/tmech.2022.3199985 · Cited by 224
Zhang, Y., Su, C., Wu, J., Liu, H., & Xie, M. (2023) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2023.109662 · Cited by 155
Li, X., Xiao, S., Li, Q., Zhu, L., Wang, T., & Chu, F. (2025) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2025.111122 · Cited by 122
Metadata
- Authors
- Ding, Y., & Jia, M.
- Affiliations
- Southeast University (CN)
- Venue
- IEEE Transactions on Instrumentation and Measurement
- Published
- 2022
- Volume
- 71
- Pages
- 1-10
- ISSN
- 0018-9456 (ISSN-L)
- 1557-9662
- OpenAlex
- W4285186957
- Cited by
- 120
- References
- 38
- Citations by year
- 2026: 18 · 2025: 39 · 2024: 39 · 2023: 21 · 2022: 3
- FWCI
- 10.28
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Transactions on Instrumentation and Measurement (publishedVersion)
- Funders
- National Natural Science Foundation of China (52075095)
- Postgraduate Research and Practice Innovation Program of Jiangsu Province (KYCX21_0082)
- Topics
- Physical Sciences › Engineering › Control and Systems Engineering › Machine Fault Diagnosis Techniques
- Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Reliability and Maintenance Optimization
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Power Transformer Diagnostics and Insulation
- Keywords
- remaining useful life prediction, bearings, convolutional transformer, attention mechanism, multi-scale convolution, self-attention, vibration signals, degradation modeling, prognostics and health management, class token, local and global feature extraction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
