A novel time–frequency Transformer based on self–attention mechanism and its application in fault diagnosis of rolling bearings
Open accessThe claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Recent advancements in supervised learning focus on sophisticated loss functions and regularization techniques to enhance performance and generalization, ensuring robustness across tasks and data types.
“Recent advancements have focused on developing sophisticated loss functions and regularization techniques, aimed at enhancing the performance and generalization capabilities of supervised learning models, ensuring they remain robust and effective across a wide range of tasks and data types [166], [167], [168].”
From From Google Gemini to OpenAI Q* (Q-Star): A Survey on Reshaping the Generative Artificial Intelligence (AI) Research Landscape (2025) · cites this paper as [167]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · see citing works
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
Squeeze-and-Excitation Networks
Hu, J., Shen, L., & Sun, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00745 · Cited by 35,667 · see citing works
(2014) · Cited by 33,419 · see citing works
(2008) · Journal of Machine Learning Research · Cited by 33,051 · see citing works
Rethinking the Inception Architecture for Computer Vision
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.308 · Cited by 32,138 · see citing works
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 21,793 · see citing works
Neural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213 · see citing works
Layer NormalizationPreprint
Ba, J., Kiros, J., & Hinton, G. E. (2016) · arXiv.org · doi:10.48550/arxiv.1607.06450 · Cited by 13,745 · see citing works
Residual Attention Network for Image Classification
Wang, F., Jiang, M., Qian, C., Yang, S., Li, C., Zhang, H., Wang, X., & Tang, X. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.683 · Cited by 4,073 · see citing works
Cubic convolution interpolation for digital image processing
Keys, R. G. (1981) · IEEE Transactions on Acoustics Speech and Signal Processing · doi:10.1109/tassp.1981.1163711 · Cited by 3,770 · see citing works
Gaussian Error Linear Units (GELUs)
Hendrycks, D., & Gimpel, K. (2016) · arXiv · doi:10.48550/arxiv.1606.08415 · Cited by 3,537 · see citing works
Self-Attention Generative Adversarial Networks
Zhang, H., Goodfellow, I., Metaxas, D., & Odena, A. (2018) · arXiv · doi:10.48550/arxiv.1805.08318 · Cited by 3,353 · see citing works
Conformer: Convolution-augmented Transformer for Speech Recognition
Gulati, A., Qin, J., Chiu, C.-C., Parmar, N., Zhang, Y., Yu, J., Han, W., Wang, S., Zhang, Z., Wu, Y., et al. (2020) · Interspeech 2020 · doi:10.21437/interspeech.2020-3015 · Cited by 3,150 · see citing works
Conformer: Convolution-augmented Transformer for Speech Recognition
Gulati, A., Qin, J., Chiu, C.-C., Parmar, N., Zhang, Y., Yu, J., Han, W., Wang, S., Zhang, Z., Wu, Y., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.08100 · Cited by 3,150 · see citing works
Gao, Z., Cecati, C., & Ding, S. X. (2015) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2015.2417501 · Cited by 3,135 · see citing works
Learning internal representations by error propagation
RUMELHART, D. E., HINTON, G. E., & WILLIAMS, R. J. (1988) · Readings in Cognitive Science · doi:10.1016/b978-1-4832-1446-7.50035-2 · Cited by 3,002 · see citing works
Synchrosqueezed wavelet transforms: An empirical mode decomposition-like tool
Daubechies, I. C., Lu, J., & Wu, H. (2010) · Applied and Computational Harmonic Analysis · doi:10.1016/j.acha.2010.08.002 · Cited by 2,496 · see citing works
Guo, M.-H., Cai, J.-X., Liu, Z.-N., Mu, T.-J., Martin, R. R., & Hu, S.-M. (2021) · Computational Visual Media · doi:10.1007/s41095-021-0229-5 · Cited by 2,100 · see citing works
Gradient Flow in Recurrent Nets: The Difficulty of Learning LongTerm Dependencies
(2009) · Institute of Electrical and Electronics Engineers eBooks · doi:10.1109/9780470544037.ch14 · Cited by 1,312 · see citing works
Efficient Transformers: A Survey
Tay, Y., Dehghani, M., Bahri, D., & Metzler, D. (2020) · arXiv · doi:10.48550/arxiv.2009.06732 · Cited by 1,267 · see citing works
A novel deep autoencoder feature learning method for rotating machinery fault diagnosis
Shao, H., Jiang, H., Zhao, H., & Wang, F. (2017) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2017.03.034 · Cited by 686 · see citing works
Fault diagnosis of rolling bearings with recurrent neural network-based autoencoders
Liu, H., Zhou, J., Zheng, Y., Jiang, W., & Zhang, Y. (2018) · ISA Transactions · doi:10.1016/j.isatra.2018.04.005 · Cited by 609 · see citing works
Jia, F., Lei, Y., Lu, N., & Xing, S. (2018) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2018.03.025 · Cited by 587 · see citing works
Zhang, X., Liang, Y., Zhou, J., & Zang, Y. (2015) · Measurement · doi:10.1016/j.measurement.2015.03.017 · Cited by 586 · see citing works
A roller bearing fault diagnosis method based on EMD energy entropy and ANN
Yang, Y., YuDejie, & Cheng, J. (2005) · Journal of Sound and Vibration · doi:10.1016/j.jsv.2005.11.002 · Cited by 566 · see citing works
Visual Transformers: Token-based Image Representation and Processing for Computer VisionPreprint
Wu, B., Xu, C., Dai, X., Wan, A., Zhang, P., Yan, Z., Tomizuka, M., Gonzalez, J., Keutzer, K., & Vajda, P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.03677 · Cited by 529 · see citing works
Rolling bearing fault diagnosis using an optimization deep belief network
Shao, H., Jiang, H., Zhang, X., & Niu, M. (2015) · Measurement Science and Technology · doi:10.1088/0957-0233/26/11/115002 · Cited by 517 · see citing works
Yan, X., & Jia, M. (2018) · Neurocomputing · doi:10.1016/j.neucom.2018.05.002 · Cited by 506 · see citing works
Self-citationLi, X., We, Z., & Ding, Q. (2019) · Signal Processing · doi:10.1016/j.sigpro.2019.03.019 · Cited by 383 · see citing works
2016 35th Chinese Control Conference (CCC)
(2016) · doi:10.1109/ccc39221.2016 · Cited by 374 · see citing works
Sequential Fault Diagnosis Based on LSTM Neural Network
Zhao, H., Sun, S., & Jin, B. (2018) · IEEE Access · doi:10.1109/access.2018.2794765 · Cited by 346 · see citing works
A Field Guide to Dynamical Recurrent Networks
Kolen, J. F., & Kremer, S. C. (2009) · Institute of Electrical and Electronics Engineers eBooks · doi:10.1109/9780470544037 · Cited by 338 · see citing works
Chen, R., Huang, X., Yang, L., Xu, X., Zhang, X., & Zhang, Y. (2019) · Computers in Industry · doi:10.1016/j.compind.2018.11.003 · Cited by 250 · see citing works
Xu, Z., Li, C., & Yang, Y. (2020) · Applied Soft Computing · doi:10.1016/j.asoc.2020.106515 · Cited by 189 · see citing works
Model-based fault diagnosis in technical processes
Frank, P. M., Ding, S. X., & Marcu, T. (2000) · Transactions of the Institute of Measurement and Control · doi:10.1177/014233120002200104 · Cited by 183 · see citing works
2016 Chinese Control and Decision Conference (CCDC)
(2016) · doi:10.1109/ccdc37849.2016 · Cited by 181 · see citing works
Li, C., Sánchez, R., Zurita, G., Cerrada, M., & Cabrera, D. R. (2015) · ISA Transactions · doi:10.1016/j.isatra.2015.10.014 · Cited by 141 · see citing works
Bearing fault diagnosis with auto-encoder extreme learning machine: A comparative study
Mao, W., He, J., Li, Y., & Yan, Y. (2016) · Proceedings of the Institution of Mechanical Engineers Part C Journal of Mechanical Engineering Science · doi:10.1177/0954406216675896 · Cited by 119 · see citing works
Long, Z., Zhang, X., Zhang, L., et al. (2020) · Measurement · doi:10.1016/j.measurement.2020.108718 · Cited by 116 · see citing works
Remaining Useful Life Estimation Under Multiple Operating Conditions via Deep Subdomain Adaptation
Ding, Y., Jia, M., & Cao, Y. (2021) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2021.3076567 · Cited by 92 · see citing works
Self-citationImbalanced Fault Diagnosis of Rolling Bearing Using Enhanced Generative Adversarial Networks
Zhang, H., Wang, R., Pan, R., & Pan, H. (2020) · IEEE Access · doi:10.1109/access.2020.3030058 · Cited by 72 · see citing works
Zhao, X., Jia, M., Bin, J., Wang, T., & Liu, Z. (2020) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2020.3041087 · Cited by 57 · see citing works
Self-citationMechanical state prediction based on LSTM neural netwok
Chen, Z., Liu, Y., & Liu, S. (2017) · Chinese Control Conference (CCC) · doi:10.23919/chicc.2017.8027963 · Cited by 55 · see citing works
2009 IEEE International Conference on Systems, Man, and Cybernetics
(2009) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2008.2012324 · Cited by 50 · see citing works
(1988) · Elsevier eBooks · doi:10.1016/c2013-0-08295-3 · Cited by 49 · see citing works
Dong, S., Xu, X., & Chen, R. (2015) · Journal of the Brazilian Society of Mechanical Sciences and Engineering · doi:10.1007/s40430-015-0455-9 · Cited by 48 · see citing works
Gear Fault Diagnosis Based on Kurtosis Criterion VMD and SOM Neural Network
Xiao, D., Ding, J., Li, X., et al. (2019) · Applied Sciences · doi:10.3390/app9245424 · Cited by 48 · see citing works
Seryasat, O. R., Shoorehdeli, M. A., Honarvar, F., et al. (2010) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/icsmc.2010.5642389 · Cited by 48 · see citing works
Yi, C., Lv, Y., Xiao, H., et al. (2018) · Measurement Science and Technology · doi:10.1088/1361-6501/aaa50a · Cited by 44 · see citing works
Liu, W., Chen, W., & Zhang, Z. (2020) · IEEE Access · doi:10.1109/access.2020.2965744 · Cited by 41 · see citing works
Fault Diagnosis of Bearing Based on KPCA and KNN Method
Wang, Q., Liu, Y. B., He, X., et al. (2014) · Advanced materials research · doi:10.4028/www.scientific.net/amr.986-987.1491 · Cited by 34 · see citing works
Salakhutdinov, R. (2014) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2623330.2630809 · Cited by 26 · see citing works
Fault diagnosis of ball bearings using Synchrosqueezed wavelet transforms and SVM
Wen, J., Gao, H., Li, S., et al. (2015) · Prognostics and System Health Management Conference · doi:10.1109/phm.2015.7380084 · Cited by 15 · see citing works
Fault diagnosis of rolling bearing based on permutation entropy and Extreme Learning Machine
Li, Y., Wang, X., & Wu, J. (2016) · Chinese Control and Decision Conference (CCDC) · doi:10.1109/ccdc.2016.7531490 · Cited by 14 · see citing works
Flagged first, then by citations
Research works printed but not linked: 7
- [26][26] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, I. Polosukhin, Attention is all you need, in: I. Guyon, U. von Luxburg, S. Bengio, H. M. Wallach, R. Fergus, S. V. N. Vishwanathan, R. Garnett (Eds.), Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, 2017, pp. 5998-6008. 4. [URL https://proceedings.neurips.cc/paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html](https://proceedings.neurips.cc/paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html) -
- [27][27] N. Shazeer, A. Mirhoseini, K. Maziarz, A. Davis, Q. V. Le, G. E. Hinton, J. Dean, Outrageously large neural networks: The sparsely-gated mixture-of-experts layer, in: 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings, OpenReview.net, 2017. 6. [URL https://openreview.net/forum?id=B1ckMDqlg](https://openreview.net/forum?id=B1ckMDqlg) -
- [35][35] K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in: 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016, IEEE Computer Society, 2016, pp. 770-778. doi:10.1109/CVPR. 2016.90 . - · DOI
- [38][38] J. Chung, C ¸ . G¨ ulc ¸ehre, K. Cho, Y. Bengio, Gated feedback recurrent neural networks, in: F. R. Bach, D. M. Blei (Eds.), Proceedings of the 32nd International Conference on Machine Learning, ICML 2015, Lille, France, 6-11 July 2015, Vol. 37 of JMLR Workshop and Conference Proceedings, JMLR.org, 2015, pp. 2067-2075. URL http://proceedings.mlr.press/v37/chung15.html -
- [40][40] Z. Yang, Z. Dai, Y. Yang, J. G. Carbonell, R. Salakhutdinov, Q. V. Le, Xlnet: Generalized autoregressive pretraining for language understanding, in: H. M. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alch´ e-Buc, E. B. Fox, R. Garnett (Eds.), Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, 2019, pp. 5754-5764. URL [https://proceedings.neurips.cc/paper/2019/hash/dc6a7e655d7e5840e66733e9ee67cc69-Abstract.html](https://proceedings.neurips.cc/paper/2019/hash/dc6a7e655d7e5840e66733e9ee67cc69-Abstract.html) -
- [41][41] T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. HerbertVoss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. M. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Berner, S. McCandlish, A. Radford, I. Sutskever, D. Amodei, Language models are few-shot learners, in: H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, H. Lin (Eds.), Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020. 2. [URL https://proceedings.neurips.cc/paper/2020/hash/1457c0d6bfcb4967418bfb8ac142f64a-Abstract.html](https://proceedings.neurips.cc/paper/2020/hash/1457c0d6bfcb4967418bfb8ac142f64a-Abstract.html) -
- [49][49] J. Zhang, T. He, S. Sra, A. Jadbabaie, Why gradient clipping accelerates training: A theoretical justification for adaptivity, in: 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020, OpenReview.net, 2020. URL https://openreview.net/forum?id=BJgnXpVYwS -
Cited by
Sorted by citations
Ding, Y., & Jia, M. (2022) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2022.3181933 · Cited by 120 · see citing works
McIntosh, T. R., Susnjak, T., Liu, T., Watters, P., Xu, D., Liu, D., & Halgamuge, M. N. (2025) · Technologies · doi:10.3390/technologies13020051 · Cited by 102 · see citing works
“They employ advanced attention mechanisms to achieve enhanced contextual processing, allowing for more subtle understanding and interaction [142], [143], [144].”
Background
- Authors
- Ding, Y., Jia, M., Miao, Q., & Cao, Y.
- Affiliations
- Southeast University (CN)
- Corresponding author
- Jia, Minping
- Venue
- Mechanical Systems and Signal Processing
- Publisher
- Elsevier BV
- Published
- 2022-04
- Volume
- 168
- Pages
- 108616
- ISSN
- 0888-3270 (ISSN-L)
- OpenAlex
- W4200049650
- Cited by
- 559
- References
- 56
- Citations by year
- 2026: 124 · 2025: 186 · 2024: 140 · 2023: 80 · 2022: 29
- FWCI
- 27.72
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- Mechanical Systems and Signal Processing (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Natural Science Foundation of China (52075095)
- Topics
- Physical Sciences › Engineering › Control and Systems Engineering › Machine Fault Diagnosis Techniques
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Power Transformer Diagnostics and Insulation
- Physical Sciences › Engineering › Control and Systems Engineering › Fault Detection and Control Systems
- Keywords
- self-attention, rolling bearing fault diagnosis, time-frequency representation, vibration signals
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
