Deep Learning for Infrared Thermal Image Based Machine Health Monitoring
Open accessThe DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.
Abstract
The condition of a machine can automatically be identified by creating and classifying features that summarize characteristics of measured signals. Currently, experts, in their respective fields, devise these features based on their knowledge. Hence, the performance and usefulness depends on the expert's knowledge of the underlying physics or statistics. Furthermore, if new and additional conditions should be detectable, experts have to implement new feature extraction methods. To mitigate the drawbacks of feature engineering, a method from the subfield of feature learning, i.e., deep learning (DL), more specifically convolutional neural networks (NNs), is researched in this paper. The objective of this paper is to investigate if and how DL can be applied to infrared thermal (IRT) video to automatically determine the condition of the machine. By applying this method on IRT data in two use cases, i.e., machine-fault detection and oil-level prediction, we show that the proposed system is able to detect many conditions in rotating machinery very accurately (i.e., 95 and 91.67% accuracy for the respective use cases), without requiring any detailed knowledge about the underlying physics, and thus having the potential to significantly simplify condition monitoring using complex sensor data. Furthermore, we show that by using the trained NNs, important regions in the IRT images can be identified related to specific conditions, which can potentially lead to new physical insights.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 3×
“We opted to use a pre-trained VGG (neural network created by the Visual Geometry Group at University of Oxford) [22] network which achieves state-of-the-art results on the imagenet data set.”
MethodInfluentialImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 1×
“[16] proposed another type of activation function called the rectified linear unit, which does not suffer from this problem.”
BackgroundLeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 8×
“This means that the CNNs were trained on recorded data from REB two, three, four and five and subsequently the CNNs were tested on data from REB one.”
BackgroundInfluentialVisualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839 · cited 3×
“An intuitive example is given in [23], where a CNN is trained to detect objects in natural”
BackgroundInfluentialHow transferable are features in deep neural networks?
Yosinski, J., Clune, J., Bengio, Y., & Lipson, H. (2014) · arXiv · doi:10.48550/arxiv.1411.1792 · Cited by 6,061 · cited 3×
“The idea behind this method is that neighboring layers coadapt during training, which can only be done when training all layers [18].”
BackgroundMethodCNN Features Off-the-Shelf: An Astounding Baseline for Recognition
Razavian, A. S., Azizpour, H., Sullivan, J., & Carlsson, S. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2014.131 · Cited by 4,451 · cited 1×
“It has been shown that such an NN will have learned general features that can be used for other tasks [18], [19].”
BackgroundSmith, W. A., & Randall, R. B. (2015) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2015.04.021 · Cited by 2,719 · cited 1×
“A lot of work and research [1], [2] has already been done to”
BackgroundConvolutional Neural Network Based Fault Detection for Rotating Machinery
Janssens, O., Slavkovikj, V., Vervisch, B., Stockman, K., Loccufier, M., Verstockt, S., Van de Walle, R., & Van Hoecke, S. (2016) · Journal of Sound and Vibration · doi:10.1016/j.jsv.2016.05.027 · Cited by 1,255 · cited 1×
“neural network (CNN), on vibration signals to detect machine faults [11].”
BackgroundSelf-citationHu, F., Xia, G.-S., Hu, J., & Zhang, L. (2015) · Remote Sensing · doi:10.3390/rs71114680 · Cited by 1,195 · cited 1×
“It has also been shown that NNs, which are trained on images of everyday scenery, can be repurposed and modified to be applicable in tasks which require domain specific images, such as medical images [20] or aerial images [21].”
BackgroundNeural-network-based motor rolling bearing fault diagnosis
Li, B., Chow, M.-Y., Tipsuwan, Y., & Hung, J. C. (2000) · IEEE Transactions on Industrial Electronics · doi:10.1109/41.873214 · Cited by 797 · cited 1×
“However, most often they are used in combination with features engineered by an expert [12], [13].”
BackgroundA summary of fault modelling and predictive health monitoring of rolling element bearings
El-Thalji, I., & Jantunen, E. (2015) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2015.02.008 · Cited by 447 · cited 1×
“A lot of work and research [1], [2] has already been done to”
BackgroundGearbox Fault Identification and Classification with Convolutional Neural Networks
Chen, Z., Li, C., & Sanchez, R.-V. (2015) · Shock and Vibration · doi:10.1155/2015/390134 · Cited by 391 · cited 1×
“However, most often they are used in combination with features engineered by an expert [12], [13].”
BackgroundStatistical analysis of sound and vibration signals for monitoring rolling element bearing condition
Heng, R. B. W., & Nor, M. J. M. (1998) · Applied Acoustics · doi:10.1016/s0003-682x(97)00018-2 · Cited by 333 · cited 1×
“But also data-driven FE is already successfully applied [3], [4] on vibration data by, for example, extracting well-known statis-”
MethodVibration Spectrum Imaging: A Novel Bearing Fault Classification Approach
Amar, M., Gondal, I., & Wilson, C. (2015) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2014.2327555 · Cited by 222 · cited 1×
“An example of such a process using NNs is given in [14], wherein vibration spectrum images”
MethodModel-based fault diagnosis in electric drives using machine learning
Murphey, Y. L., Masrur, M. A., Chen, Z., & Zhang, B. (2006) · IEEE/ASME Transactions on Mechatronics · doi:10.1109/tmech.2006.875568 · Cited by 165 · cited 1×
“But also data-driven FE is already successfully applied [3], [4] on vibration data by, for example, extracting well-known statis-”
MethodThermal image based fault diagnosis for rotating machinery
Janssens, O., Schulz, R., Slavkovikj, V., Stockman, K., Loccufier, M., Rik Van De Walle, B., & Hoecke, S. V. (2015) · Infrared Physics & Technology · doi:10.1016/j.infrared.2015.09.004 · Cited by 149 · cited 6×
“detect using vibration signals alone, for example lubrication related problems [5], other types of sensors, like thermal imaging, can be considered.”
BackgroundMethodResultSelf-citationDeep Model Based Transfer and Multi-Task Learning for Biological Image Analysis
Zhang, W., Li, R., Zeng, T., Sun, Q., Kumar, S., Ye, J., & Ji, S. (2020) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2016.2573280 · Cited by 143
Tran, V. T., Yang, B., Gu, F., & Ball, A. D. (2013) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2013.02.001 · Cited by 127 · cited 1×
“As complete physics-based modelling of a machine is difficult and requires a lot of knowledge, effort and time [6], mainly data-driven feature engineering has been applied for IRT-based condition monitoring [7], [8], [9], [5].”
Deep Model Based Transfer and Multi-Task Learning for Biological Image Analysis
Zhang, W., Li, R., Zeng, T., Sun, Q., Kumar, S., Ye, J., & Ji, S. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2783304 · Cited by 112 · cited 1×
“that NNs, which are trained on images of everyday scenery, can be repurposed and modified to be applicable in tasks that require domain specific images, such as medical images [20] or aerial”
BackgroundIntelligent condition based monitoring of rotating machines using sparse auto-encoders
Verma, N. K., Gupta, V. K., Sharma, M., & Sevakula, R. K. (2013) · 2013 IEEE Conference on Prognostics and Health Management (PHM) · doi:10.1109/icphm.2013.6621447 · Cited by 100 · cited 1×
“tion using vibration measurements, unsupervised methods using autoencoders have been used recently [15].”
MethodConfirmation of Thermal Images and Vibration Signals for Intelligent Machine Fault Diagnostics
Widodo, A., Satrijo, D., Prahasto, T., Lim, G.-M., & Choi, B.-K. (2012) · International Journal of Rotating Machinery · doi:10.1155/2012/847203 · Cited by 25 · cited 1×
“As complete physics-based modeling of a machine is difficult and requires a lot of knowledge, effort and time [6], mainly data-driven FE has been applied for IRT-based CM [5], [7]–[9].”
MethodThe Fault Diagnosis and Monitoring of Rotating Machines by Thermography
Lim, G.-M., Ali, Y., & Yang, B. (2012) · Springer eBooks · doi:10.1007/978-0-85729-493-7_43 · Cited by 12 · cited 1×
“As complete physics-based modeling of a machine is difficult and requires a lot of knowledge, effort and time [6], mainly data-driven FE has been applied for IRT-based CM [5], [7]–[9].”
MethodJanssens, O., Rennuy, M., Devos, S., Loccufier, M., Rik Van De Walle, B., & Hoecke, S. V. (2016) · The proceedings of the IEEE Conference on Control Applications/The proceedings of the ... IEEE Conference on Control Applications · doi:10.1109/cca.2016.7587991 · Cited by 10
Self-citationThermography-Assisted Bearing Condition Monitoring
Moussa, W. (2014) · uO Research (University of Ottawa) · doi:10.20381/ruor-3853 · Cited by 4 · cited 1×
“As complete physics-based modeling of a machine is difficult and requires a lot of knowledge, effort and time [6], mainly data-driven FE has been applied for IRT-based CM [5], [7]–[9].”
Method
Cited by
Applications of machine learning to machine fault diagnosis: A review and roadmap
Lei, Y., Yang, B., Jiang, X., Jia, F., Li, N., & Nandi, A. K. (2020) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2019.106587 · Cited by 2,997
A review on the application of deep learning in system health management
Khan, S., & Yairi, T. (2018) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2017.11.024 · Cited by 1,133
Machine Learning in Predictive Maintenance towards Sustainable Smart Manufacturing in Industry 4.0
Çınar, Z. M., Abdussalam Nuhu, A., Zeeshan, Q., Korhan, O., Asmael, M., & Safaei, B. (2020) · Sustainability · doi:10.3390/su12198211 · Cited by 834
“Janssens, O. et al. [75] reported a study on DL for Infrared Thermal Image Based Machine Health Monitoring, where in the study they considered Convolutional Neural Network (CNN), a Feature Learning (FL) tool, in detecting the various conditions of the machine.”
BackgroundMethodA transfer convolutional neural network for fault diagnosis based on ResNet-50
Wen, L., Li, X., & Gao, L. (2020) · Neural Computing and Applications · doi:10.1007/s00521-019-04097-w · Cited by 764
“Janssens et al. [38] investigated the transfer learning to detect various machine conditions using thermal infrared imaging data and achieved 95% and 91.67% accuracy for the respective use cases.”
MethodInfluentialA comprehensive review on convolutional neural network in machine fault diagnosis
Jiao, J., Zhao, M., Lin, J., & Liang, K. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.07.088 · Cited by 575
Metadata
- Authors
- Janssens, O., Van de Walle, R., Loccufier, M., & Van Hoecke, S.
- Affiliations
- Ghent University (BE)
- Venue
- IEEE/ASME Transactions on Mechatronics
- Published
- 2018-02
- Volume
- 23
- Issue
- 1
- Pages
- 151-159
- ISSN
- 1083-4435 (ISSN-L)
- 1941-014X
- OpenAlex
- W2724573302
- Cited by
- 207
- References
- 25
- Citations by year
- 2026: 7 · 2025: 19 · 2024: 19 · 2023: 21 · 2022: 27 · 2021: 44 · 2020: 38 · 2019: 21 · 2018: 10
- FWCI
- 12.03
- Type
- journal article
- Language
- en
- Open access
- green
- biblio.ugent.be
- Locations
- IEEE/ASME Transactions on Mechatronics (publishedVersion)
- Ghent University Academic Bibliography (Ghent University) (submittedVersion, other-oa)
- Funders
- Vlaamse innovatiesamenwerkingsverband Operations and Maintenance (VIS O&M) Excellence
- Topics
- Physical Sciences › Engineering › Control and Systems Engineering › Machine Fault Diagnosis Techniques
- Physical Sciences › Engineering › Mechanics of Materials › Thermography and Photoacoustic Techniques
- Physical Sciences › Engineering › Control and Systems Engineering › Fault Detection and Control Systems
- Keywords
- infrared thermography, machine condition monitoring, condition monitoring, equipment failure detection, rotating machinery, convolutional neural networks, representation learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.