An Incorrect Data Detection Method for Big Data Cleaning of Machinery Condition Monitoring
The DOI matches its Crossref record; 40 of the 40 references it cites were found in a registry or an index.
Abstract
The presence of incorrect data leads to the decrease of condition-monitoring big data quality. As a result, unreliable or misleading results are probably obtained by analyzing these poor-quality data. In this paper, to improve the data quality, an incorrect data detection method based on an improved local outlier factor (LOF) is proposed for data cleaning. First, a sliding window technique is used to divide data into different segments. These segments are considered as different objects and their attributes consist of time-domain statistical features extracted from each segment, such as mean, maximum and peak-to-peak value. Second, a kernel-based LOF (KLOF) is calculated using these attributes to evaluate the degree of each segment being incorrect data. Third, according to these KLOF values and a threshold value, incorrect data are detected. Finally, a simulation of vibration data generated by a defective rolling element bearing and three real cases concerning a fixed-axle gearbox, a wind turbine, and a planetary gearbox are used to verify the effectiveness of the proposed method, respectively. The results demonstrate that the proposed method is able to detect both missing segments and abnormal segments, which are two typical incorrect data, effectively, and thus is helpful for big data cleaning of machinery condition monitoring.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Chandola, V., Banerjee, A., & Kumar, V. (2009) · ACM Computing Surveys · doi:10.1145/1541880.1541882 · Cited by 11,687 · cited 1×
“Specifically, the characteristics of high-quality data are considered as contextual attributes [15].”
BackgroundLOF: identifying density-based local outliers
Breunig, M. M., Kriegel, H.-P., Ng, R. T., & Sander, J. (2000) · ACM SIGMOD Record · doi:10.1145/335191.335388 · Cited by 5,207
LOF: identifying density-based local outliers
Breunig, M. M., Kriegel, H.-P., Ng, R. T., & Sander, J. (2000) · Proceedings of the 2000 ACM SIGMOD international conference on Management of data · doi:10.1145/342009.335388 · Cited by 4,149 · cited 2×
“Being a classical-density based method, a local outlier factor (LOF) method was first proposed in [29].”
BackgroundMethodA Survey of Outlier Detection Methodologies
Hodge, V., & Austin, J. (2004) · Artificial Intelligence Review · doi:10.1007/s10462-004-4304-y · Cited by 3,596
Hawkins, D. M. (1980) · doi:10.1007/978-94-015-3994-4 · Cited by 2,982 · cited 1×
“Outlier detection [19], which is also heavily used and known as anomaly detection, is widely studied in many fields such as credit card fraud detection, cyber-security, traffic, and medical field [20].”
BackgroundChen, M., Mao, S., & Liu, Y. (2014) · Mobile Networks and Applications · doi:10.1007/s11036-013-0489-0 · Cited by 2,913
Deep learning applications and challenges in big data analytics
Najafabadi, M. M., Villanustre, F., Khoshgoftaar, T. M., Seliya, N., Wald, R., & Muharemagic, E. (2015) · Journal of Big Data · doi:10.1186/s40537-014-0007-7 · Cited by 2,594
A new convolutional neural network-based data-driven fault diagnosis method
Wen, L., Li, X., Gao, L., & Zhang, Y. (2018) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2017.2774777 · Cited by 2,222 · cited 1×
“ferent intelligent diagnosis methods have been proposed based on deep belief networks [5], recurrent neural networks [6], and convolutional neural networks [7] to extract fault features from condition-monitoring data, more effectively.”
BackgroundService Innovation and Smart Analytics for Industry 4.0 and Big Data Environment
Lee, J., Kao, H.-A., & Yang, S. (2014) · Procedia CIRP · doi:10.1016/j.procir.2014.02.001 · Cited by 1,965
A Hybrid Prognostics Approach for Estimating Remaining Useful Life of Rolling Element Bearings
Wang, B., Lei, Y., Li, N., & Li, N. (2020) · IEEE Transactions on Reliability · doi:10.1109/tr.2018.2882682 · Cited by 1,917
Self-citationLei, Y., Jia, F., Lin, J., Xing, S., & Ding, S. X. (2016) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2016.2519325 · Cited by 1,191 · cited 1×
“W ITH the collection of massive data from monitoring systems, machinery condition monitoring has entered the era of big data [1].”
BackgroundSelf-citationA survey of machine learning for big data processing
Qiu, J., Wu, Q., Ding, G., Xu, Y., & Feng, S. (2016) · EURASIP Journal on Advances in Signal Processing · doi:10.1186/s13634-016-0355-x · Cited by 894
A Model-Based Method for Remaining Useful Life Prediction of Machinery
Lei, Y., Li, N., Gontarz, S., Lin, J., Radkowski, S., & Dybala, J. (2016) · IEEE Transactions on Reliability · doi:10.1109/tr.2016.2570568 · Cited by 617
Self-citationA Survey on Wind Turbine Condition Monitoring and Fault Diagnosis—Part I: Components and Subsystems
Qiao, W., & Lu, D. (2015) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2015.2422112 · Cited by 533
OPTIMISATION OF BEARING DIAGNOSTIC TECHNIQUES USING SIMULATED AND ACTUAL BEARING FAULT SIGNALS
HO, D., & RANDALL, R. B. (2000) · Mechanical Systems and Signal Processing · doi:10.1006/mssp.2000.1304 · Cited by 531
A New Local Distance-Based Outlier Detection Approach for Scattered Real-World Data
Zhang, K., Hutter, M., & Jin, H. (2009) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-01307-2_84 · Cited by 392
Data Quality in Internet of Things: A state-of-the-art survey
Karkouch, A., Mousannif, H., Al Moatassime, H., & Noel, T. (2016) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2016.08.002 · Cited by 378 · cited 1×
“segments usually occur by replacing the actual data value with 0, NaN, NULL values, or environment noise [17], [18], which damages the completeness of data.”
BackgroundVariance-constrained filtering for uncertain stochastic systems with missing measurements
Wang, Z., Ho, D. W. C., & Liu, X. (2003) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2003.814272 · Cited by 349
Railway Track Circuit Fault Diagnosis Using Recurrent Neural Networks
de Bruin, T., Verbert, K., & Babuska, R. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2551940 · Cited by 322
Zhu, J., Ge, Z., Song, Z., & Gao, F. (2018) · Annual Reviews in Control · doi:10.1016/j.arcontrol.2018.09.003 · Cited by 308
Kim, W., Choi, B.-J., Hong, E.-K., Kim, S.-K., & Lee, D. (2003) · Data Mining and Knowledge Discovery · doi:10.1023/a:1021564703268 · Cited by 301
Shao, H., Jiang, H., Wang, F., & Wang, Y. (2017) · ISA Transactions · doi:10.1016/j.isatra.2017.03.017 · Cited by 288
Outlier Detection for Compositional Data Using Robust Methods
Filzmoser, P., & Hron, K. (2008) · Mathematical Geosciences · doi:10.1007/s11004-007-9141-5 · Cited by 225
Intelligent Fault Diagnosis and Remaining Useful Life Prediction of Rotating Machinery
(2017) · Elsevier eBooks · doi:10.1016/c2016-0-00367-4 · Cited by 224
Self-citationA Comparative Study for Outlier Detection Techniques in Data Mining
Bakar, Z., Mohemad, R., Ahmad, A., & Deris, M. (2006) · 2006 IEEE Conference on Cybernetics and Intelligent Systems · doi:10.1109/iccis.2006.252287 · Cited by 146
Dekhtiar, J., Durupt, A., Bricogne, M., Eynard, B., Rowson, H., & Kiritsis, D. (2018) · Computers in Industry · doi:10.1016/j.compind.2018.04.005 · Cited by 135
An Outlier Detection Method Based on Clustering
Pamula, R., Deka, J. K., & Nandi, S. (2011) · International Conference on Emerging Applications of Information Technology · doi:10.1109/eait.2011.25 · Cited by 120
Big Data Pre-processing: A Quality Framework
Taleb, I., Dssouli, R., & Serhani, M. A. (2015) · 2015 IEEE International Congress on Big Data · doi:10.1109/bigdatacongress.2015.35 · Cited by 119
Outlier Detection in Arbitrarily Oriented Subspaces
Kriegel, H.-P., Kroger, P., Schubert, E., & Zimek, A. (2012) · 2012 IEEE 12th International Conference on Data Mining · doi:10.1109/icdm.2012.21 · Cited by 115
Distance-based outlier detection: consolidation and renewed bearing
Orair, G. H., Teixeira, C. H. C., Meira, W., Wang, Y., & Parthasarathy, S. (2010) · Proceedings of the VLDB Endowment · doi:10.14778/1920841.1921021 · Cited by 109
Abrahamsson, P., Moser, R., Witold Pedrycz, C., Sillitti, A., & Succi, G. (2007) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1109/esem.2007.16 · Cited by 105
Robust support vector data description for outlier detection with noise or uncertain data
Chen, G., Zhang, X., Wang, Z. J., & Li, F. (2015) · Knowledge-Based Systems · doi:10.1016/j.knosys.2015.09.025 · Cited by 97
Anomaly detection in onboard-recorded flight data using cluster analysis
Li, L., Gariel, M., Hansman, R. J., & Palacios, R. (2011) · IEEE/AIAA Digital Avionics Systems Conference (DASC) · doi:10.1109/dasc.2011.6096068 · Cited by 84
An Approach to Outlier Detection of Software Measurement Data using the K-means Clustering Method
Yoon, K.-A., Kwon, O., & Bae, D. (2007) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1109/esem.2007.49 · Cited by 77
Enhancements on local outlier detection
Chiu, A. L. M., & Fu, A. W.-C. (2003) · Seventh International Database Engineering and Applications Symposium, 2003. Proceedings. · doi:10.1109/ideas.2003.1214939 · Cited by 76
Validation of vibration measurements for heavy duty machinery diagnostics
Jabłoński, A., & Barszcz, T. (2013) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2013.01.015 · Cited by 52
A Process Pattern Model for Tackling and Improving Big Data Quality
Wahyudi, A., Kuk, G., & Janssen, M. (2018) · Information Systems Frontiers · doi:10.1007/s10796-017-9822-7 · Cited by 48
A Survey of Machine Learning for Big Data Processing
Almutiri, R., Alhabeeb, S., Alhumud, S., & Khan, R. U. (2022) · Journal on big data · doi:10.32604/jbd.2022.028363 · Cited by 44
Student t‐tests for potentially abnormal data
Shuster, J. J. (2009) · Statistics in Medicine · doi:10.1002/sim.3581 · Cited by 33
Local Outlier Detection Based on Kernel Regression
Gao, J. T., Hu, W., Li, W., Zhang, Z. M., & Wu, O. (2010) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2010.148 · Cited by 15
Cited by
Applications of machine learning to machine fault diagnosis: A review and roadmap
Lei, Y., Yang, B., Jiang, X., Jia, F., Li, N., & Nandi, A. K. (2020) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2019.106587 · Cited by 2,997
Self-citationBig data analytics for smart factories of the future
Gao, R. X., Wang, L., Helu, M., & Teti, R. (2020) · CIRP Annals · doi:10.1016/j.cirp.2020.05.002 · Cited by 170
Artificial Intelligence to Power the Future of Materials Science and Engineering
Sha, W., Guo, Y., Yuan, Q. M., Tang, S., Zhang, X., Lu, S., Guo, X., Cao, Y., & Cheng, S. (2020) · Advanced Intelligent Systems · doi:10.1002/aisy.201900143 · Cited by 171
A Comprehensive Survey on Imputation of Missing Data in Internet of Things
Adhikari, D., Jiang, W., Zhan, J., He, Z., Rawat, D. B., Aickelin, U., & Khorshidi, H. A. (2022) · ACM Computing Surveys · doi:10.1145/3533381 · Cited by 196
Wang, Z., Zhao, W., Du, W., Li, N., & Wang, J. (2021) · Process Safety and Environmental Protection · doi:10.1016/j.psep.2021.03.016 · Cited by 147
Metadata
- Authors
- Xu, X., Lei, Y., & Li, Z.
- Affiliations
- Xi'an Jiaotong University (CN)
- Venue
- IEEE Transactions on Industrial Electronics
- Published
- 2020-03
- Volume
- 67
- Issue
- 3
- Pages
- 2326-2336
- ISSN
- 0278-0046 (ISSN-L)
- 1557-9948
- OpenAlex
- W2921480582
- Cited by
- 134
- References
- 40
- Citations by year
- 2026: 8 · 2025: 17 · 2024: 16 · 2023: 26 · 2022: 28 · 2021: 21 · 2020: 16
- FWCI
- 9.28
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Transactions on Industrial Electronics (publishedVersion)
- Funders
- National Natural Science Foundation of China (61673311)
- NSFC-Zhejiang Joint Fund for the Integration of Industrialization and Informatization (U1709208)
- National Program for Support of Top-Notch Young Professionals
- Topics
- Physical Sciences › Engineering › Control and Systems Engineering › Machine Fault Diagnosis Techniques
- Physical Sciences › Engineering › Control and Systems Engineering › Fault Detection and Control Systems
- Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Reliability and Maintenance Optimization
- Keywords
- machine condition monitoring, local outlier factor, sliding window segmentation, time-domain statistical features, vibration data, gearbox condition monitoring, wind turbine condition monitoring, planetary gearbox, data quality
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.