A global manufacturing big data ecosystem for fault detection in predictive maintenance
Abstract
Artificial intelligence, big data, machine learning, cloud computing, and Internet of Things (IoT) are terms which have driven the fourth industrial revolution. The digital revolution has transformed the manufacturing industry into smart manufacturing through the development of intelligent systems. In this paper, a big data ecosystem is presented for the implementation of fault detection and diagnosis in predictive maintenance with real industrial big data gathered directly from large-scale global manufacturing plants, aiming to provide a complete architecture which could be used in industrial IoT-based smart manufacturing in an industrial 4.0 system. The proposed architecture overcomes multiple challenges including big data ingestion, integration, transformation, storage, analytics, and visualization in a real-time environment using various technologies such as the data lake, NoSQL database, Apache Spark, Apache Drill, Apache Hive, OPC Collector, and other techniques. Transformation protocols, authentication, and data encryption methods are also utilized to address data and network security issues. A MapReduce-based distributed PCA model is designed for fault detection and diagnosis. In a large-scale manufacturing system, not all kinds of failure data are accessible, and the absence of labels precludes all the supervised methods in the predictive phase. Furthermore, the proposed framework takes advantage of some of the characteristics of PCA such as its ease of implementation on Spark, its simple algorithmic structure, and its real-time processing ability. All these elements are essential for smart manufacturing in the evolution to Industry 4.0. The proposed detection system has been implemented into the real-time industrial production system in a cooperated company, running for several years, and the results successfully provide an alarm warning several days before the fault happens. A test case involving several outages in 2014 is reported and analyzed in detail during the experiment section.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
We identified five online cleaning techniques [5,8,29,40,48,49].
“We identified five online cleaning techniques [5,8,29,40,48,49].”
From A Systematic Review of Data Quality in CPS and IoT for Industry 4.0 (2023) · cites this paper as [49]
Nine of these papers (i.e., [2,15,21,29,34,40,[49][50][51]) propose a data analytic solution as part of a CPS or IoT system.
“Nine of these papers (i.e., [2,15,21,29,34,40,[49][50][51]) propose a data analytic solution as part of a CPS or IoT system.”
From A Systematic Review of Data Quality in CPS and IoT for Industry 4.0 (2023) · cites this paper as [49]
Half of the data cleaning techniques [8,14,29,42,43,45,49] in the SLR have not been evaluated.
“Half of the data cleaning techniques [8,14,29,42,43,45,49] in our SLR have not been evaluated.”
From A Systematic Review of Data Quality in CPS and IoT for Industry 4.0 (2023) · cites this paper as [49]
Data warehouses and distributed file systems are ideal mediums for storing data for big data systems [32,48,49] that receive data from multiple data sources.
“Data warehouses and distributed file systems are ideal mediums for storing data for big data systems [32,48,49] that receive data from multiple data sources.”
From A Systematic Review of Data Quality in CPS and IoT for Industry 4.0 (2023) · cites this paper as [49]
Four of these studies [23,31,40,49] provide a data quality management technique (data cleaning and repair) in the architecture layers from L1 to L7.
“Four of these studies [23,31,40,49] provide a data quality management technique (data cleaning and repair) in the architecture layers from L1 to L7.”
From A Systematic Review of Data Quality in CPS and IoT for Industry 4.0 (2023) · cites this paper as [49]
Nine primary studies [4,8,10,16,23,24,31,40,49] cover edge-Cloud orchestration for data processing (not necessarily data quality management).
“Nine primary studies [4,8,10,16,23,24,31,40,49] cover edge-Cloud orchestration for data processing (not necessarily data quality management).”
From A Systematic Review of Data Quality in CPS and IoT for Industry 4.0 (2023) · cites this paper as [49]
They are all online techniques and implement data monitoring and cleaning/repair across the edge-Cloud continuum.
“They are all online techniques and implement data monitoring and cleaning/repair across the edge-Cloud continuum.”
From A Systematic Review of Data Quality in CPS and IoT for Industry 4.0 (2023) · cites this paper as [49]
Cites
Chandola, V., Banerjee, A., & Kumar, V. (2009) · ACM Computing Surveys · doi:10.1145/1541880.1541882 · Cited by 11,687 · see citing works
Internet of Things in industries: A survey
Xu, L. D., He, W., & Li, S. (2014) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2014.2300753 · Cited by 5,172 · see citing works
Industry 4.0: A survey on technologies, applications and open research issues
Lu, Y. (2017) · Journal of Industrial Information Integration · doi:10.1016/j.jii.2017.04.005 · Cited by 3,012 · see citing works
The rise of big data on cloud computing: Review and open research issues
Hashem, I. A. T., Yaqoob, I., Anuar, N. B., Mokhtar, S., Gani, A., & Ullah Khan, S. (2015) · Information Systems · doi:10.1016/j.is.2014.07.006 · Cited by 2,656 · see citing works
Industrial Internet of Things: Challenges, Opportunities, and Directions
Sisinni, E., Saifullah, A., Han, S., Jennehag, U., & Gidlund, M. (2018) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2018.2852491 · Cited by 2,380 · see citing works
Fault Detection and Diagnosis in Industrial Systems
Chiang, L. H., Russell, E. L., & Braatz, R. D. (2001) · Advanced textbooks in control and signal processing · doi:10.1007/978-1-4471-0347-9 · Cited by 1,317 · see citing works
Industry 4.0: Towards future industrial opportunities and challenges
Zhou, K., Liu, T., & Zhou, L. (2015) · International Conference on Fuzzy Systems and Knowledge Discovery (FSKD) · doi:10.1109/fskd.2015.7382284 · Cited by 1,257 · see citing works
Lei, Y., Jia, F., Lin, J., Xing, S., & Ding, S. X. (2016) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2016.2519325 · Cited by 1,191 · see citing works
Shrouf, F., Ordieres-Meré, J. B., & Miragliotta, G. (2014) · doi:10.1109/ieem.2014.7058728 · Cited by 915 · see citing works
Machine Learning for Predictive Maintenance: A Multiple Classifier Approach
Susto, G. A., Schirru, A., Pampuri, S., McLoone, S., & Beghi, A. (2015) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2014.2349359 · Cited by 763 · see citing works
CCIoT-CMfg: Cloud Computing and Internet of Things-Based Cloud Manufacturing Service System
Tao, F., Cheng, Y., Xu, L. D., Li, Z., & Li, B. H. (2014) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2014.2306383 · Cited by 724 · see citing works
Efficient IoT-based sensor BIG Data collection–processing and analysis in smart buildings
Plageras, A. P., Psannis, K. E., Stergiou, C. L., Wang, H., & Gupta, B. B. (2017) · Future Generation Computer Systems · doi:10.1016/j.future.2017.09.082 · Cited by 596 · see citing works
Manogaran, G., Varatharajan, R., Lopez, D., Kumar, P. M., Sundarasekar, R., & Thota, C. (2017) · Future Generation Computer Systems · doi:10.1016/j.future.2017.10.045 · Cited by 512 · see citing works
A Manufacturing Big Data Solution for Active Preventive Maintenance
Wan, J., Tang, S., Li, D., Wang, S., Liu, C., Abbas, H., & Vasilakos, A. V. (2017) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2017.2670505 · Cited by 443 · see citing works
A survey of open source tools for machine learning with big data in the Hadoop ecosystem
Landset, S., Khoshgoftaar, T. M., Richter, A. N., et al. (2015) · Journal Of Big Data · doi:10.1186/s40537-015-0032-1 · Cited by 438 · see citing works
An IoT-Oriented Data Storage Framework in Cloud Computing Platform
Jiang, L., Xu, L. D., Cai, H., Jiang, Z., Bu, F., & Xu, B. (2014) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2014.2306384 · Cited by 352 · see citing works
Wind Turbine Gearbox Failure Identification With Deep Neural Networks
Wang, L., Zhang, Z., Long, H., Xu, J. Y., & Liu, R. (2016) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2016.2607179 · Cited by 348 · see citing works
An Artificial Neural Network Approach for Early Fault Detection of Gearbox Bearings
Bangalore, P., & Tjernberg, L. B. (2015) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2014.2386305 · Cited by 342 · see citing works
IoT and Cloud Computing in Automation of Assembly Modeling Systems
Wang, C., Bi, Z., & Xu, L. D. (2014) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2014.2300346 · Cited by 250 · see citing works
RMER: Reliable and Energy-Efficient Data Collection for Large-Scale Wireless Sensor Networks
Dong, M., Ota, K., & Liu, A. (2016) · IEEE Internet of Things Journal · doi:10.1109/jiot.2016.2517405 · Cited by 224 · see citing works
He, Y., Gu, C., Chen, Z., et al. (2017) · International Journal of Production Research · doi:10.1080/00207543.2017.1346843 · Cited by 151 · see citing works
Xu, Z., Chen, L., Chen, C., et al. (2015) · IEEE Internet of Things Journal · doi:10.1109/jiot.2015.2482363 · Cited by 142 · see citing works
Intelligent Condition Based Monitoring Using Acoustic Signals for Air Compressors
Verma, N. K., Sevakula, R. K., Dixit, S., et al. (2015) · IEEE Transactions on Reliability · doi:10.1109/tr.2015.2459684 · Cited by 125 · see citing works
Expanded cloud plumes hiding Big Data ecosystem
Sharma, S. (2016) · Future Generation Computer Systems · doi:10.1016/j.future.2016.01.003 · Cited by 83 · see citing works
Filev, D., Chinnam, R. B., Tseng, F., & Baruah, P. (2010) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2010.2060732 · Cited by 71 · see citing works
OPC UA supporting the automated engineering of production monitoring and control systems
Schleipen, M. (2008) · IEEE International Conference on Emerging Technologies and Factory Automation (ETFA) · doi:10.1109/etfa.2008.4638464 · Cited by 57 · see citing works
On the robustness of fault detection in reciprocating compressor valves
Pichler, K., Pichler, M., Buchegger, T., et al. (2014) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc.2014.6974341 · Cited by 4 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Tackling Faults in the Industry 4.0 Era—A Survey of Machine-Learning Solutions and Key Aspects
Angelopoulos, A., Michailidis, E. T., Nomikos, N., Trakadas, P., Hatziefremidis, A., Voliotis, S., & Zahariadis, T. (2019) · Sensors · doi:10.3390/s20010109 · Cited by 365 · see citing works
Lv, Z. (2023) · Research · doi:10.34133/research.0071 · Cited by 108 · see citing works
A Systematic Review of Data Quality in CPS and IoT for Industry 4.0
Goknil, A., Nguyen, P., Sen, S., Politaki, D., Niavis, H., Pedersen, K. J., Suyuthi, A., Anand, A., & Ziegenbein, A. (2023) · ACM Computing Surveys · doi:10.1145/3593043 · Cited by 68 · see citing works
Big Data Life Cycle in Shop-Floor–Trends and Challenges
Pulikottil, T., Estrada-Jimenez, L. A., Abadía, J. J. P., Carrera-Rivera, A., Torayev, A., Rehman, H. U., Mo, F., Nikghadam-Hojjati, S., & Barata, J. (2023) · IEEE Access · doi:10.1109/access.2023.3253286 · Cited by 14 · see citing works
Development of Digital Twin for FDM Printer With Preventive Cyber-Attack and Control Algorithms
Hazrat Ali, M., Waqar Malik, A., Jyeniskhan, N., Arif Mahmood, M., Shehab, E., & Liou, F. (2024) · IEEE Access · doi:10.1109/access.2024.3516827 · Cited by 12 · see citing works
- Authors
- Yu, W., Dillon, T., Mostafa, F., Rahayu, W., & Liu, Y.
- Affiliations
- La Trobe University (AU)
- Venue
- IEEE Transactions on Industrial Informatics
- Published
- 2020-01
- Volume
- 16
- Issue
- 1
- Pages
- 183-192
- ISSN
- 1551-3203 (ISSN-L)
- 1941-0050
- OpenAlex
- W2944122571
- Cited by
- 225
- References
- 27
- Citations by year
- 2026: 17 · 2025: 30 · 2024: 41 · 2023: 45 · 2022: 28 · 2021: 38 · 2020: 25 · 2019: 1
- FWCI
- 16.98
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Transactions on Industrial Informatics (publishedVersion)
- Topics
- Physical Sciences › Engineering › Industrial and Manufacturing Engineering › Digital Transformation in Industry
- Physical Sciences › Engineering › Control and Systems Engineering › Fault Detection and Control Systems
- Physical Sciences › Engineering › Control and Systems Engineering › Machine Fault Diagnosis Techniques
- Keywords
- big data ecosystem, fault detection, predictive maintenance, smart manufacturing, Industry 4.0, Industrial Internet of Things, data lakes, NoSQL databases, Apache Spark, MapReduce, real-time data processing, data security, big data architecture, fault early warning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
