The DOI matches its Crossref record; 50 of the 50 references it cites were found in a registry or an index.
A digital twin to train deep reinforcement learning agent for smart manufacturing plants: Environment, interfaces and intelligence
Abstract
Abstract Filling the gaps between virtual and physical systems will open new doors in Smart Manufacturing. This work proposes a data-driven approach to utilize digital transformation methods to automate smart manufacturing systems. This is fundamentally enabled by using a digital twin to represent manufacturing cells, simulate system behaviors, predict process faults, and adaptively control manipulated variables. First, the manufacturing cell is accommodated to environments such as computer-aided applications, industrial Product Lifecycle Management solutions, and control platforms for automation systems. Second, a network of interfaces between the environments is designed and implemented to enable communication between the digital world and physical manufacturing plant, so that near-synchronous controls can be achieved. Third, capabilities of some members in the family of Deep Reinforcement Learning (DRL) are discussed with manufacturing features within the context of Smart Manufacturing. Trained results for Deep Q Learning algorithms are finally presented in this work as a case study to incorporate DRL-based artificial intelligence to the industrial control process. As a result, developed control methodology, named Digital Engine, is expected to acquire process knowledges, schedule manufacturing tasks, identify optimal actions, and demonstrate control robustness. The authors show that integrating a smart agent into the industrial platforms further expands the usage of the system-level digital twin, where intelligent control algorithms are trained and verified upfront before deployed to the physical world for implementation. Moreover, DRL approach to automated manufacturing control problems under facile optimization environments will be a novel combination between data science and manufacturing industries.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Playing Atari with Deep Reinforcement Learning
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013) · arXiv · doi:10.48550/arxiv.1312.5602 · Cited by 9,471
Mastering the game of Go without human knowledge
Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., et al. (2017) · Nature · doi:10.1038/nature24270 · Cited by 9,084
Digital Twin in Industry: State-of-the-Art
Tao, F., Zhang, H., Liu, A., & Nee, A. Y. C. (2019) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2018.2873186 · Cited by 5,871
Digital Twin in manufacturing: A categorical literature review and classification
Kritzinger, W., Karner, M., Traar, G., Henjes, J., & Sihn, W. (2018) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2018.08.474 · Cited by 3,612
Deep Reinforcement Learning with Double Q-Learning
Van Hasselt, H., Guez, A., & Silver, D. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10295 · Cited by 3,318
Deterministic Policy Gradient Algorithms
Silver, D., Lever, G., Heess, N., Degris, T., Wierstra, D., & Riedmiller, M. (2014) · HAL · Cited by 3,275
Schaul, T., Quan, J., Antonoglou, I., & Silver, D. (2015) · arXiv · doi:10.48550/arxiv.1511.05952 · Cited by 3,120
Digital twin-driven product design, manufacturing and service with big data
Tao, F., Cheng, J., Qi, Q., Zhang, M., Zhang, H., & Sui, F. (2018) · The International Journal of Advanced Manufacturing Technology · doi:10.1007/s00170-017-0233-1 · Cited by 2,920
Data-driven smart manufacturing
Tao, F., Qi, Q., Liu, A., & Kusiak, A. (2018) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2018.01.006 · Cited by 1,806
A Review of the Roles of Digital Twin in CPS-based Production Systems
Negri, E., Fumagalli, L., & Macchi, M. (2017) · Procedia Manufacturing · doi:10.1016/j.promfg.2017.07.198 · Cited by 1,669
Digital twin shop-floor: a new shop-floor paradigm towards smart manufacturing
Tao, F., & Zhang, M. (2017) · IEEE Access · doi:10.1109/access.2017.2756069 · Cited by 1,397
Machine learning in manufacturing: Advantages, challenges, and applications
Wuest, T., Weimer, D., Irgens, C., & Thoben, K.-D. (2016) · Production & Manufacturing Research · doi:10.1080/21693277.2016.1192517 · Cited by 1,319
Shaping the digital twin for design and production engineering
Schleich, B., Anwer, N., Mathieu, L., & Wartzack, S. (2017) · CIRP Annals · doi:10.1016/j.cirp.2017.04.040 · Cited by 1,305
Recent advances and trends in predictive manufacturing systems in big data environment
Lee, J., Lapira, E., Bagheri, B., & Kao, H.-A. (2013) · Manufacturing Letters · doi:10.1016/j.mfglet.2013.09.005 · Cited by 1,090
The Digital Twin: Realizing the Cyber-Physical Production System for Industry 4.0
Uhlemann, T. H.-J., Lehmann, C., & Steinhilper, R. (2017) · Procedia CIRP · doi:10.1016/j.procir.2016.11.152 · Cited by 994
Zhuang, C., Liu, J., & Xiong, H. (2018) · The International Journal of Advanced Manufacturing Technology · doi:10.1007/s00170-018-1617-6 · Cited by 693
Smart manufacturing must embrace big data
Kusiak, A. (2017) · Nature · doi:10.1038/544023a · Cited by 618
Smart manufacturing: Characteristics, technologies and enabling factors
Mittal, S., Khan, M. A., Romero, D., & Wuest, T. (2019) · Proceedings of the Institution of Mechanical Engineers, Part B: Journal of Engineering Manufacture · doi:10.1177/0954405417736547 · Cited by 560
Haag, S., & Anderl, R. (2018) · Manufacturing Letters · doi:10.1016/j.mfglet.2018.02.006 · Cited by 559
Digital Twin Data Modeling with AutomationML and a Communication Methodology for Data Exchange
Schroeder, G. N., Steinmetz, C., Pereira, C. E., & Espindola, D. B. (2016) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2016.11.115 · Cited by 492
The Digital Twin: Demonstrating the Potential of Real Time Data Acquisition in Production Systems
Uhlemann, T. H.-J., Schock, C., Lehmann, C., Freiberger, S., & Steinhilper, R. (2017) · Procedia Manufacturing · doi:10.1016/j.promfg.2017.04.043 · Cited by 489
Security of smart manufacturing systems
Tuptuk, N., & Hailes, S. (2018) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2018.04.007 · Cited by 456
Building blocks for a digital twin of additive manufacturing
Knapp, G. L., Mukherjee, T., Zuback, J. S., Wei, H. L., Palmer, T. A., De, A., & DebRoy, T. (2017) · Acta Materialia · doi:10.1016/j.actamat.2017.06.039 · Cited by 414
Lee, J. H., Shin, J., & Realff, M. J. (2018) · Computers & Chemical Engineering · doi:10.1016/j.compchemeng.2017.10.008 · Cited by 387
Bearing remaining useful life prediction based on deep autoencoder and deep neural networks
Ren, L., Sun, Y., Cui, J., & Zhang, L. (2018) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2018.04.008 · Cited by 380
Skin Model Shapes: A new paradigm shift for geometric variations modelling in mechanical engineering
Schleich, B., Anwer, N., Mathieu, L., & Wartzack, S. J. (2014) · Computer-Aided Design · doi:10.1016/j.cad.2014.01.001 · Cited by 316
Automated defect inspection system for metal surfaces based on deep learning and data augmentation
Yun, J. P., Shin, W. C., Koo, G., Kim, M. S., Lee, C., & Lee, S. J. (2020) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2020.03.009 · Cited by 268
A human-in-the-loop manufacturing control architecture for the next generation of production systems
Cimini, C., Pirola, F., Pinto, R., & Cavalieri, S. (2020) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2020.01.002 · Cited by 257
Learning by Playing - Solving Sparse Reward Tasks from Scratch
Riedmiller, M., Hafner, R., Lampe, T., Neunert, M., Degrave, J., Van de Wiele, T., Mnih, V., Heess, N., & Springenberg, J. T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.10567 · Cited by 242
A Cyber-Physical Machine Tools Platform using OPC UA and MTConnect
Liu, C., Vengayil, H., Lu, Y., & Xu, X. (2019) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2019.04.006 · Cited by 234
Reference architectures for smart manufacturing: A critical review
Moghaddam, M., Cadavid, M. N., Kenley, C. R., & Deshmukh, A. V. (2018) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2018.10.006 · Cited by 219
Lai, Z.-H., Tao, W., Leu, M. C., & Yin, Z. (2020) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2020.02.010 · Cited by 214
Hu, L., Liu, Z., Hu, W., Wang, Y., Tan, J., & Wu, F. (2020) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2020.02.004 · Cited by 204
Survey on the virtual commissioning of manufacturing systems
Lee, C. G., & Park, S. C. (2014) · Journal of Computational Design and Engineering · doi:10.7315/jcde.2014.021 · Cited by 203
Engineering Methods and Tools for Cyber–Physical Automation Systems
Harrison, R., Vera, D., & Ahmad, B. (2016) · Proceedings of the IEEE · doi:10.1109/jproc.2015.2510665 · Cited by 178
Günther, J., Pilarski, P. M., Helfrich, G., Shen, H., & Diepold, K. (2016) · Mechatronics · doi:10.1016/j.mechatronics.2015.09.004 · Cited by 176
A Deeper Look at Experience Replay
Zhang, S., & Sutton, R. S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.01275 · Cited by 173
System of systems engineering - New challenges for the 21st century
Jamshidi, M. (2008) · IEEE Aerospace and Electronic Systems Magazine · doi:10.1109/maes.2008.4523909 · Cited by 163
Virtual Commissioning Of Manufacturing Systems A Review And New Approaches For Simplification
Hoffmann, P., Maksoud, T. M. A., Schumann, R., & Premier, G. C. (2010) · European Conference on Modelling and Simulation (ECMS) · doi:10.7148/2010-0175-0181 · Cited by 161
Chen, L., Xu, G., Zhang, S., Yan, W., & Wu, Q. (2019) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2019.11.008 · Cited by 155
Adamson, G., Wang, L., & Moore, P. (2016) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2016.12.003 · Cited by 97
Lu, Y., & Asghar, M. R. (2020) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2020.05.001 · Cited by 93
Küçükoğlu, İ., ULUSU, H. A., GÜNDÜZ, T., & Tokçalar, Ö. (2018) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2018.10.001 · Cited by 58
An autonomous manufacturing system based on swarm of cognitive agents
Park, H., & Tran, N.-H. (2012) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2012.05.002 · Cited by 56
Simulation study on reward function of reinforcement learning in gantry work cell scheduling
Ou, X., Chang, Q., & Chakraborty, N. (2018) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2018.11.005 · Cited by 50
Konstantinov, S., Ahmad, M., Ananthanarayan, K., & Harrison, R. (2017) · Procedia CIRP · doi:10.1016/j.procir.2017.02.035 · Cited by 39
Importance Weighted Transfer of Samples in Reinforcement LearningPreprint
Tirinzoni, A., Sessa, A., Pirotta, M., & Restelli, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.10886 · Cited by 35
Shape Terra: mechanical feature recognition based on a persistent heat signature
Harik, R., Shi, Y., & Baek, S. (2016) · Computer-Aided Design and Applications · doi:10.1080/16864360.2016.1223433 · Cited by 14
Self-citationSynchro-push: A new production control paradigm
Garetti, M., Macchi, M., Pozzetti, A., Fumagalli, L., & Negri, E. (2016) · Virtual Community of Pathological Anatomy (University of Castilla La Mancha) · Cited by 5
Cited by
Digital Twins: A Survey on Enabling Technologies, Challenges, Trends and Future Prospects
Mihai, S., Yaqoob, M., Hung, D. V., Davis, W., Towakel, P., Raza, M., Karamanoglu, M., Barn, B., Shetve, D., Prasad, R. V., et al. (2022) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2022.3208773 · Cited by 1,049
Rathore, M. M., Shah, S. A., Shukla, D., Bentafat, E., & Bakiras, S. (2021) · IEEE Access · doi:10.1109/access.2021.3060863 · Cited by 616
A comprehensive review of digital twin — part 1: modeling and twinning enabling technologies
Thelen, A., Zhang, X., Fink, O., Lu, Y., Ghosh, S., Youn, B. D., Todd, M. D., Mahadevan, S., Hu, C., & Hu, Z. (2022) · Structural and Multidisciplinary Optimization · doi:10.1007/s00158-022-03425-4 · Cited by 441
A survey on AI-driven digital twins in Industry 4.0: smart manufacturing and advanced robotics
Huang, Z., Shen, Y., Li, J., Fey, M., & Brecher, C. (2021) · Sensors · doi:10.3390/s21196340 · Cited by 418
Liu, X., Jiang, D., Tao, B., Xiang, F., Jiang, G., Sun, Y., Kong, J., & Li, G. (2023) · Advanced Engineering Informatics · doi:10.1016/j.aei.2023.101876 · Cited by 351
Metadata
- Authors
- Xia, K., Sacco, C., Kirkpatrick, M., Saidy, C., Nguyen, L., Kircaliali, A., & Harik, R.
- Venue
- Journal of Manufacturing Systems
- Publisher
- Elsevier BV
- Published
- 2021-01
- Volume
- 58
- Pages
- 210-230
- ISSN
- 0278-6125 (ISSN-L)
- OpenAlex
- W3041382148
- Cited by
- 406
- References
- 50
- FWCI
- 34.36
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.