Enhancing Digital Twins through Reinforcement Learning
The DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index.
Abstract
Digital Twins are core enablers of smart and autonomous manufacturing systems. Although they strive to represent their physical counterpart as accurately as possible, slight model or data errors will remain. We present an algorithm to compensate for those residual errors through Reinforcement Learning (RL) and data fed back from the manufacturing system. When learning, the Digital Twin acts as teacher and safety policy to ensure minimal performance. We test the algorithm in a sheet metal assembly context, in which locators of the fixture are optimally adjusted for individual assemblies. Our results show a fast adaption and improved performance of the autonomous system.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Proximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625
Dropout as a bayesian approximation: Representing model uncertainty in deep learning
Gal, Y., & Ghahramani, Z. (2015) · arXiv · doi:10.48550/arxiv.1506.02142 · Cited by 6,471
Digital Twin: Mitigating Unpredictable, Undesirable Emergent Behavior in Complex Systems
Grieves, M., & Vickers, J. (2017) · Transdisciplinary Perspectives on Complex Systems · doi:10.1007/978-3-319-38756-7_4 · Cited by 3,727
Digital Twin in manufacturing: A categorical literature review and classification
Kritzinger, W., Karner, M., Traar, G., Henjes, J., & Sihn, W. (2018) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2018.08.474 · Cited by 3,612 · cited 2×
“In the manufacturing context, researchers are yet to agree on a common definition of the concept [6].”
BackgroundReinforcement learning in robotics: A survey
Kober, J., Bagnell, J. A., & Peters, J. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913495721 · Cited by 3,160 · cited 1×
“To handle that problem, function approximation has been proven successful [11].”
MethodDigital twin-driven product design, manufacturing and service with big data
Tao, F., Cheng, J., Qi, Q., Zhang, M., Zhang, H., & Sui, F. (2018) · The International Journal of Advanced Manufacturing Technology · doi:10.1007/s00170-017-0233-1 · Cited by 2,920
About The Importance of Autonomy and Digital Twins for the Future of Manufacturing
Rosen, R., von Wichert, G., Lo, G., & Bettenhausen, K. D. (2015) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2015.06.141 · Cited by 1,474 · cited 2×
“The Digital Twin is seen as a core enabler for smart and autonomous manufacturing systems [1].”
BackgroundDigital twin shop-floor: a new shop-floor paradigm towards smart manufacturing
Tao, F., & Zhang, M. (2017) · IEEE Access · doi:10.1109/access.2017.2756069 · Cited by 1,397 · cited 2×
“Another is the automatic and continuous update of models and model parameters needed over the life cycle of the system [1], [10].”
BackgroundDigital Twin—The Simulation Aspect
Boschert, S., & Rosen, R. (2016) · Mechatronic Futures · doi:10.1007/978-3-319-32156-1_5 · Cited by 1,158 · cited 1×
“[7], [8].”
Deep Q-learning From Demonstrations
Hester, T., Vecerik, M., Pietquin, O., Lanctot, M., Schaul, T., Piot, B., Horgan, D., Quan, J., Sendonaris, A., Osband, I., et al. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11757 · Cited by 889
Cited by
Digital Twins: A Survey on Enabling Technologies, Challenges, Trends and Future Prospects
Mihai, S., Yaqoob, M., Hung, D. V., Davis, W., Towakel, P., Raza, M., Karamanoglu, M., Barn, B., Shetve, D., Prasad, R. V., et al. (2022) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2022.3208773 · Cited by 1,049
Digital Twins: State of the art theory and practice, challenges, and open research questions
Sharma, A., Kosasih, E., Zhang, J., Brintrup, A., & Calinescu, A. (2022) · Journal of Industrial Information Integration · doi:10.1016/j.jii.2022.100383 · Cited by 587
A survey on AI-driven digital twins in Industry 4.0: smart manufacturing and advanced robotics
Huang, Z., Shen, Y., Li, J., Fey, M., & Brecher, C. (2021) · Sensors · doi:10.3390/s21196340 · Cited by 418
“Researchers have applied reinforcement learning algorithms, including Q-learning [96], deep reinforcement learning [95,97,267] and deep deterministic policy gradient [199,266,275] to optimize the decision-making process of box sorting, conveyor systems and other DT scenarios.”
BackgroundMethodDigital Twin: A Comprehensive Survey of Security Threats
Alcaraz, C., & Lopez, J. (2022) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2022.3171465 · Cited by 340
Deep reinforcement learning in smart manufacturing: A review and prospects
Li, C., Zheng, P., Yin, Y., Wang, B., & Wang, L. (2023) · CIRP Journal of Manufacturing Science and Technology · doi:10.1016/j.cirpj.2022.11.003 · Cited by 327
Metadata
- Authors
- Cronrath, C., Aderiani, A. R., & Lennartson, B.
- Venue
- 2019 IEEE 15th International Conference on Automation Science and Engineering (CASE)
- Publisher
- IEEE
- Published
- 2019-08
- Pages
- 293-298
- ISSN
- 2161-8070 (ISSN-L)
- OpenAlex
- W2973496471
- Cited by
- 74
- References
- 10
- FWCI
- 10.08
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Enhancing Digital Twins through Reinforcement Learning. https://app.cochat.ai/papers/10.1109/coase.2019.8842888
- The paper
- Cronrath, C., Aderiani, A. R., & Lennartson, B. (2019). Enhancing Digital Twins through Reinforcement Learning. 2019 IEEE 15th International Conference on Automation Science and Engineering (CASE). https://doi.org/10.1109/coase.2019.8842888
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.