Leveraging Task Modularity in Reinforcement Learning for Adaptable Industry 4.0 Automation
The DOI matches its Crossref record; 11 of the 11 references it cites were found in a registry or an index.
Abstract
The vision of Industry 4.0 is to materialize the notion of a lot-size of one through enhanced adaptability and resilience of manufacturing and logistics operations to dynamic changes or deviations on the shop floor. This article is motivated by the lack of formal methods for efficient transfer of knowledge across different yet interrelated tasks, with special reference to collaborative robotic operations such as material handling, machine tending, assembly, and inspection. We propose a meta reinforcement learning framework to enhance the adaptability of collaborative robots to new tasks through task modularization and efficient transfer of policies from previously learned task modules. Our experiments on the OpenAI Gym Robotics environments Reach, Push, and Pick-and-Place indicate an average 75% reduction in the number of iterations to achieve a 60% success rate as well as a 50%-80% improvement in task completion efficiency, compared to the deep deterministic policy gradient (DDPG) algorithm as a baseline. The significant improvements achieved in the jumpstart and asymptotic performance of the robot create new opportunities for investigating the current limitations of learning robots in industrial settings, associated with sample inefficiency and specialization on one task through modularization and transfer learning.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Proximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625
Lasi, H., Fettke, P., Kemper, H.-G., Feld, T., & Hoffmann, M. (2014) · Business & Information Systems Engineering · doi:10.1007/s12599-014-0334-4 · Cited by 4,276
Design Rules: The Power of Modularity
Baldwin, C. Y., & Clark, K. B. (2000) · doi:10.7551/mitpress/2366.001.0001 · Cited by 3,504
Grandmaster level in StarCraft II using multi-agent reinforcement learning
Vinyals, O., Babuschkin, I., Czarnecki, W. M., Mathieu, M., Dudzik, A., Chung, J., Choi, D. H., Powell, R., Ewalds, T., Georgiev, P., et al. (2019) · Nature · doi:10.1038/s41586-019-1724-z · Cited by 3,455
Industry 4.0: A survey on technologies, applications and open research issues
Lu, Y. (2017) · Journal of Industrial Information Integration · doi:10.1016/j.jii.2017.04.005 · Cited by 3,012
Reinforcement learning, fast and slow
Botvinick, M., Ritter, S., Wang, J. X., Kurth-Nelson, Z., Blundell, C., & Hassabis, D. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2019.02.006 · Cited by 667
Learning to reinforcement learn
Wang, J. X., Kurth-Nelson, Z., Tirumala, D., Soyer, H., Leibo, J. Z., Munos, R., Blundell, C., Kumaran, D., & Botvinick, M. (2016) · arXiv · doi:10.48550/arxiv.1611.05763 · Cited by 579
Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning
Yu, T., Quillen, D., He, Z., Julian, R., Narayan, A., Shively, H., Bellathur, A., Hausman, K., Finn, C., & Levine, S. (2019) · arXiv · doi:10.48550/arxiv.1910.10897 · Cited by 450
Reference architectures for smart manufacturing: A critical review
Moghaddam, M., Cadavid, M. N., Kenley, C. R., & Deshmukh, A. V. (2018) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2018.10.006 · Cited by 219
Self-citation
Cited by
Deep reinforcement learning in smart manufacturing: A review and prospects
Li, C., Zheng, P., Yin, Y., Wang, B., & Wang, L. (2023) · CIRP Journal of Manufacturing Science and Technology · doi:10.1016/j.cirpj.2022.11.003 · Cited by 327
Chen, Q., & Heydari, B. (2022) · Journal of Mechanical Design · doi:10.1115/1.4055057 · Cited by 12
Self-citationThe Sos Conductor: Orchestrating Resources With Iterative Agent-Based Reinforcement Learning
Chen, Q., & Heydari, B. (2024) · Systems Engineering · doi:10.1002/sys.21747 · Cited by 5
Self-citationGenerative Network Design for Multi-Agent Coordination: A VAE-Assisted Deep RL Approach
Chen, Q., & Heydari, B. (2025) · Journal of Mechanical Design · doi:10.1115/1.4068796 · Cited by 4
Self-citation
Metadata
- Authors
- Chen, Q., Heydari, B., & Moghaddam, M.
- Venue
- Journal of Mechanical Design
- Publisher
- ASME International
- Published
- 2021-01-29
- Volume
- 143
- Issue
- 7
- ISSN
- 0161-8458 (ISSN-L)
- 1050-0472
- 1528-9001
- OpenAlex
- W3119608713
- Cited by
- 36
- References
- 11
- FWCI
- 2.21
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Leveraging Task Modularity in Reinforcement Learning for Adaptable Industry 4.0 Automation. https://app.cochat.ai/papers/10.1115/1.4049531
- The paper
- Chen, Q., Heydari, B., & Moghaddam, M. (2021). Leveraging Task Modularity in Reinforcement Learning for Adaptable Industry 4.0 Automation. Journal of Mechanical Design. https://doi.org/10.1115/1.4049531
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.