The DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.
Robot learning towards smart robotic manufacturing: A review
Open accessAbstract
Robotic equipment has been playing a central role since the proposal of smart manufacturing. Since the beginning of the first integration of industrial robots into production lines, industrial robots have enhanced productivity and relieved humans from heavy workloads significantly. Towards the next generation of manufacturing, this review first introduces the comprehensive background of smart robotic manufacturing within robotics, machine learning, and robot learning. Definitions and categories of robot learning are summarised. Concretely, imitation learning, policy gradient learning, value function learning, actor-critic learning, and model-based learning as the leading technologies in robot learning are reviewed. Training tools, benchmarks, and comparisons amongst different robot learning methods are delivered. Typical industrial applications in robotic grasping, assembly, process control, and industrial human-robot collaboration are listed and discussed. Finally, open problems and future research directions are summarised.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Multilayer feedforward networks are universal approximators
Hornik, K., Stinchcombe, M., & White, H. (1989) · Neural Networks · doi:10.1016/0893-6080(89)90020-8 · Cited by 21,728
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782
The graph neural network model
Scarselli, F., Gori, M., Tsoi, A. C., Hagenbuchner, M., & Monfardini, G. (2009) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2008.2005605 · Cited by 9,582
Mastering the game of Go without human knowledge
Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., et al. (2017) · Nature · doi:10.1038/nature24270 · Cited by 9,084
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132
Soft actor-critic Off-policy maximum entropy deep reinforcement learning with a stochastic actor
Haarnoja, T., Zhou, A., Abbeel, P., & Levine, S. (2018) · arXiv · doi:10.48550/arxiv.1801.01290 · Cited by 5,594
Addressing Function Approximation Error in Actor-Critic Methods
Fujimoto, S., van Hoof, H., & Meger, D. (2018) · arXiv · doi:10.48550/arxiv.1802.09477 · Cited by 3,650
Research works printed but not linked: 82
- Kang, 2016, Smart manufacturing: past research, present findings, and future directions, Int. J. Precision Eng. Manuf.-Green Technol., 3, 111, 10.1007/s40684-016-0015-5 · DOI · Paper page
- Wang, 2019, From intelligence science to intelligent manufacturing, Engineering, 5, 615, 10.1016/j.eng.2019.04.011 · DOI
- Kaelbling, 2020, The foundation of efficient robot learning, Science, 369, 915, 10.1126/science.aaz7597 · DOI
- Cui, 2021, Toward next-generation learned robot manipulation, Sci. Robot., 6, 10.1126/scirobotics.abd9461 · DOI
- Kroemer, 2021, A review of robot learning for manipulation: challenges, representations, and algorithms, J. Mach. Learn. Res., 22
- Xie, 2020, Robot learning from demonstration for path planning: a review, Sci. China: Technol. Sci., 63, 1
- Ravichandar, 2020, Recent advances in robot learning from demonstration, Ann. Rev. Control, Robot. Autonomous Syst., 3, 297, 10.1146/annurev-control-100819-063206 · DOI · Paper page
- Wang, 2019, Learning and comfort in human–robot interaction: a review, Appl. Sci., 9, 5152, 10.3390/app9235152 · DOI
- Zhu, 2018, Robot learning from demonstration in robotic assembly: a survey, Robotics, 7, 17, 10.3390/robotics7020017 · DOI · Paper page
- Digani, 2015, Ensemble coordination approach in multi-agv systems applied to industrial warehouses, IEEE Trans. Autom. Sci. Eng., 12, 922, 10.1109/TASE.2015.2446614 · DOI
- Liu, 2019, Human-robot collaboration in disassembly for sustainable manufacturing, Int. J. Prod. Res., 57, 4027, 10.1080/00207543.2019.1578906 · DOI · Paper page
- Lu, 2020, Smart manufacturing process and system automation–a critical review of the standards and envisioned scenarios, J. Manuf. Syst., 56, 312, 10.1016/j.jmsy.2020.06.010 · DOI · Paper page
- Qu, 2019, Smart manufacturing systems: state of the art and future trends, Int. J. Adv. Manuf. Technol., 103, 3751, 10.1007/s00170-019-03754-7 · DOI · Paper page
- Wang, 2019, Symbiotic human-robot collaborative assembly, CIRP Ann. Manuf. Technol., 68, 701, 10.1016/j.cirp.2019.05.002 · DOI · Paper page
- Wang, 2021, Evaluation and optimization of dual-arm robot path planning for human–robot collaborative tasks in smart manufacturing contexts, ASME Lett. Dyn. Syst. Control, 1
- Liu, 2021, Sensorless haptic control for human-robot collaborative assembly, CIRP J. Manuf. Sci. Technol., 32, 132, 10.1016/j.cirpj.2020.11.015 · doi:10.1016/j.cirpj.2020.11.015
- Al-Yacoub, 2021, Improving human robot collaboration through force/torque based learning for object manipulation, Rob. Comput. Integr. Manuf., 69, 10.1016/j.rcim.2020.102111 · DOI
- Liu, 2021, Sensorless force estimation for industrial robots using disturbance observer and neural learning of friction approximation, Rob. Comput. Integr. Manuf., 71, 10.1016/j.rcim.2021.102168 · doi:10.1016/j.rcim.2021.102168
- Liu, 2020, Dynamic risk assessment and active response strategy for industrial human-robot collaboration, Comput. Ind. Eng., 141, 10.1016/j.cie.2020.106302 · DOI
- Liu, 2021, Deep reinforcement learning-based safe interaction for industrial human-robot collaboration using intrinsic reward function, Adv. Eng. Inf., 49, 10.1016/j.aei.2021.101360 · DOI
- Liu, 2018, Human-robot collaborative manufacturing using cooperative game: framework and implementation, Procedia CIRP, 72, 87, 10.1016/j.procir.2018.03.172 · DOI
- Billard, 2019, Trends and challenges in robot manipulation, Science, 364, 10.1126/science.aat8414 · DOI · Paper page
- Cheng, 2019, Purposive learning: robot reasoning about the meanings of human activities, Sci. Robot., 4, 10.1126/scirobotics.aav1530 · DOI
- Mahler, 2019, Learning ambidextrous robot grasping policies, Sci. Robot., 4, 10.1126/scirobotics.aau4984 · DOI · Paper page
- Ichnowski, 2020, Deep learning can accelerate grasp-optimized motion planning, Sci. Robot., 5, 10.1126/scirobotics.abd7710 · DOI
- Sundaram, 2020, Robots learn to identify objects by feeling, Sci. Robot., 5, 10.1126/scirobotics.abf1502 · DOI
- Chernova, 2014, Robot learning from human teachers, Synth. Lect. Artif. Intell. Mach. Learn., 8, 1
- Won, 2020, An adaptive deep reinforcement learning framework enables curling robots with human-like performance in real-world conditions, Sci. Robot., 5, 10.1126/scirobotics.abb9764 · DOI
- Lazaro-Gredilla, 2019, Beyond imitation: zero-shot task transfer on robots by learning concepts as cognitive programs, Sci. Robot., 4, 10.1126/scirobotics.aav3150 · DOI
- Pedersen, 2016, Robot skills for manufacturing: from concept to industrial deployment, Rob. Comput. Integr. Manuf., 37, 282, 10.1016/j.rcim.2015.04.002 · DOI · Paper page
- Bemporad, 2002, The explicit linear quadratic regulator for constrained systems, Automatica, 38, 3, 10.1016/S0005-1098(01)00174-1 · DOI · Paper page
- M. Bojarski, D. Del Testa, D. Dworakowski, B. Firner, B. Flepp, P. Goyal, et al., End to end learning for self-driving cars, arXiv preprint arXiv:160407316. (2016).
- Williams, 1992, Simple statistical gradient-following algorithms for connectionist reinforcement learning, Mach. Learn., 8, 229, 10.1023/A:1022672621406 · DOI · Paper page
- Peters, 2008, Reinforcement learning of motor skills with policy gradients, Neural Netw., 21, 682, 10.1016/j.neunet.2008.02.003 · DOI · Paper page
- J. Schulman, F. Wolski, P. Dhariwal, A. Radford, O. Klimov, Proximal policy optimization algorithms, arXiv preprint arXiv:170706347. (2017).
- D. Kalashnikov, A. Irpan, P. Pastor, J. Ibarz, A. Herzog, E. Jang, et al., Qt-opt: scalable deep reinforcement learning for vision-based robotic manipulation, arXiv preprint arXiv:180610293. (2018).
- Yao, 2018, Sensor-less external force detection for industrial manipulators to facilitate physical human-robot interaction, J. Mech. Sci. Technol., 32, 4909, 10.1007/s12206-018-0939-5 · doi:10.1007/s12206-018-0939-5
- De Boer, 2005, A tutorial on the cross-entropy method, Ann. Oper. Res., 134, 19, 10.1007/s10479-005-5724-z · DOI · Paper page
- Levine, 2016, End-to-end training of deep visuomotor policies, J. Mach. Learn. Res., 17, 1334
- Bristow, 2006, A survey of iterative learning control, IEEE Control Syst. Mag., 26, 96, 10.1109/MCS.2006.1636313 · DOI · Paper page
- James, 2020, Rlbench: the robot learning benchmark & learning environment, IEEE Robot. Autom. Lett., 5, 3019, 10.1109/LRA.2020.2974707 · DOI · Paper page
- Zhang, 2021, Digital twin-enabled reconfigurable modeling for smart manufacturing systems, Int. J. Computer Integr. Manuf., 34, 709, 10.1080/0951192X.2019.1699256 · DOI
- de Souza, 2021, Robotic grasping: from wrench space heuristics to deep learning policies, Rob. Comput. Integr. Manuf., 71, 10.1016/j.rcim.2021.102176 · DOI
- Moosmann, 2021, Separating entangled workpieces in random bin picking using deep reinforcement learning, Procedia CIRP, 104, 881, 10.1016/j.procir.2021.11.148 · DOI
- L. Berscheid, C. Friedrich, T. Kröger, Robot learning of 6 dof grasping using model-based adaptive primitives, arXiv preprint arXiv:210312810. (2021). · DOI
- Mohammed, 2020, Pick and place objects in a cluttered scene using deep reinforcement learning, Int. J. Mech. Mechatron. Eng., 20, 50
- Wang, 2021, A visual grasping strategy for improving assembly efficiency based on deep reinforcement learning, J. Sens., 2021
- Akkaladevi, 2019, Towards reinforcement based learning of an assembly process for human robot collaboration, Procedia Manuf., 38, 1491, 10.1016/j.promfg.2020.01.138 · DOI
- De Winter, 2019, Accelerating interactive reinforcement learning by human advice for an assembly task by a cobot, Robotics, 8, 104, 10.3390/robotics8040104 · DOI
- Duque, 2019, Trajectory generation for robotic assembly operations using learning by demonstration, Rob. Comput. Integr. Manuf., 57, 292, 10.1016/j.rcim.2018.12.007 · DOI
- Kim, 2020, Reinforcement learning based on movement primitives for contact tasks, Rob. Comput. Integr. Manuf., 62, 10.1016/j.rcim.2019.101863 · DOI
- Kyrarini, 2019, Robot learning of industrial assembly task via human demonstrations, Autonom. Rob., 43, 239, 10.1007/s10514-018-9725-6 · DOI
- Li, 2019, Robot skill acquisition in assembly process using deep reinforcement learning, Neurocomputing, 345, 92, 10.1016/j.neucom.2019.01.087 · DOI
- Kristensen, 2019, Towards a robot simulation framework for e-waste disassembly using reinforcement learning, Procedia Manuf., 38, 225, 10.1016/j.promfg.2020.01.030 · doi:10.1016/j.promfg.2020.01.030
- Andersen, 2019, Self-learning processes in smart factories: deep reinforcement learning for process control of robot brine injection, Procedia Manuf., 38, 171, 10.1016/j.promfg.2020.01.023 · doi:10.1016/j.promfg.2020.01.023
- Brito, 2020, A machine learning approach for collaborative robot smart manufacturing inspection for quality control systems, Procedia Manuf., 51, 11, 10.1016/j.promfg.2020.10.003 · DOI
- Duguleana, 2012, Obstacle avoidance of redundant manipulators using neural networks based reinforcement learning, Rob. Comput. Integr. Manuf., 28, 132
- M.S.A. Hameed, M.M. Khan, A. Schwung, Curiosity based reinforcement learning on robot manufacturing cell, arXiv preprint arXiv:201108743. (2020).
- Imtiaz, 2021, A comparison of two reinforcement learning algorithms for robotic pick and place with non-visual sensing, Int. J. Mech. Eng. Robot. Res., 10, 526
- Jaradat, 2011, Reinforcement based mobile robot navigation in dynamic environment, Rob. Comput. Integr. Manuf., 27, 135, 10.1016/j.rcim.2010.06.019 · DOI · Paper page
- Luo, 2019, A robot learning method with physiological interface for teleoperation systems, Appl. Sci., 9, 2099, 10.3390/app9102099 · DOI
- Maldonado-Ramirez, 2021, A visual path-following learning approach for industrial robots using drl, Rob. Comput. Integr. Manuf., 71, 10.1016/j.rcim.2021.102130 · DOI
- Tsai, 2020, Utilization of a reinforcement learning algorithm for the accurate alignment of a robotic arm in a complete soft fabric shoe tongues automation process, J. Manuf. Syst., 56, 501, 10.1016/j.jmsy.2020.07.001 · DOI
- Wang, 2020, Learning mobile manipulation through deep reinforcement learning, Sensors, 20, 939, 10.3390/s20030939 · DOI
- Wang, 2015, Nonparametric statistical learning control of robot manipulators for trajectory or contour tracking, Rob. Comput. Integr. Manuf., 35, 96, 10.1016/j.rcim.2015.03.002 · DOI
- Ye, 2020, Bagging for gaussian mixture regression in robot learning from demonstration, J. Intell. Manuf., 1
- Zeng, 2018, Robot learning human stiffness regulation for hybrid manufacture, Assembly Autom., 38, 539, 10.1108/AA-02-2018-019 · DOI
- Zhang, 2021, Robot learning system based on dynamic movement primitives and neural network, Neurocomputing, 451, 205, 10.1016/j.neucom.2021.04.034 · doi:10.1016/j.neucom.2021.04.034
- Liu, 2021, Task-level decision-making for dynamic and stochastic human-robot collaboration based on dual agents deep reinforcement learning, Int. J. Adv. Manuf. Technol., 115, 3533, 10.1007/s00170-021-07265-2 · DOI
- Oliff, 2020, Reinforcement learning for facilitating human-robot-interaction in manufacturing, J. Manuf. Syst., 56, 326, 10.1016/j.jmsy.2020.06.018 · DOI
- Rozo, 2016, Learning physical collaborative robot behaviors from human demonstrations, IEEE Trans. Rob., 32, 513, 10.1109/TRO.2016.2540623 · DOI · Paper page
- Sun, 2020, Learn how to assist humans through human teaching and robot learning in human-robot collaborative assembly, IEEE Trans. Syst., Man, Cybern.: Systems
- Wang, 2018, Facilitating human–robot collaborative tasks by teaching-learning-collaboration from human demonstrations, IEEE Trans. Autom. Sci. Eng., 16, 640, 10.1109/TASE.2018.2840345 · DOI · Paper page
- Wang, 2021, Predicting human intentions in human-robot and-over tasks through multimodal learning, IEEE Trans. Autom. Sci. Eng., 1
- Wang, 2021, Optimised learning from demonstrations for collaborative robots, Rob. Comput. Integr. Manuf., 71, 10.1016/j.rcim.2021.102169 · DOI
- Yu, 2020, Mastering the working sequence in human-robot collaborative assembly based on reinforcement learning, IEEE Access, 8, 163868, 10.1109/ACCESS.2020.3021904 · DOI
- Zhang, 2022, A reinforcement learning method for human-robot collaboration in assembly tasks, Rob. Comput. Integr. Manuf., 73, 10.1016/j.rcim.2021.102227 · doi:10.1016/j.rcim.2021.102227
- Leottau, 2018, Decentralized reinforcement learning of robot behaviors, Artif. Intell., 256, 130, 10.1016/j.artint.2017.12.001 · DOI
- Xu, 2021, Digital twin-based industrial cloud robotics: framework, control approach and implementation, J. Manuf. Syst., 58, 196, 10.1016/j.jmsy.2020.07.013 · DOI
- de Bruin, 2018, Experience selection in deep reinforcement learning for control, J. Mach. Learn. Res., 19
- T.Z. Zhao, J. Luo, O. Sushkov, R. Pevceviciute, N. Heess, J. Scholz, et al., Offline meta-reinforcement learning for industrial insertion, arXiv preprint arXiv:211004276. (2021). · DOI
- Bogert, 2018, Multi-robot inverse reinforcement learning under occlusion with estimation of state transitions, Artif. Intell., 263, 46, 10.1016/j.artint.2018.07.002 · DOI
Cited by
Human-in-the-Loop Robot Learning for Smart Manufacturing: A Human-Centric Perspective
Chen, H., Li, S., Fan, J., Duan, A., Yang, C., Navarro-Alarcon, D., & Zheng, P. (2025) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2025.3528051 · Cited by 67
“Therefore, robots should be released from the limits of old-fashioned hard-programmed solutions and point at more complicated and flexible environments to cater to personal necessities and uphold the manufacturing landscape’s imperative for adapt-ability, nimbleness, and resilience [15], [24].”
Evaluating the Reliability of Vision Transformers for Space Robotics Applications
Bodmann, P. R., Rech, P., & Saveriano, M. (2024) · 2024 International Conference on Space Robotics (iSpaRo) · doi:10.1109/isparo60631.2024.10687576 · Cited by 5
Metadata
- Authors
- Liu, Z., Liu, Q., Xu, W., Wang, L., & Zhou, Z.
- Venue
- Robotics and Computer-Integrated Manufacturing
- Publisher
- Elsevier BV
- Published
- 2022-10
- Volume
- 77
- Pages
- 102360
- ISSN
- 0736-5845 (ISSN-L)
- OpenAlex
- W4224221644
- Cited by
- 215
- References
- 31
- FWCI
- 14.38
- Type
- journal article
- Language
- en
- Open access
- hybrid
- Licence
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
- www.elsevier.com/legal/tdmrep-license (tdm)
- creativecommons.org/licenses/by/4.0/ (vor)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Robot learning towards smart robotic manufacturing: A review. https://app.cochat.ai/papers/10.1016/j.rcim.2022.102360
- The paper
- Liu, Z., Liu, Q., Xu, W., Wang, L., & Zhou, Z. (2022). Robot learning towards smart robotic manufacturing: A review. Robotics and Computer-Integrated Manufacturing. https://doi.org/10.1016/j.rcim.2022.102360
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.