Acquiring robot skills via reinforcement learning
Abstract
Skill acquisition is a difficult , yet important problem in robot performance. The authors focus on two skills, namely robotic assembly and balancing and on two classic tasks to develop these skills via learning: the peg-in hole insertion task, and the ball balancing task. A stochastic real-valued (SRV) reinforcement learning algorithm is described and used for learning control and the authors show how it can be used with nonlinear multilayer ANNs. In the peg-in-hole insertion task the SRV network successfully learns to insert to insert a peg into a hole with extremely low clearance, in spite of high sensor noise. In the ball balancing task the SRV network successfully learns to balance the ball with minimal feedback.>
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Cites
Learning Internal Representations by Error Propagation
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1985) · doi:10.21236/ada164453 · Cited by 16,501 · see citing works
Parallel Distributed Processing: Explorations in the Microstructure of Cognition: Foundations
Rumelhart, D. E., McClelland, J. L., & AU (1986) · doi:10.7551/mitpress/5236.001.0001 · Cited by 15,686 · see citing works
Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
Williams, R. J. (1992) · Machine Learning · doi:10.1007/bf00992696 · Cited by 7,936 · see citing works
Adaptive filtering, prediction and control
Ioannou, P. (1985) · Automatica · doi:10.1016/0005-1098(85)90014-7 · Cited by 4,487 · see citing works
Neuronlike adaptive elements that can solve difficult learning control problems
Barto, A. G., Sutton, R. S., & Anderson, C. W. (1983) · IEEE Transactions on Systems Man and Cybernetics · doi:10.1109/tsmc.1983.6313077 · Cited by 3,238 · see citing works
Raibert, M. H., & Tello, E. R. (1986) · IEEE Expert · doi:10.1109/mex.1986.4307016 · Cited by 2,752 · see citing works
Advances in Neural Information Processing Systems 5
Zemel, R. S., Williams, C. K. I., & Mozer, M. C. (1993) · Cited by 1,896 · see citing works
Quasi-Static Assembly of Compliantly Supported Rigid Parts
Whitney, D. E. (1982) · Journal of Dynamic Systems Measurement and Control · doi:10.1115/1.3149634 · Cited by 968 · see citing works
Automatic Synthesis of Fine-Motion Strategies for Robots
Lozano‐Pérez, T., Mason, M. T., & Taylor, R. H. (1984) · The International Journal of Robotics Research · doi:10.1177/027836498400300101 · Cited by 901 · see citing works
The joy of feedback: nonlinear and adaptive
Kokotović, P. V. (1992) · IEEE Control Systems · doi:10.1109/37.165507 · Cited by 879 · see citing works
(1991) · The MIT Press eBooks · doi:10.7551/mitpress/4939.001.0001 · Cited by 867 · see citing works
Temporal credit assignment in reinforcement learning
Sutton, R. S. (1984) · ScholarWorks@UMassAmherst (University of Massachusetts Amherst) · Cited by 769 · see citing works
Dvoretzky, A. (1956) · University of California Press eBooks · doi:10.1525/9780520313880-007 · Cited by 328 · see citing works
Pattern-recognizing stochastic learning automata
Barto, A. G., & Anandan, P. (1985) · IEEE Transactions on Systems Man and Cybernetics · doi:10.1109/tsmc.1985.6313371 · Cited by 308 · see citing works
A stochastic reinforcement learning algorithm for learning real-valued functions
Gullapalli, V. (1990) · Neural Networks · doi:10.1016/0893-6080(90)90056-q · Cited by 292 · see citing works
Self-citationUsing Backprojections for Fine Motion Planning with Uncertainty
Erdmann, M. (1986) · The International Journal of Robotics Research · doi:10.1177/027836498600500102 · Cited by 252 · see citing works
Sensor-Based Control of a Nine-Link Biped
Furusho, J., & Akihito, S. A. N. O. (1990) · The International Journal of Robotics Research · doi:10.1177/027836499000900207 · Cited by 236 · see citing works
Strategy Learning with Multilayer Connectionist Representations
Anderson, C. W. (1987) · Elsevier eBooks · doi:10.1016/b978-0-934613-41-5.50014-3 · Cited by 197 · see citing works
A heuristic approach to reinforcement learning control systems
Waltz, M. D., & Fu, K. (1965) · IEEE Transactions on Automatic Control · doi:10.1109/tac.1965.1098193 · Cited by 157 · see citing works
Admittance matrix design for force-guided assembly
Schimmels, J. M., & Peshkin, M. A. (1992) · IEEE Transactions on Robotics and Automation · doi:10.1109/70.134275 · Cited by 119 · see citing works
Asada, H. H. (2002) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.1990.126167 · Cited by 111 · see citing works
A self-learning automaton with variable resolution for high precision assembly by industrial robots
Simons, J., Brussel, H. V., Schutter, J. D., et al. (1982) · IEEE Transactions on Automatic Control · doi:10.1109/tac.1982.1103060 · Cited by 81 · see citing works
Assembly strategies for chamferless parts
Caine, M. E., Lozano‐Pérez, T., & Seering, W. P. (2003) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.1989.100031 · Cited by 81 · see citing works
Reinforcement learning and its application to control
Gullapalli, V. (1992) · Cited by 77 · see citing works
Self-citationDonald, B. R. (1986) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.1986.1087480 · Cited by 43 · see citing works
Refinement of robot motor skills through reinforcement learning
Franklin, J. A. (2003) · Proceedings of the IEEE Conference on Decision & Control, including the Symposium on Adaptive Processes · doi:10.1109/cdc.1988.194487 · Cited by 34 · see citing works
Self-citationReal-time learning: a ball on a beam
Benbrahim, H., Doleac, J., Franklin, J. A., & Selfridge, O. G. (2003) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.1992.287219 · Cited by 29 · see citing works
Self-citationAssociative reinforcement learning of real-valued functions
Gullapalli, V. (2002) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/icsmc.1991.169893 · Cited by 24 · see citing works
Self-citationA Theory for the Three-Dimensional Mating of Chamfered Cylindrical Parts
Gustavson, R. E. (1985) · Journal of Mechanisms Transmissions and Automation in Design · doi:10.1115/1.3258673 · Cited by 23 · see citing works
A tool box for approximate linearization of nonlinear systems
Kadiyala, R. R. (1993) · IEEE Control Systems · doi:10.1109/37.206985 · Cited by 23 · see citing works
Learning expert systems for robot fine motion control
Lee, S., & Kim, M. H. (2003) · IEEE International Symposium on Intelligent Control · doi:10.1109/isic.1988.65488 · Cited by 19 · see citing works
Input space representation for refinement learning control
Franklin, J. A. (2003) · IEEE International Symposium on Intelligent Control · doi:10.1109/isic.1989.238708 · Cited by 12 · see citing works
Self-citationAutomated assembly using feature localization
Gordon, S. J. (1987) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 11 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Reinforcement learning in robotics: A survey
Kober, J., Bagnell, J. A., & Peters, J. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913495721 · Cited by 3,160 · see citing works
Reinforcement learning of motor skills with policy gradients
Peters, J., & Schaal, S. (2008) · Neural Networks · doi:10.1016/j.neunet.2008.02.003 · Cited by 831 · see citing works
Salvato, E., Fenu, G., Medvet, E., & Pellegrino, F. A. (2021) · IEEE Access · doi:10.1109/access.2021.3126658 · Cited by 200 · see citing works
“In addition, policy search approaches result in more efficient trainings in terms of time needed for convergence [33]–[38].”
BackgroundSmart Congestion Control for Delay- and Disruption Tolerant Networks
Silva, A. P., Obraczka, K., Burleigh, S., & Hirata, C. M. (2016) · 2016 13th Annual IEEE International Conference on Sensing, Communication, and Networking (SECON) · doi:10.1109/sahcn.2016.7733018 · Cited by 20 · see citing works
- Authors
- Gullapalli, V., Franklin, J. A., & Benbrahim, H.
- Affiliations
- University of Massachusetts Amherst (US)
- GTE (United States) (US)
- Venue
- IEEE Control Systems
- Published
- 1994-02
- Volume
- 14
- Issue
- 1
- Pages
- 13-24
- ISSN
- 1066-033X (ISSN-L)
- 1941-000X
- OpenAlex
- W1988071341
- Cited by
- 206
- References
- 33
- Citations by year
- 2026: 2 · 2025: 2 · 2024: 6 · 2023: 3 · 2022: 15 · 2021: 11 · 2020: 10 · 2019: 11 · 2018: 3 · 2017: 4 · 2016: 5 · 2015: 5 · 2014: 2 · 2013: 5 · 2012: 5
- FWCI
- 12.36
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Control Systems (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Keywords
- robot skill learning, reinforcement learning, robotic assembly, ball balancing, peg-in-hole insertion, multilayer neural networks, learning control, sensor noise
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review