Augmenting Reinforcement Learning with Behavior Primitives for Diverse Manipulation Tasks
Open accessThe DOI matches its Crossref record; 78 of the 89 references it cites were found in a registry or an index.
Abstract
Realistic manipulation tasks require a robot to interact with an environment with a prolonged sequence of motor actions. While deep reinforcement learning methods have recently emerged as a promising paradigm for automating manipulation behaviors, they usually fall short in long-horizon tasks due to the exploration burden. This work introduces Manipulation Primitive-augmented reinforcement Learning (MAPLE), a learning framework that augments standard reinforcement learning algorithms with a pre-defined library of behavior primitives. These behavior primitives are robust functional modules specialized in achieving manipulation goals, such as grasping and pushing. To use these heterogeneous primitives, we develop a hierarchical policy that involves the primitives and instantiates their executions with input parameters. We demonstrate that MAPLE out-performs baseline approaches by a significant margin on a suite of simulated manipulation tasks. We also quantify the compositional structure of the learned behaviors and highlight our method's ability to transfer policies to new task variants and to physical hardware. Videos and code are available at https://ut-austin-rpl.github.io/maple
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Binary Codes capable of currecting deletions, insertions, and reversals
Levenshtein, V. I. (1965) · Soviet physics. Doklady · Cited by 14,804
Proximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625
Soft actor-critic Off-policy maximum entropy deep reinforcement learning with a stochastic actor
Haarnoja, T., Zhou, A., Abbeel, P., & Levine, S. (2018) · arXiv · doi:10.48550/arxiv.1801.01290 · Cited by 5,594
Sampling-based algorithms for optimal motion planning
Karaman, S., & Frazzoli, E. (2011) · The International Journal of Robotics Research · doi:10.1177/0278364911406761 · Cited by 5,275
Categorical Reparameterization with Gumbel-Softmax
Jang, E., Gu, S., & Poole, B. (2016) · arXiv · doi:10.48550/arxiv.1611.01144 · Cited by 3,616
Curiosity-Driven Exploration by Self-Supervised Prediction
Pathak, D., Agrawal, P., Efros, A. A., & Darrell, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.70 · Cited by 1,815
Learning dexterous in-hand manipulation
Andrychowicz, O. M., Baker, B., Chociej, M., Józefowicz, R., McGrew, B., Pachocki, J., Petron, A., Plappert, M., Powell, G., Ray, A., et al. (2020) · The International Journal of Robotics Research · doi:10.1177/0278364919887447 · Cited by 1,643
Dynamical Movement Primitives: Learning Attractor Models for Motor Behaviors
Ijspeert, A. J., Nakanishi, J., Hoffmann, H., Pastor, P., & Schaal, S. (2013) · Neural Computation · doi:10.1162/neco_a_00393 · Cited by 1,598
The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
Maddison, C. J., Mnih, A., & Teh, Y. W. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.00712 · Cited by 1,359
Mahler, J., Liang, J., Niyaz, S., Laskey, M., Doan, R., Liu, X., Aparicio, J., & Goldberg, K. (2017) · Robotics: Science and Systems XIII · doi:10.15607/rss.2017.xiii.058 · Cited by 1,268
Research works printed but not linked: 5
- zhu, 2020, Robosuite: A modular simulation framework and benchmark for robot learning, ArXiv Preprint
- yamada, 0, Motion planner augmented reinforcement learning for obstructed environments, CoRL
- simeonov, 0, Learning to combine primitive skills: A step towards versatile robotic manipulation, CoRL
- sharma, 0, Learning to combine primitive skills: A step towards versatile robotic manipulation, CoRL
- neumann, 2014, Learning to combine primitive skills: A step towards versatile robotic manipulation, Frontiers in Computational Neuroscience
Cited by
A Survey on Deep Reinforcement Learning Algorithms for Robotic Manipulation
Han, D., Mulyana, B., Stankovic, V., & Cheng, S. (2023) · Sensors · doi:10.3390/s23073762 · Cited by 253
Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes
Tang, C., Abbatematteo, B., Hu, J., Chandra, R., Martín-Martín, R., & Stone, P. (2025) · Annual Review of Control, Robotics, and Autonomous Systems · doi:10.1146/annurev-control-030323-022510 · Cited by 177
A review on reinforcement learning for contact-rich robotic manipulation tasks
Elguea-Aguinaco, Í., Serrano-Muñoz, A., Chrysostomou, D., Inziarte-Hidalgo, I., Bøgh, S., & Arana-Arexolaleiba, N. (2023) · Robotics and Computer-Integrated Manufacturing · doi:10.1016/j.rcim.2022.102517 · Cited by 167
Generative Skill Chaining: Long-Horizon Skill Planning with Diffusion Models
Mishra, U. A., Xue, S., Chen, Y., & Xu, D. (2024) · arXiv
Affordances from Human Videos as a Versatile Representation for Robotics
Bahl, S., Mendonca, R., Chen, L., Jain, U., & Pathak, D. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01324 · Cited by 119
“It has been common to introduce some form of prior derived from human knowledge, which could take the form of corrective interactions [23, 42, 68], structured policy spaces [2, 7, 7, 17, 50, 85, 94, 99, 108, 125], offline robotics data [25, 56, 57, 71, 97], using pretrained visual representations [84,89,106,123,124] o…”
Background
Metadata
- Authors
- Nasiriany, S., Liu, H., & Zhu, Y.
- Affiliations
- The University of Texas at Austin (US)
- ORCID
- Venue
- 2022 International Conference on Robotics and Automation (ICRA)
- Publisher
- IEEE
- Published
- 2022-05-23
- Pages
- 7477-7484
- ISSN
- 1050-4729 (ISSN-L)
- OpenAlex
- W3204657926
- Cited by
- 91
- References
- 89
- Citations by year
- 2026: 10 · 2025: 30 · 2024: 27 · 2023: 20 · 2022: 4
- FWCI
- 11.20
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Proceedings - IEEE International Conference on Robotics and Automation/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- NSF (CNS-1955523)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Keywords
- manipulation tasks, hierarchical reinforcement learning, long-horizon tasks, exploration efficiency, grasping, pushing, policy transfer, robotic manipulation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Augmenting Reinforcement Learning with Behavior Primitives for Diverse Manipulation Tasks. https://app.cochat.ai/papers/10.1109/icra46639.2022.9812140
- The paper
- Nasiriany, S., Liu, H., & Zhu, Y. (2022). Augmenting Reinforcement Learning with Behavior Primitives for Diverse Manipulation Tasks. 2022 International Conference on Robotics and Automation (ICRA). https://doi.org/10.1109/icra46639.2022.9812140
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.