The DOI matches its DataCite record; 43 of the 43 references it cites were found in a registry or an index.
Residual Policy Learning
Open accessAbstract
We present Residual Policy Learning (RPL): a simple method for improving nondifferentiable policies using model-free deep reinforcement learning. RPL thrives in complex robotic manipulation tasks where good but imperfect controllers are available. In these tasks, reinforcement learning from scratch remains data-inefficient or intractable, but learning a residual on top of the initial controller can yield substantial improvements. We study RPL in six challenging MuJoCo tasks involving partial observability, sensor noise, model misspecification, and controller miscalibration. For initial controllers, we consider both hand-designed policies and model-predictive controllers with known or learned transition models. By combining learning with control algorithms, RPL can perform long-horizon, sparse-reward tasks for which reinforcement learning alone fails. Moreover, we find that RPL consistently and substantially improves on the initial controllers. We argue that RPL is a promising approach for combining the complementary strengths of deep reinforcement learning and robotic control, pushing the boundaries of what either can achieve independently. Video and code at https://k-r-allen.github.io/residual-policy-learning/.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“the key design is a single residual-RL policy (Johannink et al., 2019;Silver et al., 2018) trained jointly across all demonstrations, which repairs kinematics rather than learning tasks, and therefore needs no per-task training and can synthesize additional physically consistent variants of each demonstration.”
From Dexterous Robot Manipulation from Human Demonstrations via Contact-Anchored Retargeting and Residual Policy Learning (2026) · cites this paper as Silver et al., 2018
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Markov decision processes: Discrete stochastic dynamic programming
Hazeghi, K., & Puterman, M. L. (1995) · Journal of the American Statistical Association · doi:10.2307/2291177 · Cited by 8,037
MuJoCo: A physics engine for model-based control
Todorov, E., Erez, T., & Tassa, Y. (2012) · 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2012.6386109 · Cited by 4,767
Trust region policy optimization
Schulman, J., Levine, S., Moritz, P., Jordan, M. I., & Abbeel, P. (2015) · arXiv · doi:10.48550/arxiv.1502.05477 · Cited by 4,578
The unscented Kalman filter for nonlinear estimation
Wan, E. A., & van der Merwe, R. (2002) · IEEE Adaptive Systems for Signal Processing, Communications, and Control Symposium (AS-SPCC) · doi:10.1109/asspcc.2000.882463 · Cited by 4,373
Apprenticeship learning via inverse reinforcement learning
Abbeel, P., & Ng, A. Y. (2004) · International Conference on Machine Learning · doi:10.1145/1015330.1015430 · Cited by 2,855
Soft Actor-Critic Algorithms and Applications
Haarnoja, T., Zhou, A., Hartikainen, K., Tucker, G., Ha, S., Tan, J., Kumar, V., Zhu, H., Gupta, A., Abbeel, P., et al. (2018) · arXiv · doi:10.48550/arxiv.1812.05905 · Cited by 1,994
A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning
Ross, S., Gordon, G. J., & Bagnell, J. A. (2018) · Carnegie Mellon University · doi:10.1184/r1/6550949 · Cited by 1,941
Maximum Entropy Inverse Reinforcement Learning
Ziebart, B. D., Maas, A. L., Bagnell, J. A., & Dey, A. K. (2018) · doi:10.1184/r1/6555512 · Cited by 1,868
Curiosity-Driven Exploration by Self-Supervised Prediction
Pathak, D., Agrawal, P., Efros, A. A., & Darrell, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.70 · Cited by 1,815
Research works printed but not linked: 5
- [2][2] Dmitry Kalashnikov, Alex Irpan, Peter Pastor, Julian Ibarz, Alexander Herzog, Eric Jang, Deirdre Quillen, Ethan Holly, Mrinal Kalakrishnan, Vincent Vanhoucke, et al. Scalable deep reinforcement learning for visionbased robotic manipulation. In Conference on Robot Learning , pages 651-673, 2018. -
- [5][5] Marcin Andrychowicz, Filip Wolski, Alex Ray, Jonas Schneider, Rachel Fong, Peter Welinder, Bob McGrew, Josh Tobin, OpenAI Pieter Abbeel, and Wojciech Zaremba. Hindsight experience replay. In Advances in Neural Information Processing Systems , pages 50485058, 2017. -
- [9][9] Shixiang Gu, Timothy Lillicrap, Ilya Sutskever, and Sergey Levine. Continuous deep Q-learning with modelbased acceleration. In International Conference on Machine Learning , pages 2829-2838, 2016. -
- [13][13] Anusha Nagabandi, Gregory Kahn, Ronald S Fearing, and Sergey Levine. Neural network dynamics for modelbased deep reinforcement learning with model-free finetuning. In IEEE International Conference on Robotics and Automation (ICRA) , pages 7559-7566. IEEE, 2018. -
- [24][24] Filipe de Avila Belbute-Peres, Kevin Smith, Kelsey Allen, Josh Tenenbaum, and J Zico Kolter. End-toend differentiable physics for learning and control. In Advances in Neural Information Processing Systems , pages 7176-7187, 2018. -
Cited by
Sorted by citations
How to train your robot with deep reinforcement learning: Lessons we have learned
Ibarz, J., Tan, J., Finn, C., Kalakrishnan, M., Pastor, P., & Levine, S. (2021) · The International Journal of Robotics Research · doi:10.1177/0278364920987859 · Cited by 570
Residual Reinforcement Learning for Robot Control
Johannink, T., Bahl, S., Nair, A., Luo, J., Kumar, A., Loskyll, M., Ojea, J. A., Solowjow, E., & Levine, S. (2019) · 2019 International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2019.8794127 · Cited by 480
TossingBot: Learning to Throw Arbitrary Objects With Residual Physics
Zeng, A., Song, S., Lee, J., Rodríguez, A., & Funkhouser, T. A. (2020) · IEEE Transactions on Robotics · doi:10.1109/tro.2020.2988642 · Cited by 299
Deep Reinforcement Learning for Industrial Insertion Tasks with Visual Inputs and Natural Rewards
Schoettler, G., Nair, A., Luo, J., Bahl, S., Ojea, J. A., Solowjow, E., & Levine, S. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9341714 · Cited by 191
Side-Tuning: A Baseline for Network Adaptation via Additive Side Networks
Zhang, J. O., (Sasha), A. S., Zamir, A., Guibas, L. J., & Malik, J. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58580-8_41 · Cited by 212
Coarse-to-Fine Imitation Learning: Robot Manipulation from a Single Demonstration
Johns, E. (2021) · 2021 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra48506.2021.9560942 · Cited by 119
TossingBot: Learning to Throw Arbitrary Objects with Residual Physics
Zeng, A., Song, S., Lee, J., Rodriquez, A., & A.Funkouser, T. (2019) · Robotics: Science and Systems · doi:10.15607/rss.2019.xv.004 · Cited by 110
Mohammed, M. Q., Kwek, L. C., & Chua, S. C. (2020) · IEEE Access · doi:10.1109/access.2020.3027923 · Cited by 111
Meta-Reinforcement Learning for Robotic Industrial Insertion Tasks
Schoettler, G., Nair, A., Ojea, J. A., Levine, S., & Solowjow, E. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9340848 · Cited by 88
Injecting knowledge in data-driven vehicle trajectory predictors
Bahari, M., Nejjar, I., & Alahi, A. (2021) · Transportation Research Part C Emerging Technologies · doi:10.1016/j.trc.2021.103010 · Cited by 80
Ohn-Bar, E., Prakash, A., Behl, A., Chitta, K., & Geiger, A. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01131 · Cited by 76
Karkaria, V. N., Tsai, Y.-K., Chen, Y.-P., & Chen, W. (2025) · Engineering Optimization · doi:10.1080/0305215x.2024.2434201 · Cited by 78
Bi-Directional Domain Adaptation for Sim2Real Transfer of Embodied Navigation Agents
Truong, J., Chernova, S., & Batra, D. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3062303 · Cited by 78
Physics-Based Dexterous Manipulations with Estimated Hand Poses and Residual Reinforcement Learning
Garcia-Hernando, G., Johns, E., & Kim, T. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9340947 · Cited by 61
Development of residual learning in deep neural networks for computer vision: A survey
Xu, G., Wang, X., Wu, X., Leng, X., & Xu, Y. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.109890 · Cited by 104
Lee, M. A., Florensa, C., Tremblay, J., Ratliff, N., Garg, A., Ramos, F., & Fox, D. (2020) · 2020 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra40945.2020.9197125 · Cited by 46
Differentiable Algorithm Networks for Composable Robot Learning
Karkus, P., Ma, X., Hsu, D., Kaelbling, L. P., Lee, W. S., & Lozano‐Pérez, T. (2019) · Robotics: Science and Systems · doi:10.15607/rss.2019.xv.039 · Cited by 57
Self-citationInsertionNet - A Scalable Solution for Insertion
Spector, O., & Castro, D. D. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3076971 · Cited by 51
Challenges of Reinforcement Learning
Ding, Z., & Dong, H. (2020) · Springer eBooks · doi:10.1007/978-981-15-4095-0_7 · Cited by 52
Robot Learning of Mobile Manipulation With Reachability Behavior Priors
Jauhri, S., Peters, J., & Chalvatzaki, G. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3188109 · Cited by 49
Luo, Y., Wang, Y., Dong, K., Zhang, Q., Cheng, E., Sun, Z., & Song, B. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.126620 · Cited by 46
Residual Policy Learning Facilitates Efficient Model-Free Autonomous Racing
Zhang, R., Hou, J. C., Chen, G., Li, Z., Chen, J., & Knoll, A. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3192770 · Cited by 42
Machine learning meets advanced robotic manipulation
Nahavandi, S., Alizadehsani, R., Nahavandi, D., Lim, C. P., Kelly, K., & Bello, F. (2024) · Information Fusion · doi:10.1016/j.inffus.2023.102221 · Cited by 46
A Comparison of Action Spaces for Learning Manipulation Tasks
Varin, P., Grossman, L., & Kuindersma, S. (2019) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8967946 · Cited by 37
Learning Hierarchical Control for Robust In-Hand Manipulation
Li, T., Srinivasan, K., Meng, M. Q. ‐., Yuan, W., & Bohg, J. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9197343 · Cited by 39
Sample-efficient Reinforcement Learning in Robotic Table Tennis
Tebbe, J., Krauch, L., Gao, Y., & Zell, A. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9560764 · Cited by 39
Adaptive Control of a Mechatronic System Using Constrained Residual Reinforcement Learning
Staessens, T., Lefebvre, T., & Crevecoeur, G. (2022) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2022.3144565 · Cited by 33
TuneNet: One-Shot Residual Tuning for System Identification and Sim-to-Real Robot Task TransferPreprint
Allevato, A., Short, E. S., Pryor, M., & Thomaz, A. L. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.11200 · Cited by 34
Physical enhanced residual learning (PERL) framework for vehicle trajectory prediction
Long, K., Sheng, Z., Shi, H., Li, X., Chen, S., & Ahn, S. (2025) · Communications in Transportation Research · doi:10.1016/j.commtr.2025.100166 · Cited by 38
Affordance Learning from Play for Sample-Efficient Policy Learning
Borja-Diaz, J., Mees, O., Kalweit, G., Hermann, L., Boedecker, J., & Burgard, W. (2022) · 2022 International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra46639.2022.9811889 · Cited by 32
Bayesian controller fusion: Leveraging control priors in deep reinforcement learning for robotics
Rana, K., Dasagi, V., Haviland, J., Talbot, B., Milford, M. J., & Sünderhauf, N. (2023) · The International Journal of Robotics Research · doi:10.1177/02783649231167210 · Cited by 40
Residual Policy Learning for Shared Autonomy
Schaff, C., & Walter, M. R. (2020) · Robotics: Science and Systems · doi:10.15607/rss.2020.xvi.072 · Cited by 30
Naunheim, S., Kuhl, Y., Schug, D., Schulz, V., & Mueller, F. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3323131 · Cited by 26
Funk, N., Schaff, C., Madan, R., Yoneda, T., Urain, J., Watson, J., Gordon, E. K., Widmaier, F., Bauer, S., Srinivasa, S. S., et al. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3129139 · Cited by 24
Evolving Rewards to Automate Reinforcement LearningPreprint
Faust, A., Francis, A. G., Jr., & Mehta, D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.07628 · Cited by 27
Rana, K., Talbot, B., Dasagi, V., Milford, M. J., & Sünderhauf, N. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9197386 · Cited by 23
Hierarchical Policies for Cluttered-Scene Grasping With Latent Plans
Wang, L., Meng, X., Yu, X., & Fox, D. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3143198 · Cited by 22
Iterative residual tuning for system identification and sim-to-real robot learning
Allevato, A., Short, E. S., Pryor, M., & Thomaz, A. L. (2020) · Autonomous Robots · doi:10.1007/s10514-020-09925-w · Cited by 21
Deep Residual Reinforcement Learning based Autonomous Blimp Control
Liu, Y. T., Price, E., Black, M. J., & Ahmad, A. (2022) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros47612.2022.9981182 · Cited by 24
Search-based task and motion planning for hybrid systems: Agile autonomous vehicles
Ajanović, Z., Regolin, E., Shyrokau, B. N., Ćatić, H., Horn, M., & Ferrara, A. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.105893 · Cited by 19
Hou, J. C., Chen, G., Li, Z., He, W., Gu, S., Knoll, A., & Jiang, C. (2023) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2023.3312647 · Cited by 19
Shaping Rewards for Reinforcement Learning with Imperfect Demonstrations using Generative Models
Wu, Y., Mozifian, M., & Shkurti, F. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9561333 · Cited by 17
Residual Physics and Post-Posed Shielding for Safe Deep Reinforcement Learning Method
Zhang, Q., Mahbod, M. H. B., Chng, C.-B., Lee, P. S., & Chui, C. (2022) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2022.3178084 · Cited by 17
Handling Sparse Rewards in Reinforcement Learning Using Model Predictive Control
Dawood, M., Dengler, N., de Heuvel, J., & Bennewitz, M. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10161492 · Cited by 14
Implicit Kinematic Policies: Unifying Joint and Cartesian Action Spaces in End-to-End Robot Learning
Ganapathi, A., Florence, P., Varley, J., Burns, K., Goldberg, K. Y., & Zeng, A. (2022) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra46639.2022.9812165 · Cited by 18
Residual Reinforcement Learning from DemonstrationsPreprint
Alakuijala, M., Dulac-Arnold, G., Mairal, J., Ponce, J., & Schmid, C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.08050 · Cited by 14
Review of Reinforcement Learning for Robotic Grasping: Analysis and Recommendations
Sekkat, H., Moutik, O., Ourabah, L., Elkari, B., Chaibi, Y., & Tchakoucht, T. A. (2023) · Statistics Optimization & Information Computing · doi:10.19139/soic-2310-5070-1797 · Cited by 17
RL-augmented MPC Framework for Agile and Robust Bipedal Footstep Locomotion Planning and Control
Bang, S. H., Jové, C. A., & Sentis, L. (2024) · IEEE-RAS International Conference on Humanoid Robots (Humanoids) · doi:10.1109/humanoids58906.2024.10769914 · Cited by 12
Sample-Efficient Learning for Industrial Assembly using Qgraph-bounded DDPG
Hoppe, S., Giftthaler, M., Krug, R., & Toussaint, M. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9341390 · Cited by 12
Residual Feedback Learning for Contact-Rich Manipulation Tasks with Uncertainty
Ranjbar, A., Vien, N. A., Ziesche, H., Boedecker, J., & Neumann, G. M. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9636176 · Cited by 12
Lv, J., Feng, Y., Zhang, C., Zhao, S., Shao, L., & Lu, C. (2023) · Robotics: Science and Systems · doi:10.15607/rss.2023.xix.040 · Cited by 22
RESPRECT: Speeding-up Multi-Fingered Grasping With Residual Reinforcement Learning
Ceola, F., Rosasco, L., & Natale, L. (2024) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3363532 · Cited by 11
Pou, B., Smith, J., Quiñones, E., Martín, M., & Gratadour, . (2024) · Optics Express · doi:10.1364/oe.530254 · Cited by 11
Qu, G., Yu, C., Low, S. H., & Wierman, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.07476 · Cited by 11
Learning Vision-based Reactive Policies for Obstacle AvoidancePreprint
Aljalbout, E., Ji, C., Ritt, K., Ulmer, M., & Haddadin, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.16298 · Cited by 11
CusADi: A GPU Parallelization Framework for Symbolic Expressions and Optimal Control
Jeon, S. H., Hong, S., Lee, H. J., Khazoom, C., & Kim, S. (2024) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3512254 · Cited by 10
Adaptive policy learning for data-driven powertrain control with eco-driving
Kerbel, L., Ayalew, B., Ivančo, A., & Loiselle, K. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.106489 · Cited by 10
Cascaded Compositional Residual Learning for Complex Interactive Behaviors
Kumar, K. N., Essa, I. A., & Ha, S. (2023) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2023.3286171 · Cited by 12
Mulet, B. P., Smith, J., Quiñones, E., Martín, M., & Gratadour, . (2022) · Adaptive Optics Systems VIII · doi:10.1117/12.2627849 · Cited by 9
Wang, Y., Bai, Y., & Svinin, M. M. (2021) · International Journal of Control Automation and Systems · doi:10.1007/s12555-020-0412-y · Cited by 9
Leveraging control priors in deep reinforcement learning for robotics
Rana, K. (2023) · Queensland University of Technology · doi:10.5204/thesis.eprints.238632 · Cited by 9
Research on Robot Massage Force Control Based on Residual Reinforcement Learning
Xiao, M., Zhang, T., Zou, Y., Chen, S., & Wu, W. R. (2023) · IEEE Access · doi:10.1109/access.2023.3347416 · Cited by 8
Liu, Q., Guo, Y., Deng, L., Liu, H., Li, D., & Sun, H. (2024) · IEEE Transactions on Sustainable Energy · doi:10.1109/tste.2024.3454080 · Cited by 8
Design and Control of Roller Grasper V3 for In-Hand Manipulation
Yuan, S., Shao, L., Feng, Y., Sun, J., Xue, T., Yako, C. L., Bohg, J., & Salisbury, J. K. (2024) · IEEE Transactions on Robotics · doi:10.1109/tro.2024.3454388 · Cited by 10
Qu, G.-N., Yu, C., Low, S., & Wierman, A. (2021) · IEEE Conference on Decision and Control · doi:10.1109/cdc45484.2021.9683735 · Cited by 16
Grasp and Motion Planning for Dexterous Manipulation for the Real Robot ChallengePreprint
Yoneda, T., Schaff, C., Maeda, T., & Walter, M. R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.02842 · Cited by 8
Deep Model Predictive Optimization
Sacks, J., Rana, R., Huang, K., Spitzer, A., Shi, G., & Boots, B. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10611492 · Cited by 7
Composable Energy Policies for Reactive Motion Generation and Reinforcement Learning
Urain, J., Liu, P., Li, A., D’Eramo, C., & Peters, J. (2021) · Robotics: Science and Systems · doi:10.15607/rss.2021.xvii.052 · Cited by 7
Trumpp, R., Javanmardi, E., Nakazato, J., Tsukada, M., & Caccamo, M. (2024) · 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros58592.2024.10801657 · Cited by 6
Bootstrapping Motor Skill Learning with Motion Planning
Abbatematteo, B., Rosen, E., Tellex, S., & Konidaris, G. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9636661 · Cited by 6
Zhang, R., Chen, G., Hou, J. C., Li, Z., & Knoll, A. (2022) · IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI) · doi:10.1109/mfi55806.2022.9913862 · Cited by 6
Adapting Object-Centric Probabilistic Movement Primitives with Residual Reinforcement Learning
Carvalho, J., Koert, D., Daniv, M., & Peters, J. (2022) · IEEE-RAS International Conference on Humanoid Robots (Humanoids) · doi:10.1109/humanoids53995.2022.10000148 · Cited by 6
Reinforcement Learning Based User-Specific Shared Control Navigation in Crowds
Zhang, B., Holloway, C., & Carlson, T. (2023) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc53992.2023.10394139 · Cited by 6
Shaping Rewards for Reinforcement Learning with Imperfect Demonstrations using Generative ModelsPreprint
Wu, Y., Mozifian, M., & Shkurti, F. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.01298 · Cited by 6
Wang, C., Su, C., Sun, B., Chen, G., & Xie, L. (2024) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2024.1355170 · Cited by 5
Lv, J., Feng, Y., Zhang, C., Zhao, S., Shao, L., & Lu, C. (2024) · The International Journal of Robotics Research · doi:10.1177/02783649241284653 · Cited by 7
Residual Model Learning for Microrobot Control
Gruenstein, J., Chen, T., Doshi, N., & Agrawal, P. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9560862 · Cited by 6
Learning hierarchical control for robust in-hand manipulation
Li, T. (2022) · Elsevier eBooks · doi:10.1016/b978-0-32-390445-2.00017-9 · Cited by 6
Tolerance-Guided Policy Learning for Adaptable and Transferrable Delicate Industrial InsertionPreprint
Niu, B., Wang, C., & Liu, C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.02303 · Cited by 5
Abbas, A. N., Chasparis, G. C., & Kelleher, J. D. (2024) · Research Square · doi:10.21203/rs.3.rs-3918353/v1 · Cited by 5
Doan, H. T. L., Arita, H., & Tahara, K. (2024) · ROBOMECH Journal · doi:10.1186/s40648-024-00273-3 · Cited by 4
Mixtures of Experts for Scaling up Neural Networks in Order Execution
Li, K., Cucuringu, M., Sánchez-Betancourt, L., & Willi, T. (2024) · ACM International Conference on AI in Finance (ICAIF) · doi:10.1145/3677052.3698691 · Cited by 4
Lee, G., Hou, B., Choudhury, S., & Srinivasa, S. S. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9636163 · Cited by 5
Efficient Multi-Task Learning via Iterated Single-Task Transfer
Zentner, K. R., Puri, U., Zhang, Y., Julian, R., & Sukhatme, G. S. (2022) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros47612.2022.9981244 · Cited by 5
Residual Policy Learning for Powertrain Control
Kerbel, L., Ayalew, B., Ivančo, A., & Loiselle, K. (2022) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2022.10.270 · Cited by 4
An offline learning of behavior correction policy for vision-based robotic manipulation
Dong, Q., Kaneko, T., & Sugiyama, M. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10610177 · Cited by 3
Residual reinforcement learning for logistics cart transportation
Matsuo, R., Yasuda, S., Kumagai, T., Sato, N., Yoshida, H., & Yairi, T. (2022) · Advanced Robotics · doi:10.1080/01691864.2022.2046504 · Cited by 3
HR-Planner: A Hierarchical Highway Tactical Planner based on Residual Reinforcement Learning
Wu, H., Li, Y., Zhuang, H., Wang, C., & Yang, M. Z. (2022) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra46639.2022.9812400 · Cited by 3
Autonomous Blimp Control via $H_{\infty}$ Robust Deep Residual Reinforcement Learning
Zuo, Y., Liu, Y. T., & Ahmad, A. (2023) · IEEE International Conference on Automation Science and Engineering (CASE) · doi:10.1109/case56687.2023.10260561 · Cited by 4
PeRP: Personalized Residual Policies For Congestion Mitigation Through Co-operative Advisory Systems
Hasan, A., Chakraborty, N., Chen, H., Cho, J.-H., Wu, C., & Driggs-Campbell, K. (2023) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc57777.2023.10422444 · Cited by 3
Long-Horizon Prediction and Uncertainty Propagation with Residual Point Contact Learners
Fazeli, N., Ajay, A., & Rodríguez, A. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9196511 · Cited by 3
Vessel-to-Vessel Motion Compensation with Reinforcement Learning
Herland, S., & Bach, K. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i13.26860 · Cited by 2
CLAMGen: Closed-Loop Arm Motion Generation via Multi-view Vision-Based RL
Akinola, I., Wang, Z., & Allen, P. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9636369 · Cited by 2
Robust nonlinear set-point control with reinforcement learning
Zhang, R., Mattsson, P., & Wigren, T. (2023) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.23919/acc55779.2023.10156038 · Cited by 2
Model Based Residual Policy Learning with Applications to Antenna Control
Möllerstedt, V. E., Russo, A., & Bouton, M. (2024) · IEEE International Conference on Machine Learning for Communication and Networking (ICMLCN) · doi:10.1109/icmlcn59089.2024.10624756 · Cited by 2
Cooperative Advisory Residual Policies for Congestion Mitigation
Hasan, A., Chakraborty, N., Chen, H., Cho, J.-H., Wu, C., & Driggs-Campbell, K. (2024) · ACM Journal on Autonomous Transportation Systems · doi:10.1145/3699519 · Cited by 3
Towards a Manipulator System for Disposal of Waste from Patients Undergoing Chemotherapy
Li, H.-Y., Ho, L. S., Athukorala, A., Lu, W. Y., Dharmawan, A. G., Guo, J. L. F., Tan, M. M. L., Wong, K. C., Ng, N. S., Tan, M. M. X., et al. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9636865 · Cited by 2
Aiding reinforcement learning for set point control
Zhang, R., Mattsson, P., & Wigren, T. (2023) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2023.10.1220 · Cited by 2
Optimizing Attention and Cognitive Control Costs Using Temporally Layered Architectures
Patel, D., B, T. J. S. A., & Siegelmann, H. T. (2024) · Neural Computation · doi:10.1162/neco_a_01718 · Cited by 2
Elaborating on Learned Demonstrations with Temporal Logic Specifications
Innes, C., & Ramamoorthy, S. (2020) · Robotics: Science and Systems · doi:10.15607/rss.2020.xvi.004 · Cited by 2
Metadata
- Authors
- Silver, T., Allen, K., Tenenbaum, J., & Kaelbling, L.
- Affiliations
- Massachusetts Institute of Technology (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2018
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1812.06298
- OpenAlex
- W2905364877
- Cited by
- 113
- References
- 40
- Citations by year
- 2026: 2 · 2025: 3 · 2024: 21 · 2023: 20 · 2022: 16 · 2021: 25 · 2020: 20 · 2019: 6
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (1723381, 1523767, 1231216)
- Companhia Brasileira de Metalurgia e Mineração (CCF-1231216)
- Natural Sciences and Engineering Research Council of Canada
- Office of Naval Research (N00014-18, N000141310333, N00014-18-1-2847, FA9550, N00014)
- Air Force Office of Scientific Research (FA9550-17, FA9550-, FA9550-17-1, FA9550-17-1-0165)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
- Physical Sciences › Engineering › Control and Systems Engineering › Adaptive Dynamic Programming Control
- Keywords
- model-free deep reinforcement learning, robotic manipulation, model predictive control, sparse rewards, long-horizon tasks, sensor noise, model misspecification, partial observability, MuJoCo
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
