Learning deep control policies for autonomous aerial vehicles with MPC-guided policy search
Open accessAbstract
Model predictive control (MPC) is an effective method for controlling robotic systems, particularly autonomous aerial vehicles such as quadcopters. However, application of MPC can be computationally demanding, and typically requires estimating the state of the system, which can be challenging in complex, unstructured environments. Reinforcement learning can in principle forego the need for explicit state estimation and acquire a policy that directly maps sensor readings to actions, but is difficult to apply to unstable systems that are liable to fail catastrophically during training before an effective policy has been found. We propose to combine MPC with reinforcement learning in the framework of guided policy search, where MPC is used to generate data at training time, under full state observations provided by an instrumented training environment. This data is used to train a deep neural network policy, which is allowed to access only the raw observations from the vehicle's onboard sensors. After training, the neural network policy can successfully control the robot without knowledge of the full state, and at a fraction of the computational cost of MPC. We evaluate our method by learning obstacle avoidance policies for a simulated quadrotor, using simulated onboard sensors and no explicit state estimation at test time.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The frequent sensing and re-planning in MPC allow the robot to react dynamically to changes in the environment while still making more intelligent choices than a purely reactive system.
“The frequent sensing and re-planning allow the robot to react dynamically to changes in the environment, while still making more intelligent choices than a purely reactive system (Erez et al., 2011; Lenz et al., 2015; Zhang et al., 2016; Liu et al., 2017; Williams et al., 2017; Gillespie et al., 2018; Amos et al., 2019; Kabzan et al., 2019).”
From Lessons for Robotics From the Control Architecture of the Octopus (2022) · cites this paper as Zhang et al., 2016
FPGAs show great improvements in both performance and power consumption in deep learning applications.
“AField-Programmable Gate Array (FPGA) is another viable solution, showing great improvements in both performance and power consumption in deep learning applications.”
From A survey of deep learning techniques for autonomous driving (2020) · cites this paper as [170]
The largest portion of papers in deep learning for self-driving cars are focused mainly on perception and End2End learning.
“Currently, the largest portion of papers in deep learning for selfdriving cars are focused mainly on perception and End2End learning [81, 124].”
From A survey of deep learning techniques for autonomous driving (2020) · cites this paper as [124]
Real-time requirements have to be fulfilled for processing large amounts of data from the car's sensors suite and for updating parameters of deep learning systems over high-speed communication lines.
“Finally, realtime requirements have to be fulfilled for processing the large amounts of data gathered from the car's sensors suite, as well as for updating the parameters of deep learning systems over high-speed communication lines [170].”
From A survey of deep learning techniques for autonomous driving (2020) · cites this paper as [170]
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 2× · see citing works
“We used default values for all learning parameters as presented in [14].”
Method(2010) · International Conference on Machine Learning · Cited by 13,995 · see citing works
Playing Atari with Deep Reinforcement Learning
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013) · arXiv · doi:10.48550/arxiv.1312.5602 · Cited by 9,471 · cited 1× · see citing works
“More recently, deep neural network policies have been used for tasks ranging from robotic control [22], [23] to video game playing [27].”
MethodMuJoCo: A physics engine for model-based control
Todorov, E., Erez, T., & Tassa, Y. (2012) · 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2012.6386109 · Cited by 4,767 · cited 2× · see citing works
“However, our derivation largely follows prior work [41].”
MethodResultReinforcement learning in robotics: A survey
Kober, J., Bagnell, J. A., & Peters, J. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913495721 · Cited by 3,160 · cited 1× · see citing works
“An overview of recent reinforcement learning methods in robotics can be found in a recent survey paper [15].”
BackgroundEnd-to-end training of deep visuomotor policies
Levine, S., Finn, C., Darrell, T., & Abbeel, P. (2016) · Journal of Machine Learning Research · Cited by 2,192 · see citing works
Self-citationA Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning
Ross, S., Gordon, G. J., & Bagnell, J. A. (2018) · Carnegie Mellon University · doi:10.1184/r1/6550949 · Cited by 1,941 · see citing works
Neural networks for control systems—A survey
Hunt, K. J., Sbarbaro, D. G., Żbikowski, R., & Gawthrop, P. J. (1992) · Automatica · doi:10.1016/0005-1098(92)90053-i · Cited by 1,866 · cited 1× · see citing works
“neural networks have been used for control for decades [12], [33], limitations on computation and algorithms have made large neural network policies very difficult to learn.”
MethodRobust model predictive control of constrained linear systems with bounded disturbances
Mayne, D. Q., Seron, M. M., & Raković, S. V. (2004) · Automatica · doi:10.1016/j.automatica.2004.08.019 · Cited by 1,680 · cited 1× · see citing works
“Model predictive control (MPC) is an effective and reliable method for controlling robotic systems, particularly autonomous aerial vehicles such as quadcopters, because of its robustness to moderate model errors [26], ability to use high-level objectives [41], and relative simplicity.”
MethodEnd-to-End Training of Deep Visuomotor PoliciesPreprint
Levine, S., Finn, C., Darrell, T., & Abbeel, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1504.00702 · Cited by 1,502 · cited 10× · see citing works
“In this paper, we use the BADMM version, which specifies the following objective for trajectory optimization: min pi(τ) T∑ t=1 Epi(xt,ut) [ `(xt,ut)− uTt λiµt + νitDKL ( pi(ut|xt)‖πθ(ut|xt) )] (1) where λµt is a Lagrange multiplier on the mean action, and the third term is a KL-divergence penalty.”
BackgroundMethodInfluentialSelf-citationALVINN, an autonomous land vehicle in a neural network
Pomerleau, D. A. (2018) · doi:10.1184/r1/6603146 · Cited by 1,403 · see citing works
ALVINN: An autonomous land vehicle in a neural network
Pomerleau, D. A. (2018) · Carnegie Mellon University · doi:10.1184/r1/6603146.v1 · Cited by 1,340 · see citing works
Reinforcement learning of motor skills with policy gradients
Peters, J., & Schaal, S. (2008) · Neural Networks · doi:10.1016/j.neunet.2008.02.003 · Cited by 831 · cited 1× · see citing works
“While this method is not as general as policy gradient RL algorithms, which can optimize arbitrary parameterized policies with arbitrary observations under unknown dynamics [32], it is fast, simple, and often effective [39].”
MethodA Survey on Policy Search for Robotics
Deisenroth, M. P. (2011) · Foundations and Trends in Robotics · doi:10.1561/2300000021 · Cited by 815 · cited 2× · see citing works
“On the other end of the spectrum from such model-based methods, reinforcement learning (RL) aims to directly learn control policies that map observations to controls [5].”
BackgroundAn integral predictive/nonlinear H ∞ control structure for a quadrotor helicopter
Raffo, G. V., Ortega, M. G., & Rubio, F. R. (2009) · Automatica · doi:10.1016/j.automatica.2009.10.018 · Cited by 795 · cited 1× · see citing works
“quently used to control autonomous aerial vehicles such as quadrotors [38], [35], [34], [3], [2], [4], [29].”
Synthesis and stabilization of complex behaviors through online trajectory optimization
Tassa, Y., Erez, T., & Todorov, E. (2012) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2012.6386025 · Cited by 774 · cited 2× · see citing works
“While this method is not as general as policy gradient RL algorithms, which can optimize arbitrary parameterized policies with arbitrary observations under unknown dynamics [32], it is fast, simple, and often effective [39].”
MethodLearning and generalization of motor skills by learning from demonstration
Pastor, P., Hoffmann, H., Asfour, T., & Schaal, S. (2009) · 2009 IEEE International Conference on Robotics and Automation · doi:10.1109/robot.2009.5152385 · Cited by 702 · cited 1× · see citing works
“RL has been used to train robotic controllers for games such as ball-in-cup and table tennis [16], manipulation [31], [6], and robotic locomotion [17], [40], [10], [8].”
MethodPolicy gradient reinforcement learning for fast quadrupedal locomotion
Kohl, N., & Stone, P. (2004) · IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004 · doi:10.1109/robot.2004.1307456 · Cited by 676 · cited 1× · see citing works
“RL has been used to train robotic controllers for games such as ball-in-cup and table tennis [16], manipulation [31], [6], and robotic locomotion [17], [40], [10], [8].”
MethodITERATIVE LINEAR QUADRATIC REGULATOR DESIGN FOR NONLINEAR BIOLOGICAL MOVEMENT SYSTEMS
Li, W., & Todorov, E. V. (2004) · International Conference on Informatics in Control, Automation and Robotics (ICINCO) · doi:10.5220/0001143902220229 · Cited by 589 · see citing works
An Application of Reinforcement Learning to Aerobatic Helicopter Flight
Abbeel, P., Coates, A., Quigley, M., & Ng, A. Y. (2007) · The MIT Press eBooks · doi:10.7551/mitpress/7503.003.0006 · Cited by 551 · cited 1× · see citing works
“While such methods have been successfully applied to aerial vehicles [1], [28], the requirement to be able to acquire an accurate model means that these methods share many of the challenges of MPC methods.”
MethodSelf-citationDifferential dynamic programming
DENHAM, W. F. (1971) · IEEE Transactions on Automatic Control · doi:10.1109/tac.1971.1099760 · Cited by 530 · see citing works
Differential dynamic programming
Jacobson, D. H., & Mayne, D. Q. (1970) · CERN Document Server (European Organization for Nuclear Research) · Cited by 510 · see citing works
Levine, S., & Koltun, V. (2013) · Cited by 481 · see citing works
Self-citationLearning monocular reactive UAV control in cluttered natural environments
Ross, S., Melik-Barkhudarov, N., Shankar, K. S., Wendel, A., Dey, D., Bagnell, J. A., & Hebert, M. (2013) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2013.6630809 · Cited by 410 · cited 1× · see citing works
“While alternative learning methods, such as learning from demonstration [36], [37], can address this”
Learning Neural Network Policies with Guided Policy Search under Unknown Dynamics
Levine, S., & Abbeel, P. (2014) · Cited by 379 · see citing works
Self-citationModel predictive quadrotor control: attitude, altitude and position experimental studies
Alexis, K., Nikolakopoulos, G., & Tzes, A. P. (2012) · IET Control Theory and Applications · doi:10.1049/iet-cta.2011.0348 · Cited by 372 · cited 1× · see citing works
“quently used to control autonomous aerial vehicles such as quadrotors [38], [35], [34], [3], [2], [4], [29].”
DeepMPC: Learning Deep Latent Features for Model Predictive Control
Lenz, I., Knepper, R. A., & Saxena, A. (2015) · Robotics: Science and Systems · doi:10.15607/rss.2015.xi.012 · Cited by 339 · cited 1× · see citing works
“Deep neural networks have also been used to learn models for MPC [18].”
MethodAugugliaro, F., Schoellig, A. P., & D’Andrea, R. (2012) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2012.6385823 · Cited by 328 · cited 1× · see citing works
“quently used to control autonomous aerial vehicles such as quadrotors [38], [35], [34], [3], [2], [4], [29].”
Vision-based autonomous mapping and exploration using a quadrotor MAV
Fraundorfer, F., Heng, L., Honegger, D., Lee, G. H., Meier, L., Tanskanen, P., & Pollefeys, M. (2012) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2012.6385934 · Cited by 319 · cited 1× · see citing works
“This challenge is conventionally addressed by using localization and mapping algorithms to map out the environment [9] and then optimizing trajectories under the resulting map [11].”
Learning contact-rich manipulation skills with guided policy search
Levine, S., Wagener, N., & Abbeel, P. (2015) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2015.7138994 · Cited by 276 · cited 2× · see citing works
“Guided policy search has been applied to locomotion [21], robotic manipulation [22], and visionbased robotic control [23], but all prior applications rely on an offline trajectory optimization phase to generate the controller that is then executed on the real system.”
MethodSelf-citationStochastic policy gradient reinforcement learning on a simple 3D biped
Tedrake, R., Zhang, T., & Seung, H. S. (2005) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2004.1389841 · Cited by 273 · cited 1× · see citing works
“RL has been used to train robotic controllers for games such as ball-in-cup and table tennis [16], manipulation [31], [6], and robotic locomotion [17], [40], [10], [8].”
MethodSelf-citationThe true role of accelerometer feedback in quadrotor control
Martin, P., & Salaun, E. (2010) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2010.5509980 · Cited by 243 · cited 1× · see citing works
“The dynamics followed the formulation described by Martin and Salaun [25].”
MethodEfficient mixed-integer planning for UAVs in cluttered environments
Deits, R., & Tedrake, R. (2015) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2015.7138978 · Cited by 228 · cited 1× · see citing works
“principle, a wide variety of methods can be used to generate guiding samples, and more sophisticated methods might afford superior robustness and obstacle avoidance [7].”
BackgroundLearning to Control a Low-Cost Manipulator using Data-Efficient Reinforcement Learning
Deisenroth, M. P., Rasmussen, C. E., & Fox, D. (2011) · Robotics: Science and Systems · doi:10.15607/rss.2011.vii.008 · Cited by 208 · cited 1× · see citing works
“RL has been used to train robotic controllers for games such as ball-in-cup and table tennis [16], manipulation [31], [6], and robotic locomotion [17], [40], [10], [8].”
MethodLearning CPG-based Biped Locomotion with a Policy Gradient Method: Application to a Humanoid Robot
Endo, G., Morimoto, J., Matsubara, T., Nakanishi, J., & Cheng, G. (2008) · The International Journal of Robotics Research · doi:10.1177/0278364907084980 · Cited by 206 · cited 1× · see citing works
“RL has been used to train robotic controllers for games such as ball-in-cup and table tennis [16], manipulation [31], [6], and robotic locomotion [17], [40], [10], [8].”
MethodNonlinear model predictive tracking control for rotorcraft-based unmanned aerial vehicles
Kim, H. J., Shim, D. H., & Sastry, S. (2002) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.1109/acc.2002.1024483 · Cited by 196 · cited 1× · see citing works
“quently used to control autonomous aerial vehicles such as quadrotors [38], [35], [34], [3], [2], [4], [29].”
Bouffard, P., Aswani, A. J., & Tomlin, C. J. (2012) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2012.6225035 · Cited by 183 · cited 1× · see citing works
“quently used to control autonomous aerial vehicles such as quadrotors [38], [35], [34], [3], [2], [4], [29].”
Reinforcement Learning to adjust Robot Movements to New Situations
Kober, J., Öztop, E., & Peters, J. (2010) · Robotics: Science and Systems · doi:10.15607/rss.2010.vi.005 · Cited by 160 · see citing works
Learning Complex Neural Network Policies with Trajectory Optimization
Levine, S., & Koltun, V. (2014) · Cited by 115 · see citing works
Self-citationA model predictive controller for quadrocopter state interception
Mueller, M. W., & D’Andrea, R. (2013) · European Control Conference (ECC) · doi:10.23919/ecc.2013.6669415 · Cited by 112 · cited 1× · see citing works
“quently used to control autonomous aerial vehicles such as quadrotors [38], [35], [34], [3], [2], [4], [29].”
Autonomous obstacle avoidance and maneuvering on a vision-guided MAV using on-board processing
Heng, L., Meier, L., Tanskanen, P., Fraundorfer, F., & Pollefeys, M. (2011) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2011.5980095 · Cited by 103 · cited 1× · see citing works
“This challenge is conventionally addressed by using localization and mapping algorithms to map out the environment [9] and then optimizing trajectories under the resulting map [11].”
Robust constrained model predictive control
Richards, A. G. (2005) · Cited by 96 · see citing works
Iterative learning of feed-forward corrections for high-performance tracking
Mueller, F. L., Schoellig, A. P., & D’Andrea, R. (2012) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2012.6385647 · Cited by 49 · cited 1× · see citing works
“While such methods have been successfully applied to aerial vehicles [1], [28], the requirement to be able to acquire an accurate model means that these methods share many of the challenges of MPC methods.”
MethodFast biped walking with a reflexive controller and real-time policy searching
Geng, T., Porr, B., & Wörgötter, F. (2005) · neural information processing systems · Cited by 18 · see citing works
Flagged first, then by citations
1 of the 45 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Efficient Processing of Deep Neural Networks: A Tutorial and Survey
Sze, V., Chen, Y.-H., Yang, T.-J., & Emer, J. S. (2017) · Proceedings of the IEEE · doi:10.1109/jproc.2017.2761740 · Cited by 4,215 · see citing works
“• Robotics DNNs have been successful in the domain of robotic tasks such as grasping with a robotic arm [32], motion planning for ground robots [33], visual navigation [4, 34], control to stabilize a quadcopter [35] and driving strategies for autonomous vehicles [36].”
A survey of deep learning techniques for autonomous driving
Grigorescu, S., Trasnea, B., Cocias, T., & Macesanu, G. (2020) · Journal of Field Robotics · doi:10.1002/rob.21918 · Cited by 1,809 · see citing works
“rently, the largest portion of papers in deep learning for selfdriving cars are focused mainly on perception and End2End learning [81, 124].”
BackgroundInfluentialData-Driven Science and Engineering
Brunton, S. L., & Kutz, J. N. (2019) · Cambridge University Press eBooks · doi:10.1017/9781108380690 · Cited by 1,177 · see citing works
Socially aware motion planning with deep reinforcement learning
Chen, Y. F., Everett, M., Liu, M., & How, J. P. (2017) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8202312 · Cited by 740 · see citing works
“While several recent works have applied deep RL to motion planning [22], [23], they are mainly focused on single agent navigation in unknown static environments, and with an emphasis on computing control inputs directly from raw”
BackgroundDecentralized non-communicating multiagent collision avoidance with deep reinforcement learning
Chen, Y. F., Liu, M., Everett, M., & How, J. P. (2017) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2017.7989037 · Cited by 673 · see citing works
“While applications of RL are typically limited to discrete, low-dimensional domains, recent advances in Deep RL [20]– [23] have demonstrated human-level performance in complex, high-dimensional spaces.”
BackgroundControl of a Quadrotor With Reinforcement Learning
Hwangbo, J., Sa, I., Siegwart, R., & Hutter, M. (2017) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2017.2720851 · Cited by 606 · see citing works
“Guided policy search with a MPC controller [5] is demonstrated in simulation.”
MethodSparse identification of nonlinear dynamics for model predictive control in the low-data limit
Kaiser, E., Kutz, J. N., & Brunton, S. L. (2018) · Proceedings of the Royal Society A Mathematical Physical and Engineering Sciences · doi:10.1098/rspa.2018.0335 · Cited by 601 · see citing works
“Nonlinear models based on machine learning, such as neural networks, are increasingly used due to advances in computing power, and recently deep reinforcement learning has been combined with MPC [36, 56], yielding impressive results in the large-data limit.”
Model-Based Reinforcement Learning for Closed-Loop Dynamic Control of Soft Robotic Manipulators
Thuruthel, T. G., Falotico, E., Renda, F., & Laschi, C. (2018) · IEEE Transactions on Robotics · doi:10.1109/tro.2018.2878318 · Cited by 409 · see citing works
“The state-of-the-art approaches using variations of this idea are concerned with using learned local models [20], compound multistep controllers [21], and deep representation of the control policies [22].”
MethodData-driven discovery of Koopman eigenfunctions for control
Kaiser, E., Kutz, J. N., & Brunton, S. L. (2021) · Machine Learning Science and Technology · doi:10.1088/2632-2153/abf0f5 · Cited by 353 · see citing works
Fan, T., Long, P., Liu, W., & Pan, J. (2020) · The International Journal of Robotics Research · doi:10.1177/0278364920916531 · Cited by 337 · see citing works
“DNNs have also been integrated with model predictive control to control robotic systems (Lenz et al., 2015; Zhang et al., 2016b).”
MethodDeep learning in robotics: A review of recent research
Pierson, H. A., & Gashler, M. S. (2017) · Advanced Robotics · doi:10.1080/01691864.2017.1365009 · Cited by 331 · see citing works
“Levine et al. [128] used reinforcement learning (Structure D) to show that this can be superior to separate systems for perception and control.”
BackgroundInfluentialFasel, U., Kutz, J. N., Brunton, B. W., & Brunton, S. L. (2022) · Proceedings of the Royal Society A Mathematical Physical and Engineering Sciences · doi:10.1098/rspa.2021.0904 · Cited by 306 · see citing works
“Deep neural network models have been increasingly used for deep MPC [83, 84]; however, they often rely on access to massive data sets, have limited ability to generalize, do not readily incorporate known physical constraints, and are computationally expensive.”
MethodEnergy Management Strategy for a Hybrid Electric Vehicle Based on Deep Reinforcement Learning
Hu, Y., Li, W., Xu, K., Zahid, T., Qin, F., & Li, C. (2018) · Applied Sciences · doi:10.3390/app8020187 · Cited by 276 · see citing works
“The experimental results are given in Section 4, followed by the conclusions in Section 5.”
Learning an Approximate Model Predictive Controller With Guarantees
Hertneck, M., Köhler, J., Trimpe, S., & Allgöwer, F. (2018) · IEEE Control Systems Letters · doi:10.1109/lcsys.2018.2843682 · Cited by 263 · see citing works
“[5, 6, 7] can be adapted similar to the proposed framework, if a robust control method is used in combination with the proposed validation method.”
BackgroundMethodApproximate model predictive building control via machine learning
Drgoňa, J., Picard, D., Kvasnica, M., & Helsen, L. (2018) · Applied Energy · doi:10.1016/j.apenergy.2018.02.156 · Cited by 260 · see citing works
Motion planning and control for mobile robot navigation using machine learning: a survey
Xiao, X., Liu, B., Warnell, G., & Stone, P. (2022) · Autonomous Robots · doi:10.1007/s10514-022-10039-8 · Cited by 294 · see citing works
“[111], who addressed the problem of partial observability of UAV obstacle avoidance by training a system with access to full state information and deployment with only partial observations.”
BackgroundMethodInfluentialVulnerability of Deep Reinforcement Learning to Policy Induction Attacks
Behzadan, V., & Munir, A. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-62416-7_19 · Cited by 247 · see citing works
“…both feature selection and Q-function approximation, hence enabling unprecedented performance in complex settings such as learning efficient playing strategies from unlabeled video frames of Atari games [7], robotic manipulation [8], and autonomous navigation of aerial [9] and ground vehicles [10].”
BackgroundData-driven science and engineering : machine learning, dynamical systems, and control
Brunton, S. L., & Kutz, J. N. (2019) · doi:10.1017/978-1-009-08951-7 · Cited by 270 · see citing works
Nonlinear and Adaptive Intelligent Control Techniques for Quadrotor UAV – A Survey
Mo, H., & Farid, G. (2018) · Asian Journal of Control · doi:10.1002/asjc.1758 · Cited by 244 · see citing works
“Reinforcement learning based studies to train a quadrotor fl ight controller for re fi ned and precise maneuverers, have also been reported in the literature [133 – 135].”
MethodResultHu, X., Liu, T., Qi, X., & Barth, M. (2019) · IEEE Industrial Electronics Magazine · doi:10.1109/mie.2019.2913015 · Cited by 240 · see citing works
“The upper level used MPC to determine the velocity trajectories to avoid red-light idling, and the lower level applied RL to optimize the energy-efficient controls for each HEV [36].”
MethodDi, X. S., & Shi, R. (2021) · Transportation Research Part C Emerging Technologies · doi:10.1016/j.trc.2021.103008 · Cited by 240 · see citing works
A Deep Reinforcement Learning Strategy for UAV Autonomous Landing on a Moving Platform
Rodríguez-Ramos, A., Sampedro, C., Bavle, H., de la Puente, P., & Campoy, P. (2018) · Journal of Intelligent & Robotic Systems · doi:10.1007/s10846-018-0891-8 · Cited by 229 · see citing works
“Nevertheless, some recent advances have proposed novel methods to learn low-level control policies from imperfect sensor data in simulation [49,21].”
BackgroundMethodSmolyanskiy, N., Kamenev, A., Smith, J., & Birchfield, S. T. (2017) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8206285 · Cited by 230 · see citing works
“[33] use DRL to learn obstacle avoidance policies for a simulated quadrotor.”
MethodDeep Reinforcement Learning Attitude Control of Fixed-Wing UAVs Using Proximal Policy optimization
Bøhn, E., Coates, E. M., Moe, S., & Johansen, T. A. (2019) · International Conference on Unmanned Aircraft Systems (ICUAS) · doi:10.1109/icuas.2019.8798254 · Cited by 219 · see citing works
“In [33], the authors use a model predictive controller (MPC) to generate training data for an RL controller, thereby guiding the policy search and avoiding the potentially catastrophic early phase before an effective policy is found.”
MethodLearning an approximate model predictive controller with guarantees
Hertneck, M. (2018) · OPUS Publication Server of the University of Stuttgart (University of Stuttgart) · doi:10.18419/opus-9722 · Cited by 213 · see citing works
Reinforcement Learning on Variable Impedance Controller for High-Precision Robotic Assembly
Luo, J., Solowjow, E., Wen, C., Ojea, J. A., Agogino, A. M., Tamar, A., & Abbeel, P. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8793506 · Cited by 195 · see citing works
“b) Related Work: Recent advances in RL have gained great success in solving a variety of problems from playing video games [9, 10, 11] to robotic locomotion [12, 13, 14, 15, 16], manipulation [17, 18, 19, 20, 4, 21, 22, 23, 24, 25].”
BackgroundSelf-citationBaccour, E., Mhaisen, N., Abdellatif, A. A., Erbad, A., Mohamed, A. M. S., Hamdi, M., & Guizani, M. (2022) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2022.3200740 · Cited by 209 · see citing works
“online learning is broadly exploited for UAVs/robots guidance, including the work in [58] where drones learn how to navigate and avoid obstacles while searching target objects.”
BackgroundDeterministic Policy Gradient With Integral Compensator for Robust Quadrotor Control
Wang, Y., Sun, J. L., He, H., & Sun, C. (2019) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2018.2884725 · Cited by 180 · see citing works
“In[27], an obstacle avoidance policy for quadrotors trained by the guided policy search with MPC is demonstrated in simulations.”
MethodTemporal Difference Models: Model-Free Deep RL for Model-Based ControlPreprint
Pong, V. H., Gu, S., Dalal, M., & Levine, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.09081 · Cited by 164 · see citing works
“…function approximators such as deep neural networks, RL can provide impressive results on tasks ranging from playing games (Mnih et al., 2015; Silver et al., 2016), to flying and driving (Lillicrap et al., 2015; Zhang et al., 2016), to controlling robotic arms (Levine et al., 2016; Gu et al., 2017).”
BackgroundSelf-citationKaufmann, E., Loquercio, A., Ranftl, R., Müller, M., Koltun, V., & Scaramuzza, D. (2020) · Robotics: Science and Systems · doi:10.15607/rss.2020.xvi.040 · Cited by 162 · see citing works
Learning nonlinear dynamic models of soft robots for model predictive control with neural networks
Gillespie, M. T., Best, C. M., Townsend, E. C., Wingate, D., & Killpack, M. D. (2018) · IEEE International Conference on Soft Robotics (RoboSoft) · doi:10.1109/robosoft.2018.8404894 · Cited by 156 · see citing works
“For the work in [21], they used a DNN to learn a MPC control policy for UAVs.”
MethodState-of-the-Art Intelligent Flight Control Systems in Unmanned Aerial Vehicles
Santoso, F., Garratt, M., & Anavatti, S. G. (2017) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2017.2651109 · Cited by 141 · see citing works
“[95] demonstrate that through the guided policy search, achieved by decomposing the search policy into a trajectory optimization phase, effective memorization and recall techniques can be achieved.”
BackgroundMethodLearning a deep neural net policy for end-to-end control of autonomous vehicles
Rausch, V., Hansen, A., Solowjow, E., Liu, C., Kreuzer, E. J., & Hedrick, J. K. (2017) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.23919/acc.2017.7963716 · Cited by 140 · see citing works
“Since explicit mapping of observations to control actions has not to be determined a priori in a design process anymore, deep learning turns out to be very efficient for certain high level controls objectives [6].”
BackgroundMultistep Prediction of Dynamic Systems With Recurrent Neural Networks
Mohajerin, N., & Waslander, S. L. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2891257 · Cited by 139 · see citing works
“In [48], a deep NN, trained by an MPC policy search, is used as a policy learner to control a quadrotor.”
MethodLiu, Y., He, X., Liu, D., & Wang, L. (2022) · Robotics and Computer-Integrated Manufacturing · doi:10.1016/j.rcim.2022.102365 · Cited by 139 · see citing works
A Survey on Policy Search Algorithms for Learning Robot Controllers in a Handful of Trials
Chatzilygeroudis, K., Vassiliades, V., Stulp, F., Calinon, S., & Mouret, J.-B. (2020) · IEEE Transactions on Robotics · doi:10.1109/tro.2019.2958211 · Cited by 160 · see citing works
“This can be realized by using MPC with full state information in some training phase, to learn neural network (NN) policies that take the raw observations as input, thus, not requiring full state information at test time [195].”
MethodPolicy Search for Model Predictive Control With Application to Agile Drone Flight
Song, Y., & Scaramuzza, D. (2022) · IEEE Transactions on Robotics · doi:10.1109/tro.2022.3141602 · Cited by 121 · see citing works
Efficient Processing of Deep Neural Networks
Sze, V., Chen, Y., Yang, T.-J., & Emer, J. (2020) · Synthesis lectures on computer architecture · doi:10.2200/s01004ed1v01y202004cac050 · Cited by 117 · see citing works
Safe deep reinforcement learning-based adaptive control for USV interception mission
Du, B., Lin, B., Zhang, C., Dong, B., & Zhang, W. (2022) · Ocean Engineering · doi:10.1016/j.oceaneng.2021.110477 · Cited by 110 · see citing works
PLATO: Policy learning using adaptive trajectory optimization
Kahn, G., Zhang, T., Levine, S., & Abbeel, P. (2017) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2017.7989379 · Cited by 108 · see citing works
“The motivation behind PLATO is most similar to the MPC variant of guided policy search (MPC-GPS) [12].”
MethodResultInfluentialSelf-citationDeep Reinforcement Learning for Robotic Assembly of Mixed Deformable and Rigid Objects
Luo, J., Solowjow, E., Wen, C., Ojea, J. A., & Agogino, A. M. (2018) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8594353 · Cited by 105 · see citing works
“Recent advances in reinforcement learning have gained great success in solving a variety of problems from playing video games [5, 6] to robotic locomotion [7, 8, 9, 10], and manipulation [11, 12, 13, 14, 15].”
BackgroundA Survey on Design, Actuation, Modeling, and Control of Continuum Robot
Zhang, J., Fang, Q., Xiang, P., Sun, D., Xue, Y., Jin, R., Qiu, K., Xiong, R., Wang, Y., & Lu, H. (2022) · Cyborg and Bionic Systems · doi:10.34133/2022/9754697 · Cited by 104 · see citing works
“The most advanced methods of using this variant of the idea involve the use of local models of learning [126], composite multistep controllers [127], and deep representations of control strategies [128].”
MethodNeural Networks and Deep Learning
Brunton, S. L., & Kutz, J. N. (2019) · Cambridge University Press eBooks · doi:10.1017/9781108380690.007 · Cited by 104 · see citing works
Near-Optimal Rapid MPC Using Neural Networks: A Primal-Dual Policy Learning Framework
Zhang, X., Bujarbaruah, M., & Borrelli, F. (2020) · IEEE Transactions on Control Systems Technology · doi:10.1109/tcst.2020.3024571 · Cited by 93 · see citing works
Neural network-based flight control systems: Present and future
Emami, S. A., Castaldi, P., & Banazadeh, A. (2022) · Annual Reviews in Control · doi:10.1016/j.arcontrol.2022.04.006 · Cited by 96 · see citing works
Autonomous Navigation for eVTOL: Review and Future Perspectives
Wei, H., Lou, B., Zhang, Z., Liang, B., Wang, F., & Lv, C. (2024) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2024.3352613 · Cited by 93 · see citing works
Review of intelligent energy management techniques for hybrid electric vehicles
Urooj, A., & Nasir, A. (2024) · Journal of Energy Storage · doi:10.1016/j.est.2024.112132 · Cited by 88 · see citing works
Flying Free: A Research Overview of Deep Learning in Drone Navigation Autonomy
Lee, T., McKeever, S., & Courtney, J. (2021) · Drones · doi:10.3390/drones5020052 · Cited by 87 · see citing works
Yang, X., & Wei, P. (2021) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2020.3048360 · Cited by 84 · see citing works
Xie, L., Wang, S., Rosa, S., Markham, A., & Trigoni, N. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8461203 · Cited by 83 · see citing works
A Benchmark Comparison of Learned Control Policies for Agile Quadrotor Flight
Kaufmann, E., Bauersfeld, L., & Scaramuzza, D. (2022) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra46639.2022.9811564 · Cited by 86 · see citing works
Guided Policy Search via Approximate Mirror DescentPreprint
Montgomery, W. H., & Levine, S. (2016) · arXiv (Cornell University) · Cited by 82 · see citing works
Self-citationDeep Visual MPC-Policy Learning for Navigation
Hirose, N., Xia, F., Martín-Martín, R., Sadeghian, A., & Savarese, S. (2019) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2019.2925731 · Cited by 81 · see citing works
SINDy with Control: A Tutorial
Fasel, U., Kaiser, E., Kutz, J. N., Brunton, B. W., & Brunton, S. L. (2021) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc45484.2021.9683120 · Cited by 81 · see citing works
Model Predictive Control for Micro Aerial Vehicles: A Survey
Nguyen, H., Kamel, M., Alexis, K., & Siegwart, R. (2021) · European Control Conference (ECC) · doi:10.23919/ecc54610.2021.9654841 · Cited by 80 · see citing works
Hu, H., Fazlyab, M., Morari, M., & Pappas, G. J. (2020) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc42340.2020.9304296 · Cited by 78 · see citing works
“In autonomous systems, DNNs are either used as feedback controllers [1], [2], motion planners [3], perception modules, or end-to-end controllers [4].”
Drgoňa, J., Kiš, K., Tuor, A. R., Vrabie, D., & Klaučo, M. (2022) · Journal of Process Control · doi:10.1016/j.jprocont.2022.06.001 · Cited by 77 · see citing works
“These approaches are known in the machine learning community as imitation learning [33], or MPC-guided policy search [34].”
BackgroundInternet of drones security: Taxonomies, open issues, and future directions
Derhab, A., Cheikhrouhou, O., Allouch, A., Koubâa, A., Qureshi, B., Ferrag, M. A., Maglaras, L. A., & Khan, F. A. (2022) · Vehicular Communications · doi:10.1016/j.vehcom.2022.100552 · Cited by 77 · see citing works
Learning control lyapunov functions from counterexamples and demonstrations
Ravanbakhsh, H., & Sankaranarayanan, S. (2018) · Autonomous Robots · doi:10.1007/s10514-018-9791-9 · Cited by 74 · see citing works
“On one hand, these are much less powerful than approaches that use neural networks [105], for instance.”
BackgroundMethodLearning quadrotor dynamics for precise, safe, and agile flight control
Saviolo, A., & Loianno, G. (2023) · Annual Reviews in Control · doi:10.1016/j.arcontrol.2023.03.009 · Cited by 76 · see citing works
Recchiuto, C. T., & Sgorbissa, A. (2017) · Journal of Field Robotics · doi:10.1002/rob.21756 · Cited by 72 · see citing works
Policy Learning for Nonlinear Model Predictive Control With Application to USVs
Wang, R., Li, H., Liang, B., Shi, Y., & Xu, D. (2023) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2023.3274869 · Cited by 71 · see citing works
“The work in [18] used MPC to guide reinforcement learning (RL) training to improve the training efficiency of RL and guarantee the safety of the RL results.”
MethodConstrained Cross-Entropy Method for Safe Reinforcement Learning
Wen, M., & Topcu, U. (2020) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2020.3015931 · Cited by 72 · see citing works
“By encoding control tasks as reward or cost functions, RL has successfully solved a wide range of tasks such as Atari games [19; 20], the game of Go [31; 32], controlling simulated robots [36; 24] and real robots [15; 37; 22].”
BackgroundModel-Based Control of Soft Actuators Using Learned Non-linear Discrete-Time Models
Hyatt, P., Wingate, D., & Killpack, M. D. (2019) · Frontiers in Robotics and AI · doi:10.3389/frobt.2019.00022 · Cited by 70 · see citing works
Greatwood, C. M., & Richards, A. G. (2019) · Autonomous Robots · doi:10.1007/s10514-019-09829-4 · Cited by 69 · see citing works
“Zhang et al. (2016) used MPC as the supervisor for their learning algorithm, resulting in a deep neural network policy for obstacle avoidance, while Aswani et al. (2013) used a learning ‘oracle’ to refine the prediction model for a general MPC. Sharma and Taylor (2012) used RL for waypoint…”
MethodHybrid Reinforcement Learning Control for a Micro Quadrotor Flight
Yoo, J., Jang, D., Kim, H. J., & Johansson, K. H. (2020) · IEEE Control Systems Letters · doi:10.1109/lcsys.2020.3001663 · Cited by 67 · see citing works
“On the other hand, similar to some RL meth-ods to use a system model [6]–[8], LQR-RL can be used if a system model is given, where LQR is an optimal control method.”
MethodMarshall, J. A., Sun, W., & L’Afflitto, A. (2021) · Annual Reviews in Control · doi:10.1016/j.arcontrol.2021.10.013 · Cited by 71 · see citing works
“Alternative approaches to the outer-loop design process require to find 𝜙 ( ⋅ ) and 𝜃 ( ⋅ ) such that the position in vehicle’s horizontal plane is regulated (Adıgüzel & Mumcu, 2019; Hamrah & Sanyal, 2020; Xiong & Zhang, 2016).”
BackgroundMethodIslam, M., Okasha, M., & Sulaeman, E. (2019) · International Journal of Control Automation and Systems · doi:10.1007/s12555-018-0860-9 · Cited by 66 · see citing works
Deep Learning in Diverse Intelligent Sensor Based Systems
Zhu, Y., Wang, M., Yin, X., Zhang, J., Meijering, E. H. W., & Hu, J. (2022) · Sensors · doi:10.3390/s23010062 · Cited by 67 · see citing works
Learning Perception-Aware Agile Flight in Cluttered Environments
Song, Y., Shi, K., Pěnička, R., & Scaramuzza, D. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10160563 · Cited by 67 · see citing works
“Some other works propose to learn end-to-end policies directly from sensory observations to control commands using imitation learning [20]–[22].”
BackgroundA Survey of Deep Network Solutions for Learning Control in Robotics: From Reinforcement to ImitationPreprint
Tai, L., Zhang, J., Liu, M., Boedecker, J., & Burgard, W. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.07139 · Cited by 95 · see citing works
“Most of the reinforcement learning applications in robotics are trained in simulation environments (Tai and Liu, 2016a; Zhang et al., 2016a).”
BackgroundMethodInfluentialMiao, Z., Li, Y., & Wang, X. (2021) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/3448016.3457258 · Cited by 63 · see citing works
Data-driven planning via imitation learning
Choudhury, S., Bhardwaj, M., Arora, S., Kapoor, A., Ranade, G. V., Scherer, S., & Dey, D. (2018) · The International Journal of Robotics Research · doi:10.1177/0278364918781001 · Cited by 63 · see citing works
“mitation of clairvoyant oracles, that has access to more information than the learner during training, to improve reinforcement learning as they offer better sample efficiency and safety. Zhang et al. [130], Kahn et al. [57] train policies that map current observation to action by extending guided policy search [73] fo…”
BackgroundToward autonomous mapping and exploration for mobile robots through deep supervised learning
Bai, S., Chen, F., & Englot, B. (2017) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8206050 · Cited by 58 · see citing works
“challenging problems such as image recognition [17], robot manipulation [18] and control [19], [20].”
BackgroundDeep-learning in Mobile Robotics - from Perception to Control Systems: A Survey on Why and Why not.Preprint
Tai, L., & Liu, M. (2016) · arXiv (Cornell University) · Cited by 59 · see citing works
Car-Following Models for Human-Driven Vehicles and Autonomous Vehicles: A Systematic Review
Wang, Z., Shi, Y., Tong, W., Gu, Z., & Cheng, Q. (2023) · Journal of Transportation Engineering Part A Systems · doi:10.1061/jtepbs.teeng-7836 · Cited by 59 · see citing works
UAV Obstacle Avoidance Algorithm to Navigate in Dynamic Building Environments
Aldao, E., González-deSantos, L. M., Michinel, H., & González-Jorge, H. (2022) · Drones · doi:10.3390/drones6010016 · Cited by 56 · see citing works
“Some of the most common are based on geometric relations [18,19], fuzzy logic [20,21], potential fields [22,23], and neural networks [24–26], among others.”
BackgroundNeural network modeling for steering control of an autonomous vehicle
Garimella, G., Funke, J., Wang, C., & Kobilarov, M. (2017) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8206084 · Cited by 56 · see citing works
Vision-Based High-Speed Driving With a Deep Dynamic Observer
Drews, P., Williams, G., Goldfain, B., Theodorou, E. A., & Rehg, J. M. (2019) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2019.2896449 · Cited by 54 · see citing works
Task-Oriented Deep Reinforcement Learning for Robotic Skill Acquisition and Control
Xiang, G., & Su, J. (2019) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2019.2949596 · Cited by 54 · see citing works
“…been extensively studied on robotic skill acquisition and control problems, such as robot locomotion [7], [8]; sensorimotor control [9], [10]; robotic manipulation [11], [12]; dextrous robot grasping [13], [14]; self-driving vehicles [15], [16]; and autonomous helicopters [17], [18], to name a few.”
BackgroundXiao, C., Lu, P., & He, Q. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3107742 · Cited by 55 · see citing works
“[3] applied guided policy search (GPS) to a quadrotor collision avoidance task.”
MethodXiong, H., Ma, T., Zhang, L., & Diao, X. (2019) · Neurocomputing · doi:10.1016/j.neucom.2019.10.020 · Cited by 54 · see citing works
OmniDrones: An Efficient and Flexible Platform for Reinforcement Learning in Drone Control
Xu, B., Gao, F., Yu, C., Zhang, R., Wu, Y., & Wang, Y. (2024) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3356168 · Cited by 61 · see citing works
“Prior works explored end-to-end training of visual-motor control policies [2], [24], [25], [26] to avoid the need for explicit dynamics modeling and hand-engineered control stack.”
BackgroundSafe and Near-Optimal Policy Learning for Model Predictive Control using Primal-Dual Neural Networks
Zhang, X., Bujarbaruah, M., & Borrelli, F. (2019) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.23919/acc.2019.8814335 · Cited by 53 · see citing works
Memory Augmented Control NetworksPreprint
Khan, A., Zhang, C., Atanasov, N., Karydis, K., Kumar, V., & Lee, D. D. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.05706 · Cited by 53 · see citing works
“Purely convolutional architectures (CNNs) perform poorly when applied to planning problems due to the reactive nature of the policies learned by them [5],[6],[7].”
BackgroundFast and flexible multilegged locomotion using learned centroidal dynamics
Kwon, T., Lee, Y., & van de Panne, M. (2020) · ACM Transactions on Graphics · doi:10.1145/3386569.3392432 · Cited by 51 · see citing works
“Trajectory optimization methods can be used to produce data for learning feedback-based controllers, typically as part of an iterative process, e.g., [Levine and Koltun 2013; Liu and Hodgins 2018; Liu et al. 2016, 2012; Mordatch et al. 2015; Zhang et al. 2016].”
MethodEstrada, P. M., Lima, D. D., Bauer, P. H., Mammetti, M., & Bruno, J. C. (2022) · Applied Energy · doi:10.1016/j.apenergy.2022.120231 · Cited by 50 · see citing works
Jiang, C., & Guo, Y. P. (2023) · IEEE Transactions on Control of Network Systems · doi:10.1109/tcns.2023.3290430 · Cited by 50 · see citing works
“An intuitive solution is to generate a reference trajectory that achieves good long-horizon performance and track the reference trajectory with receding horizon control, such as MPC [40].”
Data-driven nonlinear aeroelastic models of morphing wings for control
Fonzi, N., Brunton, S. L., & Fasel, U. (2020) · Proceedings of the Royal Society A Mathematical Physical and Engineering Sciences · doi:10.1098/rspa.2020.0079 · Cited by 50 · see citing works
“Nonlinear models based on machine learning, such as neural networks, are increasingly used due to advances in computing power, and recently deep reinforcement learning has been combined with MPC [74, 75], yielding impressive results in the large-data and high-performance computing limit.”
MethodAggressive Quadrotor Flight Using Curiosity-Driven Reinforcement Learning
Sun, Q., Fang, J., Zheng, W. X., & Tang, Y. S. (2022) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2022.3144586 · Cited by 49 · see citing works
“One of the promising methods to address this is reinforcement learning, through which UAVs can be navigated without the trajectory planning procedure [7] and which has been widely used in the control and navigation of agents [8]–[11].”
MethodCombining Reinforcement Learning with Model Predictive Control for On-Ramp Merging
Lubars, J., Gupta, H., Chinchali, S., Li, L., Raja, A., Srikant, R., & Wu, X. (2021) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc48978.2021.9564954 · Cited by 49 · see citing works
Holo-Dex: Teaching Dexterity with Immersive Mixed Reality
Arunachalam, S. P., Guzey, I., Chintala, S., & Pinto, L. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10160547 · Cited by 49 · see citing works
“fect in robotics on a wide range of domains from manipulation [1, 2], locomotion [3, 4, 5], and aerial robotics [6, 7, 8].”
BackgroundLiu, M., Peeters, S., Callaway, D. S., & Claessens, B. J. (2019) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2018.2890275 · Cited by 48 · see citing works
Deep imitation learning for 3D navigation tasks
Hussein, A. M., Elyan, E., Gaber, M. M., & Jayne, C. (2017) · Neural Computing and Applications · doi:10.1007/s00521-017-3241-z · Cited by 47 · see citing works
“Since then several papers have proposed learning autonomous aerial navigation using demonstrations [22] and reinforcement learning [15, 16, 23].”
BackgroundPolvara, R., Sharma, S. K., Wan, J., Manning, A. J., & Sutton, R. (2019) · Robotica · doi:10.1017/s0263574719000316 · Cited by 47 · see citing works
Control of Quadrotors Using the Hopf Fibration on SO(3)
Watterson, M., & Kumar, V. (2019) · Springer proceedings in advanced robotics · doi:10.1007/978-3-030-28619-4_20 · Cited by 47 · see citing works
“There has been some research on learning based control [23], but the model based approaches are more mature and can provide provable guarantees on stability.”
BackgroundLearning Heuristic Search via ImitationPreprint
Bhardwaj, M., Choudhury, S., & Scherer, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.03034 · Cited by 47 · see citing works
“[36] extend guided policy search [37] for imitating MPC.”
Efficient Processing of Deep Neural Networks: A Tutorial and SurveyPreprint
Sze, V., Chen, Y., Yang, T.-J., & Emer, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.09039 · Cited by 45 · see citing works
“• Robotics DNNs have been successful in the domain of robotic tasks such as grasping with a robotic arm [32], motion planning for ground robots [33], visual navigation [4, 34], control to stabilize a quadcopter [35] and driving strategies for autonomous vehicles [36].”
Cyber-Physical Attack Threats Analysis for UAVs from CPS Perspective
Rong-xiao, G., Ji-wei, T., Bu-hong, W., & Fu-te, S. (2020) · International Conference on Computer Engineering and Application (ICCEA) · doi:10.1109/iccea50009.2020.00063 · Cited by 44 · see citing works
“The artificial intelligence algorithms such as deep learning have been widely used to the UAVs, especially in feature extraction [32][33], task recognition and path planning [34][35], autonomous flight control [36-40] and so on.”
MethodA Novel Reinforcement Learning-Based Robust Control Strategy for a Quadrotor
Hua, H., & Fang, Y. (2022) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2022.3165288 · Cited by 43 · see citing works
- Authors
- Zhang, T., Kahn, G., Levine, S., & Abbeel, P.
- Affiliations
- University of California, Berkeley (US)
- Venue
- 2016 IEEE International Conference on Robotics and Automation (ICRA)
- Publisher
- IEEE
- Published
- 2016-05
- Pages
- 528-535
- ISSN
- 1050-4729 (ISSN-L)
- OpenAlex
- W1923344279
- Cited by
- 452
- References
- 45
- Citations by year
- 2026: 10 · 2025: 18 · 2024: 37 · 2023: 33 · 2022: 54 · 2021: 53 · 2020: 65 · 2019: 93 · 2018: 42 · 2017: 39 · 2016: 8
- FWCI
- 32.08
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Proceedings - IEEE International Conference on Robotics and Automation/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Defense Advanced Research Projects Agency (N66001)
- Army Research Office
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Computational Theory and Mathematics › Adaptive Dynamic Programming Control
- Physical Sciences › Engineering › Control and Systems Engineering › Advanced Control Systems Optimization
- Keywords
- model predictive control, deep neural network policies, autonomous aerial vehicles, quadrotor control, obstacle avoidance, reinforcement learning, state estimation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
