The DOI matches its Crossref record; 43 of the 45 references it cites were found in a registry or an index.
Learning deep control policies for autonomous aerial vehicles with MPC-guided policy search
Open accessAbstract
Model predictive control (MPC) is an effective method for controlling robotic systems, particularly autonomous aerial vehicles such as quadcopters. However, application of MPC can be computationally demanding, and typically requires estimating the state of the system, which can be challenging in complex, unstructured environments. Reinforcement learning can in principle forego the need for explicit state estimation and acquire a policy that directly maps sensor readings to actions, but is difficult to apply to unstable systems that are liable to fail catastrophically during training before an effective policy has been found. We propose to combine MPC with reinforcement learning in the framework of guided policy search, where MPC is used to generate data at training time, under full state observations provided by an instrumented training environment. This data is used to train a deep neural network policy, which is allowed to access only the raw observations from the vehicle's onboard sensors. After training, the neural network policy can successfully control the robot without knowledge of the full state, and at a fraction of the computational cost of MPC. We evaluate our method by learning obstacle avoidance policies for a simulated quadrotor, using simulated onboard sensors and no explicit state estimation at test time.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The frequent sensing and re-planning allow the robot to react dynamically to changes in the environment, while still making more intelligent choices than a purely reactive system (Erez et al., 2011; Lenz et al., 2015; Zhang et al., 2016; Liu et al., 2017; Williams et al., 2017; Gillespie et al., 2018; Amos et al., 2019; Kabzan et al., 2019).”
From Lessons for Robotics From the Control Architecture of the Octopus (2022) · cites this paper as Zhang et al., 2016
In the citing paper
“AField-Programmable Gate Array (FPGA) is another viable solution, showing great improvements in both performance and power consumption in deep learning applications.”
From A survey of deep learning techniques for autonomous driving (2020) · cites this paper as [170]
In the citing paper
“Currently, the largest portion of papers in deep learning for selfdriving cars are focused mainly on perception and End2End learning [81, 124].”
From A survey of deep learning techniques for autonomous driving (2020) · cites this paper as [124]
In the citing paper
“Finally, realtime requirements have to be fulfilled for processing the large amounts of data gathered from the car's sensors suite, as well as for updating the parameters of deep learning systems over high-speed communication lines [170].”
From A survey of deep learning techniques for autonomous driving (2020) · cites this paper as [170]
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 2×
“We used default values for all learning parameters as presented in [14].”
Method(2010) · International Conference on Machine Learning · Cited by 13,995
Playing Atari with Deep Reinforcement Learning
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013) · arXiv · doi:10.48550/arxiv.1312.5602 · Cited by 9,471 · cited 1×
“More recently, deep neural network policies have been used for tasks ranging from robotic control [22], [23] to video game playing [27].”
MethodMuJoCo: A physics engine for model-based control
Todorov, E., Erez, T., & Tassa, Y. (2012) · 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2012.6386109 · Cited by 4,767 · cited 2×
“However, our derivation largely follows prior work [41].”
MethodResultReinforcement learning in robotics: A survey
Kober, J., Bagnell, J. A., & Peters, J. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913495721 · Cited by 3,160 · cited 1×
“An overview of recent reinforcement learning methods in robotics can be found in a recent survey paper [15].”
BackgroundEnd-to-End Training of Deep Visuomotor PoliciesPreprint
Levine, S., Finn, C., Darrell, T., & Abbeel, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1504.00702 · Cited by 2,641 · cited 10×
“In this paper, we use the BADMM version, which specifies the following objective for trajectory optimization: min pi(τ) T∑ t=1 Epi(xt,ut) [ `(xt,ut)− uTt λiµt + νitDKL ( pi(ut|xt)‖πθ(ut|xt) )] (1) where λµt is a Lagrange multiplier on the mean action, and the third term is a KL-divergence penalty.”
BackgroundMethodInfluentialSelf-citationA Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning
Ross, S., Gordon, G. J., & Bagnell, J. A. (2018) · Carnegie Mellon University · doi:10.1184/r1/6550949 · Cited by 1,941
Neural networks for control systems—A survey
Hunt, K. J., Sbarbaro, D. G., Żbikowski, R., & Gawthrop, P. J. (1992) · Automatica · doi:10.1016/0005-1098(92)90053-i · Cited by 1,874 · cited 1×
“neural networks have been used for control for decades [12], [33], limitations on computation and algorithms have made large neural network policies very difficult to learn.”
MethodRobust model predictive control of constrained linear systems with bounded disturbances
Mayne, D. Q., Seron, M. M., & Raković, S. V. (2004) · Automatica · doi:10.1016/j.automatica.2004.08.019 · Cited by 1,680 · cited 1×
“Model predictive control (MPC) is an effective and reliable method for controlling robotic systems, particularly autonomous aerial vehicles such as quadcopters, because of its robustness to moderate model errors [26], ability to use high-level objectives [41], and relative simplicity.”
MethodALVINN, an autonomous land vehicle in a neural network
Pomerleau, D. (2018) · doi:10.1184/r1/6603146 · Cited by 1,369
Flagged first, then by citations
2 of the 45 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Items 101–200 of 445
Nonlinear model predictive control for hydrobatics: Experiments with an underactuated AUV
Bhat, S., Panteli, C., Stenius, I., & Dimarogonas, D. V. (2023) · Journal of Field Robotics · doi:10.1002/rob.22218 · Cited by 43
A Novel Approach to Feedback Control with Deep Reinforcement Learning
Wang, Y., Velswamy, K., & Huang, B. (2018) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2018.09.241 · Cited by 43
“…from scratch for a variety of applications, ranging from human-level performance on Atari games (Mnih et al. (2013))(Lillicrap et al. (2015)), robotics (Gu et al. (2017))(Peng et al. (2016)), building control (Sahu et al. (2017)) (Wang et al. (2017)) to autonomous vehicles (Zhang et al. (2016)).”
BackgroundSimulation-Based Evaluation and Optimization of Control Strategies in Buildings
Kontes, G. D., Giannakis, G. B., Sánchez, V., de Agustín-Camacho, P., Romero-Amorrortu, A., Panagiotidou, N., Rovas, D., Steiger, S., Mutschler, C., & Grün, G. (2018) · Energies · doi:10.3390/en11123376 · Cited by 42
Model Predictive Control for Aerial Collision Avoidance in Dynamic Environments
Castillo-Lopez, M., Sajadi-Alamdari, S. A., Sanchez-Lopez, J. L., Olivares-Mendez, M. A., & Voos, H. (2018) · 2018 26th Mediterranean Conference on Control and Automation (MED) · doi:10.1109/med.2018.8442967 · Cited by 44
“Recent work using NMPC often simplifies this problem by considering only the closest obstacle [11] or reducing the obstacles as bi-dimensional static [12], [13] or dynamic [14] constraints.”
MethodPosition‐heading quadrotor control using LPV techniques
Trapiello, C., Puig, V., & Morcego, B. (2019) · IET Control Theory and Applications · doi:10.1049/iet-cta.2018.6147 · Cited by 41
Learning Unmanned Aerial Vehicle Control for Autonomous Target Following
Li, S., Liu, T., Zhang, C., Yeung, D., & Shen, S. (2018) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/685 · Cited by 41
Whatever Does Not Kill Deep Reinforcement Learning, Makes It StrongerPreprint
Behzadan, V., & Munir, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.09344 · Cited by 41
“…su-perhuman performance various settings, ranging from learning to playing the game of Go [Silver and Hassabis, 2016] and Atari games [Mnih et al. , 2015] to robotic manipulation [Gu et al. , 2017] and autonomous navigation of aerial [Zhang et al. , 2016] and ground [Zhu et al. , 2017] vehicles.”
BackgroundOptimizing Crop Management with Reinforcement Learning and Imitation Learning
Tao, R., Zhao, P., Wu, J., Martin, N. F., Harrison, M. T., Ferreira, C., Kalantari, Z., & Hovakimyan, N. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/691 · Cited by 40
Deep Learning Quadcopter Control via Risk-Aware Active Learning
Andersson, O., Wzorek, M., & Doherty, P. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.11041 · Cited by 40
“In Mordatch et al. (2015) it was used to animate complex movement behaviors and in Zhang et al. (2016) it was used for collision avoidance of static obstacles with a simulated quadcopter.”
MethodReliably-Stabilizing Piecewise-Affine Neural Network Controllers
Fabiani, F., & Goulart, P. J. (2022) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2022.3216978 · Cited by 46
“vehicles [8], [9], [10], [11], and power electronics [12].”
BackgroundViPlanner: Visual Semantic Imperative Learning for Local Navigation
Roth, P., Nubert, J., Yang, F., Mittal, M., & Hutter, M. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10610025 · Cited by 39
“…solution goes beyond pure reactiveness by exposing the planner to different behaviors during training while not relying on a globally build map. c) Imitation Learning and Self-Supervised Learning: Another line of research aims to learn local path planning from expert demonstrations [7,34,35].”
BackgroundReal-Time Nonlinear Model Predictive Control of Robots Using a Graphics Processing Unit
Hyatt, P., & Killpack, M. D. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.2965393 · Cited by 39
Accelerated Sim-to-Real Deep Reinforcement Learning: Learning Collision Avoidance from Human Player
Niu, H., Ji, Z., Arvin, F., Lennox, B., Yin, H., & Carrasco, J. (2021) · IEEE/SICE International Symposium on System Integration (SII) · doi:10.1109/ieeeconf49454.2021.9382693 · Cited by 39
“Deep reinforcement learning algorithm has demonstrated its superior in reducing computational time to traditional control algorithms, such as MPC (Model Predictive Con trol) [12], validated on unmanned aerial vehicles for fast maneuvers.”
Model-Plant Mismatch Compensation Using Reinforcement Learning
Koryakovskiy, I., Kudruss, M., Vallery, H., Babuška, R., & Caarls, W. (2018) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2018.2800106 · Cited by 37
“Learning involving off-line planning or human-expert demonstrations [1], [19]–[22] constrains the problem space, thus reduc-”
BackgroundJagannath, J., Jagannath, A., Furman, S., & Gwin, T. (2021) · Studies in computational intelligence · doi:10.1007/978-3-030-77939-9_2 · Cited by 42
Dooraki, A. R., & Lee, D. J. (2020) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2020.103671 · Cited by 37
Information-Driven Path Planning
Bai, S., Shan, T., Chen, F., Liu, L.-T., & Englot, B. (2021) · Current Robotics Reports · doi:10.1007/s43154-021-00045-6 · Cited by 40
“Deep neural networks have been successfully applied to challenging problems such as image recognition [104], robot manipulation [96], and control [38, 136].”
Socially Aware Motion Planning with Deep Reinforcement LearningPreprint
Chen, Y. F., Everett, M., Liu, M., & How, J. P. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.08862 · Cited by 38
“While several recent works have applied deep RL to motion planning [22], [23], they are mainly focused on single agent navigation in unknown static environments, and with an emphasis on computing control inputs directly from raw”
BackgroundMachine Learning in Control Systems: An Overview of the State of the Art
Moe, S., Rustad, A. M., & Hanssen, K. G. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-04191-5_23 · Cited by 37
Learning High-Level Policies for Model Predictive Control
Song, Y., & Scaramuzza, D. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9340823 · Cited by 37
“MPC-guided policy search [13], [14], [15] are methods that study the problems of learning a deep neural network control policy using an MPC as the teacher, and hence, they transform policy search into a supervised learning fashion.”
MethodFormally Robust and Safe Trajectory Planning and Tracking for Autonomous Vehicles
Yu, Y., Shan, D., Benderius, O., Berger, C., & Kang, Y. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2022.3196623 · Cited by 36
“In recent years, deep reinforcement learning was also used as an approach to generate safe trajectories for autonomous vehicles, while in addition improving the performance over time [25], [26].”
MethodDooraki, A. R., & Lee, D. J. (2018) · Sensors · doi:10.3390/s18103575 · Cited by 36
“Zhang et al. [10] used model predictive control (MPC) to train a RL agent to learn to fly a quad-copter; MPC is necessary for training but not necessary for the testing phase.”
MethodMachine learning-based active flutter suppression for a flexible flying-wing aircraft
Mu, X., Huang, R., Zou, Q., & Hu, H. (2022) · Journal of Sound and Vibration · doi:10.1016/j.jsv.2022.116916 · Cited by 35
Machine Learning for Sparse Nonlinear Modeling and Control
Brunton, S. L., Zolman, N., Kutz, J. N., & Fasel, U. (2025) · Annual Review of Control Robotics and Autonomous Systems · doi:10.1146/annurev-control-030123-015238 · Cited by 35
Singular Value Decomposition (SVD)
Brunton, S. L., & Kutz, J. N. (2019) · Cambridge University Press eBooks · doi:10.1017/9781108380690.002 · Cited by 46
On the Stability Analysis of Deep Neural Network Representations of an Optimal State Feedback
Izzo, D., Tailor, D., & Vasileiou, T. (2020) · IEEE Transactions on Aerospace and Electronic Systems · doi:10.1109/taes.2020.3010670 · Cited by 34
“The optimal feedback of several deterministic, non-linear systems of interest in aerospace applications has been, recently, directly represented by deep neural networks trained using techniques such as imitation learning [1, 2, 3, 4] or reinforcement learning [5, 6, 7].”
BackgroundHigh-Speed Autonomous Racing Using Trajectory-Aided Deep Reinforcement Learning
Evans, B., Engelbrecht, H. A., & Jordaan, H. W. (2023) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2023.3295252 · Cited by 33
“In contrast to classical methods, deep learning agents use a neural network to map raw sensor data (LiDAR scans) directly to control commands without requiring explicit state estimation [5].”
MethodHybrid Machine Learning and Reinforcement Learning Framework for Adaptive UAV Obstacle Avoidance
Skarka, W., & Ashfaq, R. (2024) · Aerospace · doi:10.3390/aerospace11110870 · Cited by 33
“…learning SL MAV flying over a woodland [46] CNN SL Trajectory prediction [47] NN, RL SL Estimating collisions [48] DNN, RL SL State estimation [49] Regression SL Estimation of altitude [50] SVM SL Mapping a river [51] GMM, SVM SL Detecting landing sites [52] OBIA SL Precision agriculture [53]…”
Classifying COVID-19 positive X-ray using deep learning models
Rodrigues, I. R., Santos, G. L., Sadok, D. F. H., & Endo, P. T. (2021) · IEEE Latin America Transactions · doi:10.1109/tla.2021.9451232 · Cited by 32
“Há um crescimento significativo de estudos na literatura usando aprendizado de máquina em vários contextos, entre eles análise de mineração de dados [10], previsão do mercado de ações [11]; produtos de consumo [12] e veículos autônomos [13].”
Haklıdır, M., & Temeltaş, H. (2021) · IEEE Access · doi:10.1109/access.2021.3131772 · Cited by 34
“To supervise the learning of the neural network policy, Zhang et al. [73] employed MPC, which only received the full state of the system during the training phase.”
BackgroundMethodInfluentialMulti-Agent Autonomous On-Demand Free Flight Operations in Urban Air Mobility
Yang, X., Deng, L., & Wei, P. (2019) · AIAA Aviation 2019 Forum · doi:10.2514/6.2019-3520 · Cited by 30
Deep Reinforcement Learning for Autonomous Map-Less Navigation of a Flying Robot
Doukhi, O., & Lee, D. J. (2022) · IEEE Access · doi:10.1109/access.2022.3162702 · Cited by 36
“Deep learning’s success in solving the problems in artificial intelligence [8]– FIGURE 1: The developed MAV platform [10] motivates researchers in control and robotics to apply the recent algorithms in common aerial robotics problems like flight control [11] and obstacle avoidance.”
MethodSemantic-aware Active Perception for UAVs using Deep Reinforcement Learning
Bartolomei, L., Teixeira, L., & Chli, M. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9635893 · Cited by 30
“In combination with deep neural networks, deep RL provides a powerful tool capable of mapping high-dimensional sensory inputs to optimal actions, showing promising results in fields, such as autonomous driving [15] and robot navigation [16], [17].”
BackgroundChen, J., Ma, R., & Oyekan, J. O. (2023) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2023.104489 · Cited by 31
Real-Time Trajectory Adaptation for Quadrupedal Locomotion using Deep Reinforcement Learning
Gangapurwala, S., Geisert, M., Orsolino, R., Fallon, M. F., & Havoutis, I. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9561639 · Cited by 34
“A significant amount of research in RL and optimal control (OC) domains has focused on tackling shared objectives using a unified approach [16], [17] including contributions in the form of model-based RL [18], guided policy search (GPS) [19], [20] and even a reformulation of the stochastic optimal control problem in t…”
BackgroundDiesel engine air path control based on neural approximation of nonlinear MPC
Moriyasu, R., Nojiri, S., Matsunaga, A., Nakamura, T., & Jimbo, T. (2019) · Control Engineering Practice · doi:10.1016/j.conengprac.2019.104114 · Cited by 31
Multi-Agent Image Classification via Reinforcement Learning
Mousavi, H., Nazari, M., Takáč, M., & Motee, N. (2019) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8968129 · Cited by 32
Benchmarking model-free and model-based optimal control
Koryakovskiy, I., Kudruss, M., Babuška, R., Caarls, W., Kirches, C., Mombaur, K., Schlöder, J. P., & Vallery, H. (2017) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2017.02.006 · Cited by 29
“The approach was able to learn obstacle avoidance policies for a quadrotor [21].”
BackgroundDeep Reinforcement Learning With NMPC Assistance Nash Switching for Urban Autonomous Driving
Alighanbari, S., & Azad, N. L. (2022) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2022.3167616 · Cited by 31
“In[40],authorsusedMPCasaguidefor trainingofDeepNeuralNetwork(DNN).”
Son, S. H., Kim, J. W., Oh, T. H., Jeong, D. H., & Lee, J. M. (2022) · Journal of Process Control · doi:10.1016/j.jprocont.2022.04.014 · Cited by 29
Tagliabue, A., & How, J. P. (2024) · IEEE Transactions on Robotics · doi:10.1109/tro.2024.3431988 · Cited by 28
“3) Data Augmentation for Efficient/Robust IL: Guided policy search (GPS) [14], [23], [30], [31], [36] introduced first the idea to use trajectories from model-based planners, including MPC, to generate state-action samples (guiding samples) for improved sample efficiency in policy learning.”
BackgroundMethodInfluentialLearning the optimal state-feedback using deep networks
Sanchez-Sanchez, C., Izzo, D., & Hennes, D. (2016) · IEEE Symposium Series on Computational Intelligence (SSCI) · doi:10.1109/ssci.2016.7850105 · Cited by 32
“The initialization area A is x0 ∈ [−5, 5] [m], z0 ∈ [5, 20] [m], vx0 ∈ [−1, 1] [m/s], vz0 ∈ [−1, 1] [m/s], θ0 ∈ [ − π 10 , π 10 ] [rad].”
BackgroundA Scalable Model-Based Learning Algorithm With Application to UAVs
Liang, X. P., Zheng, M., & Zhang, F. (2018) · IEEE Control Systems Letters · doi:10.1109/lcsys.2018.2849576 · Cited by 28
“Recently, the data-driven learning techniques, including imitation learning [22]–[27] and reinforced learning [28]–[32], have been developed and employed in many areas including UAVs.”
BackgroundAn LSTM-Based Ensemble Learning Approach for Time-Dependent Reliability Analysis
Li, M., & Wang, Z. (2020) · Journal of Mechanical Design · doi:10.1115/1.4048625 · Cited by 28
Sample Efficient Learning of Path Following and Obstacle Avoidance Behavior for Quadrotors
Stevšić, S., Nägeli, T., Alonso–Mora, J., & Hilliges, O. (2018) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2018.2856922 · Cited by 27
State-of-the-Art Integrated Guidance and Control Systems in Unmanned Vehicles: A Review
Santoso, F., Garratt, M., & Anavatti, S. G. (2020) · IEEE Systems Journal · doi:10.1109/jsyst.2020.3007428 · Cited by 28
“While there have been numerous applications of neural network systems in machine learning (e.g., deep learning [78]–[80] supervised [81]–[83] as well as unsupervised learning [84], [85], reinforcement learning [86], [87], including Bayesian reinforcement learning techniques [88]); their combination…”
BackgroundModel Predictive Control Guided Reinforcement Learning Control Scheme
Xie, H., Xu, X., Li, Y., Hong, W., & Shi, J. (2020) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn48605.2020.9207398 · Cited by 27
“For example, in MPC-Guided Policy Search [18], the guidance is offline and based on the MPC runtime data, however, the guiding method proposed in this paper is online and directly based on the action of MPC, which makes the implementation of the algorithm simpler and results in the parallel control…”
MethodInfluentialBertram, J. R., & Wei, P. (2020) · AIAA SciTech Forum · doi:10.2514/6.2020-1371 · Cited by 27
Nurbayeva, A., Shintemirov, A., & Rubagotti, M. (2022) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2022.3217833 · Cited by 25
“Thegeneralideaofusinginformationnormallynot available to the “expert” to train the DNN follows a concept often employed in robotics applications; see, e.g., [35].”
BackgroundMethodLearning high-level policies for model predictive control
Song, Y., & Scaramuzza, D. (2020) · Zurich Open Repository and Archive (University of Zurich) · doi:10.5167/uzh-193794 · Cited by 24
Xi, C., Shi, T., Wu, Y., & Sun, L. (2020) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc45102.2020.9294679 · Cited by 24
“Combining machine learning with conventional motion planning methods has already demonstrated effectiveness and efficiency in the literature [10].”
BackgroundMethodGP-ILQG: Data-driven Robust Optimal Control for Uncertain Nonlinear Dynamical SystemsPreprint
Lee, G., Srinivasa, S. S., & Mason, M. T. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.05344 · Cited by 24
“Variants of [22] have been used to generate guiding policies for data-driven RL methods [8, 28].”
MethodLPV MPC Control of an Autonomous Aerial Vehicle
Misin, M., & Puig, V. (2020) · Mediterranean Conference on Control and Automation (MED) · doi:10.1109/med48518.2020.9183041 · Cited by 23
“Moreover, learning approaches like pure reinforcement learning [14], or MPC based deep learning [15], are being applied to quadrotor control, as well as cutting-edge applications of bioinspired flight control systems [16].”
BackgroundDemonstration-Efficient Guided Policy Search via Imitation of Robust Tube MPC
Tagliabue, A., Kim, D.-K., Everett, M., & How, J. P. (2022) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra46639.2022.9812122 · Cited by 23
Dooraki, A. R., & Lee, D. J. (2022) · Machines · doi:10.3390/machines10070500 · Cited by 23
Dual-arm aerial continuum manipulation systems: modeling, pre-grasp planning, and control
Ghorbani, S., Samadikhoshkho, Z., & Janabi‐Sharifi, F. (2023) · Nonlinear Dynamics · doi:10.1007/s11071-022-08212-w · Cited by 23
Guided Policy Search as Approximate Mirror DescentPreprint
Montgomery, W. H., & Levine, S. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1607.04614 · Cited by 23
Self-citationAljehani, F., N’Doye, I., & Laleg‐Kirati, T. (2024) · Aquacultural Engineering · doi:10.1016/j.aquaeng.2024.102511 · Cited by 22
Safety robustness of reinforcement learning policies: A view from robust control
Xiong, H., & Diao, X. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.09.055 · Cited by 23
Worst-case Satisfaction of STL Specifications Using Feedforward Neural Network Controllers
Yaghoubi, S., & Fainekos, G. (2019) · ACM Transactions on Embedded Computing Systems · doi:10.1145/3358239 · Cited by 22
Learning to Fly by CrashingPreprint
Gandhi, D., Pinto, L., & Gupta, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.05588 · Cited by 22
A deep learning framework for causal shape transformation
Lore, K. G., Stoecklein, D., Davies, M., Ganapathysubramanian, B., & Sarkar, S. (2017) · Neural Networks · doi:10.1016/j.neunet.2017.12.003 · Cited by 22
Online learning with stability guarantees: A memory-based warm starting for real-time MPC
Schwenkel, L., Gharbi, M., Trimpe, S., & Ebenbauer, C. (2020) · Automatica · doi:10.1016/j.automatica.2020.109247 · Cited by 20
“ion in MPC with an online evaluation of the explicit control law. The majority of works in this direction employ neural networks as approximate controllers [1], [12], [16], [21], [22], [29], [31] and [41], while support vector machines are used in [11], and the learning problem is formulated as quadratically constrain…”
MethodPlanning high-speed safe trajectories in confidence-rich maps
Heiden, E., Hausman, K., Sukhatme, G. S., & Agha–mohammadi, A. (2017) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8206120 · Cited by 20
“Other learning methods use high-dimensional function approximators (such as neural networks) to cope with the difficulty of planning safe motions straight from the sensory output [22].”
MethodLearning Lyapunov (Potential) Functions from Counterexamples and Demonstrations
Ravanbakhsh, H., & Sankaranarayanan, S. (2017) · Robotics: Science and Systems · doi:10.15607/rss.2017.xiii.049 · Cited by 20
“The overall framework uses a demonstrator that can in fact be a human operator [21] or a complex MPC-based control law [19, 57, 58, 34].”
BackgroundImitation Learning of Nonlinear Model Predictive Control for Emergency Collision Avoidance
Kim, S., Han, K., & Choi, S. B. (2023) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2023.3309962 · Cited by 19
“A similar NMPC-based GPS approach was used to train a neural network policy for autonomous aerial vehicles, and it achieved successful control without knowledge of the full state [32].”
MethodInformed Information Theoretic Model Predictive Control
Kusumoto, R., Palmieri, L., Spies, M., Csiszar, A., & Arras, K. O. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8793945 · Cited by 19
“Lately several algorithms have extended MPC with machine learning [10], [26] to solve more challenging problems coming from the robotics domain with more difficult nonlinear dynamics and interactions with the environment.”
BackgroundMethodDeepControl: Energy-Efficient Control of a Quadrotor using a Deep Neural Network
Varshney, P., Nagar, G., & Saha, I. (2019) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8968236 · Cited by 19
“Zhang et al. [20] proposed a methodology to train a neural network through policy search guided by the data generated from an MPC under full state observations.”
MethodWu, H., Ye, H., Xue, W., & Yang, X. (2022) · IEEE Access · doi:10.1109/access.2022.3185424 · Cited by 18
Propulsionless planar phasing of multiple satellites using deep reinforcement learning
Smith, B., Abay, R., Abbey, J., Balage, S., Brown, M., & Boyce, R. (2020) · Advances in Space Research · doi:10.1016/j.asr.2020.09.025 · Cited by 17
A Meta-MDP Approach to Exploration for Lifelong Reinforcement LearningPreprint
Garcia, F. M., & Thomas, P. S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.00843 · Cited by 18
“ent Control Flight Systems (ICFS) is an area of study that was born out of the necessity to address some of the limitations of PID controllers; where RL has gained significant traction in recent years [30, 31]. One particular scenario were our proposed problem would arise is in using RL to control autonomous vehicles […”
BackgroundObstacle Recognition and Avoidance for UAVs Under Resource-Constrained Environments
Mao, Y., Chen, M., Wei, X., & Chen, B. (2020) · IEEE Access · doi:10.1109/access.2020.3020632 · Cited by 17
“Zhang et al. collected data under full state observation based on Model predictive control (MPC) method to train a neural network to guide UAV to avoid obstacle [28].”
MethodEvaluating Guided Policy Search for Human-Robot Handovers
Kshirsagar, A., Hoffman, G., & Biess, A. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3067299 · Cited by 17
“Zhang et al. [30] augmented the original GPS algorithm with a modelpredictivecontrol(MPC)schemetogeneratetrainingdata withoutcatastrophicfailures.”
MethodYoo, J., Molin, A., Jafarian, M., Esen, H., Dimarogonas, D. V., & Johansson, K. H. (2017) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc.2017.8264468 · Cited by 16
Mitigation of Policy Manipulation Attacks on Deep Q-Networks with Parameter-Space Noise
Behzadan, V., & Munir, A. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-99229-7_34 · Cited by 17
Learning a Lattice Planner Control Set for Autonomous Vehicles
Iaco, R. D., Smith, S. L., & Czarnecki, K. (2019) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2019.8813797 · Cited by 17
“used reinforcement learning to learn a control policy for quadcopters by training on MPC outputs [12].”
MethodSmall-Scale Depthwise Separable Convolutional Neural Networks for Bacteria Classification
Mai, D.-T., & Ishibashi, K. (2021) · Electronics · doi:10.3390/electronics10233005 · Cited by 18
“Image and video classification [1–3], natural language processing [4], robotics [5], and health-care [6,7] are just a few of the fields where AI-based solutions have surpassed human accuracy and insights.”
BackgroundModel predictive control for robust quantum state preparation
Goldschmidt, A. J., DuBois, J. L., Brunton, S. L., & Kutz, J. N. (2022) · Quantum · doi:10.22331/q-2022-10-13-837 · Cited by 17
“In this work, we introduce the model predictive control (MPC) [16, 17] framework since it possess a number of structural advantages for closed-loop control of systems with imperfectly known dynamics: (i) MPC inherits a natural degree of disturbance rejection due to measurement feedback, (ii) receding finite-horizon mo…”
MethodCoverage Testing of Deep Learning Models using Dataset CharacterizationPreprint
Mani, S., Sankaran, A., Tamilselvam, S., & Sethi, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.07309 · Cited by 27
PID-based attitude control of quadrotor using robust pole assignment and LPV modeling
Kazemi, M. H., & Tarighi, R. (2024) · International Journal of Dynamics and Control · doi:10.1007/s40435-023-01372-6 · Cited by 15
Scalable FastMDP for Pre-departure Airspace Reservation and Strategic De-conflict
Bertram, J. R., Wei, P., & Zambreno, J. (2021) · AIAA SciTech Forum · doi:10.2514/6.2021-0779 · Cited by 15
Zhou, S., Helwa, M. K., & Schoellig, A. P. (2020) · The International Journal of Robotics Research · doi:10.1177/0278364920953902 · Cited by 16
“In addition to ILC, reinforcement learning (RL)-based approaches have also been proposed to iteratively optimize the tracking performance (Kiumarsi et al., 2014; Pane et al., 2016; Zhang et al., 2016a).”
BackgroundNagami, K., & Schwager, M. (2021) · Robotics: Science and Systems · doi:10.15607/rss.2021.xvii.062 · Cited by 15
“Other methods use a model-based controller to train a network, as done in [37], where MPC is used in guided policy search.”
MethodJiang, D., Cai, Z., Liu, Z., Peng, H., & Wu, Z. (2022) · Journal of Aerospace Engineering · doi:10.1061/(asce)as.1943-5525.0001426 · Cited by 15
Neural Lyapunov Differentiable Predictive Control
Mukherjee, S., Drgoňa, J., Tuor, A. R., Halappanavar, M. M., & Vrabie, D. (2022) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc51059.2022.9992386 · Cited by 15
MPC policy learning using DNN for human following control without collision
Hirose, N., Tajima, R., & Sukigara, K. (2018) · Advanced Robotics · doi:10.1080/01691864.2017.1422188 · Cited by 15
“DNNs have been successfully applied in many fields, such as robotics, computer vision [15,16], voice recognition, control [6], modeling [17,18].”
MethodMinimax Iterative Dynamic Game: Application to Nonlinear Robot Control Tasks
Ogunmolu, O. P., Gans, N., & Summers, T. (2017) · IEEE/RJS International Conference on Intelligent RObots and Systems · doi:10.1109/iros.2018.8594037 · Cited by 21
Maciel-Pearson, B. G., Carbonneau, P., & Breckon, T. P. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-96728-8_13 · Cited by 15
“Alternatively, synthetic data, from virtual environment models, could potentially replace or at least supplement hard-won real environment data [9,11,18].”
BackgroundAutomated Enemy Avoidance of Unmanned Aerial Vehicles Based on Reinforcement Learning
Cheng, Q., Wang, X., Yang, J., & Shen, L. (2019) · Applied Sciences · doi:10.3390/app9040669 · Cited by 15
RPNet: Rotational pooling net for efficient Micro Aerial Vehicle trail navigation
Agyemang, I. O., Zhang, X., Acheampong, D., Adjei-Mensah, I., Gyamfi, E. O., Arhin, J. R., Ayivi, W., & Chikwendu, I. A. (2022) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2022.105468 · Cited by 14
Handling Sparse Rewards in Reinforcement Learning Using Model Predictive Control
Dawood, M., Dengler, N., de Heuvel, J., & Bennewitz, M. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10161492 · Cited by 14
“attain partial observations from the environment which can be provided using onboard sensors [19].”
BackgroundMethodInfluentialReinforcement Learning and Attractor Neural Network Models of Associative Learning
Hamid, O. H., & Braun, J. (2019) · Studies in computational intelligence · doi:10.1007/978-3-030-16469-0_17 · Cited by 14
Derin Öğrenme Araştırma Alanlarının Literatür Taraması
Yapıcı, M. M., Tekerek, A., & Topaloğlu, N. (2019) · Gazi Journal of Engineering Sciences · doi:10.30855/gmbd.2019.03.01 · Cited by 14
Declarative vs rule-based control for flocking dynamics
Mehmood, U., Paoletti, N., Phan, D., Grosu, R., Lin, S., Stoller, S. D., Tiwari, A., Yang, J., & Smolka, S. A. (2018) · ACM Symposium on Applied Computing · doi:10.1145/3167132.3167222 · Cited by 14
“’s success using MPC to train a DNN-based controller for collision avoidance [30].”
Using Soft Actor-Critic for Low-Level UAV ControlPreprint
Barros, G. M., & Colombini, E. L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.02293 · Cited by 14
“Recent studies have employed Reinforcement Learning for UAV control to complement classic and modern control theory in model-based approaches [6] [7].”
MethodLessons for Robotics From the Control Architecture of the Octopus
Sivitilli, D. M., Smith, J. R., & Gire, D. H. (2022) · Frontiers in Robotics and AI · doi:10.3389/frobt.2022.862391 · Cited by 13
Deep Reinforcement Learning for Peg-in-hole Assembly Task Via Information Utilization Method
Wang, F., Cui, B., Liu, Y., & Ren, B. (2022) · Journal of Intelligent & Robotic Systems · doi:10.1007/s10846-022-01713-1 · Cited by 13
“Reinforcement learning has not only achieved great success in games [1], but also has been extensively studied in the field of robotics [2–4].”
BackgroundLearning Low-Frequency Motion Control for Robust and Dynamic Robot Locomotion
Gangapurwala, S., Campanaro, L., & Havoutis, I. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10160357 · Cited by 18
“A combination of model-based and model-free methods have also been proposed [22], [23], [24], [25].”
Asignacion, A., & Suzuki, S. (2024) · Drones · doi:10.3390/drones8030072 · Cited by 13
“In [95], a robust MPC method enhances safety margins compared to traditional RL methods.”
MethodGuaranteed Performances for Learning-Based Control Systems Using Robust Control Theory
Németh, B., & Gáspár, P. (2021) · Studies in computational intelligence · doi:10.1007/978-3-030-77939-9_4 · Cited by 13
Metadata
- Authors
- Zhang, T., Kahn, G., Levine, S., & Abbeel, P.
- Affiliations
- University of California, Berkeley (US)
- Venue
- 2016 IEEE International Conference on Robotics and Automation (ICRA)
- Publisher
- IEEE
- Published
- 2016-05
- Pages
- 528-535
- ISSN
- 1050-4729 (ISSN-L)
- OpenAlex
- W1923344279
- Cited by
- 452
- References
- 45
- Citations by year
- 2026: 10 · 2025: 18 · 2024: 37 · 2023: 33 · 2022: 54 · 2021: 53 · 2020: 65 · 2019: 93 · 2018: 42 · 2017: 39 · 2016: 8
- FWCI
- 32.08
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Proceedings - IEEE International Conference on Robotics and Automation/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Defense Advanced Research Projects Agency (N66001)
- Army Research Office
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Engineering › Control and Systems Engineering › Adaptive Dynamic Programming Control
- Physical Sciences › Engineering › Control and Systems Engineering › Advanced Control Systems Optimization
- Keywords
- model predictive control, deep neural network policies, autonomous aerial vehicles, quadrotor control, obstacle avoidance, reinforcement learning, state estimation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
