The DOI matches its Crossref record; 43 of the 45 references it cites were found in a registry or an index.
Learning deep control policies for autonomous aerial vehicles with MPC-guided policy search
Open accessAbstract
Model predictive control (MPC) is an effective method for controlling robotic systems, particularly autonomous aerial vehicles such as quadcopters. However, application of MPC can be computationally demanding, and typically requires estimating the state of the system, which can be challenging in complex, unstructured environments. Reinforcement learning can in principle forego the need for explicit state estimation and acquire a policy that directly maps sensor readings to actions, but is difficult to apply to unstable systems that are liable to fail catastrophically during training before an effective policy has been found. We propose to combine MPC with reinforcement learning in the framework of guided policy search, where MPC is used to generate data at training time, under full state observations provided by an instrumented training environment. This data is used to train a deep neural network policy, which is allowed to access only the raw observations from the vehicle's onboard sensors. After training, the neural network policy can successfully control the robot without knowledge of the full state, and at a fraction of the computational cost of MPC. We evaluate our method by learning obstacle avoidance policies for a simulated quadrotor, using simulated onboard sensors and no explicit state estimation at test time.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The frequent sensing and re-planning allow the robot to react dynamically to changes in the environment, while still making more intelligent choices than a purely reactive system (Erez et al., 2011; Lenz et al., 2015; Zhang et al., 2016; Liu et al., 2017; Williams et al., 2017; Gillespie et al., 2018; Amos et al., 2019; Kabzan et al., 2019).”
From Lessons for Robotics From the Control Architecture of the Octopus (2022) · cites this paper as Zhang et al., 2016
In the citing paper
“AField-Programmable Gate Array (FPGA) is another viable solution, showing great improvements in both performance and power consumption in deep learning applications.”
From A survey of deep learning techniques for autonomous driving (2020) · cites this paper as [170]
In the citing paper
“Currently, the largest portion of papers in deep learning for selfdriving cars are focused mainly on perception and End2End learning [81, 124].”
From A survey of deep learning techniques for autonomous driving (2020) · cites this paper as [124]
In the citing paper
“Finally, realtime requirements have to be fulfilled for processing the large amounts of data gathered from the car's sensors suite, as well as for updating the parameters of deep learning systems over high-speed communication lines [170].”
From A survey of deep learning techniques for autonomous driving (2020) · cites this paper as [170]
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 2×
“We used default values for all learning parameters as presented in [14].”
Method(2010) · International Conference on Machine Learning · Cited by 13,995
Playing Atari with Deep Reinforcement Learning
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013) · arXiv · doi:10.48550/arxiv.1312.5602 · Cited by 9,471 · cited 1×
“More recently, deep neural network policies have been used for tasks ranging from robotic control [22], [23] to video game playing [27].”
MethodMuJoCo: A physics engine for model-based control
Todorov, E., Erez, T., & Tassa, Y. (2012) · 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2012.6386109 · Cited by 4,767 · cited 2×
“However, our derivation largely follows prior work [41].”
MethodResultReinforcement learning in robotics: A survey
Kober, J., Bagnell, J. A., & Peters, J. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913495721 · Cited by 3,160 · cited 1×
“An overview of recent reinforcement learning methods in robotics can be found in a recent survey paper [15].”
BackgroundEnd-to-End Training of Deep Visuomotor PoliciesPreprint
Levine, S., Finn, C., Darrell, T., & Abbeel, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1504.00702 · Cited by 2,641 · cited 10×
“In this paper, we use the BADMM version, which specifies the following objective for trajectory optimization: min pi(τ) T∑ t=1 Epi(xt,ut) [ `(xt,ut)− uTt λiµt + νitDKL ( pi(ut|xt)‖πθ(ut|xt) )] (1) where λµt is a Lagrange multiplier on the mean action, and the third term is a KL-divergence penalty.”
BackgroundMethodInfluentialSelf-citationA Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning
Ross, S., Gordon, G. J., & Bagnell, J. A. (2018) · Carnegie Mellon University · doi:10.1184/r1/6550949 · Cited by 1,941
Neural networks for control systems—A survey
Hunt, K. J., Sbarbaro, D. G., Żbikowski, R., & Gawthrop, P. J. (1992) · Automatica · doi:10.1016/0005-1098(92)90053-i · Cited by 1,874 · cited 1×
“neural networks have been used for control for decades [12], [33], limitations on computation and algorithms have made large neural network policies very difficult to learn.”
MethodRobust model predictive control of constrained linear systems with bounded disturbances
Mayne, D. Q., Seron, M. M., & Raković, S. V. (2004) · Automatica · doi:10.1016/j.automatica.2004.08.019 · Cited by 1,680 · cited 1×
“Model predictive control (MPC) is an effective and reliable method for controlling robotic systems, particularly autonomous aerial vehicles such as quadcopters, because of its robustness to moderate model errors [26], ability to use high-level objectives [41], and relative simplicity.”
MethodALVINN, an autonomous land vehicle in a neural network
Pomerleau, D. (2018) · doi:10.1184/r1/6603146 · Cited by 1,369
Flagged first, then by citations
2 of the 45 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Items 201–300 of 445
Online Flight Planner with Dynamic Obstacles for Urban Air Mobility
Bertram, J. R., Yang, X., Brittain, M., & Wei, P. (2019) · AIAA AVIATION Forum · doi:10.2514/6.2019-3625 · Cited by 13
A Reinforcement Learning Approach for Control of a Nature-Inspired Aerial Vehicle
Sufiyan, D., Win, L. S. T., Win, S. K. H., Soh, G. S., & Foong, S. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8794446 · Cited by 13
“These works include learning a low level direct mapping from states to actuator outputs [4], implementing an MPC-guided policy search [5], using RL for stabilizing suspended payloads [6], and also learning a controller for a helicopter during inverted flight [7].”
BackgroundAccurate Pouring using Model Predictive Control Enabled by Recurrent Neural Network
Chen, T., Huang, Y., & Sun, Y. (2019) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8967802 · Cited by 16
“For instance, in [12], the authors combine MPC with reinforcement learning in policy search for autonomous aerial vehicles.”
BackgroundDeshpande, A. M., Kumar, R., Minai, A. A., & Kumar, M. (2020) · ASME Dynamic Systems and Control Conference (DSCC) · doi:10.1115/dscc2020-3319 · Cited by 16
Fast real-time SDRE controllers using neural networks
da Costa, R. F., Saotome, O., Rafikova, E., & Machado, R. (2021) · ISA Transactions · doi:10.1016/j.isatra.2021.02.019 · Cited by 13
Efficient Interaction-Aware Interval Analysis of Neural Network Feedback Loops
Jafarpour, S., Harapanahalli, A., & Coogan, S. (2024) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2024.3420968 · Cited by 12
Yuste, P. C., Iglesias, J. A., & Sanchis, A. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128362 · Cited by 13
Imitation learning of a model predictive controller for real-time humanoid robot walking
Porto, V. G. B. D. A., Melo, D. C., Máximo, M. R. O. A., & Afonso, R. J. M. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.109919 · Cited by 12
Sferrazza, C., Muehlebach, M., & Raffaello D’andrea, B. (2017) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc.2017.8264428 · Cited by 12
“In [21], deep learning is combined with MPC for guided policy search, where MPC is used to generate data at training time.”
MethodAdaptive Battery Control with Neural Networks
Kazhamiaka, F., Keshav, S., & Rosenberg, C. (2019) · ACM International Conference on Future Energy Systems (e-Energy) · doi:10.1145/3307772.3331032 · Cited by 12
“Our key insight is that, to circumvent this problem, deep neural networks can be trained to approximate the control policy of an MPC at a fraction of the online computational cost [16, 46].”
BackgroundMethodDeep Value of Information Estimators for Collaborative Human-Machine Information Gathering
Lore, K. G., Sweet, N., Kumar, K., Ahmed, N. R., & Sarkar, S. (2016) · ACM/IEEE International Conference on Cyber-Physical Systems (ICCPS) · doi:10.1109/iccps.2016.7479095 · Cited by 12
Eingebettete Optimierung in der Regelungstechnik – Grundlagen und Herausforderungen
Findeisen, R., Graichen, K., & Mönnigmann, M. (2018) · at - Automatisierungstechnik · doi:10.1515/auto-2018-0083 · Cited by 12
“(7) Mögliche Erweiterungen des Schätzproblems (5) betreffen die Berücksichtigung von Prozessrauschen sowie die Erweiterung von (5a) durch einen zusätzlichen Term V(x0|k) (Englisch: arrival cost), der die Konvergenz des Schätzers zu den realen Zuständen garantiert und Messwer- 2 Zur Vereinfachung der Darstellung wird i…”
InfluentialChen, Y., & Wu, S. (2023) · Journal of Transportation Engineering Part A Systems · doi:10.1061/jtepbs.0000772 · Cited by 11
Kyuroson, A., Banerjee, A., Tafanidis, N. A., Satpute, S. G., & Nikolakopoulos, G. (2024) · European Journal of Control · doi:10.1016/j.ejcon.2024.101052 · Cited by 11
“…Perceptron (MLP) network architectures and therefore are computationally efficient, DRL agents can be deployed on platforms with limited computing power, to achieve autonomous capabilities such as collision avoidance [8], path-planning, trajectory optimization [9] as well as station-keeping [10].”
BackgroundDucard, G., & Carughi, G. (2024) · Drones · doi:10.3390/drones8120727 · Cited by 11
Self-Guided Actor-Critic: Reinforcement Learning from Adaptive Expert Demonstrations
Zhang, H., Yin, C., Zhang, Y., & Jin, S. (2020) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc42340.2020.9304004 · Cited by 11
“Another kind of algorithms, including Guided Policy Search (GPS) [13][14] and Policy Learning using Adaptive Trajectory Optimization (PLATO) [15] , resorts to optimal controllers such as model-predictive controller (MPC) [18] or LQG to generate supervision data.”
MethodVid2Param: Modeling of Dynamics Parameters From Video
Asenov, M., Burke, M., Angelov, D., Davchev, T., Subr, K., & Ramamoorthy, S. (2019) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2019.2959476 · Cited by 11
“Our approach is closely related to [27] as we use supervision during the training phase of our model, and then use this learned approximation at test time.”
MethodControl of a Cable-Driven Parallel Robot via Deep Reinforcement Learning
Ma, T., Xiong, H., Zhang, L., & Diao, X. (2019) · IEEE Workshop on Advanced Robotics and its Social Impacts (ARSO) · doi:10.1109/arso46408.2019.8948792 · Cited by 11
“Many DRL algorithms, such as deep Q-network (DQN) [14], trust region policy optimization (TRPO) [15], mirror descent guided policy search (MDGPS) [16], and deep deterministic policy gradient (DDPG) [17], have recently been used to solve a variety of robot manipulation tasks [18]–[20].”
BackgroundStability Certificates for Neural Network Learning-based Controllers using Robust Control Theory
Nguyen, H. H., Zieger, T., Wells, S. C., Νικολακοπούλου, Α., Braatz, R. D., & Findeisen, R. (2021) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.23919/acc50511.2021.9482637 · Cited by 11
Parameter-Adaptive Event-Triggered Sliding Mode Control for a Mobile Robot
Tran, T. D., Nguyen, T. T., Duong, V. T., Nguyen, H. H., & Nguyen, T. T. (2022) · Robotics · doi:10.3390/robotics11040078 · Cited by 11
Drgoňa, J., Kiš, K., Tuor, A. R., Vrabie, D., & Klaučo, M. (2020) · arXiv (Cornell University) · Cited by 11
Pervasive AI for IoT Applications: Resource-efficient Distributed Artificial Intelligence.Preprint
Baccour, E., Mhaisen, N., Abdellatif, A. A., Erbad, A., Mohamed, A. M. S., Hamdi, M., & Guizani, M. (2021) · arXiv (Cornell University) · Cited by 11
DeepMPCVS: Deep Model Predictive Control for Visual ServoingPreprint
Katara, P., Harish, Y. V. S., Pandya, H., Gupta, A. K., Sanchawala, A., Kumar, G., Bhowmick, B., & Madhava Krishna, K. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.00788 · Cited by 12
“Model predictive control (MPC) approaches have been successful in learning complex skills in robotics such as controlling quadrotors [26], and humanoids [27] using accurate system dynamics.”
BackgroundA Novel Guided Deep Reinforcement Learning Tracking Control Strategy for Multirotors
Hua, H., Wang, Y., Zhong, H., Zhang, H., & Fang, Y. (2024) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2024.3374752 · Cited by 10
“The twin delayed deep deterministic policy gradient (TD3) algorithm [39] is adopted to optimize networks parameters.”
Neuromorphic Attitude Estimation and Control
Stroobants, S., Wagter, C. D., & de Croon, G. (2025) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2025.3553418 · Cited by 13
“Zhang et al. [27] have demonstrated in simulation that by using an expert privileged policy, an MLP can be trained to performend-to-endcontrol.”
Design of Approximate Explicit Model Predictive Controller Using Parametric Optimization
Tsai, Y.-K., & Malak, R. (2022) · Journal of Mechanical Design · doi:10.1115/1.4055326 · Cited by 10
Fourier and Wavelet Transforms
Brunton, S. L., & Kutz, J. N. (2019) · Cambridge University Press eBooks · doi:10.1017/9781108380690.003 · Cited by 10
Model Predictive Control for Trajectory Tracking on Differentiable ManifoldsPreprint
Lu, G., Xu, W. D., & Zhang, F. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.15233 · Cited by 10
“MPC is an inclusive framework for defining objective functions and constraints, making it amenable to complex autonomy, like learning [15], [16], obstacle avoiding [17], [18], perception awareness [19] and swarm collaboration [20].”
BackgroundDeep Reinforcement Learning-Based Hierarchical Motion Planning Strategy for Multirotors
Hua, H., Wang, Y., Zhong, H., Zhang, H., & Fang, Y. (2025) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2024.3523594 · Cited by 9
“We adopt an actor-critic network in this article, mainly because the policy gradient and value estimation are implemented in the networks [2], [19], [29].”
Approximate Model Predictive Control with Recurrent Neural Network for Autonomous Driving Vehicles
Quan, Y. S., & Chung, C. C. (2019) · Annual Conference of the Society of Instrument and Control Engineers of Japan (SICE) · doi:10.23919/sice.2019.8859955 · Cited by 9
“In [11] reinforcement learning is combined with MPC in the framework of guided policy search, while MPC is used to generate training data under full state observation.”
MethodUğurlu, H. İ., Kalkan, S., & Saranlı, A. (2021) · Journal of Intelligent & Robotic Systems · doi:10.1007/s10846-021-01412-3 · Cited by 9
“[42] has combined model-predictive control with DRL to improve performance.”
MethodDeep Reinforcement Learning-Based Diving/Pull-out Control for Bioinspired Morphing UAVs
Ye, B., Li, J., Li, J., Liu, C., Li, J., & Yang, Y. (2022) · Unmanned Systems · doi:10.1142/s2301385023410066 · Cited by 9
Learning Decentralized Control Policies for Multi-Robot Formation
Jiang, C., Chen, Z., & Guo, Y. P. (2019) · IEEE/ASME International Conference on Advanced Intelligent Mechatronics · doi:10.1109/aim.2019.8868898 · Cited by 9
“In [13], Zhang et al. proposed a learning approach based on model predictive control and guided policy search to learn deep control policies for autonomous aerial vehicles.”
BackgroundVision Based Autonomous Navigation of Quadcopter using Reinforcement Learning
Walvekar, A., Goel, Y., Jain, A., Chakrabarty, S., & Kumar, A. (2019) · IEEE International Conference on Automation, Electronics and Electrical Engineering (AUTEEE) · doi:10.1109/auteee48671.2019.9033244 · Cited by 9
Quadrotor navigation in dynamic environments with deep reinforcement learning
Fang, J., Sun, Q., Chen, Y., & Tang, Y. S. (2021) · Assembly Automation · doi:10.1108/aa-11-2020-0183 · Cited by 9
Reinforced iLQR: A Sample-Efficient Robot Locomotion Learning
Zong, T., Sun, L., & Liu, Y. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9561223 · Cited by 9
Supervised deep actor network for imitation learning in a ground-air UAV-UGVs coordination task
Nguyen, H. T., Garratt, M., Bui, L. T., & Abbass, H. A. (2017) · IEEE Symposium Series on Computational Intelligence (SSCI) · doi:10.1109/ssci.2017.8285387 · Cited by 9
“Other recent attempts looked at Deep Neural Networks (DNN) to design control policies for UAV [14].”
BackgroundTowards nominal stability certification of deep learning-based controllers
Nguyen, H. H., Matschek, J., Zieger, T., Savchenko, A., Noroozi, N., & Findeisen, R. (2020) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.23919/acc45564.2020.9147564 · Cited by 9
“Machine learning has gained popularity in recent decades in a wide field of applications, spanning from object detection, to classification and control, see e.g. [1, 8, 9, 10, 18, 19, 30].”
BackgroundDeep value of information estimators for collaborative human-machine information gatheringPreprint
Lore, K. G., Sweet, N., Kumar, K., Ahmed, N. R., & Sarkar, S. (2016) · arXiv (Cornell University) · doi:10.5555/2984464.2984467 · Cited by 9
Maharmeh, H. A., Sarhan, N. J., Ismail, M., & Alhawari, M. (2023) · IEEE Open Journal of Circuits and Systems · doi:10.1109/ojcas.2023.3332853 · Cited by 8
“…the cornerstone for modern artificial intelligence (AI) applications due to the unprecedented achieved accuracy in image classification [1], [2], object recognition/detection [3], [4], [5], speech recognition [6], [7], game playing [8], [9], healthcare [10], [11], [12], and robotics [13], [14].”
BackgroundA Survey on Learning Motion Planning and Control for Mobile Robots: Toward Embodied Intelligence
Wang, M., Niu, Y., Wang, B., Zhang, W., & Wang, C. (2026) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2026.3656889 · Cited by 9
“Zhang et al. [54] used a privileged MPC with full state information to train a vision-based obstacle avoidance policy.”
Learning to Grasp with Primitive Shaped Object Policies
Beltran-Hernandez, C. C., Petit, D., Ramírez-Alpizar, I. G., & Harada, K. (2019) · IEEE/SICE International Symposium on System Integration (SII) · doi:10.1109/sii.2019.8700399 · Cited by 8
“GPS has been applied to various robotic tasks [16], [17], [18], even in tasks that require contact-rich manipulation skills, where it has proved”
MethodYaghoubi, S., & Fainekos, G. (2020) · Information Theory and Applications Workshop (ITA) · doi:10.1109/ita50056.2020.9244969 · Cited by 8
“Model-based Reinforcement Learning methods are also another line of work close to our work [35, 54, 58].”
MethodReward Signal Design for Autonomous Racing
Evans, B., Engelbrecht, H. A., & Jordaan, H. W. (2021) · International Conference on Advanced Robotics (ICAR) · doi:10.1109/icar53236.2021.9659438 · Cited by 8
“A model predictive controller (MPC) with full state estimation, has been used as a teacher, to train a neural network to avoid obstacles [7].”
BackgroundMethodLeveraging Deep Reinforcement Learning For Active Shooting Under Open-World Setting
Tzimas, A., Passalis, N., & Tefas, A. (2020) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme46284.2020.9102966 · Cited by 8
Model-based Lookahead Reinforcement LearningPreprint
Hong, Z.-W., Pajarinen, J., & Peters, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.06012 · Cited by 13
“The follow-up works [45; 23] further adopt MPC to provide supervision for an policy.”
Guided Policy Search Model-based Reinforcement Learning for Urban Autonomous DrivingPreprint
Xu, Z., Chen, J., & Tomizuka, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.03076 · Cited by 8
“The model based or policy search based RL are more sample efficient and have been proved capable of solving a number of complicated manipulation tasks [17] and aerial vehicle [18] in both simulator and real world.”
Tagliabue, A., & How, J. P. (2024) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3386053 · Cited by 7
Papachristos, C., Dang, T., Khattak, S., Mascarich, F., Khedekar, N., & Alexis, K. (2018) · now publishers, Inc. eBooks · doi:10.1561/9781680835496 · Cited by 7
Constrained Imitation Learning for a Flapping Wing Unmanned Aerial Vehicle
Tejaswi, K., & Lee, T. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3194682 · Cited by 7
Supervised Meta-Reinforcement Learning With Trajectory Optimization for Manipulation Tasks
Wang, L., Zhang, Y., Zhu, D., Coleman, S., & Kerr, D. (2023) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2023.3286465 · Cited by 9
“In recent years, with the assistance of powerful and flexible neural network representations, RL has made great success in many fields ranging from playing games [1], [2], [3], [4], [5] to autonomous flying and driving [6], [7], [8], [9], [10], [11], [12].”
Autonomous Landing of eVTOL Vehicles for Advanced Air Mobility via Deep Reinforcement Learning
Deniz, S., Wu, Y., & Wang, Z. (2024) · AIAA SciTech Forum · doi:10.2514/6.2024-2485 · Cited by 7
Trajectory Tracking Control for Multiple Quadrotors Based on a Neurobiological-Inspired System
Valencia, D., & Kim, D. (2019) · IEEE International Conference on Robotic Computing (IRC) · doi:10.1109/irc.2019.00098 · Cited by 7
“For example, the hybrid neuro-fuzzy autopilot presents in [7], a hybrid of conventional backpropagation training for nonlinear autoregressive with exogenous inputs networks (NARX) present in [8], the hybrid control technique comprising MPC assisted with reinforcement learning introduced in [9], the spiking neural netw…”
MethodVision based real-time obstacle avoidance for drones using a time-to-collision estimation approach
Gomes, F., Hormigo, T., & Ventura, R. (2020) · IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR) · doi:10.1109/ssrr50563.2020.9292597 · Cited by 7
“Besides the classic methods, a reinforcement learning technique is shown in [11] where a neural network is trained with generated data from the MPC and online trajectory optimization resulting in a neural network policy that can successfully control the drone.”
MethodSparsity and Compressed Sensing
Brunton, S. L., & Kutz, J. N. (2019) · Cambridge University Press eBooks · doi:10.1017/9781108380690.004 · Cited by 8
MPC-guided Imitation Learning of Bayesian Neural Network Policies for the Artificial Pancreas
Chen, H., Paoletti, N., Smolka, S. A., & Lin, S. (2021) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc45484.2021.9683240 · Cited by 7
“approach, are tailored to work with MPC teachers [43], [12], [44], [45].”
BackgroundLiang, Y., Li, G., Xu, M., Zhao, J., Hao, F., & Shi, H. (2023) · Defence Technology · doi:10.1016/j.dt.2023.07.012 · Cited by 7
Evasive Maneuvering for UAVs: An MPC Approach
Castillo-López, M., Olivares-Méndez, M. A., & Voos, H. (2017) · Advances in intelligent systems and computing · doi:10.1007/978-3-319-70833-1_67 · Cited by 7
Kim, T., Lee, C., Seo, H., Choi, S., Kim, W., & Kim, H. J. (2018) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8593843 · Cited by 7
“Some of GPS papers dealt with mobile robots [15], [16], but they did not perform realworld experiments.”
BackgroundKutsuzawa, K., Sakaino, S., & Tsuji, T. (2019) · Advanced Robotics · doi:10.1080/01691864.2019.1673204 · Cited by 7
“[13] showed that neural networks trained by MPC can reduce the computation cost.”
MethodBehavior Planning For Connected Autonomous Vehicles Using Feedback Deep Reinforcement Learning.Preprint
Han, S., & Miao, F. (2020) · arXiv (Cornell University) · Cited by 7
Greedy-DAgger - A Student Rollout Efficient Imitation Learning Algorithm
Torok, M., Deghat, M., & Song, Y. (2025) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2025.3536297 · Cited by 7
Lai, J., Wu, Z., Ren, Z., Tan, Q., & Xiao, H. (2024) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2024.3520481 · Cited by 6
“The control theory method in conjunction with deep learning method has produced in several impressive break-throughs in control engineering, including those in navigation and control systems [26], [27], [28], [29], [30], [31], [32], [33].”
Zhang, Y., Wen, W., & Yan, P. (2024) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc58415.2024.10919536 · Cited by 6
“We employ CasADi [25] with IPOP [26] as the solver for the numerical optimization problem.”
Design and robustness analysis of autonomous vehicles in intersections
Szilassy, P. Á., Németh, B., & Gáspár, P. (2019) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2019.08.090 · Cited by 6
Stochastic Optimal Control for Multivariable Dynamical Systems Using Expectation Maximization
Mallick, P., & Chen, Z. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3190246 · Cited by 6
“In this article, we specifically use differential dynamic-programming-based optimal control methods such as: 1) iLQG [13]; 2) MPC [14]; and 3) BADMM [5] to carry forward the optimization routine.”
MethodResultInfluentialOutput Feedback Tube MPC-Guided Data Augmentation for Robust, Efficient Sensorimotor Policy Learning
Tagliabue, A., & How, J. P. (2022) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros47612.2022.9981888 · Cited by 6
“This approach has been used to generate efficient sensorimotor policies [7], [9]–[12] capable of producing control commands from raw sensory data, bypassing the computational cost of control and state estimation, while providing benefits in terms of latency and robustness.”
MethodRaja, M. M., Wang, H., Arshad, M. H., Kish, G. J., & Zhao, Q. Z. (2023) · IEEE Access · doi:10.1109/access.2023.3270773 · Cited by 6
“Some of them include autoregressive models [26], [27], and neural network (NN) and deep reinforcement learning models [28], [29], [30].”
BackgroundWagle, S., & Paranjape, A. A. (2020) · Winter Simulation Conference · doi:10.1109/wsc48552.2020.9383893 · Cited by 6
“…system model that can be exploited by MPC (Williams, Wagener, Goldfain, Drews, Rehg, Boots, and Theodorou 2017), or using MPC to generate training sets for neural network-based policy generators which operate at a fraction of the computational cost of MPC (Zhang, Kahn, Levine, and Abbeel 2016).”
BackgroundMethodBrunton, S. L., & Kutz, J. N. (2019) · Cambridge University Press eBooks · doi:10.1017/9781108380690.012 · Cited by 7
Intention Recognition With ProbLog
Smith, G. B., Belle, V., & Petrick, R. P. A. (2022) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2022.806262 · Cited by 6
“deep neural net (Pierson and Gashler, 2017) and that the amount of computational resources needed for neural networks are not always practical (Zhang et al., 2016).”
BackgroundNeural Flocking: MPC-Based Supervised Learning of Flocking Controllers
Mehmood, U., Roy, S., Grosu, R., Smolka, S. A., Stoller, S. D., & Tiwari, A. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-45231-5_1 · Cited by 6
“o our current nearest-neighbors formulation. We also plan to switch from 2D geometry to 3D geometry and to use a more realistic model of agent dynamics based on a quadcopter model, as given e.g. 8 in [21]. Finally, motivated by the quadcopter study of [21], we will seek to combine MPC with reinforcement learning in th…”
MethodBrunton, S. L., & Kutz, J. N. (2019) · Cambridge University Press eBooks · doi:10.1017/9781108380690.009 · Cited by 6
Neural Network Memory Architectures for Autonomous Robot NavigationPreprint
Chen, S. W., Atanasov, N., Khan, A., Karydis, K., Lee, D. D., & Kumar, V. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.08049 · Cited by 9
“Zhang et al. used Model Predictive Control (MPC) in guided policy search to train a neural network to navigate a quadrotor using lidar inputs [21].”
BackgroundDeep Learning in Robotics: A Review of Recent ResearchPreprint
Pierson, H. A., & Gashler, M. S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.07217 · Cited by 6
“Levine et al. [128] used reinforcement learning (Structure D) to show that this can be superior to separate systems for perception and control.”
BackgroundInfluentialDrgoňa, J., Tuor, A. R., & Vrabie, D. (2020) · arXiv (Cornell University) · Cited by 6
Luo, Q., Li, Y., Zeng, J., Wu, G., & Wang, Y. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.129762 · Cited by 5
Locher, L., Stai, E., & Hug, G. (2025) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2025.3617966 · Cited by 5
“In another work, [22], training datasets are constructed using solely the MPC-derived actions and the RL-based actions are taken into account only once the policy has converged.”
Multi-rotor Robot Learning to Fly in a Bio-inspired Way Using Reinforcement Learning
Dooraki, A. R., & Lee, D. J. (2019) · International Conference on Ubiquitous Robots (UR) · doi:10.1109/urai.2019.8768681 · Cited by 5
Online Multi-Objective Model-Independent Adaptive Tracking Mechanism for Dynamical Systems
Abouheaf, M., Gueaieb, W., & Spinello, D. (2019) · Robotics · doi:10.3390/robotics8040082 · Cited by 5
“Among the challenges associated with this class of control algorithms, is the need to have full or partial knowledge of the dynamics of the underlying systems, which can degrade their operation in the presence of uncertainties [5,6].”
BackgroundMethodRobust model agnostic predictive control algorithm for randomly excited dynamical systems
Tripura, T., & Chakraborty, S. (2023) · Probabilistic Engineering Mechanics · doi:10.1016/j.probengmech.2023.103517 · Cited by 5
A Trainable Multiplication Layer for Auto-correlation and Co-occurrence Extraction
Hayashi, H., & Uchida, S. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-20890-5_27 · Cited by 5
Variable Impedance Control of Manipulator Based on DQN
Hou, Y., Xu, H., Luo, J., Lei, Y., Xu, J., & Zhang, H. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-66645-3_25 · Cited by 5
Optimal real-time scheduling of battery operation using reinforcement learning
Juarez, C. Q., & Musı́lek, P. (2021) · Conference proceedings - Canadian Conference on Electrical and Computer Engineering · doi:10.1109/ccece53047.2021.9569124 · Cited by 5
Fuzzy PID Controller for UAV Based on Reinforcement Learning
Zhang, B., Zhang, W., Mou, J., Yang, R., & Zhang, Y. (2023) · Lecture notes in electrical engineering · doi:10.1007/978-981-99-0479-2_160 · Cited by 5
Control of Quadrotor Drone with Partial State Observation via Reinforcement Learning
Shan, G., Zhang, Y., Gao, Y. H., Wang, T., & Chen, J. (2019) · Chinese Automation Congress · doi:10.1109/cac48633.2019.8996394 · Cited by 5
“In [9], researchers utilize the Model Predictive Control (MPC) policy search method to construct the control model, and in [10], deterministic reinforcement method is used to train the optimal control algorithm.”
MethodChander, B. (2019) · Advances in computer and electrical engineering book series · doi:10.4018/978-1-7998-1159-6.ch001 · Cited by 5
Modeling Dynamic Systems for Multi-Step Prediction with Recurrent Neural Networks
Mohajerin, N. (2017) · UWSpace (University of Waterloo) · Cited by 5
Smith, J. S., Hwang, J.-H., Chu, F.-J., & Vela, P. A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.05132 · Cited by 5
“Similar problems may arise for methods that learn motion primitives for specific scenarios [4], as a particular motion primitive may be biased towards one topological instance of a reaction from a set of potential reactions.”
BackgroundVid2Param: Modelling of Dynamics Parameters from VideoPreprint
Asenov, M., Burke, M., Angelov, D., Davchev, T., Subr, K., & Ramamoorthy, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.06422 · Cited by 5
“Our approach is closely related to [43] as we use supervision during the training phase of our model, and then use this learned approximation at test time.”
MethodArtificial intelligence algorithm models for agents of embodiment for drone applications
Kourav, S., Verma, K., & Sundararajan, M. (2024) · Information Systems Engineering and Management · doi:10.1007/978-3-031-68256-8_4 · Cited by 4
Robust LPV design with neural network for the steering control of autonomous vehicles
Németh, B. (2019) · European Control Conference (ECC) · doi:10.23919/ecc.2019.8795979 · Cited by 4
“[2], [3].”
See What the Robot Can't See: Learning Cooperative Perception for Visual Navigation
Blumenkamp, J., Li, Q., Wang, B., Liu, Z., & Prorok, A. (2023) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros55552.2023.10342535 · Cited by 4
Linear Parameter Varying Model-Based Predictive Trajectory Tracking Control of Quadrotor
Pal, S., & Deb, A. K. (2024) · International Conference on Signal Processing and Integrated Networks (SPIN) · doi:10.1109/spin60856.2024.10511643 · Cited by 4
Guided Cooperation in Hierarchical Reinforcement Learning via Model-Based Rollout
Wang, H., Tang, Z., Sun, Y., Wang, F., Zhang, S., & Chen, Y. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3425809 · Cited by 4
“Several studies have attempted to deploy model-based HRL on various real-world control tasks, such as autonomous aerial vehicles [23] and physical humanoids [24].”
Guan, X., Hu, T., Zhang, Z., Wang, Y., Liu, Y., Wang, Y., Hao, J., & Li, G. (2024) · International Journal of Advanced Robotic Systems · doi:10.1177/17298806241301581 · Cited by 4
“26 Worse, the sampling and training are dif fi - cult and have no advantage in solving time.”
Learning Sample-Efficient Target Reaching for Mobile Robots
Khan, A., Kumar, V., & Ribeiro, A. (2018) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8594168 · Cited by 5
Input-Decoupled Q-Learning for Optimal Control
Phan, M. Q., & Azad, S. M. B. (2019) · The Journal of the Astronautical Sciences · doi:10.1007/s40295-019-00157-4 · Cited by 4
Mallick, P., Chen, Z., & Zamani, M. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.01.142 · Cited by 4
A Neural Network Engine for Resource Constrained Embedded Systems
Jelčicová, Z., Mardari, A., Andersson, O., Kasapaki, E., & Sparsø, J. (2020) · Asilomar Conference on Signals, Systems and Computers (ACSSC) · doi:10.1109/ieeeconf51394.2020.9443426 · Cited by 4
“Deep neural networks (DNNs) are ubiquitous, finding their application in various areas such as image and video processing [1], robotics [2], medicine [3], games [4] as well as audiology [5].”
Background
Metadata
- Authors
- Zhang, T., Kahn, G., Levine, S., & Abbeel, P.
- Affiliations
- University of California, Berkeley (US)
- Venue
- 2016 IEEE International Conference on Robotics and Automation (ICRA)
- Publisher
- IEEE
- Published
- 2016-05
- Pages
- 528-535
- ISSN
- 1050-4729 (ISSN-L)
- OpenAlex
- W1923344279
- Cited by
- 452
- References
- 45
- Citations by year
- 2026: 10 · 2025: 18 · 2024: 37 · 2023: 33 · 2022: 54 · 2021: 53 · 2020: 65 · 2019: 93 · 2018: 42 · 2017: 39 · 2016: 8
- FWCI
- 32.08
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Proceedings - IEEE International Conference on Robotics and Automation/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Defense Advanced Research Projects Agency (N66001)
- Army Research Office
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Engineering › Control and Systems Engineering › Adaptive Dynamic Programming Control
- Physical Sciences › Engineering › Control and Systems Engineering › Advanced Control Systems Optimization
- Keywords
- model predictive control, deep neural network policies, autonomous aerial vehicles, quadrotor control, obstacle avoidance, reinforcement learning, state estimation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
