The DOI matches its Crossref record; 43 of the 45 references it cites were found in a registry or an index.
Learning deep control policies for autonomous aerial vehicles with MPC-guided policy search
Open accessAbstract
Model predictive control (MPC) is an effective method for controlling robotic systems, particularly autonomous aerial vehicles such as quadcopters. However, application of MPC can be computationally demanding, and typically requires estimating the state of the system, which can be challenging in complex, unstructured environments. Reinforcement learning can in principle forego the need for explicit state estimation and acquire a policy that directly maps sensor readings to actions, but is difficult to apply to unstable systems that are liable to fail catastrophically during training before an effective policy has been found. We propose to combine MPC with reinforcement learning in the framework of guided policy search, where MPC is used to generate data at training time, under full state observations provided by an instrumented training environment. This data is used to train a deep neural network policy, which is allowed to access only the raw observations from the vehicle's onboard sensors. After training, the neural network policy can successfully control the robot without knowledge of the full state, and at a fraction of the computational cost of MPC. We evaluate our method by learning obstacle avoidance policies for a simulated quadrotor, using simulated onboard sensors and no explicit state estimation at test time.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The frequent sensing and re-planning allow the robot to react dynamically to changes in the environment, while still making more intelligent choices than a purely reactive system (Erez et al., 2011; Lenz et al., 2015; Zhang et al., 2016; Liu et al., 2017; Williams et al., 2017; Gillespie et al., 2018; Amos et al., 2019; Kabzan et al., 2019).”
From Lessons for Robotics From the Control Architecture of the Octopus (2022) · cites this paper as Zhang et al., 2016
In the citing paper
“AField-Programmable Gate Array (FPGA) is another viable solution, showing great improvements in both performance and power consumption in deep learning applications.”
From A survey of deep learning techniques for autonomous driving (2020) · cites this paper as [170]
In the citing paper
“Currently, the largest portion of papers in deep learning for selfdriving cars are focused mainly on perception and End2End learning [81, 124].”
From A survey of deep learning techniques for autonomous driving (2020) · cites this paper as [124]
In the citing paper
“Finally, realtime requirements have to be fulfilled for processing the large amounts of data gathered from the car's sensors suite, as well as for updating the parameters of deep learning systems over high-speed communication lines [170].”
From A survey of deep learning techniques for autonomous driving (2020) · cites this paper as [170]
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 2×
“We used default values for all learning parameters as presented in [14].”
Method(2010) · International Conference on Machine Learning · Cited by 13,995
Playing Atari with Deep Reinforcement Learning
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013) · arXiv · doi:10.48550/arxiv.1312.5602 · Cited by 9,471 · cited 1×
“More recently, deep neural network policies have been used for tasks ranging from robotic control [22], [23] to video game playing [27].”
MethodMuJoCo: A physics engine for model-based control
Todorov, E., Erez, T., & Tassa, Y. (2012) · 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2012.6386109 · Cited by 4,767 · cited 2×
“However, our derivation largely follows prior work [41].”
MethodResultReinforcement learning in robotics: A survey
Kober, J., Bagnell, J. A., & Peters, J. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913495721 · Cited by 3,160 · cited 1×
“An overview of recent reinforcement learning methods in robotics can be found in a recent survey paper [15].”
BackgroundEnd-to-End Training of Deep Visuomotor PoliciesPreprint
Levine, S., Finn, C., Darrell, T., & Abbeel, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1504.00702 · Cited by 2,641 · cited 10×
“In this paper, we use the BADMM version, which specifies the following objective for trajectory optimization: min pi(τ) T∑ t=1 Epi(xt,ut) [ `(xt,ut)− uTt λiµt + νitDKL ( pi(ut|xt)‖πθ(ut|xt) )] (1) where λµt is a Lagrange multiplier on the mean action, and the third term is a KL-divergence penalty.”
BackgroundMethodInfluentialSelf-citationA Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning
Ross, S., Gordon, G. J., & Bagnell, J. A. (2018) · Carnegie Mellon University · doi:10.1184/r1/6550949 · Cited by 1,941
Neural networks for control systems—A survey
Hunt, K. J., Sbarbaro, D. G., Żbikowski, R., & Gawthrop, P. J. (1992) · Automatica · doi:10.1016/0005-1098(92)90053-i · Cited by 1,874 · cited 1×
“neural networks have been used for control for decades [12], [33], limitations on computation and algorithms have made large neural network policies very difficult to learn.”
MethodRobust model predictive control of constrained linear systems with bounded disturbances
Mayne, D. Q., Seron, M. M., & Raković, S. V. (2004) · Automatica · doi:10.1016/j.automatica.2004.08.019 · Cited by 1,680 · cited 1×
“Model predictive control (MPC) is an effective and reliable method for controlling robotic systems, particularly autonomous aerial vehicles such as quadcopters, because of its robustness to moderate model errors [26], ability to use high-level objectives [41], and relative simplicity.”
MethodALVINN, an autonomous land vehicle in a neural network
Pomerleau, D. (2018) · doi:10.1184/r1/6603146 · Cited by 1,369
Flagged first, then by citations
2 of the 45 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Items 301–400 of 445
Periodic Intra-ensemble Knowledge Distillation for Reinforcement Learning
Hong, Z.-W., Nagarajan, P., & Maeda, G. J. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-86486-6_6 · Cited by 4
Optimal schedule generation for single-channel crude transfer using a multi-model approach
Paranjape, A. A., Baranwal, M., Wagle, S., Lotti, R., Majumder, S., & Bullière, A.-L. (2022) · Computers & Chemical Engineering · doi:10.1016/j.compchemeng.2022.107732 · Cited by 4
Deep Reinforcement Learning Based Game Decision Algorithm for Digital Media Education
Li, Z.-N., Kuang, P., Zhang, T., Yan, H.-R., & Gu, X. (2019) · International Computer Conference on Wavelet Active Media Technology and Information Processing (ICCWAMTIP) · doi:10.1109/iccwamtip47768.2019.9067731 · Cited by 4
“Zhang [7], Balduzzi [8] et al. also studied the application of strategy gradient method in deep reinforcement learning.”
MethodSelf-citationDouble Meta-Learning for Data Efficient Policy Optimization in Non-Stationary Environments
Aghapour, E., & Ayanian, N. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9561219 · Cited by 5
“RL approaches have been applied to a wide range of applications, from autonomous navigation [1], [2] to manipulation [3], [4].”
BackgroundRyad, I., Zidan, M., Rashad, N., Bakr, D., Bakr, N., Yehia, N., Ismail, Y., AbdelSalam, M., & Salem, A. (2021) · IEEE International Conference on Electronics, Circuits and Systems (ICECS) · doi:10.1109/icecs53924.2021.9665583 · Cited by 4
“Recent research work [2, 3, 4, 5, 6, 7, 8] has investigated training and testing AI Agents of self-driving cars, drones, and Automated Guided Vehicles (AGVs) with synthetically generated data in digital twin simulators, which can be produced in bulk with correct labels.”
Mahé, A., Pradalier, C., & Geist, M. (2018) · International Conference on System Theory, Control and Computing (ICSTCC) · doi:10.1109/icstcc.2018.8540719 · Cited by 4
“Model Predictive Control (MPC) is a control algorithm that is widely used in this context [9], [11], [15].”
MethodReinforcement and Imitation Learning Applied to Autonomous Aerial Robot Control
Barros, G. M., & Colombini, E. L. (2020) · doi:10.5753/wtdr_ctdr.2020.14956 · Cited by 4
“al [17] designed a low-level control using MPC with guided-policy-search.”
MethodLee, W., Liu, Y., Liu, P., Ma, S., Choi, H., Zhang, X., & Gupta, R. (2019) · Proceedings of the CGO/Proceedings of the ... CGO · doi:10.1109/cgo.2019.8661177 · Cited by 4
Arora, S., Choudhury, S., & Scherer, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.02573 · Cited by 4
“lvers — keeping track of evolving uncertainty over planning horizon. However, these approaches still need the expectation over the state space to be computed. Imitation learning based approaches [19]–[21] address this concern through data driven techniques. MDP solvers are used over sampled MDP problems to train a pol…”
Development of a SHM system by elastic guided waves applied to aeronautic structures
Kulakovskyi, A. (2019) · theses.fr (ABES) · doi:10.70675/cbf81c2azc5f6z4998zbe53z6df8f5d11bc9 · Cited by 4
Deniz, S., Barzegar, A., Sagirli, F. Y., Vasiloff, K. N., & Wang, Z. (2025) · Control Engineering Practice · doi:10.1016/j.conengprac.2025.106519 · Cited by 4
Brunton, S. L., & Kutz, J. N. (2019) · Cambridge University Press eBooks · doi:10.1017/9781108380690.010 · Cited by 3
Pushpangathan, J. V., Kandath, H., & Francis, B. (2022) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2023.03.010 · Cited by 3
Autonomous Quadcopter Trajectory Tracking with MPC
Fathani, N., Santoso, A., Sahal, M., & Arifin, I. (2024) · IEEE Student Conference on Research and Development (SCOReD) · doi:10.1109/scored64708.2024.10872633 · Cited by 3
Zhang, Y., Wang, Y., Yan, P., & Wen, W. (2025) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2025.3616580 · Cited by 3
“We employ CasADi [38] with IPOPT [39] as the solver for the numerical optimization problem.”
Deep Drone Acrobatics (Extended Abstract)
Kaufmann, E., Loquercio, A., Ranftl, R., Müller, M., Koltun, V., & Scaramuzza, D. (2021) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/650 · Cited by 3
Learning Model Predictive Control Law for Nonlinear Systems
Wang, R., Li, H., & Xu, D. (2022) · International Symposium on Autonomous Systems (ISAS) · doi:10.1109/isas55863.2022.9757271 · Cited by 3
“[26] used MPC to guide the learning of reinforcement learning algorithms and proposed MPC-Guided Policy Search to ensure the learning efficiency and safety of reinforcement learning.”
MethodNon-Cascaded Control Barrier Functions for the Safe Control of Quadrotors
Zeng, W., Cao, H., Lu, W., & Xiong, H. (2023) · Chinese Control Conference (CCC) · doi:10.23919/ccc58697.2023.10241141 · Cited by 3
“The cascaded CBFs cannot be applied to quadrotors with non-cascaded controllers (e.g., an end-to-end learning-based controller [22], [23]).”
A Supervised Reinforcement Learning Algorithm for Controlling Drone Hovering
Wu, J., Yang, Z. X., Zhuo, H., Xu, C., Zhang, C., He, N., Liao, L., & Wang, Z. (2024) · Drones · doi:10.3390/drones8030069 · Cited by 3
“At present, the flight control methods for drones mainly include linear control methods [8–10], nonlinear control methods [11–13], and intelligent control methods [14–17].”
MethodA visual programming paradigm for abstract deep learning model development
Tamilselvam, S., Panwar, N., Khare, S., Aralikatte, R., Sankaran, A., & Mani, S. (2019) · Indian Conference on Human-Computer Interaction (IndiaHCI) · doi:10.1145/3364183.3364202 · Cited by 6
“Automation has progressed significantly in various application domains such as self-driving cars [12] [13] [19], flight control systems [28], and medical diagnosis [22] [16] primarily due to the advent of deep learning technology.”
BackgroundImitation Learning for UAS Navigation in Cluttered Environments
Harris, C. M., Choi, Y., & Mavris, D. N. (2021) · AIAA SciTech Forum · doi:10.2514/6.2021-0452 · Cited by 3
“Zhang et al used model predictive control supplied with global state information as the expert to train a policy in simulation [11].”
MethodBrunton, S. L., & Kutz, J. N. (2019) · Cambridge University Press eBooks · doi:10.1017/9781108380690.008 · Cited by 4
Learning Deep Robot Controllers by Exploiting Successful and Failed Executions
Esteban, D., Rozo, L. D., & Caldwell, D. G. (2018) · IEEE-RAS International Conference on Humanoid Robots (Humanoids) · doi:10.1109/humanoids.2018.8625003 · Cited by 3
“The cost term max(dSAFE − d(ot), 0) is a hinge loss that does not penalize the robot if the distance o is further than dSAFE and favors collision avoidance behaviors [20].”
BackgroundImportance Sampling for Deep System Identification
Mahé, A., Richard, A., Mouscadet, B., Pradalier, C., & Geist, M. (2019) · International Conference on Advanced Robotics (ICAR) · doi:10.1109/icar46387.2019.8981590 · Cited by 3
“Recent attempts at black-box system identification using deep learning have shown great results, on both linear and non-linear systems [9], [10], [11].”
BackgroundDengler, C., & Lohmann, B. (2020) · Robotics · doi:10.3390/robotics9020029 · Cited by 3
“[33] for the obstacle avoidance of a simulated quadrotor.”
BackgroundGuided Policy Search Based Control of a High Dimensional Advanced Manufacturing Process
Surana, A., & Reddy, K. (2022) · IEEE Conference on Control Technology and Applications (CCTA) · doi:10.1109/ccta49430.2022.9966165 · Cited by 3
“The nominal trajectory can be constructed in a variety of ways. e.g. from demonstrations, using randomized control inputs or via solution obtained using MPC [19].”
MethodInfluentialGuided Policy Search Methods: A Review
Du, J., Fu, J. Y., & Li, C. (2021) · Journal of Physics Conference Series · doi:10.1088/1742-6596/1748/2/022039 · Cited by 3
Reinforcement Learning: An Industrial Perspective
Surana, A. (2021) · Studies in systems, decision and control · doi:10.1007/978-3-030-60990-0_21 · Cited by 3
Comparisons of RRT and MCTS for Safe Assured Path Planning in Urban Air Mobility
Wu, P., & Chen, J. (2022) · AIAA SciTech Forum · doi:10.2514/6.2022-1841 · Cited by 3
Accelerating trail navigation for unmanned aerial vehicle: A denoising deep-net with 3D-NLGL
Agyemang, I. O., Zhang, X., Adjei-Mensah, I., Agbley, B. L. Y., Mawuli, B. C., Fiasam, L. D., & Sey, C. (2022) · Journal of Intelligent & Fuzzy Systems · doi:10.3233/jifs-220693 · Cited by 3
“From the viewpoint of vision-based navigation, a UAV navigated in an obstacle environment based on Reinforcement Learning (RL) and model predictive control [15].”
Augmented Random Search for Quadcopter Control: An alternative to Reinforcement Learning
Tiwari, A. K., & Nadimpalli, S. V. (2019) · International Journal of Information Technology and Computer Science · doi:10.5815/ijitcs.2019.11.03 · Cited by 5
Evaluation of Prioritized Deep System Identification on a Path Following Task
Mahé, A., Richard, A., Aravecchia, S., Geist, M., & Pradalier, C. (2021) · Journal of Intelligent & Robotic Systems · doi:10.1007/s10846-021-01341-1 · Cited by 5
“Recent attempts at black-box system identification using deep learning showed great results, on both linear and non-linear systems [10,34,7].”
BackgroundMBB: Model-Based Baseline for Efficient Reinforcement Learning.Preprint
Lyu, X., Li, S., Siriya, S., Pu, Y., & Chen, M. (2020) · arXiv (Cornell University) · Cited by 3
Combining Reinforcement Learning with Model Predictive Control for On-Ramp MergingPreprint
Lubars, J., Gupta, H., Chinchali, S., Li, L., Raja, A., Srikant, R., & Wu, X. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.08484 · Cited by 3
“Although there has been a lot of work combining MPC with reinforcement learning and other learning-based techniques (see [12], [13], [14] and [15]), there seems to be no such work done for the problem of on-ramp merging to”
BackgroundDifferentiable Robust LQR LayersPreprint
Vien, N. A., & Neumann, G. M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.05535 · Cited by 3
Roadmap of AlphaGo to AlphaStar: Problems and challenges
Rong, Y. (2022) · 2nd International Conference on Artificial Intelligence, Automation, and High-Performance Computing (AIAHPC 2022) · doi:10.1117/12.2641824 · Cited by 2
“Reinforcement learning, as another research direction of machine learning, has achieved success in the fields of robot control [10], simulation [11], and games [12].”
BackgroundImplications of memory embedding and hierarchy on the performance of MAVeC AI accelerators
Hossain, M. R., & Rahman, M. (2025) · Memories - Materials Devices Circuits and Systems · doi:10.1016/j.memori.2025.100131 · Cited by 5
Jang, J., & Kim, J. (2023) · Ocean Engineering · doi:10.1016/j.oceaneng.2023.113620 · Cited by 2
Wang, M., Li, X., Wang, K., & Li, Y. (2022) · Chinese Control and Decision Conference (CCDC) · doi:10.1109/ccdc55256.2022.10034380 · Cited by 2
Convergence Analysis of Different Categories of Gradient Descent Algorithms
Han, R. (2023) · International Conference on Computer Science, Electronic Information Engineering and Intelligent Control Technology (CEI) · doi:10.1109/cei60616.2023.10527829 · Cited by 2
“In model parallelism, because the entire model is split into individual worker nodes, the entire training process cannot proceed when a worker node goes down [7].”
Energy-Efficient Time-Domain Computation for Edge Devices: Challenges and Prospects
Maharmeh, H. A., Ismail, M., & Alhawari, M. (2024) · Foundations and Trends® in Integrated Circuits and Systems · doi:10.1561/3500000013 · Cited by 2
Performance Comparison of Deep and Shallow Network for Quadcopter Automation
Mittall, D., Kumar, K., Hashmi, S. N., Kumar, P., Nanda, A., & Chandra, S. (2018) · IEEE International Conference on Industrial and Information Systems (ICIIS) · doi:10.1109/iciinfs.2018.8721383 · Cited by 2
“Keywords—Deep reinforcement learning, Deep Deterministic Policy Gradient (DDPG), Actor-critic, Markov Decision Process (MDP), Quadcopter Automation I. INTRODUCTION In the upcoming years, an increasing number of autonomous systems will pervade urban and domestic environments.”
MethodModel predictive control based deep neural network for dynamic manipulation
Furuta, D., Kutsuzawa, K., Okamoto, T., Sakaino, S., & Tsuji, T. (2017) · Annual Conference of the IEEE Industrial Electronics Society (IECON) · doi:10.1109/iecon.2017.8216902 · Cited by 2
“In the conventional method [7], a DNN learns MPC with only one set of parameters as shown in Fig.”
MethodDefect Sizing Using Convolution Neural Network Applied to Guided Wave Imaging
Kulakovskyi, A., Mesnil, O., Chapuis, B., D’Almeida, O., & Lhémery, A. (2019) · International Workshop on Structural Health Monitoring (IWSHM) · doi:10.12783/shm2019/32195 · Cited by 2
Multi-Model Recurrent Neural Network Control for Lane Change Systems under Speed Variation
Quan, Y. S., Kim, J. S., Lee, S.-H., & Chung, C. C. (2020) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc45102.2020.9294524 · Cited by 2
“In [2], reinforcement learning is combined with model predictive control (MPC) in the framework of guided policy search, while MPC is used to generate training data.”
MethodApprenticeship Bootstrapping Reinforcement Learning for Sky Shepherding of a Ground Swarm in Gazebo
Nguyen, H. T., Garratt, M., & Abbass, H. A. (2020) · Unmanned system technologies · doi:10.1007/978-3-030-60898-9_10 · Cited by 2
Brunton, S. L., & Kutz, J. N. (2019) · Cambridge University Press eBooks · doi:10.1017/9781108380690.006 · Cited by 2
Improved offset-free model predictive control utilizing learned model-plant mismatch map
Son, S. H., Kim, J. W., Oh, T. H., Lee, G., & Lee, J. M. (2022) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2022.07.541 · Cited by 2
“Zhang et al. (2016), Chen et al. (2018), and Hertneck et al. (2018) derive a reliable policy by supervised learning from a nominal MPC policy to avoid the system failure from exploratory policy.”
BackgroundDeep Reinforcement Learning with semi-expert distillation for autonomous UAV cinematography
Sochopoulos, A., Mademlis, I., Charalampakis, E., Papadopoulos, S., & Pitas, I. (2023) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme55011.2023.00230 · Cited by 2
“f(·), g(·) and h(·) are time-dependent vector fields defining the system dynamics, the equality constraints, and the inequality constraints, respectively [15].”
BackgroundMethodScaling simulation-to-real transfer by learning a latent space of robot skills
Julian, R., Heiden, E., He, Z., Zhang, H., Stefan Schaal, A., Lim, J. J., Sukhatme, G. S., & Hausman, K. (2020) · The International Journal of Robotics Research · doi:10.1177/0278364920944474 · Cited by 2
“Zhang et al. (2016) employed MPC to generate training data for a neural network policy.”
MethodBrunton, S. L., & Kutz, J. N. (2019) · Cambridge University Press eBooks · doi:10.1017/9781108380690.011 · Cited by 2
Chen, S., & Wen, J. T.-Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.00082 · Cited by 2
“And the NN is used either to approximate policy functions [25]–[27] and value functions [28], [29], mimic a control law [30], or approximate the dynamic systems.”
MethodFully Bayesian Recurrent Neural Networks for Safe Reinforcement LearningPreprint
Benatan, M., & Pyzer‐Knapp, E. O. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.03308 · Cited by 2
“…constraints (Garcıa and Fernández, 2015; Berkenkamp et al., 2017), and is a field of study that is becoming increasingly important as more and more automated systems are being designed to operate in safety-critical situations (Zhang et al., 2016; Chen et al., 2019; Shalev-Shwartz et al., 2016).”
BackgroundModel-plant mismatch learning offset-free model predictive controlPreprint
Son, S. H., Kim, J. W., Oh, T. H., & Lee, J. M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.02753 · Cited by 2
Deep Learning Alternative to Explicit Model Predictive Control for Unknown Nonlinear SystemsPreprint
Drgoňa, J., Kiš, K., Tuor, A. R., Vrabie, D., & Klaučo, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.03699 · Cited by 2
“These approaches are known in the machine learning community as imitation learning [33], or MPC-guided policy search [34].”
BackgroundIntelligent control of a quadrotor using reinforcement learning with proximal policy optimization
Lopes, G. C. (2018) · doi:10.47749/t/unicamp.2018.1079803 · Cited by 2
Autonomous decision-making for UAVs operating under environmental and object detection uncertainty
Sandino, J. (2022) · Queensland University of Technology · doi:10.5204/thesis.eprints.232513 · Cited by 2
AirCapRL: Autonomous Aerial Human Motion Capture Using Deep Reinforcement Learning
Tallamraju, R., Saini, N., Bonetto, E., Pabst, M., Liu, Y. T., Black, M. J., & Ahmad, A. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.3013906 · Cited by 1
“A model-predictive control guided policy search was proposed in [13] where supervised learning is used to obtain policies which map the on-board aerial robot sensor observations to control actions.”
Automatic adjustment of laparoscopic pose using deep reinforcement learning
Yu, L., Xia, Y., Wang, P., & Sun, L. (2022) · Mechanical sciences · doi:10.5194/ms-13-593-2022 · Cited by 1
Lv, F., Wu, L., Ma, T., Ren, C., & Ma, S. (2023) · Chinese Control Conference (CCC) · doi:10.23919/ccc58697.2023.10241156 · Cited by 1
Shmalko, E. Y., & Diveev, A. (2023) · International Conference on Control, Decision and Information Technologies (CoDIT) · doi:10.1109/codit58514.2023.10284376 · Cited by 1
“These methods represent a standard method of motion planning and are common in many problems for robots [2], [3].”
MethodGanesh, P., Ramos, J. H., Goecks, V. G., Paquet, J., Longmire, M. A., Waytowich, N. R., & Brink, K. (2023) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2023.10.774 · Cited by 1
Multi-Agent Guided Deep Reinforcement Learning Approach Against State Perturbed Adversarial Attacks
Çerçi, Ç., & Temeltaş, H. (2024) · IEEE Access · doi:10.1109/access.2024.3485036 · Cited by 1
“Zhang et al. [25] studied Model Predictive Control (MPC) guided policy search method.”
MethodComputationally efficient data‐driven model predictive control for modular multilevel converters
Raja, M. M., Wang, H., Arshad, M. H., Kish, G. J., & Zhao, Q. Z. (2024) · IET Electric Power Applications · doi:10.1049/elp2.12523 · Cited by 1
Robot Modeling With Autoregressive Transformers
Fañanás-Anaya, J., Nicolás, G. L., & Sagüés, C. (2024) · Iberian Robotics Conference (ROBOT) · doi:10.1109/robot61475.2024.10797406 · Cited by 1
Liu, Q. (2025) · Science and Technology of Engineering Chemistry and Environmental Protection · doi:10.61173/kj6t1269 · Cited by 1
Tsourveloudis, C., & Doitsidis, L. (2026) · Journal of Intelligent & Robotic Systems · doi:10.1007/s10846-026-02408-7 · Cited by 1
“Zhang et al. [29] used MPC to generate optimal trajectories offline, then trained RL policies to mimic these demonstrations, achieving robust tracking but inheriting MPC’s computational cost and model dependence.”
Bridging the Sim-to-Real Gap: A Control Framework for Imitation Learning of Model Predictive Control
Kim, S., Lee, J., Han, K., & Choi, S. B. (2026) · International Journal of Control Automation and Systems · doi:10.1007/s12555-026-00040-7 · Cited by 1
“Studies have shown that this method works effectively, verified by robot gait control [3], drone [4] and vehicle control [5, 6], reducing MPC’s computational load by at least 10 to 20 times while maintaining its control performance.”
A Deep Learning Approach for Gain-Scheduled PID Control Design via Advanced Supervisory Controllers
Atam, E., Zeybek, A., Özateş, Ş. B., & Güzelkaya, N. (2026) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2026.3669780 · Cited by 1
“Infact,inliteraturetherehavebeen somerelevantstudies,forexample,onapproximatingmodelpre-dictivecontrollerswithartificialneuralnetwork(ANN)-based mappingsforapplicationtosystemsrequiringfastcontrolinput calculation[22],[23],[24],[25],[26],[27],[28].”
Nazeer, M. S., Aggravi, M., Giordano, P. R., & Pacchierotti, C. (2024) · International Conference on Control, Automation and Robotics (ICCAR) · doi:10.1109/iccar61844.2024.10569813 · Cited by 1
“Further use of ML for aerial robotic applications are in autonomous navigation and path planning [30], [31], [32] as well as in obstacle avoidance [33], [34], [35], and vision-based formation control [36].”
Huang, J., Hong, H., Wang, N., Ma, H., An, H., & Lang, L. (2024) · International Journal of Control Automation and Systems · doi:10.1007/s12555-021-1105-x · Cited by 1
“The model complexity directly affects the solution method and velocity, and the solution methods are developed into linear, nonlinear, robust [23-25] and learning MPC [26,27].”
Kumar, S., & Sharma, S. (2024) · Elsevier eBooks · doi:10.1016/b978-0-443-22158-3.00009-0 · Cited by 2
Rocha, L., Bidinotto, J. H., Heintz, F., Tiger, M., & Vivaldini, K. C. T. (2024) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros58592.2024.10801910 · Cited by 1
“The Deep Q-Network (DQN) serves as an effective example, demonstrating prowess in fast tracking, exploration, and obstacle avoidance in complex and uncertain environments [4], [19].”
Approximate Bayes Optimal Policy Search using Neural Networks
Castronovo, M., François-Lavet, V., Raphael Fonteneau, D., Ernst, D., & Couëtoux, A. (2017) · International Conference on Agents and Artificial Intelligence (ICAART) · doi:10.5220/0006191701420153 · Cited by 1
“In practice, this happens for example when training a drone to fly in a safe environment before sending it on the operation field (Zhang et al., 2015).”
BackgroundRegression and Model Selection
Brunton, S. L., & Kutz, J. N. (2019) · Cambridge University Press eBooks · doi:10.1017/9781108380690.005 · Cited by 1
Deep Reinforcement Learning with Temporal-Awareness Network
Liu, Z.-Y., Liu, J., Li, W., & Zuo, X. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-63833-7_24 · Cited by 1
Model Predictive Q-Learning (MPQ-L) for Bilinear Systems
Phan, M. Q., & Azad, S. M. B. (2020) · Springer eBooks · doi:10.1007/978-3-030-55240-4_5 · Cited by 1
Han, D. (2021) · Journal of the Korea Computer Graphics Society · doi:10.15701/kcgs.2021.27.1.9 · Cited by 1
“가능할 때의 제어기를 초기화 시키고 policy search를 통해 드론의 일부 상태 관측에 기반한 제어기로 적응시키는 방법을 제안하였다[21].”
Multi-Stage Decision-Making Skill Learning for Soccer Robot
Chen, Z., He, Z.-Y., Du, H., Han, C., Wang, Y., Chen, Z., & Xiong, R. (2021) · IEEE International Conference on Real-time Computing and Robotics (RCAR) · doi:10.1109/rcar52367.2021.9517586 · Cited by 1
DMotion: Robotic Visuomotor Control with Unsupervised Forward Model Learned from Videos
Yuan, H., Wu, R., Zhao, A., Zhang, H., Ding, Z., & Dong, H. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9636362 · Cited by 1
“Model predictive control (MPC) [17], [18] is widely applied in robotics as a traditional approach for control, which has been testified to be effective in many robotic control tasks, including quadrotor control [19], dynamic legged robot control [20], robotic arm manipulations [21], SLAM of locomotory robots [22], aut…”
BackgroundMoment based model predictive control for linear systems: Additive perturbations case
Saltik, M. B., Özkan, L., & Weiland, S. (2022) · International Journal of Robust and Nonlinear Control · doi:10.1002/rnc.6279 · Cited by 1
A Lateral Acceleration Prediction Model based on Current Vehicle State
Li, S., Wang, X., Lu, X., Wang, Z., & Yu, Z. (2022) · CAA International Conference on Vehicular Control and Intelligence (CVCI) · doi:10.1109/cvci56766.2022.9964938 · Cited by 1
Strongly-Typed Agents are Guaranteed to Interact SafelyPreprint
Balduzzi, D. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.07450 · Cited by 2
“, 2016), and driving autonomous vehicles (Zhang et al., 2016).”
BackgroundDrone Path-Following in GPS-Denied Environments using Convolutional NetworksPreprint
Samy, M., Amer, K., Shaker, M., & Elhelw, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.01658 · Cited by 2
“[9] combined RL with model predictive control to train a deep neural network on obstacle avoidance when deployed on a UAV.”
MethodModelling, Design, and Control of Energy Systems: A Data-Driven Approach
Kazhamiaka, F. (2019) · UWSpace (University of Waterloo) · Cited by 1
Learning Unstable Dynamical Systems with Time-Weighted Logarithmic LossPreprint
Nar, K., Xue, Y., & Dai, A. M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.05189 · Cited by 1
“uts of the system. Identification of dynamical systems from time-series data is an important problem for various applications, such as model prediction in reinforcement learning [Lambert et al., 2019, Zhang et al., 2016], analysis of medical health records [Rubanova et al., 2019] and prediction with financial time-serie…”
MethodEmbedded Optimization in Control: An Introduction, Opportunities, and Challenges
Findeisen, R., Rose, A., Graichen, K., & Mönnigmann, M. (2025) · Elsevier eBooks · doi:10.1016/b978-0-443-14081-5.00129-x · Cited by 1
Deep Action Sequence Learning for Causal Shape TransformationPreprint
Lore, K. G., Stoecklein, D., Davies, M., Ganapathysubramanian, B., & Sarkar, S. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.05368
How to Learn a Model CheckerPreprint
Phan, D., Grosu, R., Paoletti, N., Smolka, S. A., & Stoller, S. D. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.01935
“The optimal control algorithm is typically an offline trajectory optimization procedure, or as in [72], policies are trained using an MPC controller, which makes the learned policy more robust to model errors and, compared to classical MPC, allows to circumvent the problem of state estimation.”
MethodInductive Certificate Synthesis for Control DesignPreprint
Ravanbakhsh, H. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.01822
“This allows using a variety of approaches ranging from trajectory optimizations [155], human expert demonstrations [63], and sample-based methods [76, 66], which can be probabilistically complete.”
Modeling Preemptive Behaviors for Uncommon Hazardous Situations From DemonstrationsPreprint
Parashar, P., Cosgun, A., Nakhaei, A., & Fujimura, K. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.00143
“ases in simulation and use approaches like reinforcement learning (RL), inverse RL or learning from demonstration to model the behavior. RL however either requires an accurate model of the environment[15] or large amount of exploration[9] before figuring out how http://acsweb.ucsd.edu/ pparasha/ 31st Conference on Neur…”
BackgroundAdvanced-step Stochastic Model Predictive Control using Random Forests
Wang, R., & Bao, J. (2018) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc.2018.8619533
“In [17], MPC-guided deep learning is applied to construct offline global explicit MPC policies.”
MethodNeural Flocking: MPC-based Supervised Learning of Flocking ControllersPreprint
Roy, S., Mehmood, U., Grosu, R., Smolka, S. A., Stoller, S. D., & Tiwari, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.09813
“o our current nearest-neighbors formulation. We also plan to switch from 2D geometry to 3D geometry and to use a more realistic model of agent dynamics based on a quadcopter model, as given e.g. 8 in [21]. Finally, motivated by the quadcopter study of [21], we will seek to combine MPC with reinforcement learning in th…”
MethodKutsuzawa, K., Kusano, H., Kume, A., & Yamaguchi, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.03253 · Cited by 1
Indirect Methods for Robot Skill Learning
Cabala, E. (2019) · CINECA IRIS Institutial Research Information System (University of Genoa) · doi:10.15167/esteban-cabala-domingo-francisco_phd2019-07-11
Sample-Efficient Policy Learning based on Completely Behavior Cloning
Zou, Q., Wang, L., Li, Y., & Liu, J. (2019) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc.2019.8914085
“e-varying linear-Gaussian controllers [14]. While prior applications of guided policy search rely on a learned model of system dynamic, it can cause extremely error if the learned model is inaccurate [22, 23, 24]. In order to overcome this challenge, Zhang uses MPC controller as the computational expert, and takes ful…”
MethodMPC-guided Imitation Learning of Neural Network Policies for the Artificial PancreasPreprint
Chen, H., Paoletti, N., Smolka, S. A., & Lin, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.01283
“Some of these, like our approach, are tailored to work with MPC teachers (Zhang et al., 2016; Kahn et al., 2017; Amos et al., 2018; Lowrey et al., 2019).”
BackgroundMotion Planning and Safety for Autonomous Driving
Iaco, R. D. (2019) · UWSpace (University of Waterloo)
Robustness analysis and reconfiguration strategy of autonomous vehicles in intersections
Szilassy, P. Á., Németh, B., Gáspár, P., Szauter, F., & Pup, D. (2019) · IEEE International Symposium on Applied Computational Intelligence and Informatics (SACI) · doi:10.1109/saci46893.2019.9111527
“[4], [5].”
Mallick, P., & Chen, Z. (2020) · arXiv (Cornell University)
Metadata
- Authors
- Zhang, T., Kahn, G., Levine, S., & Abbeel, P.
- Affiliations
- University of California, Berkeley (US)
- Venue
- 2016 IEEE International Conference on Robotics and Automation (ICRA)
- Publisher
- IEEE
- Published
- 2016-05
- Pages
- 528-535
- ISSN
- 1050-4729 (ISSN-L)
- OpenAlex
- W1923344279
- Cited by
- 452
- References
- 45
- Citations by year
- 2026: 10 · 2025: 18 · 2024: 37 · 2023: 33 · 2022: 54 · 2021: 53 · 2020: 65 · 2019: 93 · 2018: 42 · 2017: 39 · 2016: 8
- FWCI
- 32.08
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Proceedings - IEEE International Conference on Robotics and Automation/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Defense Advanced Research Projects Agency (N66001)
- Army Research Office
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Engineering › Control and Systems Engineering › Adaptive Dynamic Programming Control
- Physical Sciences › Engineering › Control and Systems Engineering › Advanced Control Systems Optimization
- Keywords
- model predictive control, deep neural network policies, autonomous aerial vehicles, quadrotor control, obstacle avoidance, reinforcement learning, state estimation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
