Autonomous Helicopter Aerobatics through Apprenticeship Learning
The DOI matches its Crossref record; 57 of the 57 references it cites were found in a registry or an index.
Abstract
Autonomous helicopter flight is widely regarded to be a highly challenging control problem. Despite this fact, human experts can reliably fly helicopters through a wide range of maneuvers, including aerobatic maneuvers at the edge of the helicopter’s capabilities. We present apprenticeship learning algorithms, which leverage expert demonstrations to efficiently learn good controllers for tasks being demonstrated by an expert. These apprenticeship learning algorithms have enabled us to significantly extend the state of the art in autonomous helicopter aerobatics. Our experimental results include the first autonomous execution of a wide range of maneuvers, including but not limited to in-place flips, in-place rolls, loops and hurricanes, and even auto-rotation landings, chaos and tic-tocs, which only exceptional human pilots can perform. Our results also include complete airshows, which require autonomous transitions between many of these maneuvers. Our controllers perform as well as, and often even better than, our expert pilot.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Maximum Likelihood from Incomplete Data Via the EM Algorithm
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1977.tb01600.x · Cited by 50,444
A New Approach to Linear Filtering and Prediction Problems
Kalman, R. E. (1960) · Journal of Basic Engineering · doi:10.1115/1.3662552 · Cited by 31,751
Needleman, S. B., & Wunsch, C. D. (1970) · Journal of Molecular Biology · doi:10.1016/0022-2836(70)90057-4 · Cited by 11,288 · cited 2×
“At this point, we can solve the maximization over τ using a dynamic programming algorithm known in the speech recognition literature as dynamic time warping (Sakoe and Chiba 1978) and in the biological sequence alignment literature as the Needleman–Wunsch algorithm (Needleman and Wunsch 1970).”
(1995) · Cited by 10,558
Dynamic Programming Algorithm Optimization for Spoken Word Recognition
Sakoe, H., & Chiba, S. (1978) · IEEE Transactions on Acoustics Speech and Signal Processing · doi:10.1109/tassp.1978.1163055 · Cited by 6,727
GELB, A. (1974) · CERN Document Server (European Organization for Nuclear Research) · Cited by 6,405
Proceedings of the 25th international conference on Machine learning
Cohen, W. W., McCallum, A., & Roweis, S. T. (2008) · Cited by 5,691
Preparata, F. P., & Shamos, M. I. (1985) · Springer eBooks · doi:10.1007/978-1-4612-1098-6 · Cited by 3,655
Optimal Control: Linear Quadratic Methods
Anderson, B. D. O., & Moore, J. B. (1979) · Cited by 2,823
The Journal of the Astronautical Sciences
(2013) · The Journal of the Astronautical Sciences · doi:10.1007/40295.2195-0571 · Cited by 2,775
Apprenticeship learning via inverse reinforcement learning
Abbeel, P., & Ng, A. Y. (2004) · Twenty-first international conference on Machine learning - ICML '04 · doi:10.1145/1015330.1015430 · Cited by 2,769
Self-citationProceedings of the 2004 American Control Conference
(2004) · Proceedings of the 2004 American Control Conference · doi:10.23919/acc.2004.1383555 · Cited by 2,605
Principles of Helicopter Aerodynamics
Leishman, J. G. (2000) · Cited by 2,496
Computational geometry. an introduction
Preparata, F. P., & Shamos, M. I. (1985) · Cited by 2,488
(1998) · Springer eBooks · doi:10.1007/978-94-011-5014-9 · Cited by 2,418
A View of the Em Algorithm that Justifies Incremental, Sparse, and other Variants
Neal, R. M., & Hinton, G. E. (1998) · Learning in Graphical Models · doi:10.1007/978-94-011-5014-9_12 · Cited by 2,187 · cited 1×
“(See also, e.g., Neal and Hinton (1999) for more details on EM.)”
Background2013 IEEE International Conference on Robotics and Automation
(2013) · doi:10.1109/icra19839.2013 · Cited by 2,025
(2000) · doi:10.2460/ajvr.67.2.323 · Cited by 1,621 · cited 2×
“These coupled tasks are non-trivial for systems with complex dynamics, such as helicopters.”
BackgroundKalman Filtering for Spacecraft Attitude Estimation
Lefferts, E. J., Markley, F. L., & Shuster, M. D. (1982) · Journal of Guidance, Control, and Dynamics · doi:10.2514/3.56190 · Cited by 1,344
On Learning, Representing, and Generalizing a Task in a Humanoid Robot
Calinon, S., Guenter, F., & Billard, A. (2007) · IEEE Transactions on Systems, Man and Cybernetics, Part B (Cybernetics) · doi:10.1109/tsmcb.2006.886952 · Cited by 1,090 · cited 3×
“…apprenticeship learning algorithms for modeling the dynamics of the helicopter, (iii) combining these apprenticeship learning algorithms with (a variation of) existing optimal control algorithms, namely, a receding horizon variation of linear quadratic control methods for non-linear systems (a…”
BackgroundMethodInfluentialProceedings for AIAA Guidance Navigation and Control Conference
Bisgaard, M., la Cour‐Harbo, A., & Bendtsen, J. D. (2009) · Cited by 1,027
Real-Time Motion Planning for Agile Autonomous Vehicles
Frazzoli, E., Dahleh, M. A., & Feron, E. (2002) · Journal of Guidance, Control, and Dynamics · doi:10.2514/2.4856 · Cited by 830
Cover trees for nearest neighbor
Beygelzimer, A., Kakade, S., & Langford, J. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143857 · Cited by 813
Journal of The Royal Statistical Society
Blenkinsop, I. H. (2004) · Encyclopedia of Statistical Sciences · doi:10.1002/0471667196.ess1319 · Cited by 800
Ratliff, N. D., Bagnell, J. A., & Zinkevich, M. A. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143936 · Cited by 606
Context-Specific Independence in Bayesian Networks
Boutilier, C. E., Friedman, N., Goldszmidt, M., & Koller, D. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1302.3562 · Cited by 568
Model-based control of a robot manipulator
(1988) · Choice Reviews Online · doi:10.5860/choice.26-1542 · Cited by 553
Autonomous Inverted Helicopter Flight via Reinforcement Learning
Ng, A. Y., Coates, A., Diel, M., Ganapathi, V., Schulte, J., Tse, B., Berger, E., & Liang, E. (2006) · Springer Tracts in Advanced Robotics · doi:10.1007/11552246_35 · Cited by 553
Self-citationAn Application of Reinforcement Learning to Aerobatic Helicopter Flight
Abbeel, P., Coates, A., Quigley, M., & Ng, A. Y. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0006 · Cited by 535
Self-citationRobot Learning from Demonstration
(2012) · SpringerReference · doi:10.1007/springerreference_302487 · Cited by 532
Differential dynamic programming
Denham, W. (1971) · IEEE Transactions on Automatic Control · doi:10.1109/tac.1971.1099760 · Cited by 530
Locally Weighted Learning for Control
Atkeson, C. G., Moore, A. W., & Schaal, S. (1997) · Artificial Intelligence Review · doi:10.1023/a:1006511328852 · Cited by 527
Differential dynamic programming
Jacobson, D. H., & Mayne, D. Q. (1970) · CERN Document Server (European Organization for Nuclear Research) · Cited by 521
Visually guided landing of an unmanned aerial vehicle
Saripalli, S., Montgomery, J. F., & Sukhatme, G. S. (2003) · IEEE Transactions on Robotics and Automation · doi:10.1109/tra.2003.810239 · Cited by 504
Dynamic Programming and Optimal Control. Volume 1: Dynamic Programming and Optimal Control. Volume 2
Smith, D. K. (1996) · Journal of the Operational Research Society · doi:10.1057/jors.1996.103 · Cited by 448
Bayesian Inverse Reinforcement Learning
(2025) · Partially Observed Markov Decision Processes · doi:10.1017/9781009449441.027 · Cited by 444
Maneuver-based motion planning for nonlinear systems with symmetries
Frazz, E., Dahleh, M. A., & Feron, E. (2005) · IEEE Transactions on Robotics · doi:10.1109/tro.2005.852260 · Cited by 440
Learning to Drive a Bicycle Using Reinforcement Learning and Shaping
Randløv, J., & Alstrøm, P. (1998) · Cited by 324
A Game-Theoretic Approach to Apprenticeship Learning
Umar, S. M., & Schapire, R. E. (2007) · neural information processing systems · Cited by 321
Autonomous helicopter control using reinforcement learning policy search methods
Bagnell, J. A., & Schneider, J. G. (2002) · Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164) · doi:10.1109/robot.2001.932842 · Cited by 276
Stochastic policy gradient reinforcement learning on a simple 3D biped
Tedrake, R., Zhang, T. W., & Seung, H. S. (2005) · 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566) · doi:10.1109/iros.2004.1389841 · Cited by 264
Attitude Determination from Vector Observations: Quaternion Estimation
Bar-Itzhack, I. Y., & Oshman, Y. (1985) · IEEE Transactions on Aerospace and Electronic Systems · doi:10.1109/taes.1985.310546 · Cited by 248
Learning for control from multiple demonstrations
Coates, A., Abbeel, P., & Ng, A. Y. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390175 · Cited by 228
Self-citationExploration and apprenticeship learning in reinforcement learning
Abbeel, P., & Ng, A. Y. (2005) · Proceedings of the 22nd international conference on Machine learning - ICML '05 · doi:10.1145/1102351.1102352 · Cited by 224
Self-citationAIAA Guidance, Navigation, and Control Conference
(2011) · doi:10.2514/mgnc11 · Cited by 212
Using inaccurate models in reinforcement learning
Abbeel, P., Quigley, M., & Ng, A. Y. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143845 · Cited by 207
Self-citationApprenticeship Learning using Inverse Reinforcement Learning and Gradient Methods
Neu, G., & Szepesvári, C. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1206.5264 · Cited by 186
Low-cost flight control system for a small autonomous helicopter
Roberts, J. M., Corke, P., & Buskey, G. (2004) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2003.1241651 · Cited by 135
Multiple Alignment of Continuous Time Series
Listgarten, J., Neal, R. M., Roweis, S. T., & Emili, A. (2004) · Cited by 123
AIAA/AAS Astrodynamics Specialist Conference
(2012) · doi:10.2514/mast12 · Cited by 110
Efficient Locally Weighted Polynomial Regression Predictions
Moore, A., Schneider, J., & Deng, K. (1997) · Cited by 100
Learning vehicular dynamics, with application to modeling helicopters
Abbeel, P., Ganapathi, V., & Ng, A. Y. (2005) · neural information processing systems · Cited by 91
Self-citationArtificial Intelligence Review
Kilani, A., Hamida, A. B., & Hamam, H. (2018) · Advances in computer and electrical engineering book series · doi:10.4018/978-1-5225-7368-5.ch003 · Cited by 72
Autonomous Autorotation of an RC Helicopter
Abbeel, P., Coates, A., Hunter, T., & Ng, A. (2009) · Springer tracts in advanced robotics · doi:10.1007/978-3-642-00196-3_45 · Cited by 64
Self-citationConstraint in Attitude Estimation Part I: Constrained Estimation
Shuster, M. D. (2003) · The Journal of the Astronautical Sciences · doi:10.1007/bf03546315 · Cited by 47
Learning first-order Markov models for control
Abbeel, P., & Ng, A. Y. (2004) · neural information processing systems · Cited by 42
Self-citationQuaternion Estimation and Norm Constrained Kalman Filtering
Zanetti, R., & Bishop, R. (2006) · AIAA/AAS Astrodynamics Specialist Conference and Exhibit · doi:10.2514/6.2006-6164 · Cited by 28
Cited by
(2015) · Journal of Machine Learning Research · Cited by 1,119
State of the Art and Future Directions for Lower Limb Robotic Exoskeletons
Young, A. J., & Ferris, D. P. (2017) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2016.2521160 · Cited by 901
Optimal and Autonomous Control Using Reinforcement Learning: A Survey
Kiumarsi, B., Vamvoudakis, K. G., Modares, H., & Lewis, F. L. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2017.2773458 · Cited by 891
Gibson Env: Real-World Perception for Embodied Agents
Xia, F., Zamir, A. R., He, Z., Sax, A., Malik, J., & Savarese, S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00945 · Cited by 763
Recent Advances in Robot Learning from Demonstration
Ravichandar, H., Polydoros, A. S., Chernova, S., & Billard, A. (2020) · Annual Review of Control, Robotics, and Autonomous Systems · doi:10.1146/annurev-control-100819-063206 · Cited by 752
Metadata
- Authors
- Abbeel, P., Coates, A., & Ng, A. Y.
- Venue
- The International Journal of Robotics Research
- Publisher
- SAGE Publications
- Published
- 2010-06-23
- Volume
- 29
- Issue
- 13
- Pages
- 1608-1639
- ISSN
- 0278-3649 (ISSN-L)
- 1741-3176
- OpenAlex
- W2134491302
- Cited by
- 584
- References
- 57
- FWCI
- 26.53
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.