The DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index.
Robot trajectory optimization using approximate inference
Abstract
The general stochastic optimal control (SOC) problem in robotics scenarios is often too complex to be solved exactly and in near real time. A classical approximate solution is to first compute an optimal (deterministic) trajectory and then solve a local linear-quadratic-gaussian (LQG) perturbation model to handle the system stochasticity. We present a new algorithm for this approach which improves upon previous algorithms like iLQG. We consider a probabilistic model for which the maximum likelihood (ML) trajectory coincides with the optimal trajectory and which, in the LQG case, reproduces the classical SOC solution. The algorithm then utilizes approximate inference methods (similar to expectation propagation) that efficiently generalize to non-LQG systems. We demonstrate the algorithm on a simulated 39-DoF humanoid robot.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Applied Optimal Control: Optimization, Estimation, and Control
Bryson, A. E., & Ho, Y.-C. (2018) · Routledge eBooks · doi:10.1201/9781315137667 · Cited by 6,015
J., J. E. (1979) · Technometrics · doi:10.1080/00401706.1979.10489803 · Cited by 3,073
Dynamic bayesian networks: representation, inference and learning
Murphy, K. P., & Russell, S. (2002) · Cited by 2,616
Understanding belief propagation and its generalizations
Yedidia, J. S., Freeman, W. T., & Weiss, Y. (2003) · Cited by 1,594
Expectation Propagation for approximate Bayesian inference
Minka, T. P. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.2294 · Cited by 1,392
A family of algorithms for approximate bayesian inference
Minka, T. P., & Picard, R. W. (2001) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 882
Intercontinental, B. (1985) · Cited by 682
Todorov, E., & Li, W. (2005) · Proceedings of the 2005, American Control Conference, 2005. · doi:10.1109/acc.2005.1469949 · Cited by 611
Probabilistic Inference and Influence Diagrams
Shachter, R. D. (1988) · Operations Research · doi:10.1287/opre.36.4.589 · Cited by 494
Malliaris, A. G. (1989) · Finance · doi:10.1007/978-1-349-20213-3_28 · Cited by 386
Research works printed but not linked: 7
- 10.1002/oca.4660120404 · doi:10.1002/oca.4660120404
- 10.5555/935427 · doi:10.5555/935427
- 10.5555/647235.720257 · doi:10.5555/647235.720257
- 10.5555/937223 · doi:10.5555/937223
- 10.1177/027836499801700908 · doi:10.1177/027836499801700908
- 10.5555/143802.145414 · doi:10.5555/143802.145414
- Zhang, J., & Knoll, A. (1995). An enhanced optimization approach for generating smooth robot trajectories in the presence of obstacles. Proc. of the 1995 European Chinese Automation Conference (pp. 263--268). London.
Cited by
Soft actor-critic Off-policy maximum entropy deep reinforcement learning with a stochastic actor
Haarnoja, T., Zhou, A., Abbeel, P., & Levine, S. (2018) · arXiv · doi:10.48550/arxiv.1801.01290 · Cited by 5,594
“This framework has been used in many contexts, from inverse reinforcement learning (Ziebart et al., 2008) to optimal control (Todorov, 2008; Toussaint, 2009; Rawlik et al., 2012).”
MethodSoft Actor-Critic Algorithms and Applications
Haarnoja, T., Zhou, A., Hartikainen, K., Tucker, G., Ha, S., Tan, J., Kumar, V., Zhu, H., Gupta, A., Abbeel, P., et al. (2018) · arXiv · doi:10.48550/arxiv.1812.05905 · Cited by 1,994
A Survey on Policy Search for Robotics
Deisenroth, M. P. (2011) · Foundations and Trends in Robotics · doi:10.1561/2300000021 · Cited by 798
CHOMP: Covariant Hamiltonian optimization for motion planning
Zucker, M., Ratliff, N., Dragan, A. D., Pivtoraiko, M., Klingensmith, M., Dellin, C. M., Bagnell, J. A., & Srinivasa, S. S. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913488805 · Cited by 758
How to train your robot with deep reinforcement learning: Lessons we have learned
Ibarz, J., Tan, J., Finn, C., Kalakrishnan, M., Pastor, P., & Levine, S. (2021) · The International Journal of Robotics Research · doi:10.1177/0278364920987859 · Cited by 570
(2013) · Lincoln Repository (University of Lincoln) · Cited by 503
Learning to Walk Via Deep Reinforcement Learning
Haarnoja, T., Ha, S., Zhou, A., Tan, J., Tucker, G., & Levine, S. (2019) · Robotics: Science and Systems XV · doi:10.15607/rss.2019.xv.011 · Cited by 430
Reinforcement Learning with Deep Energy-Based Policies
Haarnoja, T., Tang, H., Abbeel, P., & Levine, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.08165 · Cited by 392
A policy-blending formalism for shared control
Dragan, A. D., & Srinivasa, S. S. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913490324 · Cited by 387
“…to struggle with high-cost local minima in the complex spaces induced by manipulation tasks, this issue can be alleviated to a certain extent by learning to place the optimizer in good basins of attraction from prior experience (Jetchev and Toussaint, 2009; Dragan et al., 2011a; Dey et al., 2012).”
BackgroundMethodReinforcement Learning and Control as Probabilistic Inference: Tutorial and Review
Levine, S. (2018) · arXiv · doi:10.48550/arxiv.1805.00909 · Cited by 366
Botvinick, M., & Toussaint, M. (2012) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2012.08.006 · Cited by 292
Self-citationTrautman, P., Ma, J., Murray, R. M., & Krause, A. (2015) · The International Journal of Robotics Research · doi:10.1177/0278364914557874 · Cited by 268
A dynamical system approach to realtime obstacle avoidance
Khansari-Zadeh, S. M., & Billard, A. (2012) · Autonomous Robots · doi:10.1007/s10514-012-9287-y · Cited by 264
“Despite recent efforts at reducing the computational costs of such global searches for a feasible path (Diankov and Kuffner 2007; Burns and Brock 2005; Toussaint 2009), these methods cannot offer the reactivity sought for swiftly avoiding obstacles that appear suddenly.”
Maximum a Posteriori Policy Optimisation
Abdolmaleki, A., Springenberg, J. T., Tassa, Y., Munos, R., Heess, N., & Riedmiller, M. (2018) · arXiv · doi:10.48550/arxiv.1806.06920 · Cited by 259
The free energy principle made simpler but not too simple
Friston, K., Da Costa, L., Sajid, N., Heins, C., Ueltzhöffer, K., Pavliotis, G. A., & Parr, T. (2023) · Physics Reports · doi:10.1016/j.physrep.2023.07.001 · Cited by 163
“This underwrites prospect theory of human choice behaviour in economics [126] and modern approaches to control as inference [127–129], variously known as Kalman duality [94,130], KL control [131] and maximum entropy reinforcement learning [132].”
Data-Efficient Domain Randomization With Bayesian Optimization
Muratore, F., Eilers, C., Gienger, M., & Peters, J. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3052391 · Cited by 77
“There is a connection to control as inference approaches which interpret the cost as a log-likelihood function under an optimality criterion using a Boltzmann distribution construct [13], [14].”
MethodMaximum Entropy RL (Provably) Solves Some Robust RL Problems
Eysenbach, B., & Levine, S. (2021) · arXiv · doi:10.48550/arxiv.2103.06257 · Cited by 69
End-to-End Pixel-Based Deep Active Inference for Body Perception and Action
Sancaktar, C., van Gerven, M. A. J., & Lanillos, P. (2020) · 2020 Joint IEEE 10th International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob) · doi:10.1109/icdl-epirob48136.2020.9278105 · Cited by 58
“This prediction error variant of control as inference [25] exploits the representation learnt to indirectly generate the actions without a policy.”
BackgroundReward maximization through discrete active inference
Da Costa, L., Sajid, N., Parr, T., Friston, K., & Smith, R. (2023) · Neural Computation · doi:10.1162/neco_a_01574 · Cited by 35
Metadata
- Authors
- Toussaint, M.
- Venue
- Proceedings of the 26th Annual International Conference on Machine Learning
- Publisher
- ACM
- Published
- 2009-06-14
- Pages
- 1049-1056
- OpenAlex
- W2107464055
- Cited by
- 316
- References
- 18
- FWCI
- 7.05
- Type
- conference paper
- Language
- en
- Open access
- closed
- Funders
- Deutsche Forschungsgemeinschaft (TO 409/1-3)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.