Optimizing walking controllers for uncertain inputs and environments
Abstract
We introduce methods for optimizing physics-based walking controllers for robustness to uncertainty. Many unknown factors, such as external forces, control torques, and user control inputs, cannot be known in advance and must be treated as uncertain. These variables are represented with probability distributions, and a return function scores the desirability of a single motion. Controller optimization entails maximizing the expected value of the return, which is computed by Monte Carlo methods. We demonstrate examples with different sources of uncertainty and task constraints. Optimizing control strategies under uncertainty increases robustness and produces natural variations in style.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,263 · see citing works
Introduction to Reinforcement Learning
Sutton, R. S., & Barto, A. G. (1998) · The MIT Press eBooks · Cited by 6,855 · see citing works
Faisal, A. A., Selen, L. P. J., & Wolpert, D. M. (2008) · Nature Reviews Neuroscience · doi:10.1038/nrn2258 · Cited by 2,883 · see citing works
Signal-dependent noise determines motor planning
Harris, C. M., & Wolpert, D. M. (1998) · Nature · doi:10.1038/29528 · Cited by 2,498 · see citing works
Introduction to Stochastic Search and Optimization
Spall, J. C. (2003) · Wiley eBooks · doi:10.1002/0471722138 · Cited by 1,912 · see citing works
The CMA Evolution Strategy: A Comparing Review
Hansen, N. (2007) · Studies in fuzziness and soft computing · doi:10.1007/3-540-32494-1_4 · Cited by 1,907 · see citing works
Optimality principles in sensorimotor control
Todorov, E. (2004) · Nature Neuroscience · doi:10.1038/nn1309 · Cited by 1,746 · see citing works
INTRODUCTION TO STOCHASTIC SEARCH AND OPTIMIZATION: ESTIMATION, SIMULATION, AND CONTROL
Nowak, W. (2012) · Medical Entomology and Zoology · Cited by 1,030 · see citing works
Sims, K. (1994) · Proceedings of the 21st annual conference on Computer graphics and interactive techniques - SIGGRAPH '94 · doi:10.1145/192161.192167 · Cited by 1,005 · see citing works
An Application of Reinforcement Learning to Aerobatic Helicopter Flight
Abbeel, P., Coates, A., Quigley, M., & Ng, A. Y. (2007) · The MIT Press eBooks · doi:10.7551/mitpress/7503.003.0006 · Cited by 551 · see citing works
Towards a New Evolutionary Computation
Lozano, J. A., Larrañaga, P., Inza, I., & Bengoetxea, E. (2006) · Springer eBooks · doi:10.1007/3-540-32494-1 · Cited by 519 · see citing works
A Statistical Model of Human Pose and Body Shape
Hasler, N., Stoll, C., Sunkel, M., Rosenhahn, B., & Seidel, H. (2009) · Computer Graphics Forum · doi:10.1111/j.1467-8659.2009.01373.x · Cited by 419 · see citing works
Introduction to Stochastic Search and Optimization: Estimation, Simulation, and Control
Frey, J. C. (2004) · Journal of the American Statistical Association · doi:10.1198/jasa.2004.s362 · Cited by 383 · see citing works
PEGASUS: A Policy Search Method for Large MDPs and POMDPsPreprint
Ng, A. Y., & Jordan, M. I. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.3878 · Cited by 364 · see citing works
Yin, K., Loken, K., & van de Panne, M. (2007) · ACM Transactions on Graphics · doi:10.1145/1276377.1276509 · Cited by 340 · see citing works
Composable controllers for physics-based character animation
Faloutsos, P., van de Panne, M., & Terzopoulos, D. (2001) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/383259.383287 · Cited by 335 · see citing works
Yin, K., Loken, K., & van de Panne, M. (2007) · ACM SIGGRAPH Conference Papers (SIGGRAPH) · doi:10.1145/1275808.1276509 · Cited by 334 · see citing works
Decision theory: what “should” the nervous system do?
Körding, K. (2007) · Science · doi:10.1126/science.1142998 · Cited by 313 · see citing works
Grzeszczuk, R., Terzopoulos, D., & Hinton, G. E. (1998) · ACM SIGGRAPH Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH) · doi:10.1145/280814.280816 · Cited by 297 · see citing works
The scaling of motor noise with muscle strength and motor unit number in humans
Hamilton, A. F. D. C., Jones, K. E., & Wolpert, D. M. (2004) · Experimental Brain Research · doi:10.1007/s00221-004-1856-7 · Cited by 285 · see citing works
Stochastic policy gradient reinforcement learning on a simple 3D biped
Tedrake, R., Zhang, T., & Seung, H. S. (2005) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2004.1389841 · Cited by 273 · see citing works
Near-optimal character animation with continuous control
Treuille, A., Lee, Y.-J., & Popović, Z. (2007) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/1275808.1276386 · Cited by 208 · see citing works
van de Panne, M., & Fiume, E. L. (1993) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/166117.166159 · Cited by 203 · see citing works
Optimizing walking controllers
Wang, J. M., Fleet, D. J., & Hertzmann, A. (2009) · ACM Transactions on Graphics · doi:10.1145/1618452.1618514 · Cited by 181 · see citing works
Self-citationContact-aware nonlinear control of dynamic characters
Muico, U., Lee, Y.-J., Popović, J., & Popović, Z. (2009) · ACM Transactions on Graphics · doi:10.1145/1531326.1531387 · Cited by 174 · see citing works
Simulating biped behaviors from human motion data
Sok, K. W., Kim, M., & Lee, J. (2007) · ACM SIGGRAPH Conference Papers (SIGGRAPH) · doi:10.1145/1275808.1276511 · Cited by 171 · see citing works
Byl, K., & Tedrake, R. (2009) · The International Journal of Robotics Research · doi:10.1177/0278364909340446 · Cited by 170 · see citing works
Bouza, C. N. (2004) · Investigación operacional · Cited by 166 · see citing works
Precomputing avatar behavior from human motion data
Lee, J., & Lee, K. H. (2004) · Computer animation and simulation · doi:10.1145/1028523.1028535 · Cited by 157 · see citing works
Automated learning of muscle-actuated locomotion through control abstraction
Grzeszczuk, R., & Terzopoulos, D. (1995) · ACM SIGGRAPH Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH) · doi:10.1145/218380.218411 · Cited by 153 · see citing works
Interactive simulation of stylized human locomotion
da Silva, M. J., Abe, Y., & Popović, J. (2008) · ACM Transactions on Graphics · doi:10.1145/1360612.1360681 · Cited by 141 · see citing works
Simulating biped behaviors from human motion data
Sok, K. W., Kim, M., & Lee, J. (2007) · ACM Transactions on Graphics · doi:10.1145/1276377.1276511 · Cited by 139 · see citing works
Near-optimal character animation with continuous control
Treuille, A., Lee, Y.-J., & Popović, Z. (2007) · ACM Transactions on Graphics · doi:10.1145/1276377.1276386 · Cited by 126 · see citing works
Responsive characters from motion fragments
McCann, J., & Pollard, N. S. (2007) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/1275808.1276385 · Cited by 121 · see citing works
Robust task-based control policies for physics-based characters
Coros, S., Beaudoin, P. M., & van de Panne, M. (2009) · ACM Transactions on Graphics · doi:10.1145/1618452.1618516 · Cited by 93 · see citing works
Responsive characters from motion fragments
McCann, J., & Pollard, N. S. (2007) · ACM Transactions on Graphics · doi:10.1145/1276377.1276385 · Cited by 83 · see citing works
Precomputing avatar behavior from human motion data
Lee, J., & Lee, K. H. (2005) · Graphical Models · doi:10.1016/j.gmod.2005.03.004 · Cited by 68 · see citing works
Guided Optimization for Balanced Locomotion
van de Panne, M., & Lamouret, A. (1995) · Eurographics · doi:10.1007/978-3-7091-9435-5_13 · Cited by 64 · see citing works
Real Time Planning for Parameterized Human Motion
Lo, W., & Zwicker, M. (2008) · BORIS (University Library Bern) · Cited by 52 · see citing works
Synthesis of Controllers for Stylized Planar Bipedal Walking
Sharon, D., & van de Panne, M. (2006) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2005.1570470 · Cited by 52 · see citing works
Lee, Y.-J., Lee, S. J., & Popović, Z. (2009) · ACM Transactions on Graphics · doi:10.1145/1618452.1618515 · Cited by 37 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
MuJoCo: A physics engine for model-based control
Todorov, E., Erez, T., & Tassa, Y. (2012) · 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2012.6386109 · Cited by 4,767 · see citing works
Salvato, E., Fenu, G., Medvet, E., & Pellegrino, F. A. (2021) · IEEE Access · doi:10.1109/access.2021.3126658 · Cited by 200 · see citing works
Ensemble-CIO: Full-body dynamic motion planning that transfers to physical humanoids
Mordatch, I., Lowrey, K., & Todorov, E. (2015) · 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2015.7354126 · Cited by 135 · see citing works
Learning to schedule control fragments for physics-based characters using deep q-learning
Liu, L., & Hodgins, J. (2017) · ACM Transactions on Graphics · doi:10.1145/3083723 · Cited by 49 · see citing works
- Authors
- Wang, J. M., Fleet, D. J., & Hertzmann, A.
- Affiliations
- University of Toronto (CA)
- Venue
- ACM Transactions on Graphics
- Published
- 2010-07-26
- Volume
- 29
- Issue
- 4
- Pages
- 1-8
- ISSN
- 0730-0301 (ISSN-L)
- 1557-7368
- OpenAlex
- W1966784014
- Cited by
- 113
- References
- 41
- Citations by year
- 2026: 1 · 2025: 3 · 2024: 2 · 2023: 3 · 2022: 6 · 2021: 3 · 2020: 5 · 2019: 6 · 2018: 10 · 2017: 12 · 2016: 10 · 2015: 11 · 2014: 8 · 2013: 6 · 2012: 10
- FWCI
- 17.46
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- ACM Transactions on Graphics (publishedVersion)
- The HKU Scholars Hub (University of Hong Kong) (submittedVersion)
- Topics
- Physical Sciences › Engineering › Control and Systems Engineering › Human Motion and Animation
- Physical Sciences › Engineering › Biomedical Engineering › Robotic Locomotion and Control
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Computer Graphics and Visualization Techniques
- Keywords
- controller optimization, uncertainty, robustness, physics-based animation, stochastic modeling, Monte Carlo methods, expected return maximization, external forces, control torque, task constraints
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
