AMP: adversarial motion priors for stylized physics-based character control
Open accessThe DOI matches its Crossref record; 85 of the 85 references it cites were found in a registry or an index.
Abstract
Synthesizing graceful and life-like behaviors for physically simulated characters has been a fundamental challenge in computer animation. Data-driven methods that leverage motion tracking are a prominent class of techniques for producing high fidelity motions for a wide range of behaviors. However, the effectiveness of these tracking-based methods often hinges on carefully designed objective functions, and when applied to large and diverse motion datasets, these methods require significant additional machinery to select the appropriate motion for the character to track in a given scenario. In this work, we propose to obviate the need to manually design imitation objectives and mechanisms for motion selection by utilizing a fully automated approach based on adversarial imitation learning. High-level task objectives that the character should perform can be specified by relatively simple reward functions, while the low-level style of the character's behaviors can be specified by a dataset of unstructured motion clips, without any explicit clip selection or sequencing. For example, a character traversing an obstacle course might utilize a task-reward that only considers forward progress, while the dataset contains clips of relevant behaviors such as running, jumping, and rolling. These motion clips are used to train an adversarial motion prior, which specifies style-rewards for training the character through reinforcement learning (RL). The adversarial RL procedure automatically selects which motion to perform, dynamically interpolating and generalizing from the dataset. Our system produces high-quality motions that are comparable to those achieved by state-of-the-art tracking-based techniques, while also being able to easily accommodate large datasets of unstructured motion clips. Composition of disparate skills emerges automatically from the motion prior, without requiring a high-level motion planner or other task-specific annotations of the motion clips. We demonstrate the effectiveness of our framework on a diverse cast of complex simulated characters and a challenging suite of motor control tasks.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Auto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067
GAN(Generative Adversarial Nets)
(2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,270
Advances in neural information processing systems 7
(1996) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(96)90463-0 · Cited by 14,281
(2010) · International Conference on Machine Learning · Cited by 13,995
Unsupervised representation learning with deep convolutional generative adversarial networks
Radford, A., Metz, L., & Chintala, S. (2015) · arXiv · doi:10.48550/arxiv.1511.06434 · Cited by 12,235
Proximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625
Improved Techniques for Training GANs
Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., & Chen, X. (2016) · arXiv · doi:10.48550/arxiv.1606.03498 · Cited by 6,965
Introduction to Reinforcement Learning
Sutton, R. S., & Barto, A. G. (1998) · The MIT Press eBooks · Cited by 6,855
Improved training of Wasserstein GANs
Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., & Courville, A. (2017) · arXiv · doi:10.48550/arxiv.1704.00028 · Cited by 6,800
Dynamic Programming Algorithm Optimization for Spoken Word Recognition
Sakoe, H., & Chiba, S. (1978) · IEEE Transactions on Acoustics Speech and Signal Processing · doi:10.1109/tassp.1978.1163055 · Cited by 6,727
“Least squares generative adversarial networks,”
Mao, X., Li, Q., Xie, H., Lau, R. Y. K., Wang, Z., & Smolley, S. P. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.304 · Cited by 5,437
Progressive Growing of GANs for Improved Quality, Stability, and Variation
Karras, T., Aila, T., Laine, S., & Lehtinen, J. (2017) · arXiv · doi:10.48550/arxiv.1710.10196 · Cited by 4,727
End to End Learning for Self-Driving Cars
Bojarski, M., Del Testa, D., Dworakowski, D., Firner, B., Flepp, B., Goyal, P., Jackel, L. D., Monfort, M., Muller, U., Zhang, J., et al. (2016) · arXiv · doi:10.48550/arxiv.1604.07316 · Cited by 3,198
Apprenticeship learning via inverse reinforcement learning
Abbeel, P., & Ng, A. Y. (2004) · Twenty-first international conference on Machine learning - ICML '04 · doi:10.1145/1015330.1015430 · Cited by 2,769
Self-citationEnd-to-End Recovery of Human Shape and Pose
Kanazawa, A., Black, M. J., Jacobs, D. W., & Malik, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00744 · Cited by 2,407
Self-citationHigh-Dimensional Continuous Control Using Generalized Advantage Estimation
Schulman, J., Moritz, P., Levine, S., Jordan, M., & Abbeel, P. (2015) · arXiv · doi:10.48550/arxiv.1506.02438 · Cited by 2,103
Self-citationGenerative adversarial imitation learning
Ho, J., & Ermon, S. (2016) · arXiv · doi:10.48550/arxiv.1606.03476 · Cited by 1,942
Maximum Entropy Inverse Reinforcement Learning
Ziebart, B. D., Maas, A. L., Bagnell, J. A., & Dey, A. K. (2018) · doi:10.1184/r1/6555512 · Cited by 1,869
DeepMimic: example-guided deep reinforcement learning of physics-based character skills
Peng, X. B., Abbeel, P., Levine, S., & van de Panne, M. (2018) · ACM Transactions on Graphics · doi:10.1145/3197517.3201311 · Cited by 1,784
Self-citationALVINN: An autonomous land vehicle in a neural network
Pomerleau, D. A. (2018) · Carnegie Mellon University · doi:10.1184/r1/6603146.v1 · Cited by 1,340
ALVINN, an autonomous land vehicle in a neural network
Pomerleau, D. (2018) · doi:10.1184/r1/6603146 · Cited by 1,338
f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization
Nowozin, S., Cseke, B., & Tomioka, R. (2016) · arXiv · doi:10.48550/arxiv.1606.00709 · Cited by 1,183
Proceedings of Graphics Interface 2001
Watson, B. A., & Buchanan, J. W. (2001) · Cited by 1,025
Interactive control of avatars animated with human motion data
Lee, J., Chai, J., Reitsma, P. S. A., Hodgins, J. K., & Pollard, N. S. (2002) · Proceedings of the 29th annual conference on Computer graphics and interactive techniques · doi:10.1145/566570.566607 · Cited by 864
Which Training Methods for GANs do actually Converge?
Mescheder, L., Geiger, A., & Nowozin, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.04406 · Cited by 768
BEGAN: Boundary Equilibrium Generative Adversarial Networks
Berthelot, D., Schumm, T., & Metz, L. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.10717 · Cited by 654
Style-based inverse kinematics
Grochow, K., Martin, S. L., Hertzmann, A., & Popović, Z. (2004) · ACM Transactions on Graphics · doi:10.1145/1015706.1015755 · Cited by 642
Phase-functioned neural networks for character control
Holden, D., Komura, T., & Saito, J. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073663 · Cited by 546
peng (2017) · doi:10.1145/3072959.3073602 · Cited by 502
Tassa, Y., Doron, Y., Muldal, A., Erez, T., Li, Y., Casas, D. D. L., Budden, D., Abdolmaleki, A., Merel, J., Lefrancq, A., et al. (2018) · arXiv · doi:10.48550/arxiv.1801.00690 · Cited by 497
Practical Parameterization of Rotations Using the Exponential Map
Grassia, F. S. (1998) · Journal of Graphics Tools · doi:10.1080/10867651.1998.10487493 · Cited by 495
Interactive control of avatars animated with human motion data
Lee, J., Chai, J., Reitsma, P. S. A., Hodgins, J. K., & Pollard, N. S. (2002) · ACM Transactions on Graphics · doi:10.1145/566654.566607 · Cited by 467
Discovery of complex behaviors through contact-invariant optimization
Mordatch, I., Todorov, E., & Popović, Z. (2012) · ACM Transactions on Graphics · doi:10.1145/2185520.2185539 · Cited by 451
Learning an Embedding Space for Transferable Robot Skills
Hausman, K., Springenberg, J. T., Wang, Z., Heess, N., & Riedmiller, M. (2018) · International Conference on Learning Representations · Cited by 341
Animation of dynamic legged locomotion
Raibert, M. H., & Hodgins, J. K. (1991) · Proceedings of the 18th annual conference on Computer graphics and interactive techniques · doi:10.1145/122718.122755 · Cited by 292
Neural state machine for character-scene interactions
Starke, S., Zhang, H., Komura, T., & Saito, J. (2019) · ACM Transactions on Graphics · doi:10.1145/3355089.3356505 · Cited by 290
Optimizing locomotion controllers using biologically-based actuators and objectives
Wang, J. M., Hamner, S. R., Delp, S. L., & Koltun, V. (2012) · ACM Transactions on Graphics · doi:10.1145/2185520.2185521 · Cited by 282
Mode-adaptive neural networks for quadruped motion control
Zhang, H., Starke, S., Komura, T., & Saito, J. (2018) · ACM Transactions on Graphics · doi:10.1145/3197517.3201366 · Cited by 277
Motion capture-driven simulations that hit and react
Zordan, V. B., & Hodgins, J. K. (2002) · Proceedings of the 2002 ACM SIGGRAPH/Eurographics symposium on Computer animation · doi:10.1145/545261.545276 · Cited by 263
Flexible muscle-based locomotion for bipedal creatures
Geijtenbeek, T., van de Panne, M., & van der Stappen, A. F. (2013) · ACM Transactions on Graphics · doi:10.1145/2508363.2508399 · Cited by 256
Character controllers using motion VAEs
Ling, H. Y., Zinno, F., Cheng, G., & Van De Panne, M. (2020) · ACM Transactions on Graphics · doi:10.1145/3386569.3392422 · Cited by 253
Near-optimal character animation with continuous control
Treuille, A., Lee, Y., & Popović, Z. (2007) · ACM SIGGRAPH 2007 papers · doi:10.1145/1275808.1276386 · Cited by 207
DReCon: data-driven responsive control of physics-based characters
Bergamin, K., Clavet, S., Holden, D., & Forbes, J. R. (2019) · ACM Transactions on Graphics · doi:10.1145/3355089.3356536 · Cited by 204
A scalable approach to control diverse behaviors for physically simulated characters
Won, J., Gopinath, D., & Hodgins, J. K. (2020) · ACM Transactions on Graphics · doi:10.1145/3386569.3392381 · Cited by 204
Scalable muscle-actuated human simulation and control
Lee, S., Park, M., Lee, K., & Lee, J. (2019) · ACM Transactions on Graphics · doi:10.1145/3306346.3322972 · Cited by 203
Terrain-adaptive locomotion skills using deep reinforcement learning
Peng, X. B., Berseth, G., & van de Panne, M. (2016) · ACM Transactions on Graphics · doi:10.1145/2897824.2925881 · Cited by 200
Self-citationSFV: reinforcement learning of physical skills from videos
Peng, X. B., Kanazawa, A., Malik, J., Abbeel, P., & Levine, S. (2018) · ACM Transactions on Graphics · doi:10.1145/3272127.3275014 · Cited by 195
Self-citationOptimizing walking controllers
Wang, J. M., Fleet, D. J., & Hertzmann, A. (2009) · ACM Transactions on Graphics · doi:10.1145/1618452.1618514 · Cited by 181
Learning predict-and-simulate policies from unorganized human motion data
Park, S., Ryu, H.-S., Lee, S., Lee, S., & Lee, J. (2019) · ACM Transactions on Graphics · doi:10.1145/3355089.3356501 · Cited by 181
Continuous character control with low-dimensional embeddings
Levine, S., Wang, J. M., Haraux, A., Popović, Z., & Koltun, V. (2012) · ACM Transactions on Graphics · doi:10.1145/2185520.2185524 · Cited by 170
Self-citationSimulating biped behaviors from human motion data
Sok, K. W., Kim, M., & Lee, J. (2007) · ACM SIGGRAPH 2007 papers · doi:10.1145/1275808.1276511 · Cited by 170
Contact-aware nonlinear control of dynamic characters
Muico, U., Lee, Y., Popović, J., & Popović, Z. (2009) · ACM Transactions on Graphics · doi:10.1145/1531326.1531387 · Cited by 170
Catch & Carry: reusable neural controllers for vision-guided whole-body tasks
Merel, J., Tunyasuvunakool, S., Ahuja, A., Tassa, Y., Hasenclever, L., Pham, V., Erez, T., Wayne, G., & Heess, N. (2020) · ACM Transactions on Graphics · doi:10.1145/3386569.3392474 · Cited by 168
Imitation Learning as f-Divergence Minimization
Ke, L., Choudhury, S., Barnes, M., Sun, W. Y., Lee, G., & Srinivasa, S. S. (2021) · Springer proceedings in advanced robotics · doi:10.1007/978-3-030-66723-8_19 · Cited by 164
Sampling-based contact-rich motion control
Liu, L., Yin, K., van de Panne, M., Shao, T., & Xu, W. (2010) · ACM Transactions on Graphics · doi:10.1145/1778765.1778865 · Cited by 161
Lee, Y., Kim, S., & Lee, J. (2010) · ACM Transactions on Graphics · doi:10.1145/1778765.1781155 · Cited by 156
Learning symmetric and low-energy locomotion
Yu, W., Turk, G., & Liu, C. K. (2018) · ACM Transactions on Graphics · doi:10.1145/3197517.3201397 · Cited by 155
Simulating biped behaviors from human motion data
Sok, K. W., Kim, M., & Lee, J. (2007) · ACM Transactions on Graphics · doi:10.1145/1276377.1276511 · Cited by 139
Peng, X. B., Kanazawa, A., Toyer, S., Abbeel, P., & Levine, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.00821 · Cited by 136
Self-citationLearning human behaviors from motion capture by adversarial imitation
Merel, J., Tassa, Y., Tb, D., Srinivasan, S., Lemmon, J., Wang, Z., Wayne, G., & Heess, N. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.02201 · Cited by 134
Trajectory Optimization for Full-Body Movements with Complex Contacts
Borno, M. A., de Lasa, M., & Hertzmann, A. (2013) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2012.325 · Cited by 119
Interactive character animation by learning multi-objective control
Lee, K., Lee, S., & Lee, J. (2018) · ACM Transactions on Graphics · doi:10.1145/3272127.3275071 · Cited by 117
Physics-based motion capture imitation with deep reinforcement learning
Chentanez, N., Müller, M., Macklin, M., Makoviychuk, V., & Jeschke, S. (2018) · Motion in Games · doi:10.1145/3274247.3274506 · Cited by 113
Guided Learning of Control Graphs for Physics-Based Characters
Liu, L., Panne, M. V. D., & Yin, K. (2016) · ACM Transactions on Graphics · doi:10.1145/2893476 · Cited by 108
Robust Imitation of Diverse Behaviors
Wang, Z., Merel, J., Reed, S., Wayne, G., de Freitas, N., & Heess, N. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.02747 · Cited by 103
Synthesis of biologically realistic human motion using joint torque actuation
Jiang, Y., Wouwe, T. V., Groote, F. D., & Liu, C. K. (2019) · ACM Transactions on Graphics · doi:10.1145/3306346.3322966 · Cited by 100
Motion fields for interactive character locomotion
Lee, Y., Wampler, K., Bernstein, G., Popović, J., & Popović, Z. (2010) · ACM Transactions on Graphics · doi:10.1145/1882261.1866160 · Cited by 91
Learning and Transfer of Modulated Locomotor Controllers
Heess, N., Wayne, G., Tassa, Y., Lillicrap, T., Riedmiller, M., & Silver, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.05182 · Cited by 89
Animating human lower limbs using contact-invariant optimization
Mordatch, I., Wang, J. M., Todorov, E. V., & Koltun, V. (2013) · ACM Transactions on Graphics · doi:10.1145/2508363.2508365 · Cited by 85
Generative Adversarial Imitation from Observation
Torabi, F., Warnell, G., & Stone, P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.06158 · Cited by 83
Terrain runner: control, parameterization, composition, and planning for highly dynamic motions
Liu, L., Yin, K., van de Panne, M., & Guo, B. (2012) · ACM Transactions on Graphics · doi:10.1145/2366145.2366173 · Cited by 80
Neural probabilistic motor primitives for humanoid control
Merel, J., Hasenclever, L., Galashov, A., Ahuja, A., Pham, V., Wayne, G., Teh, Y. W., & Heess, N. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.11711 · Cited by 78
Motion fields for interactive character locomotion
Lee, Y.-J., Wampler, K., Bernstein, G., Popović, J., & Popović, Z. (2010) · ACM SIGGRAPH Conference and Exhibition on Computer Graphics and Interactive Techniques in Asia (SIGGRAPH Asia) · doi:10.1145/1866158.1866160 · Cited by 77
Tan, J., Gu, Y., Liu, C. K., & Turk, G. (2014) · ACM Transactions on Graphics · doi:10.1145/2601097.2601121 · Cited by 76
CARL: controllable agent with reinforcement learning for quadruped locomotion
Luo, Y.-S., Soeseno, J. H., Chen, T. P.-C., & Chen, W.-C. (2020) · ACM Transactions on Graphics · doi:10.1145/3386569.3392433 · Cited by 76
Synthesis of Responsive Motion Using a Dynamic Model
Ye, Y., & Liu, C. K. (2010) · Computer Graphics Forum · doi:10.1111/j.1467-8659.2009.01625.x · Cited by 67
Space-time planning with parameterized locomotion controllers
Levine, S., Lee, Y.-J., Koltun, V., & Popović, Z. (2011) · ACM Transactions on Graphics · doi:10.1145/1966394.1966402 · Cited by 61
Self-citationOptimizing walking controllers
Wang, J. M., Fleet, D. J., & Hertzmann, A. (2009) · ACM SIGGRAPH Conference and Exhibition on Computer Graphics and Interactive Techniques in Asia (SIGGRAPH Asia) · doi:10.1145/1661412.1618514 · Cited by 58
Momentum-Mapped Inverted Pendulum Models for Controlling Dynamic Human Motions
Kwon, T., & Hodgins, J. K. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.2983616 · Cited by 57
Stochastic Neural Networks for Hierarchical Reinforcement Learning
Florensa, C., Duan, Y., & Abbeel, P. (2016) · International Conference on Learning Representations · Cited by 57
Self-citationSynthesis of Controllers for Stylized Planar Bipedal Walking
Sharon, D., & van de Panne, M. (2006) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2005.1570470 · Cited by 52
Generalizing locomotion style to new animals with inverse optimal regression
Wampler, K., Popović, Z., & Popović, J. (2014) · ACM Transactions on Graphics · doi:10.1145/2601097.2601192 · Cited by 49
How to Train Your DRAGANPreprint
Kodali, N., Abernethy, J., Hays, J. H., & Kira, Z. (2017) · arXiv (Cornell University) · Cited by 40
The International Convention Centre
Smyth, H. (2005) · Taylor & Francis eBooks · doi:10.4324/9780203975954-14 · Cited by 11
Learning Omnidirectional Path Following Using Dimensionality Reduction
Kolter, J. Z., & Ng, A. Y. (2008) · The MIT Press eBooks · doi:10.7551/mitpress/7830.003.0034 · Cited by 5
Cited by
Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning
Makoviychuk, V., Wawrzyniak, L., Guo, Y., Lu, M., Storey, K., Macklin, M., Hoeller, D., Rudin, N., Allshire, A., Handa, A., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.10470 · Cited by 509
Diffusion-based Generation, Optimization, and Planning in 3D Scenes
Huang, S., Wang, Z., Li, P., Jia, B., Liu, T., Zhu, Y., Liang, W., & Zhu, S.-C. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01607 · Cited by 192
Florence, P., Lynch, C., Zeng, A., Ramirez, O., Wahid, A., Downs, L., Wong, A., Lee, J., Mordatch, I., & Tompson, J. (2021) · arXiv · doi:10.48550/arxiv.2109.00137 · Cited by 131
Trace and Pace: Controllable Pedestrian Animation via Guided Trajectory Diffusion
Rempe, D., Luo, Z., Peng, X. B., Yuan, Y., Kitani, K., Kreis, K., Fidler, S., & Litany, O. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01322 · Cited by 97
Self-citationFrom motor control to team play in simulated humanoid football
Liu, S., Lever, G., Wang, Z., Merel, J., Eslami, S. M. A., Hennes, D., Czarnecki, W. M., Tassa, Y., Omidshafiei, S., Abdolmaleki, A., et al. (2022) · Science Robotics · doi:10.1126/scirobotics.abo0235 · Cited by 88
Metadata
- Authors
- Peng, X. B., Ma, Z., Abbeel, P., Levine, S., & Kanazawa, A.
- Venue
- ACM Transactions on Graphics
- Published
- 2021-07-19
- Volume
- 40
- Issue
- 4
- Pages
- 1-20
- ISSN
- 0730-0301 (ISSN-L)
- 1557-7368
- OpenAlex
- W3147968035
- Cited by
- 396
- References
- 85
- FWCI
- 24.11
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.