Learning to schedule control fragments for physics-based characters using deep q-learning
Abstract
Given a robust control system, physical simulation offers the potential for interactive human characters that move in realistic and responsive ways. In this article, we describe how to learn a scheduling scheme that reorders short control fragments as necessary at runtime to create a control system that can respond to disturbances and allows steering and other user interactions. These schedulers provide robust control of a wide range of highly dynamic behaviors, including walking on a ball, balancing on a bongo board, skateboarding, running, push-recovery, and breakdancing. We show that moderate-sized Q-networks can model the schedulers for these control tasks effectively and that those schedulers can be efficiently learned by the deep Q-learning algorithm.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Cites
Proceedings of the 25th international conference on Machine learning - ICML '08
(2008) · doi:10.1145/1390156 · Cited by 6,865 · see citing works
Proceedings of the 29th International Conference on Machine Learning (ICML-12)Preprint
Langford, J. C., & Pineau, J. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1207.4676 · Cited by 1,687 · see citing works
Iterative Training of Dynamic Skills Inspired by Human Coaching Techniques
Ha, S., & Liu, C. K. (2014) · ACM Transactions on Graphics · doi:10.1145/2682626 · Cited by 538 · see citing works
Discovery of complex behaviors through contact-invariant optimization
Mordatch, I., Todorov, E. V., & Popović, Z. (2012) · ACM Transactions on Graphics · doi:10.1145/2185520.2185539 · Cited by 451 · see citing works
Learning from demonstration and adaptation of biped locomotion
Nakanishi, J., Morimoto, J., Endo, G., Cheng, G., Stefan Schaal, A., & Kawato, M. (2004) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2004.03.003 · Cited by 399 · see citing works
Proceedings 2000 IEEE/RSJ International Conference on Intelligent Robots and Systems [IROS 2000]
(2000) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2000.893120 · Cited by 347 · see citing works
Yin, K., Loken, K., & van de Panne, M. (2007) · ACM Transactions on Graphics · doi:10.1145/1276377.1276509 · Cited by 340 · see citing works
Generalized biped walking control
Coros, S., Beaudoin, P. M., & van de Panne, M. (2010) · ACM Transactions on Graphics · doi:10.1145/1778765.1781156 · Cited by 222 · see citing works
Reinforcement Learning With Sequences of Motion Primitives for Robust Manipulation
Stulp, F., Theodorou, E. A., & Schaal, S. (2012) · IEEE Transactions on Robotics · doi:10.1109/tro.2012.2210294 · Cited by 216 · see citing works
Terrain-adaptive locomotion skills using deep reinforcement learning
Peng, X. B., Berseth, G., & van de Panne, M. (2016) · ACM Transactions on Graphics · doi:10.1145/2897824.2925881 · Cited by 208 · see citing works
Macchietto, A., Zordan, V. B., & Shelton, C. R. (2009) · ACM Transactions on Graphics · doi:10.1145/1531326.1531386 · Cited by 181 · see citing works
Contact-aware nonlinear control of dynamic characters
Muico, U., Lee, Y.-J., Popović, J., & Popović, Z. (2009) · ACM Transactions on Graphics · doi:10.1145/1531326.1531387 · Cited by 174 · see citing works
Lee, Y., Kim, S., & Lee, J. (2010) · ACM Transactions on Graphics · doi:10.1145/1778765.1781155 · Cited by 162 · see citing works
Robust physics-based locomotion using low-dimensional planning
Mordatch, I., de Lasa, M., & Hertzmann, A. (2010) · ACM Transactions on Graphics · doi:10.1145/1778765.1778808 · Cited by 160 · see citing works
Optimal gait and form for animal locomotion
Wampler, K., & Popović, Z. (2009) · ACM Transactions on Graphics · doi:10.1145/1531326.1531366 · Cited by 148 · see citing works
Simulating biped behaviors from human motion data
Sok, K. W., Kim, M., & Lee, J. (2007) · ACM Transactions on Graphics · doi:10.1145/1276377.1276511 · Cited by 139 · see citing works
Sampling-based contact-rich motion control
Liu, L., Yin, K., van de Panne, M., Shao, T., & Xu, W. (2010) · ACM Transactions on Graphics · doi:10.1145/1778765.1778865 · Cited by 130 · see citing works
Self-citationNear-optimal character animation with continuous control
Treuille, A., Lee, Y.-J., & Popović, Z. (2007) · ACM Transactions on Graphics · doi:10.1145/1276377.1276386 · Cited by 126 · see citing works
Optimizing walking controllers for uncertain inputs and environments
Wang, J. M., Fleet, D. J., & Hertzmann, A. (2010) · ACM Transactions on Graphics · doi:10.1145/1778765.1778810 · Cited by 113 · see citing works
Trajectory Optimization for Full-Body Movements with Complex Contacts
Borno, M. A., de Lasa, M., & Hertzmann, A. (2013) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2012.325 · Cited by 112 · see citing works
Guided Learning of Control Graphs for Physics-Based Characters
Liu, L., van de Panne, M., & Yin, K. (2016) · ACM Transactions on Graphics · doi:10.1145/2893476 · Cited by 112 · see citing works
Self-citationStable Proportional-Derivative Controllers
Tan, J., Liu, K., & Turk, G. (2011) · IEEE Computer Graphics and Applications · doi:10.1109/mcg.2011.30 · Cited by 109 · see citing works
Interactive Character Animation Using Simulated Physics: A State‐of‐the‐Art Review
Geijtenbeek, T., & Pronost, N. (2012) · Computer Graphics Forum · doi:10.1111/j.1467-8659.2012.03189.x · Cited by 105 · see citing works
Optimal feedback control for character animation using an abstract model
Ye, Y., & Liu, C. K. (2010) · ACM Transactions on Graphics · doi:10.1145/1778765.1778811 · Cited by 98 · see citing works
Motion fields for interactive character locomotion
Lee, Y.-J., Wampler, K., Bernstein, G., Popović, J., & Popović, Z. (2010) · ACM Transactions on Graphics · doi:10.1145/1882261.1866160 · Cited by 96 · see citing works
Robust task-based control policies for physics-based characters
Coros, S., Beaudoin, P. M., & van de Panne, M. (2009) · ACM Transactions on Graphics · doi:10.1145/1618452.1618516 · Cited by 93 · see citing works
Online control of simulated humanoids using particle belief propagation
Hämäläinen, P., Rajamäki, J., & Liu, C. K. (2015) · ACM Transactions on Graphics · doi:10.1145/2767002 · Cited by 88 · see citing works
Responsive characters from motion fragments
McCann, J., & Pollard, N. S. (2007) · ACM Transactions on Graphics · doi:10.1145/1276377.1276385 · Cited by 83 · see citing works
Simulation and control of skeleton-driven soft body characters
Liu, L., Yin, K., Wang, B., & Guo, B. (2013) · ACM Transactions on Graphics · doi:10.1145/2508363.2508427 · Cited by 77 · see citing works
Self-citationTan, J., Gu, Y., Liu, C. K., & Turk, G. (2014) · ACM Transactions on Graphics · doi:10.1145/2601097.2601121 · Cited by 76 · see citing works
Kwon, T., & Hodgins, J. (2010) · The Royal Society of Chemistry’s Journals, Books and Databases (The Royal Society of Chemistry) · doi:10.2312/sca/sca10/129-138 · Cited by 75 · see citing works
Self-citationDynamic terrain traversal skills using reinforcement learning
Peng, X. B., Berseth, G., & van de Panne, M. (2015) · ACM Transactions on Graphics · doi:10.1145/2766910 · Cited by 65 · see citing works
Linear Bellman combination for control of character animation
da Silva, M. J., Durand, F., & Popović, J. (2009) · ACM Transactions on Graphics · doi:10.1145/1531326.1531388 · Cited by 60 · see citing works
Improving Sampling‐based Motion Control
Liu, L., Yin, K., & Guo, B. (2015) · Computer Graphics Forum · doi:10.1111/cgf.12571 · Cited by 46 · see citing works
Self-citationMomentum-Mapped Inverted Pendulum Models for Controlling Dynamic Human Motions
Kwon, T., & Hodgins, J. K. (2017) · ACM Transactions on Graphics · doi:10.1145/2983616 · Cited by 37 · see citing works
Self-citationActive learning for real-time motion controllers
Cooper, S., Hertzmann, A., & Popović, Z. (2007) · ACM Transactions on Graphics · doi:10.1145/1276377.1276384 · Cited by 30 · see citing works
Controlling physics-based characters using soft contacts
Jain, S., & Liu, C. K. (2011) · ACM Transactions on Graphics · doi:10.1145/2070781.2024197 · Cited by 29 · see citing works
Biped walking control using a trajectory library
Liu, C., Atkeson, C. G., & Su, J. (2012) · Robotica · doi:10.1017/s0263574712000203 · Cited by 23 · see citing works
Control of Rotational Dynamics for Ground and Aerial Behavior
Zordan, V. B., Brown, D. F., Macchietto, A., & Yin, K. (2014) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2014.2330610 · Cited by 22 · see citing works
Feedback control for rotational movements in feature space
Borno, M. A., Fiume, E. L., Hertzmann, A., & de Lasa, M. (2014) · Computer Graphics Forum · doi:10.1111/cgf.12321 · Cited by 18 · see citing works
Simulating 2D Gaits with a Phase-Indexed Tracking Controller
Abe, Y., & Popović, J. (2011) · IEEE Computer Graphics and Applications · doi:10.1109/mcg.2011.62 · Cited by 5 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Sim-to-Real Transfer of Robotic Control with Dynamics Randomization
Peng, X. B., Andrychowicz, M., Zaremba, W., & Abbeel, P. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8460528 · Cited by 1,309 · see citing works
Peng, X. B., Abbeel, P., Levine, S., & van de Panne, M. (2018) · ACM Transactions on Graphics · doi:10.1145/3197517.3201311 · Cited by 946 · see citing works
Mode-adaptive neural networks for quadruped motion control
Zhang, H., Starke, S., Komura, T., & Saito, J. (2018) · ACM Transactions on Graphics · doi:10.1145/3197517.3201366 · Cited by 276 · see citing works
Hierarchical motor control in mammals and machines
Merel, J., Botvinick, M., & Wayne, G. (2019) · Nature Communications · doi:10.1038/s41467-019-13239-6 · Cited by 274 · see citing works
Harvey, F. G., Yurick, M., Nowrouzezahrai, D., & Pal, C. (2020) · ACM Transactions on Graphics · doi:10.1145/3386569.3392480 · Cited by 269 · see citing works
PhysDiff: Physics-Guided Human Motion Diffusion Model
Yuan, Y., Song, J., Iqbal, U., Vahdat, A., & Kautz, J. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01467 · Cited by 246 · see citing works
Peng, X. B., Guo, Y., Halper, L., Levine, S., & Fidler, S. (2022) · ACM Transactions on Graphics · doi:10.1145/3528223.3530110 · Cited by 228 · see citing works
Peng, X. B., Kanazawa, A., Malik, J., Abbeel, P., & Levine, S. (2018) · ACM Transactions on Graphics · doi:10.1145/3272127.3275014 · Cited by 227 · see citing works
Bergamin, K., Clavet, S., Holden, D., & Forbes, J. R. (2019) · ACM Transactions on Graphics · doi:10.1145/3355089.3356536 · Cited by 209 · see citing works
Learning symmetric and low-energy locomotion
Yu, W., Turk, G., & Liu, C. K. (2018) · ACM Transactions on Graphics · doi:10.1145/3197517.3201397 · Cited by 208 · see citing works
Scalable muscle-actuated human simulation and control
Lee, S., Park, M., Lee, K., & Lee, J. (2019) · ACM Transactions on Graphics · doi:10.1145/3306346.3322972 · Cited by 205 · see citing works
Learning predict-and-simulate policies from unorganized human motion data
Park, S., Ryu, H.-S., Lee, S., Lee, S., & Lee, J. (2019) · ACM Transactions on Graphics · doi:10.1145/3355089.3356501 · Cited by 152 · see citing works
Learning basketball dribbling skills using trajectory optimization and deep reinforcement learning
Liu, L., & Hodgins, J. K. (2018) · ACM Transactions on Graphics · doi:10.1145/3197517.3201315 · Cited by 149 · see citing works
Self-citationA scalable approach to control diverse behaviors for physically simulated characters
Won, J., Gopinath, D., & Hodgins, J. K. (2020) · ACM Transactions on Graphics · doi:10.1145/3386569.3392381 · Cited by 141 · see citing works
Self-citationThe Path Planning of Mobile Robot by Neural Networks and Hierarchical Reinforcement Learning
Yu, J., Su, Y., & Liao, Y. (2020) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2020.00063 · Cited by 141 · see citing works
SimPoE: Simulated Character Control for 3D Human Pose Estimation
Yuan, Y., Wei, S.-E., Simon, T., Kitani, K., & Saragih, J. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00708 · Cited by 133 · see citing works
Ego-Pose Estimation and Forecasting As Real-Time PD Control
Yuan, Y., & Kitani, K. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.01018 · Cited by 117 · see citing works
ALLSTEPS: Curriculum‐driven Learning of Stepping Stone Skills
Xie, Z., Ling, H. Y., Kim, N. H., & van de Panne, M. (2020) · Computer Graphics Forum · doi:10.1111/cgf.14115 · Cited by 110 · see citing works
Interactive character animation by learning multi-objective control
Lee, K., Lee, S., & Lee, J. (2018) · ACM Transactions on Graphics · doi:10.1145/3272127.3275071 · Cited by 98 · see citing works
Physics-based character controllers using conditional VAEs
Won, J., Gopinath, D., & Hodgins, J. K. (2022) · ACM Transactions on Graphics · doi:10.1145/3528223.3530067 · Cited by 96 · see citing works
Self-citationMCP: Learning Composable Hierarchical Control with Multiplicative Compositional PoliciesPreprint
Peng, X. B., Chang, M., Zhang, G., Abbeel, P., & Levine, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.09808 · Cited by 91 · see citing works
Learning to dress: Synthesizing human dressing motion via deep reinforcement learning
Clegg, A., Yu, W., Tan, J., Liu, C. K., & Turk, G. (2018) · ACM Transactions on Graphics · doi:10.1145/3272127.3275048 · Cited by 89 · see citing works
Learning body shape variation in physics-based characters
Won, J., & Lee, J. (2019) · ACM Transactions on Graphics · doi:10.1145/3355089.3356499 · Cited by 88 · see citing works
A Survey on Deep Learning for Skeleton‐Based Human Animation
Mourot, L., Hoyet, L., Clerc, F., Schnitzler, F., & Pierre Hellier, J. (2021) · Computer Graphics Forum · doi:10.1111/cgf.14426 · Cited by 88 · see citing works
Learning Fast Adaptation With Meta Strategy Optimization
Yu, W., Tan, J., Bai, Y., Coumans, E., & Ha, S. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.2974685 · Cited by 86 · see citing works
Yao, H., Song, Z., Chen, B., & Liu, L. (2022) · ACM Transactions on Graphics · doi:10.1145/3550454.3555434 · Cited by 84 · see citing works
Self-citationSharp interface approaches and deep learning techniques for multiphase flows
Gibou, F., Hyde, D., & Fedkiw, R. P. (2018) · Journal of Computational Physics · doi:10.1016/j.jcp.2018.05.031 · Cited by 83 · see citing works
Recurrent transition networks for character locomotion
Harvey, F. G., & Pal, C. (2018) · SIGGRAPH Asia Technical Briefs · doi:10.1145/3283254.3283277 · Cited by 82 · see citing works
Hierarchical visuomotor control of humanoidsPreprint
Merel, J., Ahuja, A., Pham, V., Tunyasuvunakool, S., Liu, S., Tirumala, D., Heess, N., & Wayne, G. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.09656 · Cited by 82 · see citing works
The Role of Physics-Based Simulators in Robotics
Liu, C. K., & Negruţ, D. (2020) · Annual Review of Control Robotics and Autonomous Systems · doi:10.1146/annurev-control-072220-093055 · Cited by 74 · see citing works
Control strategies for physically simulated characters performing two-player competitive sports
Won, J., Gopinath, D., & Hodgins, J. K. (2021) · ACM Transactions on Graphics · doi:10.1145/3450626.3459761 · Cited by 74 · see citing works
Self-citationRhythm is a Dancer: Music-Driven Motion Synthesis With Global Structure
Aristidou, A., Yiannakidis, A., Aberman, K., Cohen‐Or, D., Shamir, A., & Chrysanthou, Y. (2022) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2022.3163676 · Cited by 73 · see citing works
Deep motifs and motion signatures
Aristidou, A., Cohen‐Or, D., Hodgins, J. K., Chrysanthou, Y., & Shamir, A. (2018) · ACM Transactions on Graphics · doi:10.1145/3272127.3275038 · Cited by 70 · see citing works
Self-citationEfficient Neural Networks for Real-time Motion Style Transfer
Smith, H. J., Cao, C., Neff, M., & Wang, Y. (2019) · Proceedings of the ACM on Computer Graphics and Interactive Techniques · doi:10.1145/3340254 · Cited by 62 · see citing works
Luo, Y.-S., Soeseno, J. H., Chen, T. P.-C., & Chen, W.-C. (2020) · ACM Transactions on Graphics · doi:10.1145/3386569.3392433 · Cited by 61 · see citing works
Fussell, L., Bergamin, K., & Holden, D. (2021) · ACM Transactions on Graphics · doi:10.1145/3478513.3480527 · Cited by 61 · see citing works
Deep learning of biomimetic sensorimotor control for biomechanical human animation
Nakada, M., Zhou, T. Y., Chen, H., Weiss, T., & Terzopoulos, D. (2018) · ACM Transactions on Graphics · doi:10.1145/3197517.3201305 · Cited by 61 · see citing works
Fluid directed rigid body control using deep reinforcement learning
Ma, P., Tian, Y., Pan, Z., Ren, B., & Manocha, D. (2018) · ACM Transactions on Graphics · doi:10.1145/3197517.3201334 · Cited by 60 · see citing works
Residual Force Control for Agile Human Behavior Imitation and Extended Motion SynthesisPreprint
Yuan, Y., & Kitani, K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.07364 · Cited by 53 · see citing works
NNWarp: Neural Network-based Nonlinear Deformation
Luo, R., Shao, T., Wang, H., Xu, W., Chen, X., Zhou, K., & Yang, Y. (2018) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2018.2881451 · Cited by 50 · see citing works
DDQN-TS: A novel bi-objective intelligent scheduling algorithm in the cloud environment
Tong, Z., Ye, F., Liu, B., Cai, J., & Mei, J. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.05.070 · Cited by 49 · see citing works
GREIL-Crowds: Crowd Simulation with Deep Reinforcement Learning and Examples
Charalambous, P., Pettré, J., Vassiliades, V., Chrysanthou, Y., & Pelechano, N. (2023) · ACM Transactions on Graphics · doi:10.1145/3592459 · Cited by 48 · see citing works
MoConVQ: Unified Physics-Based Motion Control via Scalable Discrete Representations
Yao, H., Song, Z., Zhou, Y., Ao, T., Chen, B., & Liu, L. (2024) · ACM Transactions on Graphics · doi:10.1145/3658137 · Cited by 46 · see citing works
Self-citationMohajer, A., Bavaghar, M., & Farrokhi, H. (2020) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2020.107056 · Cited by 45 · see citing works
Generative Tweening: Long-term Inbetweening of 3D Human MotionsPreprint
Zhou, Y., Lu, J., Barnes, C., Yang, J., Xiang, S., & Li, H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.08891 · Cited by 38 · see citing works
UniCon: Universal Neural Controller For Physics-based Character MotionPreprint
Wang, T., Guo, Y., Shugrina, M., & Fidler, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.15119 · Cited by 38 · see citing works
A Comparison of Action Spaces for Learning Manipulation Tasks
Varin, P., Grossman, L., & Kuindersma, S. (2019) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8967946 · Cited by 37 · see citing works
Learning time-critical responses for interactive character control
Lee, K., Min, S., Lee, S., & Lee, J. (2021) · ACM Transactions on Graphics · doi:10.1145/3450626.3459826 · Cited by 35 · see citing works
Aerobatics control of flying creatures via self-regulated learning
Won, J., Park, J., & Lee, J. (2018) · ACM Transactions on Graphics · doi:10.1145/3272127.3275023 · Cited by 34 · see citing works
Reinforcement Learning with Competitive Ensembles of Information-Constrained PrimitivesPreprint
Goyal, A., Sodhani, S., Binas, J., Peng, X. B., Levine, S., & Bengio, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.10667 · Cited by 30 · see citing works
Positioning of the Robotic Arm Using Different Reinforcement Learning Algorithms
Lindner, T., Milecki, A., & Wyrwał, D. (2021) · International Journal of Control Automation and Systems · doi:10.1007/s12555-020-0069-6 · Cited by 28 · see citing works
Learning to Sit: Synthesizing Human-Chair Interactions via Hierarchical Control
Chao, Y.-W., Yang, J., Chen, W., & Deng, J. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i7.16736 · Cited by 28 · see citing works
Learning to use chopsticks in diverse gripping styles
Yang, Z., Yin, K., & Liu, L. (2022) · ACM Transactions on Graphics · doi:10.1145/3528223.3530057 · Cited by 28 · see citing works
Self-citationNeural3Points: Learning to Generate Physically Realistic Full‐body Motion for Virtual Reality Users
Ye, Y., Liu, L., Hu, L., & Xia, S. (2022) · Computer Graphics Forum · doi:10.1111/cgf.14634 · Cited by 26 · see citing works
Self-citationAdaptNet: Policy Adaptation for Physics-Based Character Control
Xu, P., Xie, K., Andrews, S., Kry, P. G., Neff, M., McGuire, M., Karamouzas, I., & Zordan, V. B. (2023) · ACM Transactions on Graphics · doi:10.1145/3618375 · Cited by 21 · see citing works
Learning Human Dynamics in Autonomous Driving Scenarios
Wang, J., Yuan, Y., Luo, Z., Xie, K., Lin, D., Iqbal, U., Fidler, S., & Khamis, S. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01901 · Cited by 21 · see citing works
MuscleVAE: Model-Based Controllers of Muscle-Actuated Characters
Feng, Y., Xu, X., & Liu, L. (2023) · SIGGRAPH Asia Conference Papers · doi:10.1145/3610548.3618137 · Cited by 18 · see citing works
Self-citationMohajer, A., Bavaghar, M., & Farrokhi, H. (2020) · Wireless Personal Communications · doi:10.1007/s11277-020-07481-1 · Cited by 18 · see citing works
RetractedGPU-based contact-aware trajectory optimization using a smooth force model
Pan, Z., Ren, B., & Manocha, D. (2019) · ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA) · doi:10.1145/3309486.3340246 · Cited by 17 · see citing works
Autonomous Character-Scene Interaction Synthesis from Text Instruction
Jiang, N., He, Z., Wang, Z., Li, H., Chen, Y., Huang, S., & Zhu, Y. (2024) · SIGGRAPH Asia Conference Papers · doi:10.1145/3680528.3687595 · Cited by 16 · see citing works
Wu, J., Liu, M., Lu, W., & Xie, K. (2021) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2021.129597 · Cited by 15 · see citing works
Intention-based Long-Term Human Motion Anticipation
Tanke, J., Zaveri, C., & Gall, J. (2021) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv53792.2021.00069 · Cited by 14 · see citing works
VMP: Versatile Motion Priors for Robustly Tracking Motion on Physical Characters
Serifi, A., Grandia, R., Knoop, E., Groß, M. H., & Bächer, M. (2024) · Computer Graphics Forum · doi:10.1111/cgf.15175 · Cited by 13 · see citing works
Reinforcement Learning via Gaussian Processes with Neural Network Dual Kernels
Goumiri, I. R., Priest, B. W., & Schneider, M. D. (2020) · IEEE Conference on Games (CoG) · doi:10.1109/cog47356.2020.9231744 · Cited by 12 · see citing works
Strategy and Skill Learning for Physics-based Table Tennis Animation
Wang, J., Hodgins, J. K., & Won, J. (2024) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/3641519.3657437 · Cited by 11 · see citing works
Self-citationTokenHSI: Unified Synthesis of Physical Human-Scene Interactions through Task Tokenization
Pan, L., Yang, Z., Dou, Z., Wang, W., Huang, B., Dai, B., Komura, T., & Wang, J. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00506 · Cited by 10 · see citing works
Improving decision-making efficiency of image game based on deep Q-learning
Ji, Z., & Xiao, W. (2020) · Soft Computing · doi:10.1007/s00500-020-04820-z · Cited by 10 · see citing works
A causal convolutional neural network for multi-subject motion modeling and generation
Hou, S., Wang, C., Zhuang, W., Chen, Y., Wang, Y., Bao, H., Chai, J., & Xu, W. (2023) · Computational Visual Media · doi:10.1007/s41095-022-0307-3 · Cited by 10 · see citing works
Plan Arithmetic: Compositional Plan Vectors for Multi-Task ControlPreprint
Devin, C., Geng, D., Abbeel, P., Darrell, T., & Levine, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.14033 · Cited by 10 · see citing works
Low Dimensional Motor Skill Learning Using Coactivation
Ranganath, A., Xu, P., Karamouzas, I., & Zordan, V. B. (2019) · Motion in Games · doi:10.1145/3359566.3360071 · Cited by 9 · see citing works
Human Motion Control of Quadrupedal Robots using Deep Reinforcement Learning
Kim, S., Sorokin, M., Lee, J., & Ha, S. (2022) · Robotics: Science and Systems XVIII · doi:10.15607/rss.2022.xviii.021 · Cited by 9 · see citing works
DROP: Dynamics Responses from Human Motion Prior and Projective Dynamics
Jiang, Y., Won, J., Ye, Y., & Liu, C. K. (2023) · SIGGRAPH Asia Conference Papers · doi:10.1145/3610548.3618175 · Cited by 9 · see citing works
Ogum, B. N., Schomaker, L., & Carloni, R. (2024) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2024.3352416 · Cited by 9 · see citing works
Resolving Collisions in Dense 3D Crowd Animations
Gomez-Nogales, G., Prieto-Martín, M., Romero, C., Trinidad, M. C., Ramon-Prieto, P., Olivier, A., Hoyet, L., Otaduy, M. Á., Pettré, J., & Casas, D. (2024) · ACM Transactions on Graphics · doi:10.1145/3687266 · Cited by 9 · see citing works
Figure Skating Simulation from Video
Yu, R., Park, H., & Lee, J. (2019) · Computer Graphics Forum · doi:10.1111/cgf.13831 · Cited by 9 · see citing works
Muscle‐driven virtual human motion generation approach based on deep reinforcement learning
Qin, W., Tao, R., Sun, L., & Dong, K. (2022) · Computer Animation and Virtual Worlds · doi:10.1002/cav.2092 · Cited by 8 · see citing works
A Reinforcement Learning Approach To Synthesizing Climbing Movements
Naderi, K., Babadi, A., Roohi, S., & Hämäläinen, P. (2019) · IEEE Conference on Games (CoG) · doi:10.1109/cig.2019.8848127 · Cited by 7 · see citing works
Expanding Motor Skills using Relay Networks.
Kumar, V. C. V., Ha, S., & Liu, C. K. (2018) · Conference on Robot Learning · Cited by 7 · see citing works
Understanding the stability of deep control policies for biped locomotion
Park, H., Yu, R., Lee, Y., Lee, K., & Lee, J. (2022) · The Visual Computer · doi:10.1007/s00371-021-02342-9 · Cited by 7 · see citing works
Skiing Simulation Based on Skill‐Guided Motion Planning
Hu, C., Lee, C., Liou, Y., Sung, F., & Lin, W. (2019) · Computer Graphics Forum · doi:10.1111/cgf.13606 · Cited by 7 · see citing works
Evolution-Based Shape and Behavior Co-Design of Virtual Agents
Wang, Z., Beneš, B., Qureshi, A. H., & Mousas, C. (2024) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2024.3355745 · Cited by 6 · see citing works
Zhang, B., Hu, W., Cao, D., Huang, Q. P., & Chen, Z. (2021) · Asia Energy and Electrical Engineering Symposium (AEEES) · doi:10.1109/aeees51875.2021.9403176 · Cited by 6 · see citing works
Human Motion Anticipation with Symbolic LabelPreprint
Tanke, J., Weber⋆, A., & Gall, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.06079 · Cited by 6 · see citing works
Learning When to Switch: Composing Controllers to Traverse a Sequence of Terrain Artifacts
Tidd, B., Cosgun, A., Leitner, J., & Hudson, N. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9636233 · Cited by 5 · see citing works
Generating a Ball Sport Scene in a Virtual Environment
Choi, J., Kim, S. K., Kim, S.-J., & Kang, S.-J. (2019) · KSII Transactions on Internet and Information Systems · doi:10.3837/tiis.2019.11.013 · Cited by 3 · see citing works
Discovering Fatigued Movements for Virtual Character Animation
Cheema, N., Xu, R., Kim, N. H., Hämäläinen, P., Golyanik, V., Habermann, M., Theobalt, C., & Slusallek, P. (2023) · SIGGRAPH Asia Conference Papers · doi:10.1145/3610548.3618176 · Cited by 3 · see citing works
Optimizing Neural Network as Locomotion Controller With Motion Data
Guo, S., Jiang, W., Gao, X. H., Wang, M., & Liao, M. (2018) · IEEE Access · doi:10.1109/access.2018.2834380 · Cited by 3 · see citing works
Primitive Object Grasping for Finger Motion Synthesis
Hwang, J., Park, G., Suh, I. H., & Kwon, T. (2020) · Computer Graphics Forum · doi:10.1111/cgf.14187 · Cited by 3 · see citing works
Control Operators for Interactive Character Animation
Gou, R., van de Panne, M., & Holden, D. (2025) · ACM Transactions on Graphics · doi:10.1145/3763319 · Cited by 2 · see citing works
Toward Robust Long Range Policy Transfer
Tseng, W.-C., Lin, J.-S., Feng, Y.-M., & Sun, M. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i11.17196 · Cited by 2 · see citing works
Inverse Dynamics Filtering for Sampling‐based Motion Control
Xie, K., & Kry, P. G. (2021) · Computer Graphics Forum · doi:10.1111/cgf.14274 · Cited by 2 · see citing works
Research on manipulator motion planning for complex systems based on deep learning
Ding, W., Liu, Y., Hongbo, Z., Shah, M. A., & Ikbal, M. A. (2021) · International Journal of Systems Assurance Engineering and Management · doi:10.1007/s13198-021-01219-3 · Cited by 2 · see citing works
Motion Generation of a Single Rigid Body Character Using Deep Reinforcement Learning
Ahn, J., Gu, T., & Kwon, T. (2021) · Journal of the Korea Computer Graphics Society · doi:10.15701/kcgs.2021.27.3.13 · Cited by 2 · see citing works
Adaptive Policy Transfer in Reinforcement LearningPreprint
Joshi, G., & Chowdhary, G. V. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.04699 · Cited by 2 · see citing works
Character Motion Synthesis Based on Deep Learning: A Survey
Chen, A. (2023) · Highlights in Science Engineering and Technology · doi:10.54097/fg717x36 · Cited by 1 · see citing works
MDD: Masked Deconstructed Diffusion for 3D Human Motion Generation from Text
Chen, J., Liu, F., & Wang, Y. (2025) · IEEE International Conference on Artificial Intelligence and eXtended and Virtual Reality (AIxVR) · doi:10.1109/aixvr63409.2025.00017 · Cited by 1 · see citing works
Gated neural network framework for interactive character control
Wang, X., Jiang, X., Regedzai, G. R., Meng, H., & Sun, L. (2020) · Multimedia Tools and Applications · doi:10.1007/s11042-020-08792-y · Cited by 1 · see citing works
Inference-Based Deep Reinforcement Learning for Physics-Based Character Control
Chen, Y., Huang, C., Yuan, W., Wu, Y., Zhang, Z., Tang, H., & Zhang, Y. (2022) · IEEE International Conference on High Performance Computing and Communications; IEEE International Conference on Smart City; IEEE International Conference on Data Science and Systems (HPCC/SmartCity/DSS) · doi:10.1109/hpcc-dss-smartcity-dependsys57074.2022.00346 · Cited by 1 · see citing works
Dynamic Virtual Human Control Guided by Kinetic Energy Model
Mao, H., Chen, L., Guo, Y., & Wang, C. (2023) · Journal of Computer-Aided Design & Computer Graphics · doi:10.3724/sp.j.1089.2023.19270 · Cited by 1 · see citing works
Expanding Motor Skills through Relay Neural NetworksPreprint
Kumar, V. C. V., Ha, S., & Liu, C. K. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.07932 · Cited by 1 · see citing works
- Authors
- Liu, L., & Hodgins, J.
- Affiliations
- Walt Disney (United States) (US)
- Carnegie Mellon University (US)
- ORCID
- Hodgins, Jessica
- Venue
- ACM Transactions on Graphics
- Published
- 2017-06-27
- Volume
- 36
- Issue
- 3
- Pages
- 1-14
- ISSN
- 0730-0301 (ISSN-L)
- 1557-7368
- OpenAlex
- W4250901985
- Cited by
- 49
- References
- 41
- Citations by year
- 2025: 4 · 2024: 5 · 2023: 4 · 2022: 2 · 2021: 6 · 2020: 5 · 2019: 7 · 2018: 16
- FWCI
- 4.81
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- ACM Transactions on Graphics (publishedVersion)
- Funders
- University of Pennsylvania
- Topics
- Physical Sciences › Engineering › Control and Systems Engineering › Human Motion and Animation
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Computer Graphics and Visualization Techniques
- Keywords
- physics-based animation, deep Q-learning, Q-network, robust control, dynamic locomotion, balancing, skateboarding, breakdancing, push recovery
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review