Multi-Resolution Real-Time Deep Pose-Space Deformation
Open accessThe DOI matches its Crossref record; 51 of the 52 references it cites were found in a registry or an index.
Abstract
We present a hard-real-time multi-resolution mesh shape deformation technique for skeleton-driven soft-body characters. Producing mesh deformations at multiple levels of detail is very important in many applications in computer graphics. Our work targets applications where the multi-resolution shapes must be generated at fast speeds ("hard-real-time", e.g., a few milliseconds at most and preferably under 1 millisecond), as commonly needed in computer games, virtual reality and Metaverse applications. We assume that the character mesh is driven by a skeleton, and that high-quality character shapes are available in a set of training poses originating from a high-quality (slow) rig such as volumetric FEM simulation. Our method combines multi-resolution analysis, mesh partition of unity, and neural networks, to learn the pre-skinning shape deformations in an arbitrary character pose. Combined with linear blend skinning, this makes it possible to reconstruct the training shapes, as well as interpolate and extrapolate them. Crucially, we simultaneously achieve this at hard real-time rates and at multiple mesh resolution levels. Our technique makes it possible to trade deformation quality for memory and computation speed, to accommodate the strict requirements of modern real-time systems. Furthermore, we propose memory layout and code improvements to boost computation speeds. Previous methods for realtime approximations of quality shape deformations did not focus on hard real-time, or did not investigate the multi-resolution aspect of the problem. Compared to a "naive" approach of separately processing each hierarchical level of detail, our method offers a substantial memory reduction as well as computational speedups. It also makes it possible to construct the shape progressively level by level and interrupt the computation at any time, enabling graceful degradation of the deformation detail. We demonstrate our technique on several examples, including a stylized human character, human hands, and an inverse-kinematics-driven quadruped animal.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The text we found for this paper looks like another document, so the self-check wasn’t run.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
PyTorch: An Imperative Style, High-Performance Deep Learning Library
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al. (2019) · arXiv · doi:10.48550/arxiv.1912.01703 · Cited by 31,473
Loper, M., Mahmood, N., Romero, J., Pons-Moll, G., & J, B. M. (2015) · ACM Transactions on Graphics · doi:10.1145/2816795.2818013 · Cited by 3,734
Surface simplification using quadric error metrics
Garland, M., & Heckbert, P. S. (1997) · Proceedings of the 24th annual conference on Computer graphics and interactive techniques - SIGGRAPH '97 · doi:10.1145/258734.258849 · Cited by 3,412
Hoppe, H. (1996) · Proceedings of the 23rd annual conference on Computer graphics and interactive techniques · doi:10.1145/237170.237216 · Cited by 2,817
(2003) · Computer Graphics Forum · doi:10.1111/1467-8659.00652 · Cited by 1,436
Pose space deformation: a unified approach to shape interpolation and skeleton-driven deformation
Lewis, J. P., Cordner, M., & Fong, N. (2000) · Proceedings of the 27th annual conference on Computer graphics and interactive techniques - SIGGRAPH '00 · doi:10.1145/344779.344862 · Cited by 853
Special Finite Element Methods for a Class of Second Order Elliptic Problems with Rough Coefficients
Babuška, I., Caloz, G., & Osborn, J. E. (1994) · SIAM Journal on Numerical Analysis · doi:10.1137/0731051 · Cited by 661 · cited 1×
“Partitions of unity are of course not new, see, e.g. [Babuška et al. 1994; Ohtake et al. 2003]; our contribution is to combine them with neural networks and a multi-resolution scheme.”
Funkhouser, T. A., & Séquin, C. H. (1993) · Proceedings of the 20th annual conference on Computer graphics and interactive techniques · doi:10.1145/166117.166149 · Cited by 603
Geometric skinning with approximate dual quaternion blending
Kavan, L., Collins, S., Žára, J., & O'Sullivan, C. (2008) · ACM Transactions on Graphics · doi:10.1145/1409625.1409627 · Cited by 385 · cited 2×
“In Figure 15, we compare our technique to Dual Quaternion Skin-ning [Kavan et al. 2008].”
Provably good sampling and meshing of surfaces
Boissonnat, J.-D., & Oudot, S. (2005) · Graphical Models · doi:10.1016/j.gmod.2005.01.004 · Cited by 324 · cited 1×
“We then use isosurface meshing [Boissonnat and Oudot 2005] to create a quality surface triangle mesh of the skin ( 𝐷 = 0), and the shrunken skin ( 𝐷 = − 𝑑, where 𝑑 > 0 is the “fat” thickness).”
EigenSkin: real time large deformation character skinning in hardware
Kry, P. G., James, D. L., & Pai, D. K. (2002) · Proceedings of the 2002 ACM SIGGRAPH/Eurographics symposium on Computer animation · doi:10.1145/545261.545286 · Cited by 267
CHARMS: a simple framework for adaptive simulation
Grinspun, E., Krysl, P., & Schröder, P. (2002) · Proceedings of the 29th annual conference on Computer graphics and interactive techniques · doi:10.1145/566570.566578 · Cited by 259
Efficient elasticity for character skinning with contact and collisions
McAdams, A., Zhu, Y., Selle, A., Empey, M., Tamstorf, R., Teran, J., & Sifakis, E. (2011) · ACM Transactions on Graphics · doi:10.1145/2010324.1964932 · Cited by 197
Multi-level partition of unity implicits
Ohtake, Y., Belyaev, A. G., Alexa, M., Turk, G., & Seidel, H. (2003) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/1201775.882293 · Cited by 197
Learning‐Based Animation of Clothing for Virtual Try‐On
Santesteban, I., Otaduy, M. A., & Casas, D. (2019) · Computer Graphics Forum · doi:10.1111/cgf.13643 · Cited by 189
NASA Neural Articulated Shape Approximation
Deng, B., Lewis, J. P., Jeruzalski, T., Pons-Moll, G., Hinton, G., Norouzi, M., & Tagliasacchi, A. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58571-6_36 · Cited by 185
CHARMS: a simple framework for adaptive simulation
Grinspun, E., Krysl, P., & Schröder, P. (2002) · ACM Transactions on Graphics · doi:10.1145/566654.566578 · Cited by 179
Real-time deep dynamic characters
Habermann, M., Liu, L., Xu, W., Zollhöfer, M., Pons‐Moll, G., & Theobalt, C. (2021) · ACM Transactions on Graphics · doi:10.1145/3450626.3459749 · Cited by 159
A multiresolution framework for dynamic deformations
Capell, S., Green, S., Curless, B., Duchamp, T., & Popović, Z. (2002) · ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA) · doi:10.1145/545261.545268 · Cited by 145
PBNS: physically based neural simulation for unsupervised garment pose space deformation
Bertiche, H., Madadi, M., & Escalera, S. (2021) · ACM Transactions on Graphics · doi:10.1145/3478513.3480479 · Cited by 140
Efficient elasticity for character skinning with contact and collisions
McAdams, A., Zhu, Y., Selle, A. P., Empey, M., Tamstorf, R., Teran, J. M., & Sifakis, E. D. (2011) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/1964921.1964932 · Cited by 139
An efficient multigrid method for the simulation of high-resolution elastic solids
Zhu, Y., Sifakis, E. D., Teran, J. M., & Brandt, A. (2010) · ACM Transactions on Graphics · doi:10.1145/1731047.1731054 · Cited by 131 · cited 1×
“They can be combined with FEM simulation [Capell et al. 2002; Debunne et al. 2001; Grinspun et al. 2002; Zhang et al. 2022], and form the core of the multigrid method [Otaduy et al. 2007; Zhu et al. 2010].”
Latent‐space Dynamics for Reduced Deformable Simulation
Fulton, L., Modi, V., Duvenaud, D., Levin, D. I. W., & Jacobson, A. (2019) · Computer Graphics Forum · doi:10.1111/cgf.13645 · Cited by 131 · cited 1×
“Fulton et al. [Fulton et al. 2019] utilized an autoencoder neural network to generate nonlinear reduced spaces for deformation dynamics, and Holder et al. [Holden et al. 2019] used a neural network to learn the motion and interaction of deformable objects in a subspace.”
James, D. L., & Pai, D. K. (2003) · ACM Transactions on Graphics · doi:10.1145/588272.588278 · Cited by 107 · cited 1×
“…[Garland and Heckbert 1997] to create our mesh LODs, but our technique is suitable for other mesh hierarchies, as long as they permit a definition of prolongation and restriction operators [Liu et al. 2021], e.g., progressive meshes [Hoppe 1996], or mesh subdivision [James and Pai 2003].”
Subspace neural physics: fast data-driven interactive simulation
Holden, D., Duong, B. C., Datta, S., & Nowrouzezahrai, D. (2019) · ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA) · doi:10.1145/3309486.3340245 · Cited by 106
Physics-inspired upsampling for cloth simulation in games
Kavan, L., Gerszewski, D., Bargteil, A. W., & Sloan, P. (2011) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/1964921.1964988 · Cited by 101
Physics-inspired upsampling for cloth simulation in games
Kavan, L., Gerszewski, D., Bargteil, A. W., & Sloan, P. (2011) · ACM Transactions on Graphics · doi:10.1145/2010324.1964988 · Cited by 79
Pose Space Deformation: A Unified Approach to Shape Interpolation and Skeleton-Driven Deformation
Lewis, J. P., Cordner, M., & Fong, N. (2023) · Seminal Graphics Papers: Pushing the Boundaries, Volume 2 · doi:10.1145/3596711.3596796 · Cited by 76
Provably good sampling and meshing of Lipschitz surfaces
Boissonnat, J., & Oudot, S. (2006) · Annual Symposium on Computational Geometry (SoCG) · doi:10.1145/1137856.1137906 · Cited by 70
NeuroSkinning: automatic skin binding for production characters with deep graph networks
Liu, L., Zheng, Y., Tang, D., Yuan, Y., Fan, C., & Zhou, K. (2019) · ACM Transactions on Graphics · doi:10.1145/3306346.3322969 · Cited by 67
Surface multigrid via intrinsic prolongation
Liu, H.-T. D., Zhang, J. E., Ben‐Chen, M., & Jacobson, A. (2021) · ACM Transactions on Graphics · doi:10.1145/3450626.3459768 · Cited by 62
Fast and deep deformation approximations
Bailey, S. W., Otte, D., Dilorenzo, P., & O'Brien, J. F. (2018) · ACM Transactions on Graphics · doi:10.1145/3197517.3201300 · Cited by 61 · cited 11×
“As is commonly done [Bailey et al. 2018; Kry et al. 2002; Lewis et al. 2000], our method operates completely on pre-skinning displacements.”
InfluentialSoftSMPL: Data‐driven Modeling of Nonlinear Soft‐tissue Dynamics for Parametric Humans
Santesteban, I., Garcés, E., Otaduy, M. Á., & Casas, D. (2020) · Computer Graphics Forum · doi:10.1111/cgf.13912 · Cited by 61
NNWarp: Neural Network-based Nonlinear Deformation
Luo, R., Shao, T., Wang, H., Xu, W., Chen, X., Zhou, K., & Yang, Y. (2018) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2018.2881451 · Cited by 50
Fast and deep facial deformations
Bailey, S. W., Omens, D., DiLorenzo, P. C., & O’Brien, J. F. (2020) · ACM Transactions on Graphics · doi:10.1145/3386569.3392397 · Cited by 49
Learning contact corrections for handle-based subspace dynamics
Romero, C., Casas, D., Pérez, J., & Otaduy, M. Á. (2021) · ACM Transactions on Graphics · doi:10.1145/3450626.3459875 · Cited by 48
Automatic linearization of nonlinear skinning
Kavan, L., Collins, S., & O’Sullivan, C. (2009) · ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games (I3D) · doi:10.1145/1507149.1507157 · Cited by 45
Cloth and Skin Deformation with a Triangle Mesh Based Convolutional Neural Network
Chentanez, N., Macklin, M., Müller, M., Jeschke, S., & Kim, T. (2020) · Computer Graphics Forum · doi:10.1111/cgf.14107 · Cited by 40 · cited 1×
“Simulating clothing using deep learning is another well-studied field [Bertiche et al. 2021; Chentanez et al. 2020; Santesteban et al. 2019].”
Accurate face rig approximation with deep differential subspace reconstruction
Song, S. L., Shi, W., & Reed, M. (2020) · ACM Transactions on Graphics · doi:10.1145/3386569.3392491 · Cited by 37
A Deep Emulator for Secondary Motion of 3D Characters
Zheng, M., Zhou, Y., Ceylan, D., & Barbič, J. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00587 · Cited by 35
Self-citationAdaptive Deformations with Fast Tight Bounds
Otaduy, M. Á., Germann, D., Redon, S., & Groß, M. H. (2007) · HAL · doi:10.2312/sca/sca07/181-190 · Cited by 35
Contact-centric deformation learning
Romero, C., Casas, D., Chiaramonte, M. M., & Otaduy, M. Á. (2022) · ACM Transactions on Graphics · doi:10.1145/3528223.3530182 · Cited by 25
Modeling and Estimation of Nonlinear Skin Mechanics for Animated Avatars
Romero, C., Otaduy, M. Á., Casas, D., & Pérez, J. (2020) · Computer Graphics Forum · doi:10.1111/cgf.13913 · Cited by 22
Real‐time interaction of virtual and physical objects in mixed reality applications
Badías, A., González, D., Alfaro, I., Chinesta, F., & Cueto, E. (2020) · International Journal for Numerical Methods in Engineering · doi:10.1002/nme.6385 · Cited by 20 · cited 1×
“Badias et al. [Badías et al. 2020] applied model reduction to contact solutions in a reduced space, enabling real-time interaction between virtual and physical objects.”
Simulation of Hand Anatomy Using Medical Imaging
Zheng, M., Wang, B., Huang, J., & Barbič, J. (2022) · ACM Transactions on Graphics · doi:10.1145/3550454.3555486 · Cited by 13
Self-citationA survey of subdivision-based tools for surface modeling
Boier-Martin, I. M., Zorin, D., & Bernardini, F. (2005) · DIMACS series in discrete mathematics and theoretical computer science · doi:10.1090/dimacs/067/01 · Cited by 4 · cited 2×
“Our technique is grounded in multi-resolution methods extensively investigated in computer graphics [Boier-Martin et al. 2005].”
SkinCells: Sparse Skinning using Voronoi Cells
Larionov, E., Santesteban, I., Chen, H.-Y., Lin, G. W.-C., Herholz, P., Goldade, R., Kavan, L., Roble, D., & Stuyck, T. (2026) · Computer Graphics Forum · doi:10.1111/cgf.70381 · Cited by 4
Applications of Machine Learning for Character Animation
Bailey, S. W. (2020) · eScholarship (California Digital Library) · Cited by 1
Dynamic real-time deformations using space & time adaptive sampling
Debunne, G., Desbrun, M., Cani, M.-P., & Barr, A. H. (2001) · Proceedings of the 28th annual conference on Computer graphics and interactive techniques · doi:10.1145/383259.383262
Skinning: real-time shape deformation (full text not available)
Jacobson, A., Deng, Z., Kavan, L., & Lewis, J. P. (2014) · ACM SIGGRAPH 2014 Courses · doi:10.1145/2614028.2615427
Progressive Simulation for Cloth Quasistatics
Zhang, J. E., Dumas, J., Fei, Y. (., Jacobson, A., James, D. L., & Kaufman, D. M. (2022) · ACM Transactions on Graphics · doi:10.1145/3550454.3555510
Research works printed but not linked: 6
- [29] Binh Huy Le and Zhigang Deng. 2014. Robust and accurate skeletal rigging from mesh sequences. ACM Trans. on Graphics (SIGGRAPH 2014) 33, 4 (2014), 1--10. Google Scholar
- [30] Binh Huy Le and John-Peter Lewis. 2019. Direct delta mush skinning and variants. ACM Trans. on Graphics (SIGGRAPH 2019) 38, 4 (2019), 113--1. Google Scholar
- [32] Peizhuo Li, Kfir Aberman, Rana Hanocka, Libin Liu, Olga Sorkine-Hornung, and Baoquan Chen. 2021. Learning skeletal articulations with neural blend shapes. ACM Trans. on Graphics (SIGGRAPH 2021) 40, 4 (2021), 1--15. Google Scholar
- [40] Miguel A. Otaduy. 2004. 6-DoF Haptic Rendering Using Contact Levels of Detail and Haptic Textures. Ph. D. Dissertation. Department of Comp. Science, University of North Carolina at Chapel Hill. Google Scholar
- [48] Hongyi Xu and Jernej Barbič. 2014. Signed Distance Fields for Polygon Soup Meshes. Graphics Interface 2014 (2014). Digital Library Google Scholar
- [49] Zhan Xu, Yang Zhou, Evangelos Kalogerakis, Chris Landreth, and Karan Singh. 2020. Rignet: Neural rigging for articulated characters. ACM Trans. on Graphics (SIGGRAPH 2020) 39, 4 (2020), 14. Digital Library Google Scholar
Cited by
Self-supervised Learning of Latent Space Dynamics
Li, Y., Lin, G. W.-C., Larionov, E., et al. (2025) · Proceedings of the ACM on Computer Graphics and Interactive Techniques · doi:10.1145/3747854 · Cited by 3
“…et al. 2019; Li et al. 2023], deformable bodies [Barbič and James 2005; Chen et al. 2023; Harmon and Zorin 2013; Wang et al. 2015; Wu et al. 2015; Zheng and Barbic 2024], animated characters [Barbič et al. 2009, 2012; Jacobson et al. 2012] and fluids [Kim and Delaney 2013; Treuille et al. 2006].”
Metadata
- Authors
- Zheng, M., & Barbic, J.
- Affiliations
- University of Southern California (US)
- University of Southern California, Los Angeles, United States of America
- Venue
- ACM Transactions on Graphics
- Published
- 2024-11-19
- Volume
- 43
- Issue
- 6
- Pages
- 1-11
- ISSN
- 0730-0301 (ISSN-L)
- 1557-7368
- OpenAlex
- W4404527030
- Cited by
- 1
- References
- 18
- Citations by year
- 2025: 1
- FWCI
- 0.20
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM Transactions on Graphics (publishedVersion, cc-by)
- Funders
- National Science Foundation (IIS-1911224)
- Topics
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › 3D Shape Modeling and Analysis
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Computer Graphics and Visualization Techniques
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Human Motion and Animation
- Keywords
- linear blend skinning, partition of unity, artificial neural networks, memory reduction, inverse kinematics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A second model labelled each claim on 2026-10-29 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.