The DOI matches its Crossref record; 72 of the 72 references it cites were found in a registry or an index.
Deep inertial poser: learning to reconstruct human pose from sparse inertial measurements in real time
Open accessAbstract
We demonstrate a novel deep neural network capable of reconstructing human full body pose in real-time from 6 Inertial Measurement Units (IMUs) worn on the user's body. In doing so, we address several difficult challenges. First, the problem is severely under-constrained as multiple pose parameters produce the same IMU orientations. Second, capturing IMU data in conjunction with ground-truth poses is expensive and difficult to do in many target application scenarios (e.g., outdoors). Third, modeling temporal dependencies through non-linear optimization has proven effective in prior work but makes real-time prediction infeasible. To address this important limitation, we learn the temporal pose priors using deep learning. To learn from sufficient data, we synthesize IMU data from motion capture datasets. A bi-directional RNN architecture leverages past and future information that is available at training time. At test time, we deploy the network in a sliding window fashion, retaining real time capabilities. To evaluate our method, we recorded DIP-IMU, a dataset consisting of 10 subjects wearing 17 IMUs for validation in 64 sequences with 330 000 time instants; this constitutes the largest IMU dataset publicly available. We quantitatively evaluate our approach on multiple datasets and show results from a real-time implementation. DIP-IMU and the code are available for research purposes. 1
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Recent approaches addressing the lack of annotated datasets involve extraction of virtual movement data from modalities such as mocap [37, 72] and videos [44], which contain a large number of participants and diverse set of actions and environments.”
From Contrastive Predictive Coding for Human Activity Recognition (2021) · cites this paper as [37]
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
He, K., Gkioxari, G., Dollar, P., & Girshick, R. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.322 · Cited by 29,137
Bidirectional recurrent neural networks
Schuster, M., & Paliwal, K. K. (1997) · IEEE Transactions on Signal Processing · doi:10.1109/78.650093 · Cited by 10,172
Stacked Hourglass Networks for Human Pose Estimation
Newell, A., Yang, K., & Deng, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46484-8_29 · Cited by 5,634
WaveNet: a generative model for raw audio
Oord, A. V. D., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1609.03499 · Cited by 4,911
Loper, M., Mahmood, N., Romero, J., Pons-Moll, G., & J, B. M. (2015) · ACM Transactions on Graphics · doi:10.1145/2816795.2818013 · Cited by 3,734
Self-citationHuman3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments
Ionescu, C., Papava, D., Olaru, V., & Sminchisescu, C. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.248 · Cited by 3,653
DeepPose: Human Pose Estimation via Deep Neural Networks
Toshev, A., & Szegedy, C. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.214 · Cited by 3,395
Wei, S.-E., Ramakrishna, V., Kanade, T., & Sheikh, Y. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.511 · Cited by 2,966
Real-time human pose recognition in parts from single depth images
Shotton, J., Sharp, T., Kipman, A., Fitzgibbon, A. W., Finocchio, M., Blake, A., Cook, M., & Moore, R. (2013) · Communications of the ACM · doi:10.1145/2398356.2398381 · Cited by 2,050
Research works printed but not linked: 11
- 2018. Unifying Motion Capture Datasets by Automatically Solving for Full-body Shape and Motion. In preparation ( 2018 ). 2018. Unifying Motion Capture Datasets by Automatically Solving for Full-body Shape and Motion. In preparation (2018).
- 10.5555/794191.794776 · DOI
- Zhe Cao , Tomas Simon , Shih-En Wei , and Yaser Sheikh . 2016. Realtime multi-person 2d pose estimation using part affinity fields. arXiv preprint arXiv:1611.08050 ( 2016 ). Zhe Cao, Tomas Simon, Shih-En Wei, and Yaser Sheikh. 2016. Realtime multi-person 2d pose estimation using part affinity fields. arXiv preprint arXiv:1611.08050 (2016). · arXiv
- 10.5555/2919332.2919834 · DOI
- Partha Ghosh , Jie Song , Emre Aksan , and Otmar Hilliges . 2017. Learning Human Motion Models for Long-term Predictions. arXiv preprint arXiv:1704.02827 ( 2017 ). Partha Ghosh, Jie Song, Emre Aksan, and Otmar Hilliges. 2017. Learning Human Motion Models for Long-term Predictions. arXiv preprint arXiv:1704.02827 (2017). · arXiv
- Diederik P Kingma and Jimmy Ba . 2014 . Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014). Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014). · arXiv
- 10.5555/2354409.2354910 · DOI
- Daniel Roetenberg , Henk Luinge , and Per Slycke . 2007 . Moven: Full 6dof human motion tracking using miniature inertial sensors . Xsen Technologies , December (2007). Daniel Roetenberg, Henk Luinge, and Per Slycke. 2007. Moven: Full 6dof human motion tracking using miniature inertial sensors. Xsen Technologies, December (2007).
- Yu Tao , Zerong Zheng , Kaiwen Guo , Jianhui Zhao , Dai Quionhai , Hao Li , Gerard Pons-Moll , and Yebin Liu . 2018 . DoubleFusion: Real-time Capture of Human Performance with Inner Body Shape from a Depth Sensor , In IEEE Conf. on Computer Vision and Pattern Recognition. IEEE Conf. on Computer Vision and Pattern Recognition. CVPR Oral. Yu Tao, Zerong Zheng, Kaiwen Guo, Jianhui Zhao, Dai Quionhai, Hao Li, Gerard Pons-Moll, and Yebin Liu. 2018. DoubleFusion: Real-time Capture of Human Performance with Inner Body Shape from a Depth Sensor, In IEEE Conf. on Computer Vision and Pattern Recognition. IEEE Conf. on Computer Vision and Pattern Recognition. CVPR Oral.
- 10.5555/2354409.2354668 · DOI
- Jindong Wang , Yiqiang Chen , Shuji Hao , Xiaohui Peng , and Lisha Hu. 2017. Deep learning for sensor-based activity recognition: A survey. arXiv preprint arXiv:1707.03502 ( 2017 ). Jindong Wang, Yiqiang Chen, Shuji Hao, Xiaohui Peng, and Lisha Hu. 2017. Deep learning for sensor-based activity recognition: A survey. arXiv preprint arXiv:1707.03502 (2017). · arXiv
Cited by
Sorted by citations
Items 301–341 of 341
Full-Body Pose Motion Tracking From Sparse Data via Morphology-Aware Constraints
Yang, Y., Zhang, S., Wei, C., Feng, C., Liu, X., & Ye, L. (2025) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2025.3624630
Bian, Q., Wang, H. B., Alsayed, K., & Ding, Z. (2026) · Frontiers in Bioengineering and Biotechnology · doi:10.3389/fbioe.2025.1737916 · Cited by 1
Predicting gait kinetics using 3-degrees of freedom acceleration data and artificial neural networks
Gempfer, C., Santos, G. F., Hobein, A., Kuhlgatz, T., Hurschler, C., & Jakubowitz, E. (2026) · Clinical Biomechanics · doi:10.1016/j.clinbiomech.2026.106775
HeadGaze: Natural Gaze Estimation While Walking From a Single IMU on Earbuds
Takaki, K., Kamezaki, M., Sasatani, T., Murakami, H., & Kawahara, Y. (2026) · IEEE Sensors Journal · doi:10.1109/jsen.2026.3661116
ArmNet: Robust Arm Motion Tracking for IoT Interaction Using a Single IMU
Wang, K., Jia, Q., Xie, X. N., Li, K., Tao, X., Chen, S., & Li, K. (2026) · IEEE Internet of Things Journal · doi:10.1109/jiot.2026.3667973
Yu, T., Kong, L., Peng, R., & Liu, J. (2026) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2026.109906
Human-centered design-based lightweight wearable IMU human pose estimation
Wang, L., Liu, J., Xue, J., Tan, Q., & Li, M. (2026) · Scientific Reports · doi:10.1038/s41598-026-41004-5
Fang, K., Yan, W., Ma, H., & Yang, Z. (2025) · International Conference on Robotics, Automation and Artificial Intelligence (RAAI) · doi:10.1109/raai67517.2025.11423315
Taetz, B., Bleser, G., Miezal, M., & Stricker, D. (2026) · Open Research Europe · doi:10.12688/openreseurope.22844.1
Kuo, H.-Y., & Chang, K.-M. (2026) · Joint International Conference on Digital Arts, Media and Technology with ECTI Northern Section Conference on Electrical, Electronics, Computer and Telecommunications Engineering (ECTI DAMT & NCON) · doi:10.1109/ectidamtncon67592.2026.11460053
Enhancing Human-Robot Interaction with Computer Vision
Du, Y. (2023) · theses.fr (ABES) · doi:10.70675/9c6ed094z8c91z435fz8111ze1ed33a7fe92
Head and Body Pose Estimation using Ultrasound Sensing on Smart Glasses
Li, K., Kips, R., Arcadu, F., Schikora, M., Ho, J. S., Graf, C. G. C., Chen, J. H., Cai, X., Sivapalan, G., Zarei, A., et al. (2026) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3772363.3798385
Li, Z., Wang, L., Yu, C., Liu, C., Du, M., Shi, W., & Shi, Y. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791028 · Cited by 1
Amortized inverse kinematics via graph attention for real-time human avatar animation
Khan, M. S. U., Wang, C.-Y., Prokosch, T., Lorenz, M., Taetz, B., & Stricker, D. (2026) · Open Research Europe · doi:10.12688/openreseurope.24416.1
Head2Body: Body Pose Generation from Multi-Sensory Head-Mounted Inputs
Tran, M., Mao, H., Chen, Q., & Kim, Y. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00644
Hierarchical Diffusion for Sparse-to-Full Human Motion Reconstruction
Du, F., Yang, Y., Gao, X., Hou, H., & Sun, C. (2026) · IEEE Signal Processing Letters · doi:10.1109/lsp.2026.3694693
Balasubramanyam, A., Velpanur, S. M., Sushma, E. J., J, T. C., Honnavalli, P. B., & Srinivasa, G. (2026) · Sensors · doi:10.3390/s26103262
Physical Self-Supervised Learning: IMU Sensing without Manual Labels
Leng, Y., Liu, R., Hu, S., Zhao, P., Chen, C.-F., Chen, S., & Yao, S. (2026) · International Conference on Mobile Systems, Applications, and Services · doi:10.1145/3745756.3809252
Coherent Human Animation Generation Based on Cross-Modal Contrastive LearningPreprint
(14029), X. Z., Ye, L., Song, W., Ding, Y., Chai, Z., Li, S., Hou, X., Hao, A., & Qin, H. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6876305
Huang, L., & Xia, C. (2026) · International Conference on Multimedia Retrieval · doi:10.1145/3805622.3810642
PMP: Plug-and-Play Physics Momentum Prior for Contact-Stable 3D Human Pose Estimation
Tao, W. F. (2026) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg67764.2026.11557004
From Sensor Data to Sequence of Scenes: Diffusion Model-Based Visualization of Human Activity
Matsuishi, K., Ukita, K., & Okita, T. (2026) · International Conference on Activity and Behavior Computing (ABC) · doi:10.1109/abc68169.2026.11567210
Towards Real-World Wearable Motion Reconstruction
Camiletto, A. B., Dabral, R., Alvarado, E., Beeler, T., Habermann, M., & Theobalt, C. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37281-9_17 · Cited by 1
Application-Oriented In-Cabin Human Pose Estimation With Six Wearable IMUs and Sensor-Aware Training
Natale, T., Ndozi, F., Russo, F., Polonelli, T., Dindelli, L., Dell’Olio, F., & Magno, M. (2026) · IEEE Sensors Journal · doi:10.1109/jsen.2026.3711871
Hierarchical spatiotemporal fusion for full-body pose estimation using sparse wearable sensors
Yang, Y., Zhang, S., Wei, C., Li, S., & Ye, L. (2026) · Information Fusion · doi:10.1016/j.inffus.2026.104625
EgoExoMoCap: Distributed Ego-Exo Human Motion Capture
Jiang, J., Bhatnagar, B. L., Yang, N., Ma, L., Starke, S., Kips, R., Bertsch, N., Holz, C., & Bogo, F. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37422-6_6
Beyond the Skeleton: Cross-Modal Photorealistic Human Reconstruction from Sparse Inertial Signals
Li, J., Tao, C., Zhang, C., Wang, R., Jiao, J., & Wang, C. (2026) · Youth Academic Annual Conference of Chinese Association of Automation (YAC) · doi:10.1109/yac71005.2026.11615403
A Sensor Fusion Approach to Mitigate Lift-Off- and Tilt Angle-Induced Noise in Eddy-Current Testing
Li, J., Li, S. Y., Shi, Z., & Xie, Y. (2026) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2026.3720802
Hieu, N. Q., Alem, Y. F., & Alsheikh, M. A. (2026) · IEEE Internet of Things Journal · doi:10.1109/jiot.2026.3723210
Transpose-Seq: Sequential 3D Human Pose Estimation with Six Inertial Measurement Units (IMUS)
Wu, Y., Teh, M. H., Li, T., Zhang, X., Bian, Q., & Yin, J. (2026) · International Conference on Computer, Control and Robotics (ICCCR) · doi:10.1109/icccr69988.2026.11642756
A Quantitative Comparison of Human Body Modeling Methods to Synthesize IMU Data
Uhlenberg, L., Kempfle, J., Haeusler, L. O., & Amft, O. (2026) · IEEE Access · doi:10.1109/access.2026.3726408
Lu, Y., Li, X., Wang, S., Xu, Y., Zhou, X., Yao, Z., Sun, Y., & Yang, X. (2026) · Pattern Recognition · doi:10.1016/j.patcog.2026.114826
Mamba Inertial Poser: Dual-temporal causal human motion tracking from sparse inertial sensors
Han, S., Zhou, X., Zhou, C., Lu, F., & Yang, J. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.134383
Upper-Limb Pose Estimation From Sparse Body-Worn Visual-Inertial Sensors
Pompanin, F., Martini, E., Calanca, A., Fummi, F., & Bombieri, N. (2026) · IEEE Signal Processing Letters · doi:10.1109/lsp.2026.3733600
WaviTrack: Exploiting Body-Area RF Channels for Wearable Human Motion Reconstruction
Wu, F. (., Lai, G., Mills, X., Shi, W., Gilbert, T., & Liu, R. (2026) · ACM International Symposium on Wearable Computers (ISWC) · doi:10.1145/3830727.3834831
Generating Human Pose and Activity Videos from Wearable Ultra-Wideband Sensors
Toha, T. R., Wang, Z., Jagadeesh, D., Zhang, G., Chen, T., & Nirjon, S. (2026) · International Workshop on Human-Centered Sensing, Modeling, and Intelligent Systems · doi:10.1145/3842436.3843802
EchoMotion: Towards Full-Body Pose Tracking with a Single Smartwatch Using Active Acoustic Sensing
Mahmud, S., Kim, J., Yu, T. C., Liang, Q., Parikh, V. N., Guimbretière, F. V., & Zhang, C. (2026) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3832001
Ndozi, F., Russo, F., Polonelli, T., & Magno, M. (2026) · IEEE Workshop on Design Automation for CPS and IoT (DESTION) · doi:10.1109/destion71208.2026.00010
MARIO: Motion-Augmented Real-Time Multi-Sensor Inertial Odometry
Li, Y., Yeon, T., Gao, C., Xu, V., Liu, X., & Ahuja, K. (2026) · doi:10.1109/cvprf71909.2026.00336
Unlocking Practical Cardiac Monitoring Capabilities on True Wireless Stereo Earbuds
Fu, Y., Sun, K., Zhang, X., Wang, R., Li, X., Pan, H., Zhang, Y., & Ren, J. (2026) · ACM Annual International Conference on Mobile Computing and Networking (MobiCom) · doi:10.1145/3795866.3796678
ViPoser: Sparse-IMU Based Human Pose Estimation with Distilled Vision Foundation Priors
Zeng, H., Ji, H., & Zhou, P. (2026) · ACM Annual International Conference on Mobile Computing and Networking (MobiCom) · doi:10.1145/3795866.3844462
Metadata
- Authors
- Huang, Y., Kaufmann, M., Aksan, E., Black, M. J., Hilliges, O., & Pons-Moll, G.
- Affiliations
- Max Planck Institute for Intelligent Systems (DE)
- ETH Zurich (CH)
- Max Planck Institute for Informatics, Saarbrücken, Germany
- Venue
- ACM Transactions on Graphics
- Published
- 2018-12-04
- Volume
- 37
- Issue
- 6
- Pages
- 1-15
- ISSN
- 0730-0301 (ISSN-L)
- 1557-7368
- OpenAlex
- W2898073175
- Cited by
- 342
- References
- 72
- Citations by year
- 2026: 43 · 2025: 54 · 2024: 69 · 2023: 56 · 2022: 38 · 2021: 44 · 2020: 26 · 2019: 12
- FWCI
- 10.25
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- ACM Transactions on Graphics (publishedVersion)
- Funders
- Nvidia
- H2020 European Research Council (717054)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Human Motion and Animation
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › 3D Shape Modeling and Analysis
- Keywords
- inertial measurement unit, full-body pose estimation, real-time pose estimation, bidirectional recurrent neural network, motion capture, wearable sensors
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
