LimbMotion: Decimeter-level Limb Tracking for Wearable-based Human-Computer Interaction
The DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.
Abstract
Wearable-based human-computer interaction is a promising technology to enable various applications. This paper aims to track the 3D posture of the entire limb, both wrist/ankle and elbow/knee, of a user wearing a smart device. This limb tracking technology can trace the geometric motion of the limb, without introducing any training stage usually required in gesture recognition approaches. Nonetheless, the tracked limb motion can also be used as a generic input for gesture-based applications. The 3D posture of a limb is defined by the relative positions among main joints, e.g., shoulder, elbow, and wrist for an arm. When a smartwatch is worn on the wrist of a user, its position is affected by both elbow and shoulder motions. It is challenging to infer the entire 3D posture when only given a single point of sensor data from the smartwatch. In this paper, we propose LimbMotion, an accurate and real-time limb tracking system. The performance gain of LimbMotion comes from multiple key technologies, including an accurate attitude estimator based on a novel two-step filter, fast acoustic ranging, and point clouds-based positioning. We implemented LimbMotion and evaluated its performance using extensive experiments, including different gestures, moving speeds, users, and limbs. Results show that LimbMotion achieves real-time tracking with a median error of 7.5cm to 8.9cm, which outperforms the state-of-the-art approach by about 32%.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Numerical Methods for Least Squares Problems
Björck, Å. (1996) · Society for Industrial and Applied Mathematics eBooks · doi:10.1137/1.9781611971484 · Cited by 3,257
Estimation of IMU and MARG orientation using a gradient descent algorithm
Madgwick, S. O. H., Harrison, A. J. L., & Vaidyanathan, R. (2011) · 2011 IEEE International Conference on Rehabilitation Robotics · doi:10.1109/icorr.2011.5975346 · Cited by 2,068 · cited 2×
“Traditionally, researchers have applied sensor fusion methods to fuse outputs from multiple IMU sensors, including accelerometers, gyroscopes, and compasses [5, 7, 10, 25].”
BackgroundNonlinear Complementary Filters on the Special Orthogonal Group
Mahony, R., Hamel, T., & Pflimlin, J.-M. (2008) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2008.923738 · Cited by 1,751 · cited 2×
“Traditional methods [26] use a 9-axis IMU for attitude estimation, which consists of a 3-axis accelerometer, a 3-axis compass, and a 3-axis gyroscope.”
MethodVision based hand gesture recognition for human computer interaction: a survey
Rautaray, S. S., & Agrawal, A. (2015) · Artificial Intelligence Review · doi:10.1007/s10462-012-9356-9 · Cited by 1,526 · cited 1×
“Many research groups and start-ups have studied this human-computer interaction (HCI) technology using various methodologies, e.g., wearable-based approaches [18, 19, 23, 39] and computer vision-based approaches [22, 34, 41].”
BackgroundGait Analysis Using Wearable Sensors
Tao, W., Liu, T., Zheng, R., & Feng, H. (2012) · Sensors · doi:10.3390/s120202255 · Cited by 1,165 · cited 1×
“Step counting techniques are utilized in many commercial products [11], and further gait analysis is investigated for user authentication [44].”
MethodSurvey of Nonlinear Attitude Estimation Methods
Crassidis, J. L., Markley, F. L., & Cheng, Y. (2007) · Journal of Guidance, Control, and Dynamics · doi:10.2514/1.22452 · Cited by 1,094 · cited 2×
“Traditionally, researchers have applied sensor fusion methods to fuse outputs from multiple IMU sensors, including accelerometers, gyroscopes, and compasses [5, 7, 10, 25].”
BackgroundQuaternion-Based Extended Kalman Filter for Determining Orientation by Inertial and Magnetic Sensing
Sabatini, A. M. (2006) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2006.875664 · Cited by 887 · cited 1×
“First, we implemented a Mahony passive complementary filter and an EKF that are widely used [26, 37].”
MethodA reliable and accurate indoor localization method using phone inertial sensors
Li, F., Zhao, C., Ding, G., Gong, J., Liu, C., & Zhao, F. (2012) · Proceedings of the 2012 ACM Conference on Ubiquitous Computing · doi:10.1145/2370216.2370280 · Cited by 574 · cited 1×
“We thought the reason could be the magnetic disturbance in the indoor environment, which has been noticed by existing works [9, 21, 49].”
BackgroundBeepBeep: a high accuracy acoustic ranging system using COTS mobile devices
Peng, C., Shen, G., Zhang, Y., Li, Y., & Tan, K. (2007) · Proceedings of the 5th international conference on Embedded networked sensor systems · doi:10.1145/1322263.1322265 · Cited by 473 · cited 3×
“BeepBeep [31] addresses this synchronization problem by a two-way sensing method, which converts the calculation of TOA with different device clocks to device local clocks.”
MethodInfluentialFingerIO: Using Active Sonar for Fine-Grained Finger Tracking
Nandakumar, R., Iyer, V., Tan, D., & Gollakota, S. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858580 · Cited by 408 · cited 1×
“There are also echo-based ranging techniques, such as FingerIO [29].”
Method
Research works printed but not linked: 8
- Chen Min, 2011, Body area networks: A survey. Mobile networks and applications, 16(2):171--193
- Jiang L., 2018, Safedrive: Detecting distracted driving behaviors using wrist-worn devices, ACM IMWUT, 1, 1
- Maciejasz P., 2014, A survey on robotic devices for upper limb rehabilitation. Journal of neuroengineering and rehabilitation, 11(1):3
- Ranganathan Ananth, 2004, The levenberg-marquardt algorithm, Tutoral on LM algorithm, 11, 101
- Roetenberg Daniel, 2009, Full 6dof human motion tracking using miniature inertial sensors. Xsens Motion Technologies BV, Tech. Rep, 1
- Ruder Sebastian, 2016, An overview of gradient descent optimization algorithms. arXiv preprint arXiv:1609.04747 · arXiv
- 10.5555/1082243.1082256 · doi:10.5555/1082243.1082256
- Zhang C., 2017, Soundtrak: Continuous 3d tracking of a finger using active acoustics, ACM IMWUT, 1, 30
Cited by
Xu, H., Zhou, P., Tan, R., Li, M., & Shen, G. (2021) · Proceedings of the 19th ACM Conference on Embedded Networked Sensor Systems · doi:10.1145/3485730.3485937 · Cited by 150
Gao, L., Zhang, G., Yu, B., Qiao, Z., & Wang, J. (2020) · Measurement · doi:10.1016/j.measurement.2020.108252 · Cited by 104
HOOV: Hand Out-Of-View Tracking for Proprioceptive Interaction using Inertial Sensing
Streli, P., Armani, R., Cheng, Y. F., & Holz, C. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581468 · Cited by 50
SmartPoser: Arm Pose Estimation with a Smartphone and Smartwatch Using UWB and IMU Data
DeVrio, N., Mollyn, V., & Harrison, C. (2023) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3586183.3606821 · Cited by 42
“In the context of body tracking, worn systems have combined IMU data with magnetic feld sensors [39, 55], ultrasonic transmitters and micro-phones [19, 26, 47], infrared distance sensors [56], micro-fow sensors [27], acoustic ranging [61], and optical linear encoders [33].”
BackgroundMethodReal-Time Limb Motion Tracking with a Single IMU Sensor for Physical Therapy Exercises
Wei, W., Kurita, K., Kuang, J., & Gao, A. (2021) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc46164.2021.9630480 · Cited by 24
“LimbMotion [15] uses acoustic ranging to detect the distance between the smartwatch and an edge device.”
BackgroundMethodInfluentialBoudlal, H., Serrhini, M., & Tahiri, A. (2024) · Signal Image and Video Processing · doi:10.1007/s11760-024-03031-5 · Cited by 27
Real-Time 3D Arm Motion Tracking Using the 6-axis IMU Sensor of a Smartwatch
Wei, W., Kurita, K., Kuang, J., & Gao, A. J. (2021) · Wearable and Implantable Body Sensor Networks · doi:10.1109/bsn51625.2021.9507012 · Cited by 21
Posture Tracking Meets Fitness Coaching: A Two-Phase Optimization Approach with Wearable Devices
Zhou, H., Gao, Y. C., Liu, W., Jiang, Y., & Dong, W. (2020) · IEEE International Conference on Mobile Ad-Hoc and Sensor Systems (MASS) · doi:10.1109/mass50613.2020.00070 · Cited by 19
Self-citationSkeleton-Based Human Pose Recognition Using Channel State Information: A Survey
Wang, Z., Ma, M., Feng, X., Li, X., Liu, F., Guo, Y., & Chen, D. (2022) · Sensors · doi:10.3390/s22228738 · Cited by 19
“FMCW Imaging WiFi Sensing (Section 2.2) FMCW Imaging WiFi Sensing (Section 2.2) (Section 2.1) (Section 2.1)”
InfluentialGomes, P. R. I., Castro, M., & Nascimento, T. H. (2023) · Brazilian Symposium on Human Factors in Computing Systems (IHC) · doi:10.1145/3638067.3638082 · Cited by 11
Toward Sensor-In-the-Loop LLM Agent: Benchmarks and Implications
Ren, Z., Li, J., Zhang, M., Wang, D., Fan, X., & Shangguan, L. (2025) · ACM Conference on Embedded Networked Sensor Systems (SenSys) · doi:10.1145/3715014.3722082 · Cited by 17
AccessWear: Making Smartphone Applications Accessible to Blind Users
Khanna, P., Feiz, S., Xu, J., Ramakrishnan, I. V., Jain, S., Bi, X., & Balasubramanian, A. (2023) · ACM Annual International Conference on Mobile Computing and Networking (MobiCom) · doi:10.1145/3570361.3592495 · Cited by 9
Nie, J., Fan, Y., Xuan, Z., Zhao, M., Wan, R., Preindl, M., & Jiang, X. (2025) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3729486 · Cited by 7
Chang, T.-C., Wu, Y., Han, C., & Chang, C.-S. (2024) · International Conference on Internet of Things: Systems, Management and Security (IOTSMS) · doi:10.1109/iotsms62296.2024.10710280 · Cited by 5
UltraMotion: High-precision Ultrasonic Arm Tracking for Real-world Exercises
Niu, X., Zou, K., Shen, D., Drew, S., Wu, S., Guo, G., & Chen, R. (2023) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2023.3241077 · Cited by 6
Jia, H., Hu, J., & Hu, W. S. (2021) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3478082 · Cited by 11
3D Hand Tracking with Induced Magnetic Field
Bian, S., Zhou, B., Lukowicz, P., Liu, M., Magno, M., & Guo, K. (2023) · IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops) · doi:10.1109/percomworkshops56833.2023.10150286 · Cited by 4
Towards Customizable Foundation Models for Human Activity Recognition with Wearable Devices
Qiu, M., Weng, C., Fan, M., & Wu, K. (2025) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3749479 · Cited by 4
VIHand: Enhancing 3D Hand Pose Estimation with Visual-Inertial Benchmark
Wang, X., Ren, P., Zhang, H., Sheng, X., Li, D., Xie, L., Gao, Y., & Yin, E. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3758215 · Cited by 6
Self-citationMembership Inference Against Self-supervised IMU Sensing Applications
Zhao, T., Wang, N., & Wang, X. (2025) · ACM Conference on Embedded Networked Sensor Systems (SenSys) · doi:10.1145/3715014.3722060 · Cited by 4
Zhao, X., Yang, S., Deng, J., & Zhang, J. (2022) · Reactive and Functional Polymers · doi:10.1016/j.reactfunctpolym.2022.105241 · Cited by 1
Technical Report: Edge-centric Programming for IoT Applications with EdgeProgPreprint
Li, B., & Dong, W. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.15098 · Cited by 1
Self-citationExperience Paper: Adopting Activity Recognition in On-demand Food Delivery Business
Xu, H., Zhang, Y., Gao, W., Shen, G., & Li, M. (2025) · ACM Annual International Conference on Mobile Computing and Networking (MobiCom) · doi:10.1145/3680207.3765261 · Cited by 5
Zhao, Y., Song, X., Huang, H., & Braud, T. (2025) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3770631
Self-citationA Survey on Multi-Device Systems Involving Smartwatches
Liu, Y., Huang, J., Gu, Y., Polidoro, F., Yu, L., & Liu, Y. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2026.2643339
Adaptive Human Intent Recognition in Collaborative Robots Using Wearable AI
Tilawat, R., Saleheen, N., & Akther, S. (2025) · Conference on AI, Science, Engineering, and Technology (AIxSET) · doi:10.1109/aixset65682.2025.00016
Learning From Decoupled Sensor Axes: Enhancing Cross-Dataset Human Activity Recognition
Tang, Z.-P., He, S.-B., Wang, C., Chen, Q., Pan, J., & Gu, C.-J. (2026) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2026.3723993
Metadata
- Authors
- Zhou, H., Gao, Y., Song, X., Liu, W., & Dong, W.
- Affiliations
- Zhejiang University (CN)
- Venue
- Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
- Published
- 2019-12-11
- Volume
- 3
- Issue
- 4
- Pages
- 1-24
- ISSN
- 2474-9567 (ISSN-L)
- OpenAlex
- W3003717288
- Cited by
- 27
- References
- 42
- Citations by year
- 2026: 2 · 2025: 8 · 2024: 2 · 2023: 6 · 2022: 2 · 2021: 5 · 2020: 2
- FWCI
- 2.13
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies (publishedVersion)
- Funders
- Zhejiang Provincial Natural Science Foundation for Distinguished Young Scholars (No. LR19F020001)
- Fundamental Research Funds for the Central Universities (No. 2018FZA5013)
- National Science Foundation of China (No.61872437, 61772465)
- Topics
- Physical Sciences › Computer Science › Human-Computer Interaction › Hand Gesture Recognition Systems
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Indoor and Outdoor Localization Technologies
- Keywords
- smartwatches, gesture recognition, attitude estimation, acoustic ranging, real-time tracking
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.