Real-time Arm Skeleton Tracking and Gesture Inference Tolerant to Missing Wearable Sensors
The DOI matches its Crossref record; 57 of the 60 references it cites were found in a registry or an index.
Abstract
This paper presents ArmTroi, a wearable system for understanding and analyzing the detailed arm motions of people primarily by using the motion sensors from wrist-worn wearable devices. ArmTroi can achieve real-time 3D arm skeleton tracking and reliable gesture inference tolerant to missing wearable sensors for enabling numerous useful application designs. We have coped with two major challenges through ArmTroi. First, the skeleton of each arm is determined from the locations of the elbow and wrist, whereas a wearable device only senses a single point from the wrist. We find that the potential solution space is huge. This underconstrained nature fundamentally challenges the achievement of accurate and real-time arm skeleton tracking. Second, wearable sensors may not reliably provide sensory data. For example, devices are not worn by the user, yet the learning tools for gesture inference, such as deep learning, typically have static network structures, which require nontrivial network adaptation to match the input's varying availability and ensure reliable gesture inference. We propose effective techniques to address above challenges, and all computations can be conducted on the user's smartphone. ArmTroi is thus a fully lightweight and portable system. We develop a prototype and extensive evaluation shows the efficacy of the ArmTroi design.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“As a result, the available devices are an arbitrary subset of the total devices, which is unpredictable and would severely impact the recognition performance of existing methods.”
From Spatial-Temporal Masked Autoencoder for Multi-Device Wearable Human Activity Recognition (2023) · cites this paper as [45]
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
(2014) · Cited by 33,419
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766 · cited 1×
“The attention technique is widely used in the fields of natural language generation and visual object recognition [45].”
MethodBidirectional recurrent neural networks
Schuster, M., & Paliwal, K. K. (1997) · IEEE Transactions on Signal Processing · doi:10.1109/78.650093 · Cited by 10,172 · cited 8×
“Its output is yr = (y 0 r , · · · ,y T−1 r ), and each ytr = o t r ◦ fy (c t r ), where ctr = f t r ◦ c t−1 r + i t r ◦ fc (W c r x t r +W y r y t−1 r + b c r ) is the cell in an LSTM model; otr , f t r and i t r are gates in an LSTM model;W c r and W y r are the weights; b c r is a bias vector; fy and fc are the acti…”
MethodInfluentialEffective Approaches to Attention-based Neural Machine Translation
Luong, T., Pham, H., & Manning, C. D. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1166 · Cited by 8,382 · cited 2×
“We introduce an attention-based design to achieve a virtual network adaptation to address this possible missing input data issue, where attention [27] is a novel technique for adjusting certain parameters of a network (by multiplying a weighting vector where each weight can vary) to enhance the output on the basis of…”
MethodShow, Attend and Tell: Neural Image Caption Generation with Visual Attention
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.03044 · Cited by 8,248 · cited 5×
“To reduce computation, fatt (·) is typically realized as a single-layer multilayer perceptron, such as tanh(·) and ReLu(·) [1, 53]; however, it still follows the same principle as a complete neural network, i.”
BackgroundMethodInfluentialRealtime Multi-person 2D Pose Estimation Using Part Affinity Fields
Cao, Z., Simon, T., Wei, S.-E., & Sheikh, Y. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.143 · Cited by 7,539 · cited 1×
“Recently, people can utilize cameras to track user’s 2D upper limbs or even the whole body with errors of about 2 cm to enable person re-identification, video re-targeting and robot teaching applications [3, 56].”
BackgroundRecurrent neural network based language model
Mikolov, T., Karafiát, M., Burget, L., Černocký, J., & Khudanpur, S. (2010) · Interspeech 2010 · doi:10.21437/interspeech.2010-343 · Cited by 5,390 · cited 2×
“With the aforementioned inputs, we propose to first apply a recurrent neural network (RNN) to extract the input’s individual dynamic motion features [30] and then gradually concatenate them with additional RNNs for further extracting characteristics from an aggregated view, which are finally used for the gesture infer…”
BackgroundMethod(2011) · Cited by 2,752
A Survey on Human Activity Recognition using Wearable Sensors
Lara, O. D., & Labrador, M. A. (2013) · IEEE Communications Surveys & Tutorials · doi:10.1109/surv.2012.110112.00192 · Cited by 2,662 · cited 1×
“Deep learning is widely applied in applications [6, 9, 14, 16, 24, 50, 52] without using handcrafted features [2, 22].”
Background
Research works printed but not linked: 9
- Bahdanau Dzmitry, 2015, Proc. of ICLR .
- Du Yong, 2015, Proc. of IEEE CVPR .
- Hammerla Nils Yannick, 2015, Proc. of AAAI .
- Ngiam Jiquan, 2011, Proc. of ICML .
- 10.5555/1460257.1460283 · doi:10.5555/1460257.1460283
- Nancy Berryman Reese and William D Bandy. 2016. Joint Range of Motion and Muscle Length Testing-E-Book .Elsevier Health Sciences. Nancy Berryman Reese and William D Bandy. 2016. Joint Range of Motion and Muscle Length Testing-E-Book .Elsevier Health Sciences.
- Shan Chew Zhen, 2015, Proc. of IEEE ASCC .
- Sutskever Ilya, 2014, Proc. of NIPS .
- Xu Kelvin, 2015, Proc. of ICML .
Cited by
Xu, H., Zhou, P., Tan, R., Li, M., & Shen, G. (2021) · Proceedings of the 19th ACM Conference on Embedded Networked Sensor Systems · doi:10.1145/3485730.3485937 · Cited by 150
“, accelerometer, gyroscope, and magnetometer), which are widely used in various applications including estimating the attitudes ofmobile devices [34, 59], tracking users’ arm [24, 35], and implementing secure text pin [3].”
BackgroundAdversarial Attacks and Defenses on Cyber–Physical Systems: A Survey
Li, J., Liu, Y., Chen, T., Xiao, Z., Li, Z., & Wang, J. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.2975654 · Cited by 95
“4) Other Sensors: In addition to the three types of sensors stated above, some other sensors have also been utilized in the CPS and IoT systems, e.g., the inertial measurement unit (IMU) sensors for smart health [41] and the activity recognition [31], [49] applications.”
MethodSelf-citationYin, Y., Xie, L., Jiang, Z., Xiao, F., Cao, J., & Lu, S. (2024) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2024.3357591 · Cited by 108
“…walking upstairs and downstairs, jogging, watching TV, eating, washing dishes, shaking hands, making a call etc; a series of activities in a specific scenario (e.g., activities in drinking [106] or toothbrushing [52]); and the vital sign changes [89] like breathing, heartbeat-ing and coughs.”
InfluentialArtificial Intelligence of Things: A Survey
Siam, M. S. I., Ahn, H., Liu, L., Alam, S., Shen, H., Cao, Z., Shroff, N. B., Krishnamachari, B., Srivastava, M. B., & Zhang, M. (2024) · ACM Transactions on Sensor Networks · doi:10.1145/3690639 · Cited by 123
On-Device Deep Learning for Mobile and Wearable Sensing Applications: A Review
İncel, Ö. D., & Bursa, S. Ö. (2023) · IEEE Sensors Journal · doi:10.1109/jsen.2023.3240854 · Cited by 110
Metadata
- Authors
- Liu, Y., Li, Z., Liu, Z., & Wu, K.
- Affiliations
- City University of Hong Kong (HK)
- Shenzhen University (CN)
- Venue
- Proceedings of the 17th Annual International Conference on Mobile Systems, Applications, and Services
- Publisher
- ACM
- Published
- 2019-06-12
- Pages
- 287-299
- OpenAlex
- W2953033606
- Cited by
- 71
- References
- 60
- Citations by year
- 2026: 4 · 2025: 8 · 2024: 12 · 2023: 15 · 2022: 12 · 2021: 11 · 2020: 7 · 2019: 2
- FWCI
- 6.48
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Conference on Mobile Systems, Applications, and Services (publishedVersion)
- Funders
- ECS grant (CityU 21203516)
- NSFC Grant (No.61802261 and No.61872248)
- Guangdong NSF (2017A030312008)
- Fok Ying-Tong Education Foundation for Young Teachers in the Higher Education Institutions of China (No.161064)
- NSF Grant of Shenzhen University (No.2018061)
- GRF grant (CityU 11217817)
- Topics
- Physical Sciences › Computer Science › Human-Computer Interaction › Hand Gesture Recognition Systems
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Physical Sciences › Computer Science › Human-Computer Interaction › Interactive and Immersive Displays
- Keywords
- wearable sensors, missing sensor data, wrist-worn devices, real-time tracking
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Real-time Arm Skeleton Tracking and Gesture Inference Tolerant to Missing Wearable Sensors. https://app.cochat.ai/papers/10.1145/3307334.3326109
- The paper
- Liu, Y., Li, Z., Liu, Z., & Wu, K. (2019). Real-time Arm Skeleton Tracking and Gesture Inference Tolerant to Missing Wearable Sensors. Proceedings of the 17th Annual International Conference on Mobile Systems, Applications, and Services. https://doi.org/10.1145/3307334.3326109
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.