CoChat

Real-time Arm Skeleton Tracking and Gesture Inference Tolerant to Missing Wearable Sensors

Liu, Yang, Li, Zhenjiang, Liu, Zhidan, Wu, Kaishun · 2019 · Proceedings of the 17th Annual International Conference on Mobile Systems, Applications, and Services

The DOI matches its Crossref record; 57 of the 60 references it cites were found in a registry or an index.

Abstract

This paper presents ArmTroi, a wearable system for understanding and analyzing the detailed arm motions of people primarily by using the motion sensors from wrist-worn wearable devices. ArmTroi can achieve real-time 3D arm skeleton tracking and reliable gesture inference tolerant to missing wearable sensors for enabling numerous useful application designs. We have coped with two major challenges through ArmTroi. First, the skeleton of each arm is determined from the locations of the elbow and wrist, whereas a wearable device only senses a single point from the wrist. We find that the potential solution space is huge. This underconstrained nature fundamentally challenges the achievement of accurate and real-time arm skeleton tracking. Second, wearable sensors may not reliably provide sensory data. For example, devices are not worn by the user, yet the learning tools for gesture inference, such as deep learning, typically have static network structures, which require nontrivial network adaptation to match the input's varying availability and ensure reliable gesture inference. We propose effective techniques to address above challenges, and all computations can be conducted on the user's smartphone. ArmTroi is thus a fully lightweight and portable system. We develop a prototype and extensive evaluation shows the efficacy of the ArmTroi design.

Overview

Integrity signals

Registry

Crossref · 10.1145/3307334.3326109

Cites

57 of 57 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

  • Long Short-Term Memory

    Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318

  • Untitled work

    (2014) · Cited by 33,419

  • Sequence to Sequence Learning with Neural Networks

    Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766 · cited 1×

    “The attention technique is widely used in the fields of natural language generation and visual object recognition [45].”
    Method
  • Bidirectional recurrent neural networks

    Schuster, M., & Paliwal, K. K. (1997) · IEEE Transactions on Signal Processing · doi:10.1109/78.650093 · Cited by 10,172 · cited 8×

    “Its output is yr = (y 0 r , · · · ,y T−1 r ), and each ytr = o t r ◦ fy (c t r ), where ctr = f t r ◦ c t−1 r + i t r ◦ fc (W c r x t r +W y r y t−1 r + b c r ) is the cell in an LSTM model; otr , f t r and i t r are gates in an LSTM model;W c r and W y r are the weights; b c r is a bias vector; fy and fc are the acti…”
    MethodInfluential
  • Effective Approaches to Attention-based Neural Machine Translation

    Luong, T., Pham, H., & Manning, C. D. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1166 · Cited by 8,382 · cited 2×

    “We introduce an attention-based design to achieve a virtual network adaptation to address this possible missing input data issue, where attention [27] is a novel technique for adjusting certain parameters of a network (by multiplying a weighting vector where each weight can vary) to enhance the output on the basis of…”
    Method
  • Show, Attend and Tell: Neural Image Caption Generation with Visual Attention

    Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.03044 · Cited by 8,248 · cited 5×

    “To reduce computation, fatt (·) is typically realized as a single-layer multilayer perceptron, such as tanh(·) and ReLu(·) [1, 53]; however, it still follows the same principle as a complete neural network, i.”
    BackgroundMethodInfluential
  • Realtime Multi-person 2D Pose Estimation Using Part Affinity Fields

    Cao, Z., Simon, T., Wei, S.-E., & Sheikh, Y. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.143 · Cited by 7,539 · cited 1×

    “Recently, people can utilize cameras to track user’s 2D upper limbs or even the whole body with errors of about 2 cm to enable person re-identification, video re-targeting and robot teaching applications [3, 56].”
    Background
  • Recurrent neural network based language model

    Mikolov, T., Karafiát, M., Burget, L., Černocký, J., & Khudanpur, S. (2010) · Interspeech 2010 · doi:10.21437/interspeech.2010-343 · Cited by 5,390 · cited 2×

    “With the aforementioned inputs, we propose to first apply a recurrent neural network (RNN) to extract the input’s individual dynamic motion features [30] and then gradually concatenate them with additional RNNs for further extracting characteristics from an aggregated view, which are finally used for the gesture infer…”
    BackgroundMethod
  • Untitled work

    (2011) · Cited by 2,752

  • A Survey on Human Activity Recognition using Wearable Sensors

    Lara, O. D., & Labrador, M. A. (2013) · IEEE Communications Surveys & Tutorials · doi:10.1109/surv.2012.110112.00192 · Cited by 2,662 · cited 1×

    “Deep learning is widely applied in applications [6, 9, 14, 16, 24, 50, 52] without using handcrafted features [2, 22].”
    Background

Show all 57 references

Research works printed but not linked: 9

  1. Bahdanau Dzmitry, 2015, Proc. of ICLR .
  2. Du Yong, 2015, Proc. of IEEE CVPR .
  3. Hammerla Nils Yannick, 2015, Proc. of AAAI .
  4. Ngiam Jiquan, 2011, Proc. of ICML .
  5. 10.5555/1460257.1460283 · doi:10.5555/1460257.1460283
  6. Nancy Berryman Reese and William D Bandy. 2016. Joint Range of Motion and Muscle Length Testing-E-Book .Elsevier Health Sciences. Nancy Berryman Reese and William D Bandy. 2016. Joint Range of Motion and Muscle Length Testing-E-Book .Elsevier Health Sciences.
  7. Shan Chew Zhen, 2015, Proc. of IEEE ASCC .
  8. Sutskever Ilya, 2014, Proc. of NIPS .
  9. Xu Kelvin, 2015, Proc. of ICML .

Cited by

See all 71

Metadata

Authors
Liu, Y., Li, Z., Liu, Z., & Wu, K.
Affiliations
  • City University of Hong Kong (HK)
  • Shenzhen University (CN)
Venue
Proceedings of the 17th Annual International Conference on Mobile Systems, Applications, and Services
Publisher
ACM
Published
2019-06-12
Pages
287-299
OpenAlex
W2953033606
Cited by
71
References
60
Citations by year
2026: 4 · 2025: 8 · 2024: 12 · 2023: 15 · 2022: 12 · 2021: 11 · 2020: 7 · 2019: 2
FWCI
6.48
Type
conference paper
Language
en
Open access
closed
Locations
International Conference on Mobile Systems, Applications, and Services (publishedVersion)
Funders
  • ECS grant (CityU 21203516)
  • NSFC Grant (No.61802261 and No.61872248)
  • Guangdong NSF (2017A030312008)
  • Fok Ying-Tong Education Foundation for Young Teachers in the Higher Education Institutions of China (No.161064)
  • NSF Grant of Shenzhen University (No.2018061)
  • GRF grant (CityU 11217817)
Topics
  • Physical Sciences › Computer Science › Human-Computer Interaction › Hand Gesture Recognition Systems
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
  • Physical Sciences › Computer Science › Human-Computer Interaction › Interactive and Immersive Displays
Keywords
wearable sensors, missing sensor data, wrist-worn devices, real-time tracking

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Real-time Arm Skeleton Tracking and Gesture Inference Tolerant to Missing Wearable Sensors. https://app.cochat.ai/papers/10.1145/3307334.3326109
The paper
Liu, Y., Li, Z., Liu, Z., & Wu, K. (2019). Real-time Arm Skeleton Tracking and Gesture Inference Tolerant to Missing Wearable Sensors. Proceedings of the 17th Annual International Conference on Mobile Systems, Applications, and Services. https://doi.org/10.1145/3307334.3326109

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.