CoChat

Real-time Arm Skeleton Tracking and Gesture Inference Tolerant to Missing Wearable Sensors

Liu, Yang, Li, Zhenjiang, Liu, Zhidan, Wu, Kaishun · 2019 · Proceedings of the 17th Annual International Conference on Mobile Systems, Applications, and Services

The DOI matches its Crossref record; 57 of the 60 references it cites were found in a registry or an index.

Abstract

This paper presents ArmTroi, a wearable system for understanding and analyzing the detailed arm motions of people primarily by using the motion sensors from wrist-worn wearable devices. ArmTroi can achieve real-time 3D arm skeleton tracking and reliable gesture inference tolerant to missing wearable sensors for enabling numerous useful application designs. We have coped with two major challenges through ArmTroi. First, the skeleton of each arm is determined from the locations of the elbow and wrist, whereas a wearable device only senses a single point from the wrist. We find that the potential solution space is huge. This underconstrained nature fundamentally challenges the achievement of accurate and real-time arm skeleton tracking. Second, wearable sensors may not reliably provide sensory data. For example, devices are not worn by the user, yet the learning tools for gesture inference, such as deep learning, typically have static network structures, which require nontrivial network adaptation to match the input's varying availability and ensure reliable gesture inference. We propose effective techniques to address above challenges, and all computations can be conducted on the user's smartphone. ArmTroi is thus a fully lightweight and portable system. We develop a prototype and extensive evaluation shows the efficacy of the ArmTroi design.

Overview

Integrity signals

Registry

Crossref · 10.1145/3307334.3326109

Cites

57 of 57 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Research works printed but not linked: 9

  1. Bahdanau Dzmitry, 2015, Proc. of ICLR .
  2. Du Yong, 2015, Proc. of IEEE CVPR .
  3. Hammerla Nils Yannick, 2015, Proc. of AAAI .
  4. Ngiam Jiquan, 2011, Proc. of ICML .
  5. 10.5555/1460257.1460283 · doi:10.5555/1460257.1460283
  6. Nancy Berryman Reese and William D Bandy. 2016. Joint Range of Motion and Muscle Length Testing-E-Book .Elsevier Health Sciences. Nancy Berryman Reese and William D Bandy. 2016. Joint Range of Motion and Muscle Length Testing-E-Book .Elsevier Health Sciences.
  7. Shan Chew Zhen, 2015, Proc. of IEEE ASCC .
  8. Sutskever Ilya, 2014, Proc. of NIPS .
  9. Xu Kelvin, 2015, Proc. of ICML .

Cited by

See all 71

Metadata

Authors
Liu, Y., Li, Z., Liu, Z., & Wu, K.
Affiliations
  • City University of Hong Kong (HK)
  • Shenzhen University (CN)
Venue
Proceedings of the 17th Annual International Conference on Mobile Systems, Applications, and Services
Publisher
ACM
Published
2019-06-12
Pages
287-299
OpenAlex
W2953033606
Cited by
71
References
60
Citations by year
2026: 4 · 2025: 8 · 2024: 12 · 2023: 15 · 2022: 12 · 2021: 11 · 2020: 7 · 2019: 2
FWCI
6.48
Type
conference paper
Language
en
Open access
closed
Locations
International Conference on Mobile Systems, Applications, and Services (publishedVersion)
Funders
  • ECS grant (CityU 21203516)
  • NSFC Grant (No.61802261 and No.61872248)
  • Guangdong NSF (2017A030312008)
  • Fok Ying-Tong Education Foundation for Young Teachers in the Higher Education Institutions of China (No.161064)
  • NSF Grant of Shenzhen University (No.2018061)
  • GRF grant (CityU 11217817)
Topics
  • Physical Sciences › Computer Science › Human-Computer Interaction › Hand Gesture Recognition Systems
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
  • Physical Sciences › Computer Science › Human-Computer Interaction › Interactive and Immersive Displays
Keywords
wearable sensors, missing sensor data, wrist-worn devices, real-time tracking

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.