CoChat

Feature Representation and Data Augmentation for Human Activity Classification Based on Wearable IMU Sensor Data Using a Deep LSTM Neural Network

Open access
Steven Eyobu, Odongo, Han, Dong Seog · 2018 · Sensors

The DOI matches its Crossref record; 50 of the 50 references it cites were found in a registry or an index.

Abstract

Wearable inertial measurement unit (IMU) sensors are powerful enablers for acquisition of motion data. Specifically, in human activity recognition (HAR), IMU sensor data collected from human motion are categorically combined to formulate datasets that can be used for learning human activities. However, successful learning of human activities from motion data involves the design and use of proper feature representations of IMU sensor data and suitable classifiers. Furthermore, the scarcity of labelled data is an impeding factor in the process of understanding the performance capabilities of data-driven learning models. To tackle these challenges, two primary contributions are in this article: first; by using raw IMU sensor data, a spectrogram-based feature extraction approach is proposed. Second, an ensemble of data augmentations in feature space is proposed to take care of the data scarcity problem. Performance tests were conducted on a deep long term short term memory (LSTM) neural network architecture to explore the influence of feature representations and the augmentations on activity recognition accuracy. The proposed feature extraction approach combined with the data augmentation ensemble produces state-of-the-art accuracy results in HAR. A performance evaluation of each augmentation approach is performed to show the influence on classification accuracy. Finally, in addition to using our own dataset, the proposed data augmentation technique is evaluated against the University of California, Irvine (UCI) public online HAR dataset and yields state-of-the-art accuracy results at various learning rates.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3390/s18092892

Cites

50 of 50 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 Human activity recognition system workflow.
Figure 1 Human activity recognition system workflow.CC BY · PubMed Central
Figure 2 Data collection architecture.
Figure 2 Data collection architecture.CC BY · PubMed Central
Figure 3 An example of 3D raw data for (a) sitting and (b) walking based on 1IMU sensor tied on the left-hand wrist.
Figure 3 An example of 3D raw data for (a) sitting and (b) walking based on 1IMU sensor tied on the left-hand wrist.CC BY · PubMed Central
Figure 4 (a) An example of time domain data and their spectrogram representing walking data; (b) Proposed feature extraction algorithm.
Figure 4 (a) An example of time domain data and their spectrogram representing walking data; (b) Proposed feature extraction algorithm.CC BY · PubMed Central
Figure 5 Data augmentation workflow.
Figure 5 Data augmentation workflow.CC BY · PubMed Central
Figure 6 Set 1: OR dataset.
Figure 6 Set 1: OR dataset.CC BY · PubMed Central
Figure 7 Set 1: Generating local averages.
Figure 7 Set 1: Generating local averages.CC BY · PubMed Central
Figure 8 Set 2: OR + LA1 dataset.
Figure 8 Set 2: OR + LA1 dataset.CC BY · PubMed Central
Figure 9 Randomly shuffled (OR + LA1 + SH) feature set 2.
Figure 9 Randomly shuffled (OR + LA1 + SH) feature set 2.CC BY · PubMed Central
Figure 10 Generating local averages of the shuffled feature set.
Figure 10 Generating local averages of the shuffled feature set.CC BY · PubMed Central
Figure 11 Set 3: OR + LA1 + SH + LA2 feature set with local averages.
Figure 11 Set 3: OR + LA1 + SH + LA2 feature set with local averages.CC BY · PubMed Central
Figure 12 Visualizing the variance of data to check the influence of each data augmentation block. (a,b,c) represent the variance of the unaugmented dataset, augmented dataset after the first local averaging and that of the augmented dataset after the first local averaging, shuffling and the second local averaging procedure respectively for the standing activity. (d,e,(f) represent the variance of the unaugmented dataset, augmented dataset after the first local averaging and that of the augmented dataset after the first local averaging, shuffling and the second local averaging procedure respectively for the sitting activity. (g,h,i) represent the variance of the unaugmented dataset, augmented dataset after the first local averaging and that of the augmented dataset after the first local averaging, shuffling and the second local averaging procedure respectively for the walking activity.
Figure 12 Visualizing the variance of data to check the influence of each data augmentation block. (a,b,c) represent the variance of the unaugmented dataset, augmented dataset after the first local averaging and that of the augmented dataset after the first local averaging, shuffling and the second local averaging procedure respectively for the standing activity. (d,e,(f) represent the variance of the unaugmented dataset, augmented dataset after the first local averaging and that of the augmented dataset after the first local averaging, shuffling and the second local averaging procedure respectively for the sitting activity. (g,h,i) represent the variance of the unaugmented dataset, augmented dataset after the first local averaging and that of the augmented dataset after the first local averaging, shuffling and the second local averaging procedure respectively for the walking activity.CC BY · PubMed Central
Figure 13 LSTM cell.
Figure 13 LSTM cell.CC BY · PubMed Central
Figure 14 (a) Accuracy versus learning rate based on only the OR dataset without augmentation, (b) precision versus learning rate based on only the OR dataset without augmentation, (c) recall versus learning rate based on only the OR dataset without augmentation and (d) f1_score versus learning rate based on only the OR dataset without augmentation.
Figure 14 (a) Accuracy versus learning rate based on only the OR dataset without augmentation, (b) precision versus learning rate based on only the OR dataset without augmentation, (c) recall versus learning rate based on only the OR dataset without augmentation and (d) f1_score versus learning rate based on only the OR dataset without augmentation.CC BY · PubMed Central
Figure 15 100-feature vector dataset: (a) Accuracy versus learning rate, (b) precision versus learning rate, (c) recall versus learning rate and (d) f1_score versus learning rate.
Figure 15 100-feature vector dataset: (a) Accuracy versus learning rate, (b) precision versus learning rate, (c) recall versus learning rate and (d) f1_score versus learning rate.CC BY · PubMed Central
Figure 16 200-feature vector dataset: (a) Accuracy versus learning rate, (b) precision versus learning rate, (c) recall versus learning rate and (d) f1_score versus learning rate.
Figure 16 200-feature vector dataset: (a) Accuracy versus learning rate, (b) precision versus learning rate, (c) recall versus learning rate and (d) f1_score versus learning rate.CC BY · PubMed Central
Figure 17 UCI’s 128-feature vector dataset: (a) Accuracy versus learning rate, (b) precision versus learning rate, (c) recall versus learning rate and (d) f1_score versus learning rate.
Figure 17 UCI’s 128-feature vector dataset: (a) Accuracy versus learning rate, (b) precision versus learning rate, (c) recall versus learning rate and (d) f1_score versus learning rate.CC BY · PubMed Central
Figure 18 Comparing the performance of OR and OR + LA1 using the UCI dataset and our dataset at various learning rates of: (a) 0.0002 (b) 0.003 (c) 0.006 and (d) 0.01.
Figure 18 Comparing the performance of OR and OR + LA1 using the UCI dataset and our dataset at various learning rates of: (a) 0.0002 (b) 0.003 (c) 0.006 and (d) 0.01.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 50 references

Research works printed but not linked: 5

  1. Eyobu, O.S., Kim, Y., and Han, D.S. (2017, January 21–23). Activity Recognition for Infant Health Care Based on Wireless Inertial Measurement Unit Motion Data. Proceedings of the Korean Institute of Communication Sciences Conference, Jeju, Korea.
  2. Ohashi, H., Al-Nasser, M., Ahmed, S., Akiyama, T., Sato, T., Nguyen, P., Nakamura, K., and Dengel, A. (2017, January 6–11). Augmenting Wearable Sensor Data with Physical Constraint for DNN-Based Human-Action Recognition. Presented at ICML 2017 Times Series Workshop, Sydney, Australia.
  3. Bächlin, M., Förster, K., and Tröster, G. (October, January 30). SwimMaster: A wearable assistant for swimmer. Proceedings of the 11th International Conference on Ubiquitous Computing, Orlando, FL, USA.
  4. Cui, Z., Chen, W., and Chen, Y. (arXiv, 2016). Multi-scale convolutional neural networks for time series classification, arXiv.
  5. DeVries, T., and Taylor, G.W. (arXiv, 2017). Dataset Augmentation in Feature Space, arXiv.

Cited by

See all 267

Metadata

Authors
Steven Eyobu, O., & Han, D. S.
Affiliations
  • School of Electronics Engineering, Kyungpook National University, 80 Daehak-ro, Buk-gu, Daegu 41566, Korea
  • Kyungpook National University (KR)
Corresponding author
Han, Dong Seog
Venue
Sensors
Publisher
MDPI AG
Published
2018-08-31
Volume
18
Issue
9
Pages
2892
ISSN
1424-8220 (ISSN-L)
OpenAlex
W2889505988
Cited by
259
References
50
Citations by year
2026: 15 · 2025: 27 · 2024: 36 · 2023: 45 · 2022: 32 · 2021: 55 · 2020: 31 · 2019: 17 · 2018: 1
FWCI
13.01
Type
journal article
Language
en
Open access
Locations
  • Sensors (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, other-oa)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
Institute for Information and communications Technology Promotion (R7124-16- 0004)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
  • Physical Sciences › Engineering › Biomedical Engineering › Gait Recognition and Analysis
  • Physical Sciences › Computer Science › Signal Processing › Time Series Analysis and Forecasting
Keywords
human activity recognition, inertial measurement unit, spectrogram feature extraction, data augmentation, feature representation, LSTM neural network, labeled data scarcity, UCI HAR dataset
MeSH terms
Wearable Electronic Devices, Deep Learning, Humans, Human Activities, Movement, Neural Networks, Computer

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Feature Representation and Data Augmentation for Human Activity Classification Based on Wearable IMU Sensor Data Using a Deep LSTM Neural Network. https://app.cochat.ai/papers/10.3390/s18092892
The paper
Steven Eyobu, O., & Han, D. S. (2018). Feature Representation and Data Augmentation for Human Activity Classification Based on Wearable IMU Sensor Data Using a Deep LSTM Neural Network. Sensors. https://doi.org/10.3390/s18092892

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.