CoChat

The DOI matches its Crossref record; 37 of the 38 references it cites were found in a registry or an index.

Translating Videos into Synthetic Training Data for Wearable Sensor-Based Activity Recognition Systems Using Residual Deep Convolutional Networks

Open access
Fortes Rey, Vitor, Garewal, Kamalveer Kaur, Lukowicz, Paul · 2021 · Applied Sciences

Abstract

Human activity recognition (HAR) using wearable sensors has benefited much less from recent advances in Deep Learning than fields such as computer vision and natural language processing. This is, to a large extent, due to the lack of large scale (as compared to computer vision) repositories of labeled training data for sensor-based HAR tasks. Thus, for example, ImageNet has images for around 100,000 categories (based on WordNet) with on average 1000 images per category (therefore up to 100,000,000 samples). The Kinetics-700 video activity data set has 650,000 video clips covering 700 different human activities (in total over 1800 h). By contrast, the total length of all sensor-based HAR data sets in the popular UCI machine learning repository is less than 63 h, with around 38 of those consisting of simple mode of locomotion activities like walking, standing or cycling. In our research we aim to facilitate the use of online videos, which exist in ample quantities for most activities and are much easier to label than sensor data, to simulate labeled wearable motion sensor data. In previous work we already demonstrated some preliminary results in this direction, focusing on very simple, activity specific simulation models and a single sensor modality (acceleration norm). In this paper, we show how we can train a regression model on generic motions for both accelerometer and gyro signals and then apply it to videos of the target activities to generate synthetic Inertial Measurement Units (IMU) data (acceleration and gyro norms) that can be used to train and/or improve HAR models. We demonstrate that systems trained on simulated data generated by our regression model can come to within around 10% of the mean F1 score of a system trained on real sensor data. Furthermore, we show that by either including a small amount of real sensor data for model calibration or simply leveraging the fact that (in general) we can easily generate much more simulated data from video than we can collect its real version, the advantage of the latter can eventually be equalized.

Overview

Integrity signals

Registry

Crossref · 10.3390/app11073094

Cites

37 of 37 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 38 references

Research works printed but not linked: 7

  1. Smaira, L., Carreira, J., Noland, E., Clancy, E., Wu, A., and Zisserman, A. (2020). A Short Note on the Kinetics-700-2020 Human Action Dataset. arXiv.
  2. Smith, M., Moore, T., Hill, C., Noakes, C., and Hide, C. (2003, January 22–23). Simulation of GNSS/IMU measurements. Proceedings of the ISPRS International Workshop. Working Group I/5: Theory, Technology and Realities of Inertial/GPS Sensor Orientation, Castelldefels, Spain.
  3. Parés, M., Rosales, J., and Colomina, I. (February, January 30). Yet another IMU simulator: Validation and applications. Proceedings of the Eurocow, Castelldefels, Spain.
  4. Murthy, P., Kovalenko, O., Elhayek, A., Gava, C.C., and Stricker, D. (2017). 3D Human Pose Tracking inside Car using Single RGB Spherical Camera. Proceedings of the ACM Chapters Computer Science in Cars Symposium (CSCS), ACM.
  5. Raissi, M., Perdikaris, P., and Karniadakis, G.E. (2017). Physics informed deep learning (part i): Data-driven solutions of nonlinear partial differential equations. arXiv.
  6. Bai, S., Kolter, J.Z., and Koltun, V. (2018). An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv.
  7. Kingma, D.P., and Ba, J. (2014). Adam: A Method for Stochastic Optimization. arXiv.

Cited by

See all 18

Metadata

Authors
Fortes Rey, V., Garewal, K. K., & Lukowicz, P.
Affiliations
  • Embedded Intelligence, German Research Center for Artificial Intelligence (DFKI), 67663 Kaiserslautern, Germany
  • German Research Centre for Artificial Intelligence (DE)
  • University of Kaiserslautern (DE)
Corresponding author
  • Garewal, Kamalveer Kaur
  • Lukowicz, Paul
Venue
Applied Sciences
Publisher
MDPI AG
Published
2021-03-31
Volume
11
Issue
7
Pages
3094
ISSN
2076-3417 (ISSN-L)
OpenAlex
W3146293857
Cited by
17
References
38
Citations by year
2025: 5 · 2024: 1 · 2023: 5 · 2022: 4 · 2021: 2
FWCI
1.08
Type
journal article
Language
en
Open access
Locations
  • Applied Sciences (publishedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
Bundesministerium für Bildung und Forschung (01IW18001)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
  • Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Physical Activity and Health
  • Physical Sciences › Computer Science › Human-Computer Interaction › Hand Gesture Recognition Systems
Keywords
human activity recognition, wearable sensors, synthetic training data, inertial measurement unit, data augmentation, model calibration

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Translating Videos into Synthetic Training Data for Wearable Sensor-Based Activity Recognition Systems Using Residual Deep Convolutional Networks. https://app.cochat.ai/papers/10.3390/app11073094
The paper
Fortes Rey, V., Garewal, K. K., & Lukowicz, P. (2021). Translating Videos into Synthetic Training Data for Wearable Sensor-Based Activity Recognition Systems Using Residual Deep Convolutional Networks. Applied Sciences. https://doi.org/10.3390/app11073094

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.