CoChat

The DOI matches its Crossref record; 81 of the 81 references it cites were found in a registry or an index; 11 of the 32 sources its claims cite were checked for support.

Towards Learning Discrete Representations via Self-Supervision for Wearables-Based Human Activity Recognition

Open access

Abstract

Human activity recognition (HAR) in wearable and ubiquitous computing typically involves translating sensor readings into feature representations, either derived through dedicated pre-processing procedures or integrated into end-to-end learning approaches. Independent of their origin, for the vast majority of contemporary HAR methods and applications, those feature representations are typically continuous in nature. That has not always been the case. In the early days of HAR, discretization approaches had been explored—primarily motivated by the desire to minimize computational requirements on HAR, but also with a view on applications beyond mere activity classification, such as, for example, activity discovery, fingerprinting, or large-scale search. Those traditional discretization approaches, however, suffer from substantial loss in precision and resolution in the resulting data representations with detrimental effects on downstream analysis tasks. Times have changed, and in this paper, we propose a return to discretized representations. We adopt and apply recent advancements in vector quantization (VQ) to wearables applications, which enables us to directly learn a mapping between short spans of sensor data and a codebook of vectors, where the index comprises the discrete representation, resulting in recognition performance that is at least on par with their contemporary, continuous counterparts—often surpassing them. Therefore, this work presents a proof of concept for demonstrating how effective discrete representations can be derived, enabling applications beyond mere activity classification but also opening up the field to advanced tools for the analysis of symbolic sequences, as they are known, for example, from domains such as natural language processing. Based on an extensive experimental evaluation of a suite of wearable-based benchmark HAR tasks, we demonstrate the potential of our learned discretization scheme and discuss how discretized sensor data analysis can lead to substantial changes in HAR.

Overview

CoChat Summary

TL;DR

The paper proposes learning discrete representations for wearable-based human activity recognition by adding vector quantization to an Enhanced CPC self-supervised framework, then classifying the resulting symbolic sequences. Across six benchmarks, the learned discrete representations perform comparably to or better than continuous self-supervised and end-to-end baselines on several tasks, especially locomotion and daily-living activities.

Key findings

  • VQ-CPC discrete representations performed best or similar to the best on three benchmark datasets: Mobiact reached a mean of 77.8%, and HHAR improved over Enhanced CPC by around 1.5%.
  • Discretization was detrimental for fine-grained gym activities: on Myogym, performance dropped significantly compared with other self-supervised methods, and on MHEALTH it dropped by around 4.8% relative to Enhanced CPC.
  • Learned VQ-CPC outperformed traditional SAX and SAX-REPEAT discretization; SAX showed reductions of over 10% for HHAR, Myogym, and Motionsense.
  • Adding RoBERTa pre-training on the discrete symbols improved all target datasets: small RoBERTa increased HHAR by 2%, Myogym by 2.8%, Mobiact by 1.6%, Motionsense by 2.5%, and leg-based datasets by around 2.5%; medium RoBERTa increased HHAR by 3% and Myogym by 3.5%.

Methods

  • Pre-training used the Capture-24 dataset: around 2500 h from 151 participants, single wrist-worn accelerometer, downsampled to 50 Hz; evaluation used six target datasets across wrist, waist, and leg locations, with two datasets per location.
  • The framework combines Enhanced CPC self-supervision with online K-means vector quantization; base setup used G=2 groups and V=100 codebook variables, with a GRU/LSTM classifier and MLP backend, evaluated via five-fold cross-validation and five randomized runs.
  • Additional experiments varied learned alphabet size, encoder output frequency, base self-supervised method, and RoBERTa small/medium pre-training on discrete sequences.

Limitations

  • The paper states this is a proof of concept and that discretization is detrimental for fine-grained movements, as seen on Myogym; it also notes that the medium RoBERTa model did not improve over the small one, likely because Capture-24 is not large enough to leverage the bigger architecture.
  • The paper states that lower encoder output frequency substantially reduces performance, with drops around 10% for HHAR, over 15% for Myogym/MHEALTH/PAMAP2, and over 20% for waist-based datasets.
  • Evident from the text: pre-training and evaluation use a single accelerometer, and evaluation covers two datasets per sensor location rather than the three used in the referenced assessment framework.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.3390/s24041238

Cites

81 of 81 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Figure A1 Using RoBERTa embeddings for classifying discrete representations: first, we pre-train a RoBERTa model on discrete representations (indexed) from Capture-24. Subsequently, we replace the learnable embeddings with frozen RoBERTa embeddings in order to study the impact of the NLP-based pre-training. A GRU network is used along with an MLP for activity recognition.
Figure A1 Using RoBERTa embeddings for classifying discrete representations: first, we pre-train a RoBERTa model on discrete representations (indexed) from Capture-24. Subsequently, we replace the learnable embeddings with frozen RoBERTa embeddings in order to study the impact of the NLP-based pre-training. A GRU network is used along with an MLP for activity recognition.CC BY · PubMed Central
Figure A2 Visualizing the histograms of discrete representations per class for the first fold of Motionsense. This is the full figure from which Figure 5 is obtained by truncating the x-axis to 50 symbols.
Figure A2 Visualizing the histograms of discrete representations per class for the first fold of Motionsense. This is the full figure from which Figure 5 is obtained by truncating the x-axis to 50 symbols.CC BY · PubMed Central
Figure A3 Visualizing how the weights of the VQ-CPC architecture are updated using different parts of the overall loss. As detailed in Section 3.1.1, the Aggregator network is updated using (∑)k=1^KLk^CPC whereas the codebook vectors utilize ∥sg(z)−z^(∥)². Finally, the encoder is updated using both (∑)k=1^KLk^CPC and γ∥z−sg(z^)(∥)².
Figure A3 Visualizing how the weights of the VQ-CPC architecture are updated using different parts of the overall loss. As detailed in Section 3.1.1, the Aggregator network is updated using (∑)k=1^KLk^CPC whereas the codebook vectors utilize ∥sg(z)−z^(∥)². Finally, the encoder is updated using both (∑)k=1^KLk^CPC and γ∥z−sg(z^)(∥)².CC BY · PubMed Central
Figure 1 Discrete representation learning: given a window of accelerometer data as input, the output is a collection of enumerated symbols. We map short spans of continuous-valued sensor data to a list of ‘symbols’ (i.e., the numbers in the figure). Therefore, we obtain the “strings of motion”, as the symbols are discrete.
Figure 1 Discrete representation learning: given a window of accelerometer data as input, the output is a collection of enumerated symbols. We map short spans of continuous-valued sensor data to a list of ‘symbols’ (i.e., the numbers in the figure). Therefore, we obtain the “strings of motion”, as the symbols are discrete.CC BY · PubMed Central
Figure 2 Overview of discrete representation learning for HAR using wearables: we combine a contrastive future timestep prediction problem with vector quantization to map spans of sensor data to a codebook of vectors. The index of the codebook vector closest to each zt functions as the discrete representation.
Figure 2 Overview of discrete representation learning for HAR using wearables: we combine a contrastive future timestep prediction problem with vector quantization to map spans of sensor data to a codebook of vectors. The index of the codebook vector closest to each zt functions as the discrete representation.CC BY · PubMed Central
Figure 3 Visualizing K-means-based quantization: for each z-vector, the l2 distance is computed to the codebook of vectors ei. The index of the nearest codebook vector comprises the discrete representation, whereas the nearest vector itself is passed as the output (i.e., z^-vector). This figure has been adopted and adapted from [27].
Figure 3 Visualizing K-means-based quantization: for each z-vector, the l2 distance is computed to the codebook of vectors ei. The index of the nearest codebook vector comprises the discrete representation, whereas the nearest vector itself is passed as the output (i.e., z^-vector). This figure has been adopted and adapted from [27].CC BY · PubMed Central
Figure 4 Performing classification using the discrete representations: the sequences of symbolic representations (indexed) are first passed through a learnable embedding layer. Subsequently, an RNN network (GRU or LSTM) is utilized along with an MLP network for classifying the sequences into the activities of interest.
Figure 4 Performing classification using the discrete representations: the sequences of symbolic representations (indexed) are first passed through a learnable embedding layer. Subsequently, an RNN network (GRU or LSTM) is utilized along with an MLP network for classifying the sequences into the activities of interest.CC BY · PubMed Central
Figure 5 Plotting the histograms per class of the derived discrete representations for the train set from the first fold of Motionsense: The y-axis corresponds to the fraction of the number of occurrences for the symbols computed against the available data. The x-axis comprises the discrete representations, which are numbered. We observe that standing and sitting are covered by only a few symbols, which can be reasoned by the lack of movements in these classes. Walking up and down the stairs is also similar, yet hard to distinguish via visual examination from both walking and running. For clarity, we truncate the y-axis to 0.6 and x-axis to 50 symbols in order to show the plot in more detail. The full figure is available in the Appendix (refer to Figure A2).
Figure 5 Plotting the histograms per class of the derived discrete representations for the train set from the first fold of Motionsense: The y-axis corresponds to the fraction of the number of occurrences for the symbols computed against the available data. The x-axis comprises the discrete representations, which are numbered. We observe that standing and sitting are covered by only a few symbols, which can be reasoned by the lack of movements in these classes. Walking up and down the stairs is also similar, yet hard to distinguish via visual examination from both walking and running. For clarity, we truncate the y-axis to 0.6 and x-axis to 50 symbols in order to show the plot in more detail. The full figure is available in the Appendix (refer to Figure A2).CC BY · PubMed Central

Verification

Claims the paper makes

9 Valid · 0 Not valid · 13 Not checked · 97 Own results and interpretations

51 more claims

Create a free account to see all 119 claims and to verify the 21 references still open.

See all claims — free account
  • Probably not a substantial claim. 26
  • It may state two separate claims as one. 45
  • The quote only partly states this claim. 1

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

1 of 1

Citations

Cites

Show all 77 references

Research works printed but not linked: 22

  1. Bock, M., Kuehne, H., Van Laerhoven, K., and Moeller, M. (2023). WEAR: An Outdoor Sports for Wearable and Egocentric Activity Recognition. arXiv.
  2. Baevski, A., Schneider, S., and Auli, M. (2019). vq-wav2vec: Self-supervised learning of discrete speech representations. arXiv.
  3. Van Den Oord, A., and Vinyals, O. (2017). Advances in Neural Information Processing Systems, MIT Press.
  4. Chan Chang, S., and Doherty, A. (2021). Capture-24: Activity Tracker Dataset for Human Activity Recognition, University of Oxford.
  5. Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov, V. (2019). Roberta: A robustly optimized bert pretraining approach. arXiv.
  6. Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv.
  7. Shieh, J., and Keogh, E. (2008, January 24–27). i SAX: Indexing and mining terabyte sized time series. Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Las Legas, NV, USA.
  8. Ye, L., and Keogh, E. (July, January 28). Time series shapelets: A new primitive for data mining. Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Paris, France.
  9. Razavi, A., Van den Oord, A., and Vinyals, O. (2019). Advances in Neural Information Processing Systems, MIT Press.
  10. Jang, E., Gu, S., and Poole, B. (2016). Categorical reparameterization with gumbel-softmax. arXiv.
  11. Cuervo, S., ańcucki, A., Marxer, R., Rychlikowski, P., and Chorowski, J. (2022). Variable-rate hierarchical CPC leads to acoustic unit discovery in speech. arXiv.
  12. Dieleman, S., Nash, C., Engel, J., and Simonyan, K. (2021). Variable-rate discrete representation learning. arXiv.
  13. Kukačka, J., Golkov, V., and Cremers, D. (2017). Regularization for deep learning: A taxonomy. arXiv.
  14. Oord, A.v.d., Li, Y., and Vinyals, O. (2018). Representation learning with contrastive predictive coding. arXiv.
  15. Bengio, Y., Léonard, N., and Courville, A. (2013). Estimating or propagating gradients through stochastic neurons for conditional computation. arXiv.
  16. Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., and Antiga, L. (2019). Advances in Neural Information Processing Systems, MIT Press.
  17. Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv.
  18. Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., and Soricut, R. (2019). Albert: A lite bert for self-supervised learning of language representations. arXiv.
  19. Sanh, V., Debut, L., Chaumond, J., and Wolf, T. (2019). DistilBERT, a distilled version of BERT: Smaller, faster, cheaper and lighter. arXiv.
  20. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., and Polosukhin, I. (2017). Advances in Neural Information Processing Systems, MIT Press.
  21. Radford, 2019, Language models are unsupervised multitask learners, Openai Blog, 1, 9
  22. Gokhan, T., Smith, P., and Lee, M. (2021, January 15–16). Extractive financial narrative summarisation using sentencebert based clustering. Proceedings of the 3rd Financial Narrative Processing Workshop, Lancaster, UK.

Cited by

See all 18

Metadata

Authors
Haresamudram, H., Essa, I., & Plötz, T.
Affiliations
Georgia Institute of Technology (US)
Corresponding author
Haresamudram, Harish
Venue
Sensors
Publisher
MDPI AG
Published
2024-02-15
Volume
24
Issue
4
Pages
1238
ISSN
1424-8220 (ISSN-L)
OpenAlex
W4391846608
Cited by
14
References
77
Citations by year
2026: 4 · 2025: 5 · 2024: 4
FWCI
1.82
Type
journal article
Language
en
Open access
Locations
  • Sensors (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
  • arXiv (Cornell University)
Funders
NSF (IIS-2112633)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
  • Physical Sciences › Computer Science › Signal Processing › Time Series Analysis and Forecasting
  • Physical Sciences › Engineering › Biomedical Engineering › Gait Recognition and Analysis
Keywords
human activity recognition, wearable sensors, vector quantization, discrete representations, self-supervised learning, codebook learning, activity classification
MeSH terms
Machine Learning, Wearable Electronic Devices, Humans, Human Activities, Natural Language Processing

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-05, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-05 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Towards Learning Discrete Representations via Self-Supervision for Wearables-Based Human Activity Recognition. https://app.cochat.ai/papers/10.3390/s24041238
The paper
Haresamudram, H., Essa, I., & Plötz, T. (2024). Towards Learning Discrete Representations via Self-Supervision for Wearables-Based Human Activity Recognition. Sensors. https://doi.org/10.3390/s24041238

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.