CoChat

Contrastive Predictive Coding for Human Activity Recognition

Open access
Haresamudram, Harish, Essa, Irfan, Plötz, Thomas · 2021 · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies

The DOI matches its Crossref record; 117 of the 117 references it cites were found in a registry or an index; 20 of the 41 sources its claims cite were checked for support.

Abstract

Feature extraction is crucial for human activity recognition (HAR) using body-worn movement sensors. Recently, learned representations have been used successfully, offering promising alternatives to manually engineered features. Our work focuses on effective use of small amounts of labeled data and the opportunistic exploitation of unlabeled data that are straightforward to collect in mobile and ubiquitous computing scenarios. We hypothesize and demonstrate that explicitly considering the temporality of sensor data at representation level plays an important role for effective HAR in challenging scenarios. We introduce the Contrastive Predictive Coding (CPC) framework to human activity recognition, which captures the temporal structure of sensor data streams. Through a range of experimental evaluations on real-life recognition tasks, we demonstrate its effectiveness for improved HAR. CPC-based pre-training is self-supervised, and the resulting learned representations can be integrated into standard activity chains. It leads to significantly improved recognition performance when only small amounts of labeled training data are available, thereby demonstrating the practical value of our approach. Through a series of experiments, we also develop guidelines to help practitioners adapt and modify the framework towards other mobile and ubiquitous computing scenarios.

Overview

CoChat Summary

TL;DR

The paper introduces Contrastive Predictive Coding (CPC) as self-supervised pre-training for wearable human activity recognition, learning representations by predicting multiple future timesteps in latent space. Across four benchmark datasets, CPC-based frozen features outperform prior unsupervised methods on three datasets and improve recognition when only small amounts of labeled data are available.

Key findings

  • CPC unsupervised mean F1-scores were Mobiact 80.97, Motionsense 89.05, UCI-HAR 81.65, and USC-HAD 52.01; it outperformed all state-of-the-art unsupervised baselines on Mobiact, Motionsense, and USC-HAD, and was comparable to masked reconstruction on UCI-HAR (81.89).
  • Relative to supervised DeepConvLSTM, CPC improved performance by approximately 4% on Motionsense and over 7% on USC-HAD; on Mobiact and UCI-HAR, CPC scored below DeepConvLSTM in Table 2 (80.97 vs 82.40; 81.65 vs 82.83).
  • With limited labeled data, CPC improved over DeepConvLSTM on all datasets: Mobiact was consistently higher by over 5% and over 15% above random-initialized features when more than ten samples per class were available; Motionsense reached a maximum improvement of approximately 10% at 50 samples; UCI-HAR peaked at over 10% at ten samples; USC-HAD improved even with one sample per class.
  • For pre-training choices, k=12 future steps was best for Mobiact, Motionsense, and UCI-HAR (about 400 ms), while k=8 was best for USC-HAD (about 270 ms); kernel size 9 was around 13% lower than kernel size 3, recurrent encoders were around 42% lower, and using only the first two encoder layers gave Mobiact 85.22% and UCI-HAR 82.58%.

Methods

  • Self-supervised CPC pre-training on unlabeled sliding windows: an encoder \(g_{enc}\) maps sensor windows to latent representations, a GRU autoregressive network \(g_{ar}\) produces context representations, and linear predictors \(W_k\) predict k future timesteps using the InfoNCE/NCE loss. Frozen learned weights are used as a feature extractor with a three-layer MLP classifier (256, 128, softmax; batch normalization, ReLU, dropout p=0.2).
  • Four benchmark datasets were used: Mobiact (61 users, 11 activities), Motionsense (24 users, 6 activities), UCI-HAR (30 users, 6 activities), and USC-HAD (14 users, 12 activities). Raw accelerometer and gyroscope data were downsampled to 30 Hz, normalized, and segmented into 1 s windows with 50% overlap; splits were participant-based, with 20% test and 20% of remaining data for validation, except USC-HAD used participants 1–10 for training, 11–12 for validation, and 13–14 for testing.
  • Pre-training ran for 150 epochs with Adam, learning rates tuned over {1e-3, 5e-4}, and k over {2, 4, 8, 12, 16}; the classifier was trained for 150 epochs with cross-entropy, learning rates over {5e-4, 1e-4} decayed by 0.8 every 25 epochs. The primary metric was test set mean F1-score.

Limitations

  • The paper does not state a dedicated limitations section. Evident from the text: evaluation is limited to four benchmark datasets and primarily locomotion-style activities, and the authors note open challenges for sensor placements different from the source dataset and domain shift when using virtual movement data.
  • Evident from the text: CPC pre-training uses the entire unlabeled dataset for each benchmark, and the paper does not report cross-dataset transfer or deployment on new sensor placements; it frames these as future research agenda.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1145/3463506

Cites

117 of 117 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

14 Valid · 7 Not valid · 7 Not checked · 84 Own results and interpretations

77 more claims · 2 with their cited sources checked

Create a free account to see all 112 claims and to verify the 4 references still open.

See all claims — free account
  • Probably not a substantial claim. 21
  • It may state two separate claims as one. 15

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

12 more claims about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

1 of 1

Citations

Cites

Show all 117 references

Research works printed but not linked: 4

  1. 10.5555/1460232.1460247 · doi:10.5555/1460232.1460247
  2. William Lotter , Gabriel Kreiman , and David Cox . 2016. Deep predictive coding networks for video prediction and unsupervised learning. arXiv preprint arXiv:1605.08104 ( 2016 ). William Lotter, Gabriel Kreiman, and David Cox. 2016. Deep predictive coding networks for video prediction and unsupervised learning. arXiv preprint arXiv:1605.08104 (2016). · arXiv
  3. Aaqib Saeed , Victor Ungureanu , and Beat Gfeller . 2020. Sense and Learn: Self-Supervision for Omnipresent Sensors. arXiv preprint arXiv:2009.13233 ( 2020 ). Aaqib Saeed, Victor Ungureanu, and Beat Gfeller. 2020. Sense and Learn: Self-Supervision for Omnipresent Sensors. arXiv preprint arXiv:2009.13233 (2020). · arXiv
  4. Neo Wu , Bradley Green , Xue Ben , and Shawn O'Banion . 2020. Deep transformer models for time series forecasting: The influenza prevalence case. arXiv preprint arXiv:2001.08317 ( 2020 ). Neo Wu, Bradley Green, Xue Ben, and Shawn O'Banion. 2020. Deep transformer models for time series forecasting: The influenza prevalence case. arXiv preprint arXiv:2001.08317 (2020). · arXiv

Cited by

Items 101–152 of 152

Metadata

Authors
Haresamudram, H., Essa, I., & Plötz, T.
Affiliations
Georgia Institute of Technology (US)
Venue
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
Published
2021-06-23
Volume
5
Issue
2
Pages
1-26
ISSN
2474-9567 (ISSN-L)
OpenAlex
W3111841649
Cited by
150
References
99
Citations by year
2026: 15 · 2025: 42 · 2024: 39 · 2023: 27 · 2022: 21 · 2021: 6
FWCI
8.35
Type
journal article
Language
en
Open access
Locations
  • Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies (publishedVersion, public-domain)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Social Sciences › Transportation › Human Mobility and Location-Based Analysis
Keywords
human activity recognition, contrastive predictive coding, self-supervised learning, wearable sensors, motion sensors, learned representations, pre-training, limited labeled data, wearable computing

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.