CoChat

Multi-task Self-Supervised Learning for Human Activity Detection

Open access
Saeed, Aaqib, Ozcelebi, Tanir, Lukkien, Johan · 2019 · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies

The DOI matches its Crossref record; 85 of the 85 references it cites were found in a registry or an index.

Abstract

Deep learning methods are successfully used in applications pertaining to ubiquitous computing, pervasive intelligence, health, and well-being. Specifically, the area of human activity recognition (HAR) is primarily transformed by the convolutional and recurrent neural networks, thanks to their ability to learn semantic representations directly from raw input. However, in order to extract generalizable features massive amounts of well-curated data are required, which is a notoriously challenging task; hindered by privacy issues and annotation costs. Therefore, unsupervised representation learning (i.e., learning without manually labeling the instances) is of prime importance to leverage the vast amount of unlabeled data produced by smart devices. In this work, we propose a novel self-supervised technique for feature learning from sensory data that does not require access to any form of semantic labels, i.e., activity classes. We learn a multi-task temporal convolutional network to recognize transformations applied on an input signal. By exploiting these transformations, we demonstrate that simple auxiliary tasks of the binary classification result in a strong supervisory signal for extracting useful features for the down-stream task. We extensively evaluate the proposed approach on several publicly available datasets for smartphone-based HAR in unsupervised, semi-supervised and transfer learning settings. Our method achieves performance levels superior to or comparable with fully-supervised networks trained directly with activity labels, and it performs significantly better than unsupervised learning through autoencoders. Notably, for the semi-supervised case, the self-supervised features substantially boost the detection rate by attaining a kappa score between 0.7 - 0.8 with only 10 labeled examples per class. We get similar impressive performance even if the features are transferred from a different data source. Self-supervision drastically reduces the requirement of labeled activity data, effectively narrowing the gap between supervised and unsupervised techniques for learning meaningful representations. While this paper focuses on HAR as the application domain, the proposed approach is general and could be applied to a wide variety of problems in other areas.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3328932

Cites

85 of 85 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

19 more claims about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

  • ImageNet classification with deep convolutional neural networks

    Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 1×

    “These techniques have achieved indisputable empirical success across a broad spectrum of problems [4, 21, 28, 36, 43, 56, 65, 66] thanks to the increasing dataset sizes and computing power availability.”
    Background
  • Adam: A Method for Stochastic Optimization

    Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 3×

    “We use ReLU as non-linearity in all the layers (except the output) and train a network with Adam optimizer [26] for a maximum of 30 epochs with a learning rate of 0.”
    MethodInfluential
  • Deep learning

    LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126

  • Untitled work

    (2008) · Journal of Machine Learning Research · Cited by 33,051

  • A survey on transfer learning

    Pan, S. J., & Yang, Q. (2010) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2009.191 · Cited by 23,387 · cited 1×

    “owledge transfer se−ings (such as features, instances, weights) from existing labeled data of some related task or domain. For a detailed review of transfer learning, we refer an interested reader to [53]. 2.4 Semi-supervised Learning Semi-supervised learning provides a compelling framework for leveraging unlabeled da…”
    BackgroundInfluential
  • Sequence to Sequence Learning with Neural Networks

    Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766 · cited 1×

    “These techniques have achieved indisputable empirical success across a broad spectrum of problems [4, 21, 28, 36, 43, 56, 65, 66] thanks to the increasing dataset sizes and computing power availability.”
    Background
  • Untitled work

    (2010) · International Conference on Machine Learning · Cited by 13,995

  • Representation Learning: A Review and New Perspectives

    Bengio, Y., Courville, A., & Vincent, P. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.50 · Cited by 13,001 · cited 3×

    “Œis inherent limitation of the fully-supervised learning paradigm emphasizes the importance of unsupervised learning to leverage a large amount of unlabeled data for representation learning [8] that can be easily acquired in a real-world seŠing.”
    BackgroundInfluential
  • Multitask Learning

    Caruana, R. (1997) · Machine Learning · doi:10.1023/a:1007379606734 · Cited by 6,636 · cited 2×

    “Œe goal of multi-task learning (MTL) is to enhance the learning eciency and accuracy through simultaneously optimizing multiple objectives based on shared representations and exploiting relations among the tasks [10].”
    Background
  • DeepFace: Closing the Gap to Human-Level Performance in Face Verification

    Taigman, Y., Yang, M., Ranzato, M., & Wolf, L. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.220 · Cited by 6,542 · cited 1×

    “Œese techniques have achieved indisputable empirical success across a broad spectrum of problems [4, 21, 28, 35, 43, 56, 65, 66] thanks to the increasing dataset sizes and computing power availability.”
    Background

Show all 85 references

Research works printed but not linked: 7

  1. Baldi Pierre, 2012, Proceedings of ICML workshop on unsupervised and transfer learning. 37--49
  2. Chatzaki Charikleia, 2016, International Conference on Information and Communication Technologies for Ageing Well and e-Health. Springer, 100--118
  3. 10.5555/3104322.3104425 · doi:10.5555/3104322.3104425
  4. 10.5555/3305890.3305968 · doi:10.5555/3305890.3305968
  5. Plötz Thomas, 2011, IJCAI Proceedings-International Joint Conference on Artificial Intelligence, 22
  6. Razavian Narges, 2016, Machine Learning for Healthcare Conference. 73--100
  7. Wawrzyniak S., 2015 Federated Conference on Computer Science and Information Systems (FedCSIS). 411--416

Cited by

Items 101–200 of 345

Metadata

Authors
Saeed, A., Ozcelebi, T., & Lukkien, J.
Affiliations
  • Eindhoven University of Technology (NL)
  • Eindhoven University of Technology, Eindhoven, The Netherlands
Venue
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
Published
2019-06-21
Volume
3
Issue
2
Pages
1-30
ISSN
2474-9567 (ISSN-L)
OpenAlex
W3099025572
Cited by
346
References
85
Citations by year
2026: 20 · 2025: 65 · 2024: 82 · 2023: 71 · 2022: 49 · 2021: 42 · 2020: 17
FWCI
11.94
Type
journal article
Language
en
Open access
Locations
  • Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies (publishedVersion, public-domain)
  • TU/e Research Portal (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • TU/e Research Portal (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Funders
Horizon 2020 Framework Programme (737422)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Physical Sciences › Computer Science › Signal Processing › Time Series Analysis and Forecasting
Keywords
human activity recognition, self-supervised learning, unsupervised feature learning, multi-task learning, temporal convolutional network, smartphone sensing, semi-supervised learning, transfer learning, unlabeled data

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.