CoChat

Time-Series Representation Learning via Temporal and Contextual Contrasting

Open access
Eldele, Emadeldeen, Ragab, Mohamed, Chen, Zhenghua, Wu, Min, Kwoh, Chee Keong, Li, Xiaoli, Guan, Cuntai · 2021 · Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence

The DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index; 9 of the 21 sources its claims cite were checked for support.

Abstract

Learning decent representations from unlabeled time-series data with temporal dynamics is a very challenging task. In this paper, we propose an unsupervised Time-Series representation learning framework via Temporal and Contextual Contrasting (TS-TCC), to learn time-series representation from unlabeled data. First, the raw time-series data are transformed into two different yet correlated views by using weak and strong augmentations. Second, we propose a novel temporal contrasting module to learn robust temporal representations by designing a tough cross-view prediction task. Last, to further learn discriminative representations, we propose a contextual contrasting module built upon the contexts from the temporal contrasting module. It attempts to maximize the similarity among different contexts of the same sample while minimizing similarity among contexts of different samples. Experiments have been carried out on three real-world time-series datasets. The results manifest that training a linear classifier on top of the features learned by our proposed TS-TCC performs comparably with the supervised training. Additionally, our proposed TS-TCC shows high efficiency in few-labeled data and transfer learning scenarios. The code is publicly available at https://github.com/emadeldeen24/TS-TCC.

Overview

CoChat Summary

TL;DR

TS-TCC is an unsupervised time-series representation learning framework that combines weak/strong augmentations, a cross-view temporal contrasting module, and a contextual contrasting module. It learns representations that perform comparably to supervised training with a linear classifier and are effective in few-label and transfer settings.

Key findings

  • On linear evaluation, TS-TCC achieved HAR ACC 90.37 ± 0.34 / MF1 90.38 ± 0.39, Sleep-EDF ACC 83.00 ± 0.71 / MF1 73.57 ± 0.74, and Epilepsy ACC 97.23 ± 0.10 / MF1 95.54 ± 0.08, outperforming SSL-ECG, CPC, and SimCLR and matching or exceeding supervised training on two of three datasets.
  • In semi-supervised settings, TS-TCC fine-tuning reached around 70% and 90% for HAR and Epilepsy with only 1% labeled data, and with 10% labeled data achieved performance comparable to supervised training with 100% labeled data on the three datasets.
  • In fault-diagnosis transfer learning, TS-TCC fine-tuning averaged 67.82% accuracy versus 63.83% for supervised pretraining, outperforming supervised in 8 of 12 cross-domain scenarios and improving by about 4% overall.
  • Ablation showed the cross-view prediction task improved performance by more than 5% on HAR and about 1% on Sleep-EDF and Epilepsy; adding contextual contrasting further improved results.

Methods

  • The framework creates two views per sample using weak augmentation (jitter-and-scale) and strong augmentation (permutation-and-jitter), encodes them with a 3-block convolutional encoder, and applies a Transformer-based autoregressive temporal contrasting module with a cross-view prediction task plus a contextual contrasting loss.
  • Experiments used UCI HAR, Sleep-EDF, Epilepsy, and a fault diagnosis dataset; data were split 60%/20%/20% for train/validation/test, with subject-wise split for Sleep-EDF, and experiments were repeated 5 times with 5 seeds.
  • Evaluation included linear classifier on frozen features, semi-supervised training with 1%, 5%, 10%, 50%, and 75% labeled data, cross-domain transfer learning, ablation, and sensitivity analysis; training used 40 epochs, Adam with lr 3e-4, batch size 128 (32 for few-label), and dataset-specific augmentation/Transformer hyperparameters.

Limitations

  • Not stated in the paper: the paper does not include an explicit limitations section.
  • Evident from the text: augmentation hyperparameters must be chosen carefully according to the nature of the time-series data, and the main evaluation covers three time-series datasets plus one fault-diagnosis transfer dataset.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.24963/ijcai.2021/324

Cites

28 of 28 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Figure 1 Overall architecture of proposed TS-TCC model.
Figure 1 Overall architecture of proposed TS-TCC model.From the PDF
Figure 2 Architecture of Transformer model used in Temporal Contrasting module. The token c in the output is sent next to the Contextual Contrasting module.
Figure 2 Architecture of Transformer model used in Temporal Contrasting module. The token c in the output is sent next to the Contextual Contrasting module.From the PDF
Figure 3 Comparison between supervised training vs. TS-TCC fine-tuning for different few-labeled data scenarios in terms of MF1.
Figure 3 Comparison between supervised training vs. TS-TCC fine-tuning for different few-labeled data scenarios in terms of MF1.From the PDF
Figure 4 Three sensitivity analysis experiments on HAR dataset.
Figure 4 Three sensitivity analysis experiments on HAR dataset.From the PDF
Image 5
From the PDF
Image 6
From the PDF

Verification

Claims the paper makes

8 Valid · 2 Not valid · 5 Not checked · 43 Own results and interpretations

32 more claims

Create a free account to see all 58 claims and every source verdict.

See all claims — free account
  • Probably not a substantial claim. 12
  • It may state two separate claims as one. 12

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

1 of 1

Citations

Cites

Show all 28 references

Research works printed but not linked: 1

  1. [Aggarwal et al. , 2019] Karan Aggarwal, Shafiq R. Joty, Luis Fern´ andez-Luque, and Jaideep Srivastava. Adversarial unsupervised representation learning for activity timeseries. AAAI , 2019.

Cited by

Items 301–400 of 640

Metadata

Authors
Eldele, E., Ragab, M., Chen, Z., Wu, M., Kwoh, C. K., Li, X., & Guan, C.
Affiliations
  • Nanyang Technological University (SG)
  • Agency for Science, Technology and Research (SG)
Venue
Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence
Published
2021-08
Pages
2352-2359
OpenAlex
W3190152617
Cited by
696
References
25
Citations by year
2026: 152 · 2025: 213 · 2024: 181 · 2023: 117 · 2022: 31 · 2021: 2
FWCI
50.23
Type
conference paper
Language
en
Open access
Licence
cc-by
Locations
  • International Joint Conference on Artificial Intelligence (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion)
  • DR-NTU (Nanyang Technological University) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Funders
Agency for Science, Technology and Research (A20H6b0151, C210112046)
Topics
  • Physical Sciences › Computer Science › Signal Processing › Time Series Analysis and Forecasting
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › Advanced Graph Neural Networks
Keywords
time series representation learning, unsupervised learning, contrastive learning, data augmentation, temporal dynamics, limited labeled data, transfer learning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.