CoChat

Siamese masked reconstruction with temporal rotation consistency for inertial-based human activity recognition

Open access
Calatrava-Nicolás, Francisco M., Stoyanov, Todor, Martinez Mozos, Oscar · 2026 · Scientific Reports

The DOI matches its Crossref record; 37 of the 37 references it cites were found in a registry or an index; 8 of the 21 sources its claims cite were checked for support.

Abstract

Inertial-based Human Activity Recognition (HAR) aims at inferring activities performed by an individual from data on wearable inertial sensors. However, HAR often suffers from poor generalization to unseen users due to data heterogeneity across different individuals and conditions. While collecting and curating large labeled datasets could help alleviate this limitation, it is also costly. To reduce this gap without relying on extensive annotated data, we propose a self-supervised framework grounded in how IMU signals transform under rotation: applying a rotation matrix changes how motion projects onto the sensor axes without altering the underlying motion. A model trained to align representations of a signal and its rotated counterpart is therefore encouraged to capture motion content rather than sensor-frame patterns. We exploit this property by training a Siamese masked convolutional autoencoder that learns by reconstructing masked inputs while aligning representations of original signals and its rotated version, further regularized by a temporal consistency loss that enforces agreement between the temporal structure of original and rotated representations within each window. We evaluate on four public HAR benchmarks covering diverse scenarios, including activities of the daily living (ADL) and sports activities, using cross-subject evaluation, where our method yields improvements of +1.4 and +1.2 percentage points over the strongest baseline with linear and MLP probes respectively, averaged across datasets and sensor positions. We further show that our method obtains competitive results against the fully supervised baseline in low-data regimes. The code is available on https://github.com/FranciscoCalatrava/Rotation_Siamese_Masked.git .

Overview

CoChat Summary

## TL;DR

The paper proposes a self-supervised framework for inertial HAR that combines masked reconstruction with rotation-based Siamese consistency and a temporal Gram constraint, using rotations as structured supervision rather than just augmentation. Across four public datasets, it generally outperforms baselines and reports average gains of +1.4 percentage points with a linear head and +1.2 with an MLP head in macro F1-score.

## Key findings

  • Across 24 dataset–placement–head configurations, the method achieved the best average Macro F1-score in 19 cases and the second-best in the remaining 4.
  • Averaged across datasets and body positions, the proposed method improved over the strongest competitor by +1.4 percentage points with a linear classifier and +1.2 percentage points with an MLP classifier (macro F1-score).
  • In the low-data study, it remained competitive with a frozen encoder and generally surpassed the supervised baseline when fully fine-tuned; the paper reports clearer gains in the lowest-data regimes of 10 and 20 samples per class.
  • The ablation study found that \(L_{rec}+L_{gram}\) ranked first or second in 14 of 24 configurations, while the full objective \(L_{rec}+L_{siam}+L_{gram}\) was first or second in 21 of 24 configurations.

## Methods

  • Two-stage design: self-supervised pretraining on unlabeled inertial windows, followed by downstream HAR evaluation using a frozen encoder and a lightweight classifier.
  • Data/sample: four public datasets (PAMAP2, MHEALTH, REALDISP, WEAR), using subject-wise cross-validation; input windows were 128 samples with 50% overlap (2.56 s per window), and the main experiments used single-sensor wrist/ankle placements.
  • Main analysis: a self-supervised objective combining masked reconstruction, a Siamese-style consistency loss between original and rotated views, and a temporal Gram-matrix loss; evaluation used Macro F1-score with Linear and MLP probes.

## Limitations

  • The paper states that the framework “does not yet fully capture all the information that rotations can provide” and does not explicitly model orientation-aware relationships or the geometric structure induced by successive rotations.
  • It notes that this is especially important because real-world sensor orientation varies across users, sessions, wearing conditions, and placements.
  • Evident from the text: the study is limited to a single-sensor, single-position setup and to the four benchmark datasets considered; the paper itself notes that further experiments are needed to confirm some hypotheses about dataset- and modality-specific effects.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1038/s41598-026-64846-5

Cites

37 of 37 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Fig. 1 General overview of the proposed method. Given an inertial window (accelerometer + gyroscope), we sample K random rotations to generate rotated views. The original window and each rotated view are then independently masked by hiding subsets of their temporal segments, and the resulting masked windows are encoded by f into latent representations. A Siamese loss encourages each rotated-view representation to align with that of the original signal. To additionally preserve temporal motion structure, a temporal Gram loss matches the time–time similarity matrices of the features extracted from the original and rotated signals, enforcing consistency in their pairwise temporal relationships. Finally, a decoder is trained with an MSE reconstruction loss to reconstruct the masked portions of each input window. The network is trained end-to-end by jointly optimizing the three losses.
Fig. 1 General overview of the proposed method. Given an inertial window (accelerometer + gyroscope), we sample K random rotations to generate rotated views. The original window and each rotated view are then independently masked by hiding subsets of their temporal segments, and the resulting masked windows are encoded by f into latent representations. A Siamese loss encourages each rotated-view representation to align with that of the original signal. To additionally preserve temporal motion structure, a temporal Gram loss matches the time–time similarity matrices of the features extracted from the original and rotated signals, enforcing consistency in their pairwise temporal relationships. Finally, a decoder is trained with an MSE reconstruction loss to reconstruct the masked portions of each input window. The network is trained end-to-end by jointly optimizing the three losses.CC BY · PubMed Central
Fig. 2 Rotation preserves temporal structure in raw inertial signals but requires explicit enforcement in learned representations. (A) Original 3-axis accelerometer signal exhibiting periodic motion patterns. (B) The same signal after applying a random 3D rotation—individual axes appear completely different, with information redistributed across spatial dimensions. (C) Temporal Gram matrices, which capture time-to-time correlation patterns, remain nearly identical between original and rotated signals, demonstrating that rotation preserves temporal dynamics at the raw signal level.
Fig. 2 Rotation preserves temporal structure in raw inertial signals but requires explicit enforcement in learned representations. (A) Original 3-axis accelerometer signal exhibiting periodic motion patterns. (B) The same signal after applying a random 3D rotation—individual axes appear completely different, with information redistributed across spatial dimensions. (C) Temporal Gram matrices, which capture time-to-time correlation patterns, remain nearly identical between original and rotated signals, demonstrating that rotation preserves temporal dynamics at the raw signal level.CC BY · PubMed Central
Algorithm 1 Self-supervised pre-training.
Algorithm 1 Self-supervised pre-training.CC BY · PubMed Central
Fig. 3 Macro F1-scores for the right/left wrist and right/left ankle across MHEALTH, PAMAP2, REALDISP, and WEAR at varying samples-per-class. We compare our method with a fully supervised baseline trained end-to-end from scratch, using the same encoder architecture and MLP classification head in both cases. For our method, we evaluate two settings: a frozen-encoder setting, in which only the classifier is trained while the encoder weights learned through the SSL task are frozen, and a fine-tuned setting, in which the encoder is further optimized on the downstream task together with a newly initialized MLP classification head. Our method remains competitive with the supervised baseline in the frozen-encoder setting and typically outperforms it in the fine-tuned setting, consistently across sample regimes.
Fig. 3 Macro F1-scores for the right/left wrist and right/left ankle across MHEALTH, PAMAP2, REALDISP, and WEAR at varying samples-per-class. We compare our method with a fully supervised baseline trained end-to-end from scratch, using the same encoder architecture and MLP classification head in both cases. For our method, we evaluate two settings: a frozen-encoder setting, in which only the classifier is trained while the encoder weights learned through the SSL task are frozen, and a fine-tuned setting, in which the encoder is further optimized on the downstream task together with a newly initialized MLP classification head. Our method remains competitive with the supervised baseline in the frozen-encoder setting and typically outperforms it in the fine-tuned setting, consistently across sample regimes.CC BY · PubMed Central
Fig. 4 Confusion matrices for left and right wrist sensors. (A) REALDISP: while the left wrist maintains relatively distinct class separations, the right wrist shows predictions collapsing primarily into three classes (waist bending, knee bending, and rowing), particularly for activities beyond basic locomotion (classes 0–7). (B) WEAR: symmetric confusion patterns can be observed. Values below 10% are omitted for readability.
Fig. 4 Confusion matrices for left and right wrist sensors. (A) REALDISP: while the left wrist maintains relatively distinct class separations, the right wrist shows predictions collapsing primarily into three classes (waist bending, knee bending, and rowing), particularly for activities beyond basic locomotion (classes 0–7). (B) WEAR: symmetric confusion patterns can be observed. Values below 10% are omitted for readability.CC BY · PubMed Central
Fig. 5 Comparison of random rotation and random noise as data augmentation techniques on right-wrist data from PAMAP2 and MHEALTH. The encoder is kept frozen, and both an MLP and a linear classifier are trained on top using all available training data.
Fig. 5 Comparison of random rotation and random noise as data augmentation techniques on right-wrist data from PAMAP2 and MHEALTH. The encoder is kept frozen, and both an MLP and a linear classifier are trained on top using all available training data.CC BY · PubMed Central
Fig. 6 Weight sensitivity study for the self-supervised objective defined in Equation 13, conducted using the MHEALTH right-wrist data. Two loss weights are kept fixed at their original values, while the remaining weight is swept from 0.5 to 10.0 in increments of 0.5. Experiments follow the previous experimental protocol: after pre-training, the encoder is kept frozen and an MLP classification head is trained on top using all available training data.
Fig. 6 Weight sensitivity study for the self-supervised objective defined in Equation 13, conducted using the MHEALTH right-wrist data. Two loss weights are kept fixed at their original values, while the remaining weight is swept from 0.5 to 10.0 in increments of 0.5. Experiments follow the previous experimental protocol: after pre-training, the encoder is kept frozen and an MLP classification head is trained on top using all available training data.CC BY · PubMed Central
Fig. 7 Signal ablation study. We compare the full pipeline using right-wrist acceleration and gyroscope signals, corresponding to the original implementation, against an acceleration-only configuration to isolate the effect of gyroscope data. In both settings, the encoder is first pretrained with the proposed self-supervised method. The pretrained encoder is then frozen, and either a linear or MLP classification head is fine-tuned using all available training data. Results are reported in terms of macro F1-score.
Fig. 7 Signal ablation study. We compare the full pipeline using right-wrist acceleration and gyroscope signals, corresponding to the original implementation, against an acceleration-only configuration to isolate the effect of gyroscope data. In both settings, the encoder is first pretrained with the proposed self-supervised method. The pretrained encoder is then frozen, and either a linear or MLP classification head is fine-tuned using all available training data. Results are reported in terms of macro F1-score.CC BY · PubMed Central

Verification

Claims the paper makes

5 Valid · 2 Not valid · 7 Not checked · 97 Own results and interpretations

58 more claims

Create a free account to see all 111 claims and to verify the 13 references still open.

See all claims — free account
  • It may state two separate claims as one. 32
  • Probably not a substantial claim. 21

Create a free account to see these claims.

Citations

Cites

Show all 37 references

Research works printed but not linked: 9

  1. Bock, M., Kuehne, H., Van Laerhoven, K. & Moeller, M. Wear: An outdoor sports dataset for wearable and egocentric activity recognition. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 8, 1–21 (2024).
  2. Cheng, D. et al. Maskcae: Masked convolutional autoencoder via sensor data reconstruction for self-supervised human activity recognition. IEEE J. Biomed. Heal. Informatics 28, 2687–2698 (2024).
  3. Grill, J.-B. et al. Bootstrap your own latent-a new approach to self-supervised learning. Adv. neural information processing systems 33, 21271–21284 (2020).
  4. Mahmud, S. et al. Human activity recognition from wearable sensor data using self-attention. In ECAI 2020 - 24th European Conference on Artificial Intelligence, 29 August–8 September 2020, Vol. 325, 1332–1339 (Santiago de Compostela, Spain, 2020).
  5. Franceschi, J.-Y., Dieuleveut, A. & Jaggi, M. Unsupervised scalable representation learning for multivariate time series. Adv. Neural Inf. Process. Syst. 32, 4652–4663 (2019).
  6. Chen, T., Kornblith, S., Norouzi, M. & Hinton, G. A simple framework for contrastive learning of visual representations. In International conference on machine learning Vol. 119, 1597–1607 (PmLR, 2020).
  7. Rodrigues, O. Des lois géométriques qui régissent les déplacements d’un système solide dans l’espace, et de la variation des coordonnées provenant de ces déplacements considérés indépendamment des causes qui peuvent les produire. Journal de mathématiques pures et appliquées 5, 380–440 (1840).
  8. Reiss, A. & Stricker, D. Introducing a new benchmarked dataset for activity monitoring. In International Symposium on Wearable Computers, 108–109 (2012). · DOI · Paper page
  9. Pedregosa, F. et al. Scikit-learn: Machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011).

Cited by

Still being checked.

Metadata

Authors
Calatrava-Nicolás, F. M., Stoyanov, T., & Martinez Mozos, O.
Affiliations
Örebro University (SE)
Corresponding author
Calatrava-Nicolás, Francisco M.
Venue
Scientific Reports
Published
2026-08-14
Volume
16
Issue
1
ISSN
2045-2322 (ISSN-L)
OpenAlex
W7203468076
Cited by
0
References
34
FWCI
0.00
Type
journal article
Language
en
Open access
Licence
Locations
  • Scientific Reports (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • PubMed Central (publishedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
Funders
Örebro University
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
  • Physical Sciences › Engineering › Biomedical Engineering › Gait Recognition and Analysis
  • Physical Sciences › Computer Science › Signal Processing › Time Series Analysis and Forecasting
Keywords
wearable inertial sensors, cross-subject generalization, self-supervised learning, masked reconstruction, temporal consistency loss, data heterogeneity, low-data regimes, activities of daily living, sports activities
MeSH terms
Wearable Electronic Devices, Autoencoder, Activities of Daily Living, Algorithms, Humans, Human Activities, Rotation, Sports

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (qwen3.8-omni-flash and gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Siamese masked reconstruction with temporal rotation consistency for inertial-based human activity recognition. https://app.cochat.ai/papers/10.1038/s41598-026-64846-5
The paper
Calatrava-Nicolás, F. M., Stoyanov, T., & Martinez Mozos, O. (2026). Siamese masked reconstruction with temporal rotation consistency for inertial-based human activity recognition. Scientific Reports. https://doi.org/10.1038/s41598-026-64846-5

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.