Siamese masked reconstruction with temporal rotation consistency for inertial-based human activity recognition
Open accessThe DOI matches its Crossref record; 37 of the 37 references it cites were found in a registry or an index; 8 of the 21 sources its claims cite were checked for support.
Abstract
Inertial-based Human Activity Recognition (HAR) aims at inferring activities performed by an individual from data on wearable inertial sensors. However, HAR often suffers from poor generalization to unseen users due to data heterogeneity across different individuals and conditions. While collecting and curating large labeled datasets could help alleviate this limitation, it is also costly. To reduce this gap without relying on extensive annotated data, we propose a self-supervised framework grounded in how IMU signals transform under rotation: applying a rotation matrix changes how motion projects onto the sensor axes without altering the underlying motion. A model trained to align representations of a signal and its rotated counterpart is therefore encouraged to capture motion content rather than sensor-frame patterns. We exploit this property by training a Siamese masked convolutional autoencoder that learns by reconstructing masked inputs while aligning representations of original signals and its rotated version, further regularized by a temporal consistency loss that enforces agreement between the temporal structure of original and rotated representations within each window. We evaluate on four public HAR benchmarks covering diverse scenarios, including activities of the daily living (ADL) and sports activities, using cross-subject evaluation, where our method yields improvements of +1.4 and +1.2 percentage points over the strongest baseline with linear and MLP probes respectively, averaged across datasets and sensor positions. We further show that our method obtains competitive results against the fully supervised baseline in low-data regimes. The code is available on https://github.com/FranciscoCalatrava/Rotation_Siamese_Masked.git .
Overview
CoChat Summary
## TL;DR
The paper proposes a self-supervised framework for inertial HAR that combines masked reconstruction with rotation-based Siamese consistency and a temporal Gram constraint, using rotations as structured supervision rather than just augmentation. Across four public datasets, it generally outperforms baselines and reports average gains of +1.4 percentage points with a linear head and +1.2 with an MLP head in macro F1-score.
## Key findings
- Across 24 dataset–placement–head configurations, the method achieved the best average Macro F1-score in 19 cases and the second-best in the remaining 4.
- Averaged across datasets and body positions, the proposed method improved over the strongest competitor by +1.4 percentage points with a linear classifier and +1.2 percentage points with an MLP classifier (macro F1-score).
- In the low-data study, it remained competitive with a frozen encoder and generally surpassed the supervised baseline when fully fine-tuned; the paper reports clearer gains in the lowest-data regimes of 10 and 20 samples per class.
- The ablation study found that \(L_{rec}+L_{gram}\) ranked first or second in 14 of 24 configurations, while the full objective \(L_{rec}+L_{siam}+L_{gram}\) was first or second in 21 of 24 configurations.
## Methods
- Two-stage design: self-supervised pretraining on unlabeled inertial windows, followed by downstream HAR evaluation using a frozen encoder and a lightweight classifier.
- Data/sample: four public datasets (PAMAP2, MHEALTH, REALDISP, WEAR), using subject-wise cross-validation; input windows were 128 samples with 50% overlap (2.56 s per window), and the main experiments used single-sensor wrist/ankle placements.
- Main analysis: a self-supervised objective combining masked reconstruction, a Siamese-style consistency loss between original and rotated views, and a temporal Gram-matrix loss; evaluation used Macro F1-score with Linear and MLP probes.
## Limitations
- The paper states that the framework “does not yet fully capture all the information that rotations can provide” and does not explicitly model orientation-aware relationships or the geometric structure induced by successive rotations.
- It notes that this is especially important because real-world sensor orientation varies across users, sessions, wearing conditions, and placements.
- Evident from the text: the study is limited to a single-sensor, single-position setup and to the four benchmark datasets considered; the paper itself notes that further experiments are needed to confirm some hypotheses about dataset- and modality-specific effects.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures








Verification
Claims the paper makes
5 Valid · 2 Not valid · 7 Not checked · 97 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“In addition to this, the rotation transformation has been shown to be an effective IMU augmentation, improving robustness30.”
§ Siamese masked reconstruction with temporal rotation consistency for inertial-based human activity recognition > Introduction
What the sources say
- Supportedcites [30] Um, T. T. Data augmentation of wearable sensor data for parkinson’s disease monitoring using convolutional neural networks. In Proceedings of the 19th ACM International Conference on Multimodal Interaction 216–220 (2017). source
The Results (4.3, p. 4) state that 'rotation, permutation, and time-warping methods improve the performance of PD motor state classification. The best performance among the single data augmentation methods is achieved by rotation ... improve the performance by 3.6-5.1% accuracy', and that 'rotation can be used to alleviate sensor pose variability'; the Conclusion (p. 4) reports Rot+Perm lifting the 77.52% baseline to 86.88%, with Fig. 4 showing better generalization from the augmentation's regularization effect. The Methods (3.2, p. 2) motivate rotation precisely as simulating different sensor placements on wearable sensor data. Two wording caveats that do not defeat the paraphrase: our copy calls the data 'wearable sensor data' rather than IMU, and it frames the gain as accuracy/generalization and reduced pose variability rather than using 'robustness' for rotation (that word appears in 3.2 for jittering/scaling/magnitude-warping). (LLM, conf=0.85)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Physically, a 3-dimensional rotation deterministically maps accelerometer and gyroscope signals across axes via a rotation matrix29.”
§ Siamese masked reconstruction with temporal rotation consistency for inertial-based human activity recognition > Introduction
What the sources say
- Supportedcites [29] Xu, H., Zhou, P., Tan, R. & Li, M. Practically adopting human activity recognition. In Proceedings of the 29th Annual International Conference on Mobile Computing and Networking, 1–15 (2023). source
The Local rotation section derives deterministic transformations for both accelerometer and gyroscope readings under a known rotation. The Discussion expresses those same transformations with a rotation matrix, giving a′ = ΔR⁻¹a and ω′ = ΔR⁻¹ω (Equations 21–22). (LLM, conf=0.94)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“Deep learning has greatly advanced HAR in recent years, yet supervised models trained only on activity labeled data still struggle to generalise to unseen users.”
§ Siamese masked reconstruction with temporal rotation consistency for inertial-based human activity recognition > Introduction
What the sources say
- Could not verifycites [16] Calatrava-Nicolás, F. M. & Mozos, O. M. Light residual network for human activity recognition using wearable sensor data. IEEE Sensors Lett. 7, 1–4 (2023). sourceExperimental — accuracy not yet measured
Our copy of Calatrava-Nicolás & Mozos (2023) never states that supervised, activity-label-trained HAR models struggle to generalise to unseen users. Its Introduction/Related Work (II) and Abstract frame the leave-one-person-out benchmark as a way to standardise and make comparisons more reliable and fair, because previous works reuse the UCI-HAR fixed 70/30 participant split which 'limits the details on performance' — an evaluation-protocol argument, not a cross-user generalisation failure. On the contrary, V. EXPERIMENTS reports 97.6% F1 under LOOCV and says Tables 2 and 5 'confirm the robustness of our model in different training and test sets', with the one weak per-participant result (Participant 14, Table 4) attributed to a data-collection problem. Searched: generalisation to unseen users/subjects, leave-one-person-out, cross-validation performance per participant; no passage addresses the claim's point. (LLM, conf=0.62)
- Not checked yetcites [17] Calatrava-Nicolás, F. M., Miyauchi, S. & Mozos, O. M. Deep adversarial learning with activity-based user discrimination task for human activity recognition. arXiv preprint arXiv:2410.12819 (2024).
- Supportedcites [18] Suh, S., Rey, V. F. & Lukowicz, P. Adversarial deep feature extraction network for user independent human activity recognition. In IEEE International Conference on Pervasive Computing and Communications (PerCom), 217–226 (2022). source
The Introduction (I. INTRODUCTION, p. 1) states that 'deep learning methods have achieved state-of-the-art performance in HAR' (CNNs, LSTM-CNNs, transformers) yet 'creating models that can generalize to unseen subjects is still a major challenge', attributing this to interpersonal variability that 'makes user recognition possible, but activity recognition more challenging' and to the observed performance gap under leave-subjects-out evaluation. The Abstract (p. 1) reinforces this by calling user dependence 'one of the most difficult general problems in HAR'. The citing sentence's qualifier about models trained on activity labels alone is the paper's own framing, echoed in its discussion of [10]-[12] (Introduction p. 1, Related Work pp. 2-3), where exploiting user labels or adversarial generalization is needed because activity-label-only representations do not extend to unseen subjects. (LLM, conf=0.85)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“In inertial-based HAR, this shift is produced by the way different individuals implement the same activity (inter-person variability), the sensor position (placement variability), and hardware differences between sensors18,19,37, among others.”
§ Siamese masked reconstruction with temporal rotation consistency for inertial-based human activity recognition > Related works
What the sources say
- Partly supportedcites [18] Suh, S., Rey, V. F. & Lukowicz, P. Adversarial deep feature extraction network for user independent human activity recognition. In IEEE International Conference on Pervasive Computing and Communications (PerCom), 217–226 (2022). sourceExperimental — accuracy not yet measured
The Abstract (p. 1) and Introduction (p. 1) of Suh et al. (2022) state two of the three factors the claim attributes to them: user dependence arises from 'the huge variability of the way different people execute even the simplest actions' plus 'detailed sensor fixtures and placement will be different for different people or even at different times for the same users', with the method framed as realigning 'the data distribution over multiple subjects' (also Related Work, p. 2, on domain adaptation measuring 'data distribution heterogeneity'). Hardware differences between sensors are not stated by this paper as a source of the shift: the closest passage is the Conclusion (p. 10), which only observes that the tested datasets 'used different types and numbers of sensors on different positions of the body' and defers sensor-invariant extraction to future work, while sensor/device heterogeneity in Related Work (p. 3) is credited to other studies (e.g. SenseHAR's 'heterogeneous data distribution'). So the inter-person and placement components are supported, the hardware-difference component is not. (LLM, conf=0.75)
- Not checked yetcites [19] Plötz, T. If only we had more data!: Sensor-based human activity recognition in challenging scenarios. In IEEE International Conference on Pervasive Computing and Communications (PerCom) Workshops 565–570 (2023). source
- Partly supportedcites [37] Calatrava-Nicolás, F. M. et al. Embedded inter-subject variability in adversarial learning for inertial sensor-based human activity recognition. In 2025 IEEE 35th International Workshop on Machine Learning for Signal Processing (MLSP), 1–6 (IEEE, 2025). sourceExperimental — accuracy not yet measured
Our copy squarely supports the inter-person component: the Introduction (p.1) states the generalization gap 'arises from the heterogeneity in data distributions due to inter-subject variability, i.e., different individuals perform the same activity differently,' echoed in the Abstract and Section 5/6 (Wasserstein-distance analysis of train-vs-test participants). However, the paper nowhere attributes distribution shift to sensor position/placement variability or to hardware differences between sensors — Section 4 (Datasets and Setup) mentions sensor locations (dominant wrist, chest, dominant leg; chest, right wrist, left ankle; 9 sensors on various body parts) only as descriptive metadata of PAMAP2, MHEALTH and REALDISP, not as drivers of shift, and no passage discusses device/hardware heterogeneity. Those two listed causes are therefore unsupported by this source. (LLM, conf=0.72)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“our method yields improvements of +1.4 and +1.2 percentage points over the strongest baseline with linear and MLP probes respectively, averaged across datasets and sensor positions.”
- Also stated in: § Introduction · plain
“Our approach yields more informative representations than the baselines, with improvements of +1.4 and +1.2 percentage points over the strongest competitor with linear and MLP heads respectively, averaged over datasets and body positions.”
§ Siamese masked reconstruction with temporal rotation consistency for inertial-based human activity recognition > Abstract Inertial-based Human Activity Recognition (HAR) aims at inferring activities performed by an individual from data on wearable inertial sensors. However, HAR often suffers from poor generalization to unseen users due to data heterogeneity across different individuals and conditions. While collecting and curating large labeled datasets could help alleviate this limitation, it is also costly. To reduce this gap without relying on extensive annotated data, we propose a self-supervised framework grounded in how IMU signals transform under rotation: applying a rotation matrix changes how motion projects onto the sensor axes without altering the underlying motion. A model trained to align representations of a signal and its rotated counterpart is therefore encouraged to capture motion content rather than sensor-frame patterns. We exploit this property by training a Siamese masked convolutional autoencoder that learns by reconstructing masked inputs while aligning representations of original signals and its rotated version, further regularized by a temporal consistency loss that enforces agreement between the temporal structure of original and rotated representations within each window. We evaluate on four public HAR benchmarks covering diverse scenarios, including activities of the daily living (ADL) and sports activities, using cross-subject evaluation, where our method yields improvements of +1.4 and +1.2 percentage points over the strongest baseline with linear and MLP probes respectively, averaged across datasets and sensor positions. We further show that our method obtains competitive results against the fully supervised baseline in low-data regimes. The code is available on https://github.com/FranciscoCalatrava/Rotation_Siamese_Masked.git.
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Introduction · plain
58 more claims
Create a free account to see all 111 claims and to verify the 13 references still open.
- It may state two separate claims as one. 32
- Probably not a substantial claim. 21
Citations
Cites
Representation Learning: A Review and New Perspectives
Bengio, Y., Courville, A., & Vincent, P. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.50 · Cited by 13,001
Exploring Simple Siamese Representation Learning
Chen, X., & He, K. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01549 · Cited by 3,725
Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition
Ordóñez, F., & Roggen, D. (2016) · Sensors · doi:10.3390/s16010115 · Cited by 2,761
A Survey on Human Activity Recognition using Wearable Sensors
Lara, O. D., & Labrador, M. A. (2013) · IEEE Communications Surveys & Tutorials · doi:10.1109/surv.2012.110112.00192 · Cited by 2,662
ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoders
Woo, S., Debnath, S., Hu, R., Chen, X., Liu, Z., Kweon, I. S., & Xie, S. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01548 · Cited by 1,791
Introducing a New Benchmarked Dataset for Activity Monitoring
Reiss, A., & Stricker, D. (2012) · 2012 16th International Symposium on Wearable Computers · doi:10.1109/iswc.2012.13 · Cited by 1,456
Um, T. T., Pfister, F. M. J., Pichler, D., Endo, S., Lang, M., Hirche, S., Fietzek, U., & Kulić, D. (2017) · Proceedings of the 19th ACM International Conference on Multimodal Interaction · doi:10.1145/3136755.3136817 · Cited by 729 · cited 1×
“Prior work has shown that rotation augmentation, i.e., applying a 3-dimensional rotation (via a rotation matrix) to inertial measurements at each time step, is particularly effective for diversifying training samples in HAR 26,30 .”
mHealthDroid: A Novel Framework for Agile Development of Mobile Health Applications
Banos, O., Garcia, R., Holgado-Terriza, J. A., Damas, M., Pomares, H., Rojas, I., Saez, A., & Villalonga, C. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-13105-4_14 · Cited by 583
Demrozi, F., Pravadelli, G., Bihorac, A., & Rashidi, P. (2020) · IEEE Access · doi:10.1109/access.2020.3037715 · Cited by 368
Multi-task Self-Supervised Learning for Human Activity Detection
Saeed, A., Ozcelebi, T., & Lukkien, J. (2019) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3328932 · Cited by 346 · cited 4×
“Single-task transformation prediction 25 : We train the encoder with a single proxy task that performs binary classification of whether an input window has been transformed by a rotation or not.”
Influential
Research works printed but not linked: 9
- Bock, M., Kuehne, H., Van Laerhoven, K. & Moeller, M. Wear: An outdoor sports dataset for wearable and egocentric activity recognition. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 8, 1–21 (2024).
- Cheng, D. et al. Maskcae: Masked convolutional autoencoder via sensor data reconstruction for self-supervised human activity recognition. IEEE J. Biomed. Heal. Informatics 28, 2687–2698 (2024).
- Grill, J.-B. et al. Bootstrap your own latent-a new approach to self-supervised learning. Adv. neural information processing systems 33, 21271–21284 (2020).
- Mahmud, S. et al. Human activity recognition from wearable sensor data using self-attention. In ECAI 2020 - 24th European Conference on Artificial Intelligence, 29 August–8 September 2020, Vol. 325, 1332–1339 (Santiago de Compostela, Spain, 2020).
- Franceschi, J.-Y., Dieuleveut, A. & Jaggi, M. Unsupervised scalable representation learning for multivariate time series. Adv. Neural Inf. Process. Syst. 32, 4652–4663 (2019).
- Chen, T., Kornblith, S., Norouzi, M. & Hinton, G. A simple framework for contrastive learning of visual representations. In International conference on machine learning Vol. 119, 1597–1607 (PmLR, 2020).
- Rodrigues, O. Des lois géométriques qui régissent les déplacements d’un système solide dans l’espace, et de la variation des coordonnées provenant de ces déplacements considérés indépendamment des causes qui peuvent les produire. Journal de mathématiques pures et appliquées 5, 380–440 (1840).
- Reiss, A. & Stricker, D. Introducing a new benchmarked dataset for activity monitoring. In International Symposium on Wearable Computers, 108–109 (2012). · DOI · Paper page
- Pedregosa, F. et al. Scikit-learn: Machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011).
Cited by
Still being checked.
Metadata
- Authors
- Calatrava-Nicolás, F. M., Stoyanov, T., & Martinez Mozos, O.
- Affiliations
- Örebro University (SE)
- Corresponding author
- Calatrava-Nicolás, Francisco M.
- Venue
- Scientific Reports
- Published
- 2026-08-14
- Volume
- 16
- Issue
- 1
- ISSN
- 2045-2322 (ISSN-L)
- PMC
- PMC13476224
- OpenAlex
- W7203468076
- Semantic Scholar
- af6ddb753f430ce92750d5519202d8d87f0b8b9c
- Cited by
- 0
- References
- 34
- FWCI
- 0.00
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Scientific Reports (publishedVersion, cc-by)
- PubMed (publishedVersion)
- PubMed Central (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Funders
- Örebro University
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Physical Sciences › Engineering › Biomedical Engineering › Gait Recognition and Analysis
- Physical Sciences › Computer Science › Signal Processing › Time Series Analysis and Forecasting
- Keywords
- wearable inertial sensors, cross-subject generalization, self-supervised learning, masked reconstruction, temporal consistency loss, data heterogeneity, low-data regimes, activities of daily living, sports activities
- MeSH terms
- Wearable Electronic Devices, Autoencoder, Activities of Daily Living, Algorithms, Humans, Human Activities, Rotation, Sports
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash and gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Siamese masked reconstruction with temporal rotation consistency for inertial-based human activity recognition. https://app.cochat.ai/papers/10.1038/s41598-026-64846-5
- The paper
- Calatrava-Nicolás, F. M., Stoyanov, T., & Martinez Mozos, O. (2026). Siamese masked reconstruction with temporal rotation consistency for inertial-based human activity recognition. Scientific Reports. https://doi.org/10.1038/s41598-026-64846-5
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.