The DOI matches its Crossref record; 55 of the 55 references it cites were found in a registry or an index; 1 of the 12 sources its claims cite were checked for support.
A tiny inertial transformer for human activity recognition via multimodal knowledge distillation and explainable AI
Open accessPublished version of a preprint
Abstract
Human activity recognition (HAR) is essential for applications such as healthcare monitoring, fitness tracking, and smart environments, yet deploying accurate and interpretable models on resource-constrained devices remains challenging. In this paper, we propose XTinyHAR, a lightweight, transformer-based unimodal framework trained via cross-modal knowledge distillation from a multimodal teacher. Our model incorporates temporal positional embeddings and attention rollout to enhance sequential feature extraction and interpretability. Evaluated on UTD-MHAD and MM-Fit, XTinyHAR achieves test accuracies of 98.71% and 98.55% with F1-scores that match these results and Cohen's Kappa above 0.98. The model remains lightweight (2.45 MB) with fast inference (3.1 ms on CPU, 1.2 ms on GPU) and low computational cost (11.3M FLOPs). Extensive ablation studies confirm the contribution of each component, and subject-wise evaluations demonstrate strong generalization across users. These results highlight XTinyHAR's potential as a high-performance, interpretable, and deployable solution for real-time HAR on edge devices. Our codes are available at: https://github.com/Ism-ail11/XTinyHAR.
Overview
CoChat Summary
## TL;DR
XTinyHAR is a lightweight inertial-only transformer for human activity recognition trained by distilling knowledge from a multimodal skeleton+inertial teacher. On UTD-MHAD and MM-Fit, it reports 98.71% and 98.55% test accuracy, respectively, while staying small and fast enough for edge deployment.
## Key findings
- XTinyHAR achieves test accuracies of 98.71% on UTD-MHAD and 98.55% on MM-Fit; the abstract also reports F1-scores that match these results and Cohen’s Kappa above 0.98.
- The model is reported as 2.45 MB with 11.3M FLOPs, and inference time is 3.1 ms on CPU and 1.2 ms on GPU.
- In an ablation on UTD-MHAD, removing knowledge distillation reduces accuracy from 98.71% to 96.84%; removing positional embeddings reduces it to 97.42%; removing attention rollout reduces it to 98.11%.
- Distilling from a multimodal teacher outperforms a unimodal teacher: student accuracy improves by 1.5–2.4% absolute, with 98.71% vs 96.98% on UTD-MHAD and 98.55% vs 96.13% on MM-Fit.
- Subject-wise evaluation reports accuracies ranging from 97.8% to 99.2% on UTD-MHAD and 96.8% to 98.5% on MM-Fit, with standard deviations of 0.54% and 0.52%, respectively.
## Methods
- The paper proposes XTinyHAR, an inertial-only student transformer trained via offline multimodal-to-unimodal knowledge distillation from a multimodal teacher (ST-ConvT) that uses skeleton and inertial streams.
- Experiments are run on UTD-MHAD (8 subjects, 27 actions, 861 labeled samples) and MM-Fit (more than 70 participants, 12 exercises), using subject-independent evaluation; UTD-MHAD uses leave-one-subject-out, and MM-Fit uses a fixed subject-wise split.
- Inputs are segmented into 3-second windows with 50% overlap, inertial signals are resampled to 50 Hz and z-score normalized, and the student uses a patch-based Transformer with L = 2 encoder layers, D = 128, and P = 20.
- Main analysis includes classification metrics, ablation studies, subject-wise testing, noise/missing-data robustness, edge-device deployment tests on Raspberry Pi 4B and Jetson Nano, and XAI methods: attention analysis, Integrated Gradients, attention rollout, and attention similarity.
## Limitations
- The paper states that UTD-MHAD and MM-Fit lack diversity in age and physical ability; in MM-Fit, subjects aged 60+ show a 3.2% accuracy drop.
- The model is designed for fixed wrist-mounted IMUs; simulated sensor-placement shifts in MM-Fit cause performance degradation of up to 5.6%.
- The paper notes a lack of personalization mechanisms for individual sensor variation and routines.
- The text makes evident, and the paper states, that no cross-dataset generalization experiments were conducted (for example, training on UTD-MHAD and testing on MM-Fit).
- The paper also states that true deployment on ultra-constrained platforms such as Cortex-M microcontrollers was not covered.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures





















Verification
Claims the paper makes
1 Valid · 6 Not valid · 11 Not checked · 199 Own results and interpretations
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The proposed Inertial Transformer (IT) is compact (0.62M parameters), fast (1.2 ms latency), and suitable for edge deployment on resource-constrained devices.”
§ A tiny inertial transformer for human activity recognition via multimodal knowledge distillation and explainable AI > Introduction
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“XTinyHAR achieves 98.71% on UTD-MHAD and 98.55% on MM-Fit datasets using only inertial data, closely matching the multimodal teacher while significantly reducing hardware requirements.”
§ A tiny inertial transformer for human activity recognition via multimodal knowledge distillation and explainable AI > Introduction
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Evaluated on UTD-MHAD and MM-Fit, XTinyHAR achieves test accuracies of 98.71% and 98.55% with F1-scores that match these results and Cohen’s Kappa above 0.98.”
§ A tiny inertial transformer for human activity recognition via multimodal knowledge distillation and explainable AI > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Evaluated on UTD-MHAD and MM-Fit, XTinyHAR achieves test accuracies of 98.71% and 98.55% with F1-scores that match these results and Cohen’s Kappa above 0.98.”
- Also stated in: § Introduction · plain
“XTinyHAR achieves 98.71% on UTD-MHAD and 98.55% on MM-Fit datasets using only inertial data, closely matching the multimodal teacher while significantly reducing hardware requirements.”
§ A tiny inertial transformer for human activity recognition via multimodal knowledge distillation and explainable AI > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Introduction · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“XTinyHAR achieves 98.71% on UTD-MHAD and 98.55% on MM-Fit datasets using only inertial data, closely matching the multimodal teacher while significantly reducing hardware requirements.”
§ A tiny inertial transformer for human activity recognition via multimodal knowledge distillation and explainable AI > Introduction
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
104 more claims
Create a free account to see all 217 claims and to verify the 11 references still open.
- It may state two separate claims as one. 73
- Probably not a substantial claim. 44
Citations
Cites
Knowledge Distillation: A Survey
Gou, J., Yu, B., Maybank, S. J., & Tao, D. (2021) · International Journal of Computer Vision · doi:10.1007/s11263-021-01453-z · Cited by 3,931 · cited 1×
“In the proposed XTinyHAR framework, we adopt an offline knowledge distillation 45 paradigm where the multimodal teacher is pre-trained and fixed prior to student training 46 .”
Transformers in Vision: A Survey
Khan, S., Naseer, M., Hayat, M., Zamir, S. W., Khan, F. S., & Shah, M. (2022) · ACM Computing Surveys · doi:10.1145/3505244 · Cited by 3,595
Explainable AI (XAI): Core Ideas, Techniques, and Solutions
Dwivedi, R., Dave, D., Naik, H., Singhal, S., Omer, R., Patel, P., Qian, B., Wen, Z., Shah, T., Morgan, G., et al. (2023) · ACM Computing Surveys · doi:10.1145/3561048 · Cited by 1,386 · cited 1×
“Given that XTinyHAR is designed to function solely with inertial data in a lightweight and deployable manner, it becomes imperative to incorporate Explainable AI (XAI) techniques 51 that provide transparency into the model’s internal reasoning.”
Transformer Interpretability Beyond Attention Visualization
Chefer, H., Gur, S., & Wolf, L. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00084 · Cited by 903
Chen, C., Jafari, R., & Kehtarnavaz, N. (2015) · 2015 IEEE International Conference on Image Processing (ICIP) · doi:10.1109/icip.2015.7350781 · Cited by 765 · cited 1×
“The UTD-MHAD dataset 31 is a benchmark dataset designed for HAR tasks involving multiple sensing modalities.”
Human Action Recognition From Various Data Modalities: A Review
Sun, Z., Ke, Q., Rahmani, H., Bennamoun, M., Wang, G., & Liu, J. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3183112 · Cited by 601
Opportunities and Challenges in Explainable Artificial Intelligence (XAI): A Survey
Das, A., & Rad, P. (2020) · arXiv · doi:10.48550/arxiv.2006.11371 · Cited by 504
Recent trends in machine learning for human activity recognition—A survey
Ramasamy Ramamurthy, S., & Roy, N. (2018) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1254 · Cited by 299
A Deep-Learned Embedding Technique for Categorical Features Encoding
Dahouda, M. K., & Joe, I. (2021) · IEEE Access · doi:10.1109/access.2021.3104357 · Cited by 294
Deep CNN-LSTM With Self-Attention Model for Human Activity Recognition Using Wearable Sensor
Khatun, M. A., Yousuf, M. A., Ahmed, S., Uddin, M. Z., Alyami, S. A., Al-Ashhab, S., Akhdar, H. F., Khan, A., Azad, A., & Moni, M. A. (2022) · IEEE Journal of Translational Engineering in Health and Medicine · doi:10.1109/jtehm.2022.3177710 · Cited by 288 · cited 2×
“…deep learning approaches such as CNN-LSTM hybrids, attention-enhanced recurrent networks, and self-supervised convolutional models have achieved strong results, with reported accuracies ranging from 90.89 to 99.93% across datasets like MHEALTH, UCI-HAR, and custom sensor-based benchmarks 19–23 .”
Influential
Research works printed but not linked: 3
- Bastola, A., Luitel, N., Wang, H., Paudel, D. P., Poudel, R. & Razi, A. RobustFormer: Noise-Robust Pre-Training for Images and Videos. arXiv preprint arXiv:2411.13040 (2024). · arXiv
- Zhu, J. Comparative study of sequence-to-sequence models: From RNNs to transformers. Appl. Comput. Eng. 42(67), 2755–2721 (2023).
- Lin, J. et al. MCUNet: Tiny deep learning on IoT devices. Adv. Neural Inf. Process. Syst. (NeurIPS) 33, 11711–11722 (2020).
Cited by
Lamaakal, I., Yahyati, C., Maleh, Y., et al. (2026) · Internet of Things · doi:10.1016/j.iot.2026.101889 · Cited by 9
Self-citationLamaakal, I., Yahyati, C., Maleh, Y., Makkaoui, K. E., & Ouahbi, I. (2026) · Scientific Reports · doi:10.1038/s41598-026-38478-8 · Cited by 13
Self-citationA Comprehensive Survey of TinyML-Based Biometric Recognition for IoT Edge Devices
Essahraui, S., & Lamaakal, I. (2026) · IEEE Internet of Things Journal · doi:10.1109/jiot.2026.3656932 · Cited by 13
“Beyond e ffi ciency, on-device processing confers security and privacy advantages by minimizing raw biometric data movement and enabling template protection mechanisms to reside within a trusted boundary [13], [14].”
Self-citationYahyati, C., Lamaakal, I., Maleh, Y., Makkaoui, K. E., & Ouahbi, I. (2026) · IEEE Access · doi:10.1109/access.2026.3656854 · Cited by 7
“As a result, Tiny Machine Learning (TinyML) models [7], [8], [12], [20], [37] that execute end-to-end on-device with integer arithmetic and small, deterministic memory footprints has emerged as the most appropriate paradigm for DMS at the edge.”
Self-citationLamaakal, I., Yahyati, C., Maleh, Y., Makkaoui, K. E., & Ouahbi, I. (2026) · Machine Learning and Knowledge Extraction · doi:10.3390/make8030055 · Cited by 8
Self-citation
Metadata
- Authors
- Lamaakal, I., Yahyati, C., Maleh, Y., El Makkaoui, K., Ouahbi, I., El-Latif, A. A. A., Zomorodi, M., & El-Rahiem, B. A.
- Affiliations
- Mohamed I University (MA)
- Université Sultan Moulay Slimane (MA)
- Menoufia University (EG)
- Corresponding author
- Maleh, Yassine
- El-Rahiem, Basma Abd
- Venue
- Scientific Reports
- Published
- 2025-11-27
- Volume
- 15
- Issue
- 1
- Pages
- 42335-42335
- ISSN
- 2045-2322 (ISSN-L)
- PMC
- PMC12660865
- OpenAlex
- W4416740529
- Cited by
- 26
- References
- 49
- Citations by year
- 2026: 25 · 2025: 1
- FWCI
- 9.08
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by-nc-nd
- creativecommons.org/licenses/by-nc-nd/4.0 (tdm)
- creativecommons.org/licenses/by-nc-nd/4.0 (vor)
- Locations
- Scientific Reports (publishedVersion, cc-by-nc-nd)
- Research Square (acceptedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Research, Development, and Innovation Authority (13325-psu-2023-PSNU-R-3-1-EF)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Physical Sciences › Engineering › Biomedical Engineering › Gait Recognition and Analysis
- Keywords
- human activity recognition, inertial sensors, lightweight transformer, cross-modal knowledge distillation, explainable AI, attention rollout, edge deployment, resource-constrained devices, real-time activity recognition
- MeSH terms
- Algorithms, Artificial Intelligence, Humans, Human Activities
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: A tiny inertial transformer for human activity recognition via multimodal knowledge distillation and explainable AI. https://app.cochat.ai/papers/10.1038/s41598-025-26297-2
- The paper
- Lamaakal, I., Yahyati, C., Maleh, Y., El Makkaoui, K., Ouahbi, I., El-Latif, A. A. A., Zomorodi, M., & El-Rahiem, B. A. (2025). A tiny inertial transformer for human activity recognition via multimodal knowledge distillation and explainable AI. Scientific Reports. https://doi.org/10.1038/s41598-025-26297-2
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.