Inertial-Measurement-Unit-Based Novel Human Activity Recognition Algorithm Using Conformer
Open accessThe DOI matches its Crossref record; 48 of the 48 references it cites were found in a registry or an index.
Abstract
Inertial-measurement-unit (IMU)-based human activity recognition (HAR) studies have improved their performance owing to the latest classification model. In this study, the conformer, which is a state-of-the-art (SOTA) model in the field of speech recognition, is introduced in HAR to improve the performance of the transformer-based HAR model. The transformer model has a multi-head self-attention structure that can extract temporal dependency well, similar to the recurrent neural network (RNN) series while having higher computational efficiency than the RNN series. However, recent HAR studies have shown good performance by combining an RNN-series and convolutional neural network (CNN) model. Therefore, the performance of the transformer-based HAR study can be improved by adding a CNN layer that extracts local features well. The model that improved these points is the conformer-based-model model. To evaluate the proposed model, WISDM, UCI-HAR, and PAMAP2 datasets were used. A synthetic minority oversampling technique was used for the data augmentation algorithm to improve the dataset. From the experiment, the conformer-based HAR model showed better performance than baseline models: the transformer-based-model and the 1D-CNN HAR models. Moreover, the performance of the proposed algorithm was superior to that of algorithms proposed in recent similar studies which do not use RNN-series.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures



Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“As a result, they achieved SOTA performance on a variety of HAR datasets.”
From TS2ACT: Few-Shot Human Activity Sensing with Cross-Modal Co-Learning (2023) · cites this paper as [25]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 2×
“The transformer model is known to have good computational efficiency while extracting temporal dependency similar to the RNN model [17] and has been recently introduced into IMU-based HAR research.”
BackgroundMethodHochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 1×
“Long short-term memory (LSTM) and Sensors 2022 , 22 , 3932 2 of 13 gated recurrent units (GRUs), which are types of RNN series models, feature a gate structure that can memorize the long-term state of the previous input and learn the sequential context of time-series data [13,14].”
BackgroundSMOTE synthetic minority over-sampling technique
Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002) · Journal of Artificial Intelligence Research · doi:10.1613/jair.953 · Cited by 32,865 · cited 15×
“The SMOTE algorithm augments windowed data because it has little effect on high-dimensional data for most trained classifiers [42].”
MethodInfluentialLearning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545 · cited 1×
“Long short-term memory (LSTM) and Sensors 2022 , 22 , 3932 2 of 13 gated recurrent units (GRUs), which are types of RNN series models, feature a gate structure that can memorize the long-term state of the previous input and learn the sequential context of time-series data [13,14].”
BackgroundHan (2023) · doi:10.1109/tpami.2022.3152247 · Cited by 4,682 · cited 1×
“The transformer’s efficient, global information capture and interpretable advantages were applied to the vision field and showed similar or superior performance to conventional CNN and RNN-based models [19–21].”
MethodTemporal Fusion Transformers for interpretable multi-horizon time series forecasting
Lim, B., Arık, S. Ö., Loeff, N., & Pfister, T. (2021) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2021.03.012 · Cited by 3,291
Conformer: Convolution-augmented Transformer for Speech Recognition
Gulati, A., Qin, J., Chiu, C.-C., Parmar, N., Zhang, Y., Yu, J., Han, W., Wang, S., Zhang, Z., Wu, Y., et al. (2020) · Interspeech 2020 · doi:10.21437/interspeech.2020-3015 · Cited by 3,150
Conformer: Convolution-augmented Transformer for Speech Recognition
Gulati, A., Qin, J., Chiu, C.-C., Parmar, N., Zhang, Y., Yu, J., Han, W., Wang, S., Zhang, Z., Wu, Y., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.08100 · Cited by 3,150 · cited 3×
“Unlike the transformer, however, the conformer incorporates a CNN structure to extract local features with slightly better performance [24–26].”
BackgroundSurvey on deep learning with class imbalance
Johnson, J. M., & Khoshgoftaar, T. M. (2019) · Journal of Big Data · doi:10.1186/s40537-019-0192-5 · Cited by 3,032 · cited 4×
“(1) The windowed data are 2D, and these 2D data are converted into a vector before applying the SMOTE to solve this problem is to equalize the amount of data in the majority and minority classes through data augmentation, which generates synthetic data similar to the original data [39–41].”
BackgroundMethodInfluentialActivity recognition using cell phone accelerometers
Kwapisz, J. R., Weiss, G. M., & Moore, S. A. (2011) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1964897.1964918 · Cited by 2,912 · cited 2×
“The execution time of participants’ motions were different from each other, and the data were imbalanced [34].”
Background
Research works printed but not linked: 3
- Kuang X. He J. Hu Z. Zhou Y. Comparison of deep feature learning methods for human activity recognition Appl. Res. Comput. 2018 35 2815 2817
- Boin J.B. Roth N. Doshi J. Llueca P. Borensztein N. Multi-class segmentation under severe class imbalance: A case study in roof damage assessment arXiv 2020 2010.07151
- Abedin A. Rezatofighi H. Ranasinghe D.C. Guided-GAN: Adversarial Representation Learning for Activity Recognition with Wearables arXiv 2021 2110.05732
Cited by
Transformer-based models to deal with heterogeneous environments in Human Activity Recognition
Ek, S., Portet, F., & Lalanda, P. (2023) · Personal and Ubiquitous Computing · doi:10.1007/s00779-023-01776-3 · Cited by 103
“Comparison with recent Transformer based HAR models [50, 22] was not directly possible since most papers do not publicly release their code and there is no consensual benchmark in HAR for mobile phones.”
InfluentialChu, X., Kim, S., Cho, Y., & Park, K.-S. (2024) · Electronics · doi:10.3390/electronics13040747 · Cited by 42
Conformer-Based Human Activity Recognition Using Inertial Measurement Units
Seenath, S., & Dharmaraj, M. (2023) · Sensors · doi:10.3390/s23177357 · Cited by 22
“The key contributions of this paper are as follows:”
MethodTS2ACT: Few-Shot Human Activity Sensing with Cross-Modal Co-Learning
Xia, K., Li, W., Gan, S., & Lu, S. (2023) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3631445 · Cited by 19
Haresamudram, H., Essa, I., & Plötz, T. (2024) · Sensors · doi:10.3390/s24041238 · Cited by 14
Liu, X., Cheng, D., Fu, Z., Zhang, L., Wu, H., & Song, A. (2026) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2026.3664424 · Cited by 17
“Kim et al. [28] introducedahybridtransformer-basedmodelnamedConformer, whichcancombinetheuseofsyntheticminorityoversampling techniqueforaugmentingsensordatatoimproveactivityrecog-nition performance; 3) Deep learning with manual features: This paradigm employs the features manually extracted from…”
Ryu, S., Yun, S., Lee, S., & Jeong, I. C. (2024) · Sensors · doi:10.3390/s24051610 · Cited by 13
“Kim et al. [28] employed Conformer, a state-of-the-art model in speech recognition, to improve HAR performance.”
BackgroundSelf-citationA Method of Self-Supervised Denoising and Classification for Sensor-Based Human Activity Recognition
Lu, L., & Deng, T. (2023) · IEEE Sensors Journal · doi:10.1109/jsen.2023.3323314 · Cited by 10
“Some researchers have integrated attention modules with classification models, and they have made significant contributions in their research endeavors, resulting in improved performance of the entire system [41], [42], [43], [44], [45], [46].”
Efficient Human Gait Activity Recognition Based on Sensor Fusion and Intelligent Stacking Framework
Tarekegn, A. N., Sajjad, M., Cheikh, F. A., Ullah, M., & Muhammad, K. (2023) · IEEE Sensors Journal · doi:10.1109/jsen.2023.3319353 · Cited by 13
“The methods used in recent studies, such as DeepCNN-RF [46], multiheaded CNN [49], conformer-based model [47], ensemble DL [48], CNN-BiLSTM [50], CNN-GRU [23], LSTM-CNN [27], and our stacking approach, were compared in terms of accuracy and FAR test.”
MethodLightweight human activity recognition method based on the MobileHARC model
Gong, X., Zhang, X., & Li, N. (2024) · Systems Science & Control Engineering · doi:10.1080/21642583.2024.2328549 · Cited by 13
“The Conformer involves adding CNN layers within the Transformer to enhance the capability of local feature extraction (Kim et al., 2022).”
Bochniewicz, E. M., Emmer, G., Dromerick, A. W., Barth, J., & Lum, P. S. (2023) · Sensors · doi:10.3390/s23063111 · Cited by 12
Data Valuation Algorithm for Inertial Measurement Unit-Based Human Activity Recognition
Kim, Y.-W., & Lee, S. (2022) · Sensors · doi:10.3390/s23010184 · Cited by 11
Self-citationHierarchical Deep Learning for Human Activity Recognition Integrating Postural Transitions
Tilley, D. J., & Martínez-Hernández, U. (2024) · IEEE Sensors Journal · doi:10.1109/jsen.2024.3491352 · Cited by 10
“A notable advancement is the conformer-based HAR model, which integrates convolutional layers, multiple conformer blocks with self-attention and residual connections, and classification layers to enhance recognition accuracy [27], [31].”
InfluentialHuman Activity Recognition Based on Feature Fusion of Millimeter Wave Radar and Inertial Navigation
Shi, J., Zhu, Y., He, J., Xu, Z., Chu, L., Braun, R., & Shi, Q. (2025) · IEEE Journal of Microwaves · doi:10.1109/jmw.2025.3539957 · Cited by 11
“Conformer [39] fuses local features and global features at different resolutions in an interactive manner.”
Seeking optimal and explainable deep learning models for inertial-based posture recognition
Martins, D. R., Cerqueira, S. M., & Santos, C. P. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.112700 · Cited by 8
Chu, X., Kim, S., & Park, K.-S. (2023) · ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics (ACM-BCB) · doi:10.1145/3584371.3612999 · Cited by 6
“Studies also explore the use of other sensor data, like inertial measurement units, combined with various machine learning (ML) techniques [2-5].”
Advances in E-Health and Mobile Health Monitoring
Haghighi, P. D., & Burstein, F. (2022) · Sensors · doi:10.3390/s22228621 · Cited by 4
Conditional Human Activity Signal Generation and Generative Classification with a GPT-2 Model
Zilelioglu, H., Khodabandelou, G., Chibani, A., & Amirat, Y. (2023) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn54540.2023.10191464 · Cited by 5
“19% F1 [38] Transformer-CNN WISDM 80 70% / 30% split 98.”
MethodInfluentialPostural Sway Classification using Modified Vision Transformer
Nehary, E. A., Rajan, S., & Andò, B. (2023) · IEEE Biomedical Circuits and Systems Conference (BioCAS) · doi:10.1109/biocas58349.2023.10389098 · Cited by 3
Transformer Model for Human Activity Recognition Using IoT Wearables
Sowmiya, S., & Menaka, D. (2023) · Lecture notes in electrical engineering · doi:10.1007/978-981-99-6690-5_21 · Cited by 3
Ariaeimehr, M., & Ravanmehr, R. (2025) · Multimedia Tools and Applications · doi:10.1007/s11042-025-21086-5 · Cited by 3
“Its hybrid neural network structure enables robust sequential modeling with improved computational perfor - mance compared to other architectures [17 ].”
Hyperbolic Transformers with LLMs for Multimodal Human Activity Recognition
Soleimani, F., Khodabandelou, G., Chibani, A., & Amirat, Y. (2025) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros60139.2025.11247069 · Cited by 3
“Prior approaches include CNN-GRU [21] (96.20% accuracy, 96.19% F1), and Transformer-CNN with self-attention [22] (99.30% accuracy).”
InfluentialSenseMLP: a parallel MLP architecture for sensor-based human activity recognition
Li, W., Guo, J., & Wu, H. (2024) · Multimedia Systems · doi:10.1007/s00530-024-01384-y · Cited by 3
Miss-placement Prediction of Multiple On-body Devices for Human Activity Recognition
Dönnebrink, R., Rueda, F. M., Grzeszick, R., & Stach, M. (2023) · International Workshop on Sensor-based Activity Recognition and Interaction (iWOAR) · doi:10.1145/3615834.3615838 · Cited by 2
“RNNs, transformers, and conformers [3, 7, 22].”
Multi-Task Learning U-Net for Functional Shoulder Sub-Task Segmentation
Chu, E.-P., Liu, K.-C., Hsieh, C.-Y., Chang, C.-Y., Tsao, Y., & Chan, C.-T. (2023) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc40787.2023.10341137 · Cited by 3
SigFormer: Sparse Signal-guided Transformer for Multi-modal Action Segmentation
Liu, Q., Liu, X., Liu, K., Gu, X., & Liu, W. (2024) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3657296 · Cited by 3
“In this section, we introduce the ablation studies to evaluate the effectiveness of each module in SigFormer on the OpenPack dataset.”
Xia, K., Li, W., Shao, Y., & Lu, S. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681664 · Cited by 3
Identification of the Advanced Data Exfiltration by Human Activity Recognition using Transformer
Martins, J. D. C., Castro, G. A., Souza, L. R., Weigang, L., & Costa, P. (2023) · Annual Conference of the IEEE Industrial Electronics Society (IECON) · doi:10.1109/iecon51785.2023.10312631
“Like [28], Khaertdinov et al. [31] also used data augmentation, but they proposed the use of random data transformations consisting of several simple augmentations of time series.”
BackgroundMethodInfluentialRefined Human Activity Recognition Algorithm Using Oversampling and Combinatorial Loss in 1DCLA-Net
Xu, Z., Liu, Q., Peng, N., & Meng, W. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-0777-8_15
Individual Risky Action Warning of Wearable Sensors on Personalized Health Profiles via LLM
Xie, Y., Jing, Y., Sun, H., Qi, Q., He, B., Wang, C., & Wang, J. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11464652
Activity recognition via Riemannian optimization and multimodal Transformers
Soleimani, F., Khodabandelou, G., Chibani, A., & Amirat, Y. (2026) · Machine Learning with Applications · doi:10.1016/j.mlwa.2026.101021
Metadata
- Authors
- Kim, Y.-W., Cho, W.-H., Kim, K.-S., & Lee, S.
- Affiliations
- Inha University (KR)
- Corresponding author
- Lee, Sangmin
- Venue
- Sensors
- Publisher
- MDPI AG
- Published
- 2022-05-23
- Volume
- 22
- Issue
- 10
- Pages
- 3932
- ISSN
- 1424-8220 (ISSN-L)
- PMC
- PMC9144209
- OpenAlex
- W4282928124
- Cited by
- 31
- References
- 46
- Citations by year
- 2026: 3 · 2025: 4 · 2024: 9 · 2023: 13 · 2022: 2
- FWCI
- 2.29
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Sensors (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Inha University (NRF-2018R1A6A1A03025523)
- Basic Science Research Program of the National Research Foundation of Korea (NRF-2018R1A6A1A03025523)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Physical Sciences › Engineering › Biomedical Engineering › Gait Recognition and Analysis
- Physical Sciences › Computer Science › Signal Processing › Time Series Analysis and Forecasting
- Keywords
- human activity recognition, inertial measurement unit, conformational isomers, transformer models, recurrent neural networks, temporal dependencies, local feature extraction, SMOTE, data augmentation, wearable sensor data
- MeSH terms
- Algorithms, Humans, Human Activities, Neural Networks, Computer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.