Multiscale Deep Feature Learning for Human Activity Recognition Using Wearable Sensors
Abstract
Deep convolutional neural networks (CNNs) achieve state-of-the-art performance in wearable human activity recognition (HAR), which has become a new research trend in ubiquitous computing scenario. Increasing network depth or width can further improve accuracy. However, in order to obtain the optimal HAR performance on mobile platform, it has to consider a reasonable tradeoff between recognition accuracy and resource consumption. Improving the performance of CNNs without increasing memory and computational burden is more beneficial for HAR. In this article, we first propose a new CNN that uses hierarchical-split (HS) idea for a large variety of HAR tasks, which is able to enhance multiscale feature representation ability via capturing a wider range of receptive fields of human activities within one feature layer. Experiments conducted on benchmarks demonstrate that the proposed HS module is an impressive alternative to baseline models with similar model complexity, and can achieve higher recognition performance (e.g., 97.28%, 93.75%, 99.02%, and 79.02% classification accuracies) on UCI-HAR, PAMAP2, WISDM, and UNIMIB-SHAR. Extensive ablation studies are performed to evaluate the effect of the variations of receptive fields on classification performance. Finally, we demonstrate that multiscale receptive fields can help to learn more discriminative features (achieving 94.10% SOTA accuracy) in weakly labeled HAR dataset.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Most existing deep learning-based HAR methods assume that training and test data are drawn from the same distribution, which limits their ability to generalize to unseen users.
“Most existing methods assume that the training and test data are drawn from the same distribution [4,5,6,7,8], which limits their ability to generalize to unseen users.”
From Towards User-Generalizable Wearable-Sensor-Based Human Activity Recognition: A Multi-Task Contrastive Learning Approach (2025) · cites this paper as [8]
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 74,248 · see citing works
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562 · see citing works
Aggregated Residual Transformations for Deep Neural Networks
Xie, S., Girshick, R., Dollar, P., Tu, Z., & He, K. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.634 · Cited by 12,465 · see citing works
Res2Net: A New Multi-Scale Backbone Architecture
Gao, S., Cheng, M., Zhao, K., Zhang, X., Yang, M., & Torr, P. H. S. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2938758 · Cited by 3,423 · see citing works
Activity recognition using cell phone accelerometers
Kwapisz, J. R., Weiss, G. M., & Moore, S. A. (2011) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1964897.1964918 · Cited by 2,912 · cited 9× · see citing works
“In this section, the proposed HS model is evaluated on four public benchmark HAR datasets, namely UCI-HAR, WISDM, PAMAP2, and UNIMIB-SHAR.”
MethodInfluentialDeep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition
Ordóñez, F., & Roggen, D. (2016) · Sensors · doi:10.3390/s16010115 · Cited by 2,761 · cited 5× · see citing works
“To ensure fair comparisons, we utilize the same values used in the literature [5], where sensor time series are divided into 500-ms windows that have an 50% overlap rate.”
BackgroundMethodInfluentialDeep learning for sensor-based activity recognition: A survey
Wang, J., Chen, Y., Hao, S., Peng, X., & Hu, L. (2019) · Pattern Recognition Letters · doi:10.1016/j.patrec.2018.02.010 · Cited by 2,014 · cited 2× · see citing works
“A. Motivation D UE to the rapid progress of mobile sensing and ubiquitous computing, Human activity recognition (HAR) using inertial sensors has seen wide applications in various fields [1], [2], such as activity tracking, health-care, and human–computer interaction.”
BackgroundIntroducing a New Benchmarked Dataset for Activity Monitoring
Reiss, A., & Stricker, D. (2012) · 2012 16th International Symposium on Wearable Computers · doi:10.1109/iswc.2012.13 · Cited by 1,456 · cited 2× · see citing works
“The experimental results show an outstanding performance improvement on four HAR benchmarks consisting of UCI-HAR [11], PAMAP2 [12], WISDM [13], and UNIMIB-SHAR [14].”
BackgroundMethod(2015) · Cited by 989 · see citing works
Convolutional Neural Networks for Human Activity Recognition using Mobile Sensors
Zeng, M., Nguyen, L. T., Yu, B., Mengshoel, O. J., Zhu, J., Wu, P., & Zhang, J. (2014) · Proceedings of the 6th International Conference on Mobile Computing, Applications and Services · doi:10.4108/icst.mobicase.2014.257786 · Cited by 876 · see citing works
Real-time human activity recognition from accelerometer data using Convolutional Neural Networks
Ignatov, A. (2017) · Applied Soft Computing · doi:10.1016/j.asoc.2017.09.027 · Cited by 817 · see citing works
The Opportunity challenge: A benchmark database for on-body sensor-based activity recognition
Chavarriaga, R., Sagha, H., Calatroni, A., Digumarti, S. T., Tröster, G., Millán, J. D. R., & Roggen, D. (2013) · Pattern Recognition Letters · doi:10.1016/j.patrec.2012.12.014 · Cited by 742 · see citing works
Deep Learning Models for Real-time Human Activity Recognition with Smartphones
Wan, S., Qi, L., Xu, X., Tong, C., & Gu, Z. (2019) · Mobile Networks and Applications · doi:10.1007/s11036-019-01445-x · Cited by 597 · see citing works
UniMiB SHAR: A Dataset for Human Activity Recognition Using Acceleration Data from Smartphones
Micucci, D., Mobilio, M., & Napoletano, P. (2017) · Applied Sciences · doi:10.3390/app7101101 · Cited by 596 · see citing works
UniMiB SHAR: A Dataset for Human Activity Recognition Using Acceleration Data from Smartphones
Micucci, D., Mobilio, M., & Napoletano, P. (2017) · doi:10.20944/preprints201706.0033.v1 · Cited by 596 · cited 2× · see citing works
“The experimental results show an outstanding performance improvement on four HAR benchmarks consisting of UCI-HAR [11], PAMAP2 [12], WISDM [13], and UNIMIB-SHAR [14].”
BackgroundMethodEnsembles of Deep LSTM Learners for Activity Recognition using Wearables
Guan, Y., & Plötz, T. (2017) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3090076 · Cited by 511 · see citing works
A CNN-LSTM Approach to Human Activity Recognition
Mutegeki, R., & Han, D. S. (2020) · 2020 International Conference on Artificial Intelligence in Information and Communication (ICAIIC) · doi:10.1109/icaiic48513.2020.9065078 · Cited by 452 · see citing works
Comparison of Feature Learning Methods for Human Activity Recognition Using Wearable Sensors
Li, F., Shirahama, K., Nisar, M., Köping, L., & Grzegorzek, M. (2018) · Sensors · doi:10.3390/s18020679 · Cited by 302 · see citing works
Ravi, D., Wong, C., Lo, B., & Yang, G.-Z. (2016) · 2016 IEEE 13th International Conference on Wearable and Implantable Body Sensor Networks (BSN) · doi:10.1109/bsn.2016.7516235 · Cited by 269 · see citing works
AttnSense: Multi-level Attention Mechanism For Multimodal Human Activity Recognition
Ma, H., Li, W., Zhang, X., Gao, S., & Lu, S. (2019) · Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/431 · Cited by 266 · see citing works
Deep Activity Recognition Models with Triaxial Accelerometers
Alsheikh, M. A., Selim, A., Niyato, D. T., Doyle, L., Lin, S., & Tan, H.-P. (2015) · DR-NTU (Nanyang Technological University) · Cited by 191 · see citing works
Activity Recognition Using Temporal Optical Flow Convolutional Features and Multilayer LSTM
Ullah, A., Muhammad, K., Ser, J. D., Baik, S. W., & de Albuquerque, V. H. C. (2018) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2018.2881943 · Cited by 191 · see citing works
Attention induced multi-head convolutional neural network for human activity recognition
Khan, Z. N., & Ahmad, J. (2021) · Applied Soft Computing · doi:10.1016/j.asoc.2021.107671 · Cited by 187 · see citing works
Zeng, M., Gao, H., Yu, T., Mengshoel, O. J., Langseth, H., Lane, I., & Liu, X. (2018) · Proceedings of the 2018 ACM International Symposium on Wearable Computers · doi:10.1145/3267242.3267286 · Cited by 185 · cited 1× · see citing works
“Zeng et al. [18] built a new network using LSTM as backbone, which incorporates two attention ideas to highlight more important components of sensor time series and sensor modalities.”
MethodTeng, Q., Wang, K., Zhang, L., & He, J. (2020) · IEEE Sensors Journal · doi:10.1109/jsen.2020.2978772 · Cited by 183 · see citing works
Self-citationHS-ResNet: Hierarchical-Split Block on Convolutional Neural NetworkPreprint
Yuan, P., Lin, S., Cui, C., Du, Y., Guo, R., He, D., Ding, E., & Han, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.07621 · Cited by 59 · see citing works
Asymmetric Residual Neural Network for Accurate Human Activity Recognition
Long, J., Sun, W., Yang, Z., & Raymond, O. I. (2019) · Information · doi:10.3390/info10060203 · Cited by 56 · see citing works
Wang, K., He, J., & Zhang, L. (2021) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2021.3086008 · Cited by 53 · see citing works
Self-citationEvidential Reasoning With Hesitant Fuzzy Belief Structures for Human Activity Recognition
Dong, Y., Li, X., Dezert, J., Zhou, R., Zhu, C., Wei, L., & Ge, S. S. (2021) · IEEE Transactions on Fuzzy Systems · doi:10.1109/tfuzz.2021.3079495 · Cited by 50 · see citing works
Multi-agent Attentional Activity Recognition
Chen, K., Yao, L., Zhang, D., Guo, B., & Yu, Z. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/186 · Cited by 44 · see citing works
Xia, S., Chu, L., Pei, L., Zhang, Z., Yu, W., & Qiu, R. C. (2021) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2021.3111996 · Cited by 40 · see citing works
Cho, H., & Yoon, S. M. (2019) · Electronics Letters · doi:10.1049/el.2018.6117 · Cited by 32 · see citing works
Weighted Support Tensor Machines for Human Activity Recognition With Smartphone Sensors
Ma, Z., Yang, L. T., Lin, M., et al. (2021) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2021.3061559 · Cited by 21 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
TS2ACT: Few-Shot Human Activity Sensing with Cross-Modal Co-Learning
Xia, K., Li, W., Gan, S., & Lu, S. (2023) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3631445 · Cited by 19 · see citing works
Guo, P., & Nakayama, M. (2025) · Sensors · doi:10.3390/s25226988 · Cited by 6 · see citing works
- Authors
- Tang, Y., Zhang, L., Min, F., & He, J.
- Affiliations
- Nanjing Normal University (CN)
- Nanjing University of Information Science and Technology (CN)
- Venue
- IEEE Transactions on Industrial Electronics
- Published
- 2023-02
- Volume
- 70
- Issue
- 2
- Pages
- 2106-2116
- ISSN
- 0278-0046 (ISSN-L)
- 1557-9948
- OpenAlex
- W4286681770
- Cited by
- 224
- References
- 34
- Citations by year
- 2026: 30 · 2025: 54 · 2024: 68 · 2023: 54 · 2022: 18
- FWCI
- 15.88
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Transactions on Industrial Electronics (publishedVersion)
- Funders
- National Natural Science Foundation of China (61971228)
- Natural Science Foundation of Jiangsu Province (BK20191371)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Physical Sciences › Engineering › Biomedical Engineering › Gait Recognition and Analysis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Keywords
- human activity recognition, wearable sensors, multiscale feature representation, convolutional neural networks, receptive fields, weakly labeled data, UCI-HAR, PAMAP2, WISDM
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
