A Novel Multi-Stage Training Approach for Human Activity Recognition From Multimodal Wearable Sensor Data Using Deep Neural Network
Open accessAbstract
Deep neural network is an effective choice to automatically recognize human actions utilizing data from various wearable sensors. These networks automate the process of feature extraction relying completely on data. However, various noises in time series data with complex inter-modal relationships among sensors make this process more complicated. In this article, we have proposed a novel multi-stage training approach that increases diversity in this feature extraction process to make accurate recognition of actions by combining varieties of features extracted from diverse perspectives. Initially, instead of using single type of transformation, numerous transformations are employed on time series data to obtain variegated representations of the features encoded in raw data. An efficient deep CNN architecture is proposed that can be individually trained to extract features from different transformed spaces. Later, these CNN feature extractors are merged into an optimal architecture finely tuned for optimizing diversified extracted features through a combined training stage or multiple sequential training stages. This approach offers the opportunity to explore the encoded features in raw sensor data utilizing multifarious observation windows with immense scope for efficient selection of features for final convergence. Extensive experimentations have been carried out in three publicly available datasets that provide outstanding performance consistently with average five-fold cross-validation accuracy of 99.29% on UCI HAR database, 99.02% on USC HAR database, and 97.21% on SKODA database outperforming other state-of-the-art approaches.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Only a few multimodal-based multi-stage detection frameworks have been developed and have achieved outstanding detection performance on benchmark datasets.
“More recently, only a few multimodal-based multi-stage detection frameworks [191–193] have been developed and have achieved outstanding detection performance on benchmark datasets.”
From A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets (2022) · cites this paper as [193]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Part of this paper’s reference list could not be split into single entries.
Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition
Ordóñez, F., & Roggen, D. (2016) · Sensors · doi:10.3390/s16010115 · Cited by 2,761 · see citing works
(2013) · RECERCAT (Consorci de Serveis Universitaris de Catalunya) · Cited by 1,959 · see citing works
Mallat, S. (2011) · arXiv.org · doi:10.1002/cpa.21413 · Cited by 1,176 · see citing works
Um, T. T., Pfister, F. M. J., Pichler, D., Endo, S., Lang, M., Hirche, S., Fietzek, U., & Kulić, D. (2017) · Proceedings of the 19th ACM International Conference on Multimodal Interaction · doi:10.1145/3136755.3136817 · Cited by 729 · see citing works
Human Activity Recognition Using Wearable Sensors by Deep Convolutional Neural Networks
Jiang, W., & Yin, Z. (2015) · Proceedings of the 23rd ACM international conference on Multimedia · doi:10.1145/2733373.2806333 · Cited by 676 · see citing works
Um, T. T. (n.d.) · mediaTUM – the media and publications repository of the Technical University Munich (Technical University Munich) · Cited by 602 · see citing works
USC-HAD: a daily activity dataset for ubiquitous activity recognition using wearable sensors
Zhang, M., & Sawchuk, A. A. (2012) · Proceedings of the 2012 ACM Conference on Ubiquitous Computing · doi:10.1145/2370216.2370438 · Cited by 554 · see citing works
Deep Recurrent Neural Networks for Human Activity Recognition
Murad, A., & Pyun, J.-Y. (2017) · Sensors · doi:10.3390/s17112556 · Cited by 524 · see citing works
Ensembles of Deep LSTM Learners for Activity Recognition using Wearables
Guan, Y., & Plötz, T. (2017) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3090076 · Cited by 511 · see citing works
Bianchi, V., Bassoli, M., Lombardo, G., Fornacciari, P., Mordonini, M., & De Munari, I. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2019.2920283 · Cited by 466 · see citing works
Robust human activity recognition from depth video using spatiotemporal multi-fused features
Jalal, A., Kim, Y.-H., Kim, Y.-J., Kamal, S., & Kim, D. (2016) · Pattern Recognition · doi:10.1016/j.patcog.2016.08.003 · Cited by 392 · see citing works
Wang, Z., & Oates, T. (2015) · National Conference on Artificial Intelligence · Cited by 382 · see citing works
A Deep Learning Approach to Human Activity Recognition Based on Single Accelerometer
Chen, Y., & Xue, Y. (2015) · 2015 IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc.2015.263 · Cited by 378 · see citing works
Classification of time-series images using deep convolutional neural networks
Debayle, J., Hatami, N., & Gavet, Y. (2018) · International Conference on Machine Vision · doi:10.1117/12.2309486 · Cited by 374 · see citing works
Activity Recognition from On-Body Sensors: Accuracy-Power Trade-Off by Dynamic Sensor Selection
Zappi, P., Lombriser, C., Stiefmeier, T., Farella, E., Roggen, D., Benini, L., & Tröster, G. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-77690-1_2 · Cited by 329 · see citing works
Ravi, D., Wong, C., Lo, B., & Yang, G.-Z. (2016) · 2016 IEEE 13th International Conference on Wearable and Implantable Body Sensor Networks (BSN) · doi:10.1109/bsn.2016.7516235 · Cited by 269 · see citing works
On the use of ensemble of classifiers for accelerometer-based activity recognition
Catal, C., Tufekci, S., Pirmit, E., & Kocabag, G. (2015) · Applied Soft Computing · doi:10.1016/j.asoc.2015.01.025 · Cited by 240 · see citing works
Human Activity Classification in Smartphones Using Accelerometer and Gyroscope Sensors
Jain, A., & Kanhangad, V. (2017) · IEEE Sensors Journal · doi:10.1109/jsen.2017.2782492 · Cited by 217 · see citing works
Chung, S., Lim, J., Noh, K. J., Kim, G., & Jeong, H. (2019) · Sensors · doi:10.3390/s19071716 · Cited by 215 · see citing works
Deep Activity Recognition Models with Triaxial Accelerometers
Alsheikh, M. A., Selim, A., Niyato, D. T., Doyle, L., Lin, S., & Tan, H.-P. (2015) · DR-NTU (Nanyang Technological University) · Cited by 191 · see citing works
Human Physical Activity Recognition Using Smartphone Sensors
Voicu, R.-A., Dobre, C., Băjenaru, L., & Ciobanu, R. (2019) · Sensors · doi:10.3390/s19030458 · Cited by 173 · see citing works
Seto, S., Zhang, W., & Zhou, Y. (2015) · IEEE Symposium Series on Computational Intelligence (SSCI) · doi:10.1109/ssci.2015.199 · Cited by 163 · see citing works
A Hybrid Deep Learning Model for Human Activity Recognition Using Multimodal Body Sensing Data
Gumaei, A., Hassan, M. M., Alelaiwi, A., & Alsalman, H. (2019) · IEEE Access · doi:10.1109/access.2019.2927134 · Cited by 157 · see citing works
Islam, N., Faheem, Y., Din, I. U., Talha, M., Guizani, M., & Khalil, M. (2019) · Future Generation Computer Systems · doi:10.1016/j.future.2019.05.059 · Cited by 154 · see citing works
Deep Activity Recognition Models with Triaxial Accelerometers
Alsheikh, M. A., Selim, A., Niyato, D., Doyle, L., Lin, S., & Tan, H.-P. (2015) · arXiv · doi:10.48550/arxiv.1511.04664 · Cited by 126 · see citing works
Smartphone-Based Activity Recognition for Indoor Localization Using a Convolutional Neural Network
Zhou, B., Yang, J., & Li, Q. (2019) · Sensors · doi:10.3390/s19030621 · Cited by 88 · see citing works
Wearable Computing for Internet of Things: A Discriminant Approach for Human Activity Recognition
Lu, W., Fan, F., Chu, J., Jing, P., & Su, Y. (2018) · IEEE Internet of Things Journal · doi:10.1109/jiot.2018.2873594 · Cited by 82 · see citing works
Lv, M., Xu, W., & Chen, T. (2019) · Neurocomputing · doi:10.1016/j.neucom.2019.06.051 · Cited by 64 · see citing works
Human Activity Recognition Based on the Hierarchical Feature Selection and Classification Framework
Zheng, Y. (2015) · Journal of Electrical and Computer Engineering · doi:10.1155/2015/140820 · Cited by 47 · see citing works
Seto, S., Zhang, W., & Zhou, Y. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1512.06747 · Cited by 31 · see citing works
Yu, H., Pan, G., Pan, M., Li, C., Jia, W., Zhang, L., & Sun, M. (2019) · Sensors · doi:10.3390/s19030546 · Cited by 21 · see citing works
PEMAR: A pervasive middleware for activity recognition with smart phones
Vaka, P., Shen, F., Chandrashekar, M., & Lee, Y. (2015) · 2015 IEEE International Conference on Pervasive Computing and Communication Workshops (PerCom Workshops) · doi:10.1109/percomw.2015.7134073 · Cited by 16 · see citing works
Human activity recognition in cognitive environments using sequential ELM
Chandan Kumar, R., Bharadwaj, S. S., Sumukha, B. N., & George, K. (2016) · 2016 Second International Conference on Cognitive Computing and Information Processing (CCIP) · doi:10.1109/ccip.2016.7802880 · Cited by 8 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Bayoudh, K., Knani, R., Hamdaoui, F., & Mtibaa, A. (2022) · The Visual Computer · doi:10.1007/s00371-021-02166-7 · Cited by 456 · see citing works
Machine Learning and AI Technologies for Smart Wearables
Seng, K. P., Ang, L.-M., Peter, E., & Mmonyi, A. (2023) · Electronics · doi:10.3390/electronics12071509 · Cited by 90 · see citing works
Hoai Thu, N. T., & Han, D. S. (2021) · IEEE Access · doi:10.1109/access.2021.3122298 · Cited by 54 · see citing works
Cross-Sensor Masked Self-Supervised Transformers for Domain-Adaptive Human Activity Recognition
Karim, M. M., Kabir, S., & Fattah, S. A. (2026) · 2026 IEEE Region 10 Symposium (TENSYMP) · doi:10.1109/tensymp68909.2026.11704026
Self-citation
- Authors
- Mahmud, T., Sazzad Sayyed, A. Q. M., Fattah, S. A., & Kung, S.-Y.
- Affiliations
- Bangladesh University of Engineering and Technology (BD)
- Princeton University (US)
- Venue
- IEEE Sensors Journal
- Published
- 2021-01-15
- Volume
- 21
- Issue
- 2
- Pages
- 1715-1726
- ISSN
- 1530-437X (ISSN-L)
- 1558-1748
- 2379-9153
- OpenAlex
- W3048480463
- Cited by
- 58
- References
- 33
- Citations by year
- 2026: 10 · 2025: 5 · 2024: 12 · 2023: 14 · 2022: 10 · 2021: 7
- FWCI
- 3.01
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Sensors Journal (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Physical Sciences › Computer Science › Signal Processing › Time Series Analysis and Forecasting
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Keywords
- human activity recognition, wearable sensors, multimodal sensor data, multi-stage training, feature extraction, time series transformation, feature fusion, UCI-HAR, Skoda
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
