Harmonic Loss Function for Sensor-Based Human Activity Recognition Based on LSTM Recurrent Neural Networks
Open accessAbstract
Human activity recognition (HAR) has been a very popular field in both real practice and theoretical research. Over the years, a number of many-vs-one Long Short-Term Memory (LSTM) models have been proposed for the sensor-based HAR problem. However, how to utilize sequence outputs of them to improve the HAR performance has not been studied seriously. To solve this problem, we present a novel loss function named harmonic loss, which is utilized to improve the overall classification performance of HAR based on baseline LSTM networks. First, label replication method is presented to duplicate true labels at each sequence step in many-vs-one LSTM networks, thus each sequence step can generate a local error and a local output. Then, considering the importance of different local errors and inspired by the Ebbinghaus memory curve, the harmonic loss is proposed to give unequal weights to different local errors based on harmonic series equation. Additionally, to improve the overall classification performance of HAR, integrated methods are utilized to exploit the sequence outputs of LSTM models based on harmonic loss and ensemble learning strategy. Finally, based on the LSTM model construction and hyper-parameter setting, extensive experiments are conducted. A series of experimental results demonstrate that our harmonic loss significantly achieves higher macro-F1 and accuracy than strong baselines on two public HAR benchmarks. Compared with previous state-of-art methods, our proposed methods can achieve competitive classification performance.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
[200] proposes a harmonic loss function and [207] combines LSTM with batch normalization to achieve 92% accuracy with raw accelerometer and gyroscope data.
“For example, [200] proposes a harmonic loss function and [207] combines LSTM with batch normalization to achieve 92% accuracy with raw accelerometer and gyroscope data.”
From Deep Learning in Human Activity Recognition with Wearable Sensors: A Review on Advances (2022) · cites this paper as [200]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Focal Loss for Dense Object Detection
Lin, T.-Y., Goyal, P., Girshick, R., He, K., & Dollar, P. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.324 · Cited by 30,581 · see citing works
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545 · see citing works
Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1406.1078 · Cited by 3,317 · see citing works
Recent Trends in Deep Learning Based Natural Language Processing [Review Article]
Young, T., Hazarika, D., Poria, S., & Cambria, E. (2018) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2018.2840738 · Cited by 2,921 · see citing works
Activity recognition using cell phone accelerometers
Kwapisz, J. R., Weiss, G. M., & Moore, S. A. (2011) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1964897.1964918 · Cited by 2,912 · cited 1× · see citing works
“Public datasets refer to researchers performed work on public HAR datasets, such as Opportunity dataset [17], public domain UCI dataset [10], [39], and WISDM dataset [40].”
MethodDeep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition
Ordóñez, F., & Roggen, D. (2016) · Sensors · doi:10.3390/s16010115 · Cited by 2,761 · cited 2× · see citing works
“…(AI) technology [16], massive deep learning methods including forward neural networks (FNNs) [17], [18], recurrent neural networks (RNNs) [19]–[22], autoencoder (AE) [23], and convolutional neural networks (CNNs) [18], [24]–[27] have been proposed to recognize different types of human activities.”
BackgroundMethodDeep Clustering for Unsupervised Learning of Visual Features
Caron, M., Bojanowski, P., Joulin, A., & Douze, M. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01264-9_9 · Cited by 2,674 · see citing works
Unsupervised Representation Learning by Predicting Image Rotations
Gidaris, S., Singh, P., & Komodakis, N. (2018) · arXiv · doi:10.48550/arxiv.1803.07728 · Cited by 2,477 · see citing works
Beyond short snippets: Deep networks for video classification
Ng, J. Y.-H., Hausknecht, M., Vijayanarasimhan, S., Vinyals, O., Monga, R., & Toderici, G. D. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7299101 · Cited by 2,121 · see citing works
Deep learning for sensor-based activity recognition: A survey
Wang, J., Chen, Y., Hao, S., Peng, X., & Hu, L. (2019) · Pattern Recognition Letters · doi:10.1016/j.patrec.2018.02.010 · Cited by 2,014 · see citing works
(2013) · RECERCAT (Consorci de Serveis Universitaris de Catalunya) · Cited by 1,959 · see citing works
Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels
Zhang, Z., & Sabuncu, M. R. (2018) · arXiv · doi:10.48550/arxiv.1805.07836 · Cited by 1,907 · see citing works
Deeply-Supervised NetsPreprint
Lee, C., Xie, S., Gallagher, P., Zhang, Z., & Tu, Z. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1409.5185 · Cited by 1,661 · see citing works
Human activity recognition with smartphone sensors using deep learning neural networks
Ronao, C. A., & Cho, S.-B. (2016) · Expert Systems with Applications · doi:10.1016/j.eswa.2016.04.032 · Cited by 1,215 · see citing works
(2015) · Cited by 989 · see citing works
Nweke, H. F., Teh, Y. W., Al-garadi, M. A., & Alo, U. R. (2018) · Expert Systems with Applications · doi:10.1016/j.eswa.2018.03.056 · Cited by 930 · see citing works
Learning to Diagnose with LSTM Recurrent Neural Networks
Lipton, Z. C., Kale, D. C., Elkan, C., & Wetzel, R. (2015) · arXiv · doi:10.48550/arxiv.1511.03677 · Cited by 879 · see citing works
Real-time human activity recognition from accelerometer data using Convolutional Neural Networks
Ignatov, A. (2017) · Applied Soft Computing · doi:10.1016/j.asoc.2017.09.027 · Cited by 817 · see citing works
A robust human activity recognition system using smartphone sensors and deep learning
Hassan, M. M., Uddin, M. Z., Mohamed, A. M. S., & Almogren, A. (2017) · Future Generation Computer Systems · doi:10.1016/j.future.2017.11.029 · Cited by 659 · see citing works
Deep, Convolutional, and Recurrent Models for Human Activity Recognition Using Wearables
Hammerla, N. Y., Halloran, S., & Ploetz, T. (2016) · arXiv · doi:10.48550/arxiv.1604.08880 · Cited by 606 · see citing works
Recent Advances in Recurrent Neural NetworksPreprint
Salehinejad, H., Sankar, S., Barfett, J., Colak, E., & Valaee, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.01078 · Cited by 584 · see citing works
A Study on Human Activity Recognition Using Accelerometer Data from Smartphones
Bayat, A., Pomplun, M., & Tran, D. A. (2014) · Procedia Computer Science · doi:10.1016/j.procs.2014.07.009 · Cited by 531 · see citing works
Deep Recurrent Neural Networks for Human Activity Recognition
Murad, A., & Pyun, J.-Y. (2017) · Sensors · doi:10.3390/s17112556 · Cited by 524 · see citing works
Ensembles of Deep LSTM Learners for Activity Recognition using Wearables
Guan, Y., & Plötz, T. (2017) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3090076 · Cited by 511 · see citing works
A Deep Learning Approach to Human Activity Recognition Based on Single Accelerometer
Chen, Y., & Xue, Y. (2015) · 2015 IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc.2015.263 · Cited by 378 · see citing works
Deep Residual Bidir-LSTM for Human Activity Recognition Using Wearable Sensors
Zhao, Y., Yang, R., Chevalier, G., Xu, X., & Zhang, Z. (2018) · Mathematical Problems in Engineering · doi:10.1155/2018/7316954 · Cited by 347 · see citing works
Structured AutoEncoders for Subspace Clustering
Peng, X., Feng, J., Xiao, S., Yau, W., Zhou, J. T., & Yang, S. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2848470 · Cited by 345 · see citing works
Human Activity Recognition from Accelerometer Data Using a Wearable Device
Casale, P., Pujol, O., & Radeva, P. I. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-21257-4_36 · Cited by 341 · see citing works
A Comparative Study on Human Activity Recognition Using Inertial Sensors in a Smartphone
Wang, A., Chen, G., Yang, J., Zhao, S., & Chang, C.-Y. (2016) · IEEE Sensors Journal · doi:10.1109/jsen.2016.2545708 · Cited by 324 · see citing works
Bidirectional RNN for Medical Event Detection in Electronic Health Records
Jagannatha, A. N., & Yu, H. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/n16-1056 · Cited by 314 · see citing works
Real-Time Prediction of Taxi Demand Using Recurrent Neural Networks
Xu, J., Rahmatizadeh, R., Bölöni, L. L., & Turgut, D. (2017) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2017.2755684 · Cited by 301 · see citing works
Recent trends in machine learning for human activity recognition—A survey
Ramamurthy, S. R., & Roy, N. (2018) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.1254 · Cited by 301 · see citing works
Deep Computational Phenotyping
Che, Z., Kale, D. C., Li, W., Bahadori, M. T., & Liu, Y. (2015) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2783258.2783365 · Cited by 247 · see citing works
Unsupervised learning for human activity recognition using smartphone sensors
Kwon, Y., Kang, K., & Bae, C. (2014) · Expert Systems with Applications · doi:10.1016/j.eswa.2014.04.037 · Cited by 207 · see citing works
JUCS - Journal of Universal Computer Science
:none (2020) · CINECA IRIS Institutial research information system (University of Pisa) · doi:10.3217/jucs-019-09 · Cited by 188 · see citing works
From smart to deep: Robust activity recognition on smartwatches using deep learning
Bhattacharya, S., & Lane, N. D. (2016) · IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops) · doi:10.1109/percomw.2016.7457169 · Cited by 175 · see citing works
A Hybrid CNN–LSTM Network for the Classification of Human Activities Based on Micro-Doppler Radar
Zhu, J., Chen, H., & Ye, W. (2020) · IEEE Access · doi:10.1109/access.2020.2971064 · Cited by 168 · see citing works
Deep Clustering With Sample-Assignment Invariance Prior
Peng, X., Zhu, H., Feng, J., Shen, C., Zhang, H., & Zhou, J. T. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2958324 · Cited by 156 · see citing works
Regression and classification using extreme learning machine based on L1-norm and L2-norm
Luo, X., Chang, X., & Ban, X. (2015) · Neurocomputing · doi:10.1016/j.neucom.2015.03.112 · Cited by 147 · see citing works
A Review on Deep Learning Approaches to Image Classification and Object Segmentation
Wu, H., Liu, Q., & Liu, X. (2019) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2019.03595 · Cited by 136 · see citing works
Emulating the Ebbinghaus forgetting curve of the human brain with a NiO-based memristor
Hu, S., Liu, Y., Chen, T. P., Liu, Z., Yu, Q., Deng, L. J., Yin, Y., & Hosaka, S. (2013) · Applied Physics Letters · doi:10.1063/1.4822124 · Cited by 132 · see citing works
Antar, A. D., Ahmed, M., & Ahad, M. A. R. (2019) · International Conference on Informatics, Electronics & Vision (ICIEV) · doi:10.1109/iciev.2019.8858508 · Cited by 111 · see citing works
Benchmarking classification techniques using the Opportunity human activity dataset
Sagha, H., Digumarti, S. T., Millán, J. D. R., et al. (2011) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/icsmc.2011.6083628 · Cited by 110 · see citing works
Recognition of Human Activities Using Continuous Autoencoders with Wearable Sensors
Wang, L. (2016) · Sensors · doi:10.3390/s16020189 · Cited by 87 · see citing works
Activity Recognition with Evolving Data Streams
Abdallah, Z. S., Gaber, M. M., Srinivasan, B., & Krishnaswamy, S. P. (2018) · ACM Computing Surveys · doi:10.1145/3158645 · Cited by 85 · see citing works
ACTIVITY RECOGNITION USING K-NEAREST NEIGHBOR ALGORITHM ON SMARTPHONE WITH TRI-AXIAL ACCELEROMETER
Kaghyan, S., & Sarukhanyan, H. G. (2012) · Cited by 80 · see citing works
Outlier Detection in Wearable Sensor Data for Human Activity Recognition (HAR) Based on DRNNs
Muñoz-Organero, M. (2019) · IEEE Access · doi:10.1109/access.2019.2921096 · Cited by 70 · see citing works
Dynamic Computation Offloading for Low-Power Wearable Health Monitoring Systems
Kalantarian, H., Sideris, C., Mortazavi, B. J., Alshurafa, N. I., & Sarrafzadeh, M. (2016) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2016.2570210 · Cited by 63 · see citing works
Anomaly detection in traffic using L1-norm minimization extreme learning machine
Wang, Y., Li, D., Du, Y., & Pan, Z. (2014) · Neurocomputing · doi:10.1016/j.neucom.2014.04.073 · Cited by 61 · see citing works
An integrated framework for human activity classification
Cao, H., Nguyen, M. N., Phua, C., et al. (2012) · doi:10.1145/2370216.2370268 · Cited by 49 · see citing works
A Novel Deep Neural Network Model for Multi-Label Chronic Disease Prediction
Zhang, X., Zhao, H., Zhang, S., & Li, R. (2019) · Frontiers in Genetics · doi:10.3389/fgene.2019.00351 · Cited by 45 · see citing works
Self-citationRonaoo, C. A., & Cho, S. (2015) · 한국정보과학회 학술발표논문집 · Cited by 43 · see citing works
Chen, K., Yao, L., Wang, X., Zhang, D., Gu, T., Yu, Z., & Yang, Z. (2018) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2018.8489767 · Cited by 36 · see citing works
Okai, J., Paraschiakos, S., Beekman, M., Knobbe, A., & de Sa, C. R. (2019) · 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc.2019.8857288 · Cited by 29 · see citing works
Hashtag Recommendation Using LSTM Networks with Self-Attention
Shen, Y., Li, Y., Sun, J., et al. (2019) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2019.06104 · Cited by 22 · see citing works
Adaptive Scaling for Sparse Detection in Information Extraction
Lin, H., Lu, Y., Han, X., et al. (2018) · Annual Meeting of the Association for Computational Linguistics (ACL) · doi:10.18653/v1/p18-1095 · Cited by 13 · see citing works
Evolutionary Design of Recurrent Neural Network Architecture for Human Activity Recognition
Viswambaran, R. A., Chen, G., Xue, B., & Nekooei, M. (2019) · IEEE Congress on Evolutionary Computation · doi:10.1109/cec.2019.8790050 · Cited by 12 · see citing works
SECERBEGOVIC, A., Gogić, A., Suljanović, N., et al. (2018) · Advances in Electrical and Computer Engineering · doi:10.4316/aece.2018.04001 · Cited by 11 · see citing works
Detecting Domain Generation Algorithms with Bi-LSTM
Ding, L., Li, L., Han, J., et al. (2019) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2019.06160 · Cited by 7 · see citing works
Flagged first, then by citations
Research works printed but not linked: 5
- D. Anguita, A. Ghio, L. Oneto, X. Parra, and J. R. Reyes-Ortiz, ''Energy efficient smartphone-based activity recognition using fixed-point arith- metic,'' J. Universal Comput. Sci., vol. 19, no. 9, pp. 1295-1314, 2013.
- M. MacKay, P. Vicol, J. Ba, and R. B. Grosse, ''Reversible recurrent neural networks,'' in Proc. Adv. Neural Inf. Process. Syst., 2018, pp. 9029-9040.
- E. Choi, M. T. Bahadori, A. Schuetz, W. F. Stewart, and J. Sun, ''Doctor AI: Predicting clinical events via recurrent neural networks,'' in Proc. Mach. Learn. Healthcare Conf., 2016, pp. 301-318.
- H.-Y. Chen, P.-H. Wang, C.-H. Liu, S.-C. Chang, J.-Y. Pan, Y.-T. Chen, W. Wei, and D.-C. Juan, ''Complement objective training,'' in Proc. Int. Conf. Learn. Represent., 2019, pp. 1-11. [Online]. Available: https://openreview.net/forum?id=HyM7AiA5YX
- WEI SHE (Member, IEEE) received the B.S. degree in control engineering from the Air Defence Academy of PLA, in 2000, the M.S. degree in software engineering from Hunan Uni- versity, in 2008, and the Ph.D. degree in computer software and theory from Zhengzhou University, China, in 2013. He is currently an Associate Pro- fessor with the College of Software, Zhengzhou University. His research interests include informa- tion security, the energy Internet, and the Internet healthcare.
Cited by
Sorted by citations
Deep Learning in Human Activity Recognition with Wearable Sensors: A Review on Advances
Zhang, S., Li, Y., Zhang, S., Shahabi, F., Xia, S., Deng, Y., & Alshurafa, N. (2022) · Sensors · doi:10.3390/s22041476 · Cited by 625 · see citing works
“The GRU introduces a reset and update gate to control the flow of inputs to a cell [78, 129, 135, 206, 342].”
BackgroundMethod
- Authors
- Hu, Y., Zhang, X.-Q., Xu, L., Xian He, F., Tian, Z., She, W., & Liu, W.
- Affiliations
- Zhengzhou University (CN)
- Venue
- IEEE Access
- Published
- 2020
- Volume
- 8
- Pages
- 135617-135627
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W3045503191
- Cited by
- 36
- References
- 59
- Citations by year
- 2026: 2 · 2025: 5 · 2024: 6 · 2023: 9 · 2022: 9 · 2021: 4 · 2020: 1
- FWCI
- 2.24
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- public-domain
- creativecommons.org/licenses/by/4.0/legalcode (vor)
- Locations
- IEEE Access (publishedVersion, public-domain)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Open Project Fund of the Research Center on Levee Safety Disaster Prevention (2018007)
- National Basic Research Program of China (2018YFB1201403)
- CERNET Innovation Project (NGII20180702)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Physical Sciences › Computer Science › Signal Processing › Time Series Analysis and Forecasting
- Physical Sciences › Engineering › Biomedical Engineering › Gait Recognition and Analysis
- Keywords
- human activity recognition, sensor-based activity recognition, ensemble learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
