The DOI matches its Crossref record; 49 of the 49 references it cites were found in a registry or an index.
LSTM Networks Using Smartphone Data for Sensor-Based Human Activity Recognition in Smart Homes
Open accessAbstract
Human Activity Recognition (HAR) employing inertial motion data has gained considerable momentum in recent years, both in research and industrial applications. From the abstract perspective, this has been driven by an acceleration in the building of intelligent and smart environments and systems that cover all aspects of human life including healthcare, sports, manufacturing, commerce, etc. Such environments and systems necessitate and subsume activity recognition, aimed at recognizing the actions, characteristics, and goals of one or more individuals from a temporal series of observations streamed from one or more sensors. Due to the reliance of conventional Machine Learning (ML) techniques on handcrafted features in the extraction process, current research suggests that deep-learning approaches are more applicable to automated feature extraction from raw sensor data. In this work, the generic HAR framework for smartphone sensor data is proposed, based on Long Short-Term Memory (LSTM) networks for time-series domains. Four baseline LSTM networks are comparatively studied to analyze the impact of using different kinds of smartphone sensor data. In addition, a hybrid LSTM network called 4-layer CNN-LSTM is proposed to improve recognition performance. The HAR method is evaluated on a public smartphone-based dataset of UCI-HAR through various combinations of sample generation processes (OW and NOW) and validation protocols (10-fold and LOSO cross validation). Moreover, Bayesian optimization techniques are used in this study since they are advantageous for tuning the hyperparameters of each LSTM network. The experimental results indicate that the proposed 4-layer CNN-LSTM network performs well in activity recognition, enhancing the average accuracy by up to 2.24% compared to prior state-of-the-art approaches.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
The Elements of Statistical Learning
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7 · Cited by 25,970
TensorFlow: A system for large-scale machine learning
Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.08695 · Cited by 11,822
Learning long-term dependencies with gradient descent is difficult
Bengio, Y., Simard, P., & Frasconi, P. (1994) · IEEE Transactions on Neural Networks · doi:10.1109/72.279181 · Cited by 8,542 · cited 1×
“[41] inform that RNN networks can recognize the data for only a moment, owing to the dissolving gradient issue.”
BackgroundA survey of cross-validation procedures for model selection
Arlot, S., & Célisse, A. (2010) · LillOA (Université de Lille (University Of Lille)) · Cited by 3,669
A survey of cross-validation procedures for model selection
Arlot, S., & Celisse, A. (2010) · Statistics Surveys · doi:10.1214/09-ss054 · Cited by 3,522
Activity Recognition from User-Annotated Acceleration Data
Bao, L., & Intille, S. S. (2004) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-24646-6_1 · Cited by 3,147
Activity recognition using cell phone accelerometers
Kwapisz, J. R., Weiss, G. M., & Moore, S. A. (2011) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1964897.1964918 · Cited by 2,912
Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition
Ordóñez, F., & Roggen, D. (2016) · Sensors · doi:10.3390/s16010115 · Cited by 2,761 · cited 1×
“To provide promising results in recognition performance, Ordóñez and Roggen [32] combined the convolutional layer with LSTM layers.”
BackgroundDeep learning for sensor-based activity recognition: A survey
Wang, J., Chen, Y., Hao, S., Peng, X., & Hu, L. (2019) · Pattern Recognition Letters · doi:10.1016/j.patrec.2018.02.010 · Cited by 2,014 · cited 2×
“When applied to time-series classification-related HAR, the CNN has superiority over other conventional ML approaches, due to its local dependency and scale invariance [26].”
BackgroundMethodA tutorial on human activity recognition using body-worn inertial sensors
Bulling, A., Blanke, U., & Schiele, B. (2014) · ACM Computing Surveys · doi:10.1145/2499621 · Cited by 1,668
Karantonis, D. M., Narayanan, M. R., Mathie, M., Lovell, N. H., & Celler, B. G. (2006) · IEEE Transactions on Information Technology in Biomedicine · doi:10.1109/titb.2005.856864 · Cited by 1,236
Real-time human activity recognition from accelerometer data using Convolutional Neural Networks
Ignatov, A. (2018) · Applied Soft Computing · doi:10.1016/j.asoc.2017.09.027 · Cited by 814
Sensor-based and vision-based human activity recognition: A comprehensive survey
Minh Dang, L., Min, K., Wang, H., Jalil Piran, M., Hee Lee, C., & Moon, H. (2020) · Pattern Recognition · doi:10.1016/j.patcog.2020.107561 · Cited by 730 · cited 1×
“The HAR methods can be categorized into two categories according to data sources: visual-based and sensor-based [12].”
MethodA CNN-LSTM Approach to Human Activity Recognition
Mutegeki, R., & Han, D. S. (2020) · 2020 International Conference on Artificial Intelligence in Information and Communication (ICAIIC) · doi:10.1109/icaiic48513.2020.9065078 · Cited by 452 · cited 4×
“Lee [25] 1D-CNN 92.71% 2017 Hernandez [45] Bidir-LSTM 92.67% 2019 Mutegeki [31] CNN-LSTM 92.13% 2020 Ni [54] SDAE 97.15% 2020 The proposed approach 4-layer CNN-LSTM 99.39% -”
MethodInfluentialThe need to approximate the use-case in clinical machine learning
Saeb, S., Lonini, L., Jayaraman, A., Mohr, D. C., & Kording, K. P. (2017) · GigaScience · doi:10.1093/gigascience/gix019 · Cited by 367 · cited 2×
“Various approaches, such as leave-one-out, leave-p-out, and k-fold cross validation, can be used to separate the data for training and testing [49,50].”
BackgroundMethodDeep Residual Bidir-LSTM for Human Activity Recognition Using Wearable Sensors
Zhao, Y., Yang, R., Chevalier, G., Xu, X., & Zhang, Z. (2018) · Mathematical Problems in Engineering · doi:10.1155/2018/7316954 · Cited by 348
Human activity recognition from accelerometer data using Convolutional Neural Network
Lee, S.-M., Yoon, S. M., & Cho, H. (2017) · 2017 IEEE International Conference on Big Data and Smart Computing (BigComp) · doi:10.1109/bigcomp.2017.7881728 · Cited by 331
A Review on Human Activity Recognition Using Vision-Based Method
Zhang, S., Wei, Z., Nie, J., Huang, L., Wang, S., & Li, Z. (2017) · Journal of Healthcare Engineering · doi:10.1155/2017/3090343 · Cited by 328
Recent trends in machine learning for human activity recognition—A survey
Ramasamy Ramamurthy, S., & Roy, N. (2018) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1254 · Cited by 299
Human Activity Recognition: A Survey
Jobanputra, C., Bavishi, J., & Doshi, N. (2019) · Procedia Computer Science · doi:10.1016/j.procs.2019.08.100 · Cited by 274
Stride and cadence as a biometric in automatic person identification and verification
BenAbdelkader, C., Cutler, R., & Davis, L. (2003) · Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition · doi:10.1109/afgr.2002.1004182 · Cited by 252
A review and categorization of techniques on device-free human activity recognition
Hussain, Z., Sheng, Q. Z., & Zhang, W. E. (2020) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2020.102738 · Cited by 241
Mekruksavanich, S., & Jitpattanakul, A. (2021) · Electronics · doi:10.3390/electronics10030308 · Cited by 219
Self-citationSensor-Based Datasets for Human Activity Recognition – A Systematic Review of Literature
De-La-Hoz-Franco, E., Ariza-Colpas, P., Quero, J. M., & Espinilla, M. (2018) · IEEE Access · doi:10.1109/access.2018.2873502 · Cited by 209
Deep ConvLSTM With Self-Attention for Human Activity Decoding Using Wearable Sensors
Singh, S. P., Sharma, M. K., Lay-Ekuakille, A., Gangwar, D., & Gupta, S. (2021) · IEEE Sensors Journal · doi:10.1109/jsen.2020.3045135 · Cited by 199
LSTM Networks for Mobile Human Activity Recognition
Chen, Y., Zhong, K., Zhang, J., Sun, Q., & Zhao, X. (2016) · Proceedings of the 2016 International Conference on Artificial Intelligence: Technologies and Applications · doi:10.2991/icaita-16.2016.13 · Cited by 197
JUCS - Journal of Universal Computer Science
:none (2020) · CINECA IRIS Institutial research information system (University of Pisa) · doi:10.3217/jucs-019-09 · Cited by 188
Gyllensten, I. C., & Bonomi, A. G. (2011) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2011.2160723 · Cited by 183
Stacked Lstm Network for Human Activity Recognition Using Smartphone Data
Ullah, M., Ullah, H., Khan, S. D., & Cheikh, F. A. (2019) · European Workshop on Visual Information Processing · doi:10.1109/euvip47703.2019.8946180 · Cited by 143
Qi, J., Yang, P., Hanneghan, M., Tang, S., & Zhou, B. (2018) · IEEE Internet of Things Journal · doi:10.1109/jiot.2018.2846359 · Cited by 120
Human Activity Recognition on Smartphones Using a Bidirectional LSTM Network
Castrillón, F., Suárez, L. F. D., Villamizar, J., & Altuve, M. (2019) · Symposium on Image, Signal Processing and Artificial Vision (STSIVA) · doi:10.1109/stsiva.2019.8730249 · Cited by 108
Enhanced Hand-Oriented Activity Recognition Based on Smartwatch Sensor Data Using LSTMs
Mekruksavanich, S., Jitpattanakul, A., Youplao, P., & Yupapin, P. P. (2020) · Symmetry · doi:10.3390/sym12091570 · Cited by 97
Self-citationBaldominos, A., Cervantes, A., Sáez, Y., & Isasi, P. (2019) · Sensors · doi:10.3390/s19030521 · Cited by 86
Personalized Human Activity Recognition Based on Integrated Wearable Sensor and Transfer Learning
Fu, Z., He, X., Wang, E., Huo, J., Huang, J., & Wu, D. (2021) · Sensors · doi:10.3390/s21030885 · Cited by 79
Smartwatch-based sitting detection with human activity recognition for office workers syndrome
Mekruksavanich, S., Hnoohom, N., & Jitpattanakul, A. (2018) · Joint International Conference on Digital Arts, Media and Technology with ECTI Northern Section Conference on Electrical, Electronics, Computer and Telecommunications Engineering (ECTI DAMT & NCON) · doi:10.1109/ecti-ncon.2018.8378302 · Cited by 66
Self-citationDevice-free single-user activity recognition using diversified deep ensemble learning
Cui, W., Li, B., Zhang, L., & Chen, Z. (2021) · Applied Soft Computing · doi:10.1016/j.asoc.2020.107066 · Cited by 54
Exercise Activity Recognition with Surface Electromyography Sensor using Machine Learning Approach
Mekruksavanich, S., & Jitpattanakul, A. (2020) · Joint International Conference on Digital Arts, Media and Technology with ECTI Northern Section Conference on Electrical, Electronics, Computer and Telecommunications Engineering (ECTI DAMT & NCON) · doi:10.1109/ectidamtncon48261.2020.9090711 · Cited by 53
Self-citationPattern Recognition and Image Analysis
Martínez-Usó, A., Sotoca, J. M., Pla, F., & Anaya-Sánchez, H. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-21257-4 · Cited by 52
Human Activity Recognition Using Triaxial Acceleration Data from Smartphone and Ensemble Learning
Hnoohom, N., Mekruksavanich, S., & Jitpattanakul, A. (2017) · International Conference on Signal-Image Technology & Internet-Based Systems (SITIS) · doi:10.1109/sitis.2017.73 · Cited by 49
Self-citationDeep Learning for Activity Recognition in Older People Using a Pocket-Worn Smartphone
Nan, Y., Lovell, N. H., Redmond, S. J., Wang, K., Delbaere, K., & van Schooten, K. S. (2020) · Sensors · doi:10.3390/s20247195 · Cited by 46
Sargano, A. B., Angelov, P. P., & Habib, Z. (2016) · Applied Sciences · doi:10.3390/app6100309 · Cited by 35
Leveraging Wearable Sensors for Human Daily Activity Recognition with Stacked Denoising Autoencoders
Ni, Q., Fan, Z., Zhang, L., Nugent, C., Cleland, I., Zhang, Y., & Zhou, N. Q. (2020) · Sensors · doi:10.3390/s20185114 · Cited by 33
A robust convolutional neural network for online smartphone-based human activity recognition
Almaslukh, B., Al-Muhtadi, J. F., & Artoli, A. M. (2018) · Journal of Intelligent & Fuzzy Systems · doi:10.3233/jifs-169699 · Cited by 32
Eating detection and chews counting through sensing mastication muscle contraction
Wang, S., Zhou, G., Ma, Y., Hu, L., Chen, Z., Chen, Y., Zhao, H., & Jung, W. (2018) · Smart Health · doi:10.1016/j.smhl.2018.07.004 · Cited by 31
Less Annotation on Personalized Activity Recognition Using Context Data
Hu, L., Chen, Y., Wang, S., Wang, J., Shen, J., Jiang, X., & Shen, Z. (2016) · IEEE Smart World Congress and International Conferences on Ubiquitous Intelligence & Computing, Advanced and Trusted Computing, Scalable Computing and Communications, Cloud and Big Data Computing, Internet of People (SmartWorld/UIC/ATC/ScalCom/CBDCom/IoP) · doi:10.1109/uic-atc-scalcom-cbdcom-iop-smartworld.2016.0066 · Cited by 22
Agac, S., Shoaib, M. A., & İncel, Ö. D. (2021) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2020.106949 · Cited by 19
Zhang, P., Zhang, Z., & Chao, H. (2020) · Sensors · doi:10.3390/s20144016 · Cited by 7
Research works printed but not linked: 9
- Shih, 2018, WuKong: Secure Run-Time environment and data-driven IoT applications for Smart Cities and Smart Buildings, J. Internet Serv. Inf. Secur., 8, 1
- Atapour, 2018, Modeling Advanced Persistent Threats to enhance anomaly detection techniques, J. Wirel. Mob. Netw. Ubiquitous Comput. Dependable Appl., 9, 71
- Park, 2018, Situational Awareness Framework for Threat Intelligence Measurement of Android Malware, J. Wirel. Mob. Netw. Ubiquitous Comput. Dependable Appl., 9, 25
- Kotenko, 2018, Applying Big Data Processing and Machine Learning Methods for Mobile Internet of Things Security Monitoring, J. Internet Serv. Inf. Secur., 8, 54
- Afza, 2021, A framework of human action recognition using length control features fusion and weighted entropy-variances based feature selection, Image Vis. Comput., 106, 104090, 10.1016/j.imavis.2020.104090 · doi:10.1016/j.imavis.2020.104090
- Hussain, Z., Sheng, M., and Zhang, W.E. (2019). Different Approaches for Human Activity Recognition: A Survey. CoRR, abs:1906.05074.
- Hammerla, N.Y., Halloran, S., and Plötz, T. (2016). Deep, Convolutional, and Recurrent Models for Human Activity Recognition Using Wearables, AAAI Press.
- Anguita, 2013, Energy efficient smartphone-based activity recognition using fixed-point arithmetic, J. Univers. Comput. Sci., 19, 1295
- Dewancker, I., McCourt, M., Clark, S., Hayes, P., Johnson, A., and Ke, G. (2016). A Stratified Analysis of Bayesian Optimization Methods. arXiv.
Cited by
Qiu, S., Zhao, H., Jiang, N., Wang, Z., Liu, L., An, Y., Zhao, H., Miao, X., Liu, R., & Fortino, G. (2022) · Information Fusion · doi:10.1016/j.inffus.2021.11.006 · Cited by 671
Deep Learning for Time Series Classification and Extrinsic Regression: A Current Survey
Mohammadi Foumani, N., Miller, L., Tan, C. W., Webb, G. I., Forestier, G., & Salehi, M. (2024) · ACM Computing Surveys · doi:10.1145/3649448 · Cited by 268
Human Activity Recognition Based on Residual Network and BiLSTM
Li, Y., & Wang, L. (2022) · Sensors · doi:10.3390/s22020635 · Cited by 156
Mekruksavanich, S., & Jitpattanakul, A. (2021) · Electronics · doi:10.3390/electronics10141685 · Cited by 138
Self-citationMekruksavanich, S., & Jitpattanakul, A. (2023) · Scientific Reports · doi:10.1038/s41598-023-39080-y · Cited by 75
Self-citation
Metadata
- Authors
- Mekruksavanich, S., & Jitpattanakul, A.
- Venue
- Sensors
- Publisher
- MDPI AG
- Published
- 2021-02-26
- Volume
- 21
- Issue
- 5
- Pages
- 1636
- ISSN
- 1424-8220 (ISSN-L)
- OpenAlex
- W3135100418
- Cited by
- 335
- References
- 49
- FWCI
- 22.05
- Type
- journal article
- Language
- en
- Open access
- gold
- Funders
- University of Phayao (FF64-UoE008)
- King Mongkut's University of Technology North Bangkok (KMUTNB-BasicR-64-33-2)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.