Deep Learning for Sensor-based Human Activity Recognition
The DOI matches its Crossref record; 151 of the 151 references it cites were found in a registry or an index.
Abstract
The vast proliferation of sensor devices and Internet of Things enables the applications of sensor-based activity recognition. However, there exist substantial challenges that could influence the performance of the recognition system in practical scenarios. Recently, as deep learning has demonstrated its effectiveness in many areas, plenty of deep methods have been investigated to address the challenges in activity recognition. In this study, we present a survey of the state-of-the-art deep learning methods for sensor-based human activity recognition. We first introduce the multi-modality of the sensory data and provide information for public datasets that can be used for evaluation in different challenge tasks. We then propose a new taxonomy to structure the deep methods by challenges. Challenges and challenge-related deep methods are summarized and analyzed to form an overview of the current research progress. At the end of this work, we discuss the open issues and provide some insights for future directions.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“This renders activity labeling difficult, reducing the number of available labels [8,23,30].”
From Spatial-Temporal Masked Autoencoder for Multi-Device Wearable Human Activity Recognition (2023) · cites this paper as [8]
Citations
Cites
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
Pan, S. J., & Yang, Q. (2010) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2009.191 · Cited by 23,387 · cited 1×
“Before taking a closer look at the factors that cause distribution discrepancy in sensory data, we briefly introduce transfer learning [102].”
BackgroundGreff, K., Srivastava, R. K., Koutnik, J., Steunebrink, B. R., & Schmidhuber, J. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2582924 · Cited by 6,955 · cited 19×
“A multi-label RNN was created with each RNN cell responding to activity recognition of one occupant.”
BackgroundMethodInfluentialProceedings of the 19th International Joint Conference on Artificial Intelligence
Pujol, J. M., Delgado, J., Ramon Sangüesa, A., & Flache, A. (2005) · International Joint Conference on Artificial Intelligence · Cited by 5,771
Combining labeled and unlabeled data with co-training
Blum, A., & Mitchell, T. (1998) · Proceedings of the eleventh annual conference on Computational learning theory · doi:10.1145/279943.279962 · Cited by 5,495
Recurrent neural network based language model
Mikolov, T., Karafiát, M., Burget, L., Černocký, J., & Khudanpur, S. (2010) · Interspeech 2010 · doi:10.21437/interspeech.2010-343 · Cited by 5,390
Activity recognition using cell phone accelerometers
Kwapisz, J. R., Weiss, G. M., & Moore, S. A. (2011) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1964897.1964918 · Cited by 2,912
Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition
Ordóñez, F., & Roggen, D. (2016) · Sensors · doi:10.3390/s16010115 · Cited by 2,761 · cited 1×
“eep hybrid model to explore different views of temporal dynamics is another attractive trend in the human activity recognition community. In light of the advantages of CNN and RNN, Ordóñez and Roggen [106] proposed to combine CNNs and LSTMs for both local and global temporal feature extraction. Xu et al. [162] adopted…”
A Survey on Human Activity Recognition using Wearable Sensors
Lara, O. D., & Labrador, M. A. (2013) · IEEE Communications Surveys & Tutorials · doi:10.1109/surv.2012.110112.00192 · Cited by 2,662 · cited 1×
“data facilitates recognition and which part deteriorates that. 1.2 Deep Learning in Human Activity Recognition. Numerous previous works adopted machine learning methods in human activity recognition [79]. They highly rely on feature extraction techniques including time-frequency transformation [62], statistical approa…”
MethodInfluential
Research works printed but not linked: 37
- Alsheikh Mohammad Abu, 2016, Proceedings of the Workshops at the 30th AAAI Conference on Artificial Intelligence.
- Bengio Yoshua, 2012, Proceedings of ICML Workshop on Unsupervised and Transfer Learning. 17–36
- Brophy Eoin, 2018, Zhengwei Wang, Alan F. Smeaton, and Tomas E. Ward.
- 10.5555/3367032.3367223 · doi:10.5555/3367032.3367223
- Chen Ling, 2020, METIER: A deep multi-task learning based activity and user recognition model using wearable sensors. Proc. ACM Interact., Mob, Wear. Ubiq. Technol., 4, 1
- Dong Mingtao, 2018, Proceedings of the International Conference on Signal and Information Processing, Networking and Computers. Springer, 32–40
- Goodfellow Ian, 2014, Proceedings of the International Conference on Advances in Neural Information Processing Systems. 2672–2680
- Hammerla Nils Yannick, 2015, Proceedings of the 29th AAAI Conference on Artificial Intelligence.
- Hammerla Nils Y., 2016, Proceedings of the 25th International Joint Conference on Artificial Intelligence. 1533–1540
- Karpathy Andrej, 2016, Proceedings of the 4th International Conference on Learning Representations Workshop.
- Al Hafiz Khan Md Abdullah, 2018, Proceedings of the International Conference on Pervasive Computing and Communications. IEEE, 1–9.
- 10.5555/3164150.3164213 · doi:10.5555/3164150.3164213
- Krizhevsky Alex, Proceedings of the International Conference on Advances in Neural Information Processing Systems. 1097–1105
- Nicholas, Proceedings of the 16th International Workshop on Mobile Computing Systems and Applications. ACM, 117–122
- Lee Song-Mi, 2017, Proceedings of the IEEE International Conference on Big Data and Smart Computing (BigComp’17)
- Li Fei, 2011, Proceedings of the Workshops at the 25th AAAI Conference on Artificial Intelligence.
- Li Xinyu, Proceedings of the 8th Wireless of the Students, by the Students, and for the Students Workshop (S3@MobiCom’16)
- Li Xinyu, Proceedings of the 14th ACM Conference on Embedded Network Sensor Systems CD-ROM. ACM, 164–175
- Li Xinyu, 2017, Burd
- Ordóñez Morales Francisco Javier, 2016, Proceedings of the ACM International Symposium on Wearable Computers. ACM, 92–99
- Vishvak, Proceedings of the ACM International Symposium on Wearable Computers. ACM, 100–103
- Phan NhatHai, 2016, Proceedings of the 30th AAAI Conference on Artificial Intelligence.
- Pouyanfar Samira, 2018, A survey on deep learning: Algorithms, techniques, and applications, Comput. Surv., 51, 92
- Press Ofir, 2017, Language generation with recurrent generative adversarial networks without pre-training. arXiv preprint arXiv:1706.01399 · arXiv
- Qian Hangwei, 2018, Proceedings of the AAAI Conference on Artificial Intelligence.
- Saeedi Ramyar, Proceedings of the IEEE International Conference on Big Data. IEEE, 473–479
- Serrano Sofia, Proceedings of the 57th Conference of the Association for Computational Linguistics (ACL’19)
- van Kasteren Tim L. M., 2011, Kröse
- Xue Li, 2018, Understanding and improving deep neural network for activity recognition. arXiv preprint arXiv:1805.07020 · arXiv
- 10.5555/2832747.2832806 · doi:10.5555/2832747.2832806
- Zebin Tahmina, Proceedings of the IEEE Conference on Sensors (SENSORS’16)
- Zhang Dalin, 2018, Proceedings of the AAAI Conference on Artificial Intelligence, 32
- Zhang Mi, Proceedings of the ACM Conference on Ubiquitous Computing. ACM, 1036–1043
- 10.5555/3304889.3305093 · doi:10.5555/3304889.3305093
- Zhang Yanyi, Proceedings of the 16th ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN’17)
- Zhang Yong, 2018, Human activity recognition based on time series analysis using U-Net. arXiv preprint arXiv:1809.08113 · arXiv
- Zheng Yi, Proceedings of the International Conference on Web-age Information Management. Springer, 298–310
Cited by
Items 501–503 of 503
Sensor-based human activity recognition using wrist and ankle wearables
Gorjani, O. M., Jaroš, R., & Bilík, P. (2026) · Frontiers in Bioengineering and Biotechnology · doi:10.3389/fbioe.2026.1926076
“For example, Chen et al. (2022) took a deep dive into applying deep learning to sensor-based HAR.”
Boylu, D., Polat, S., Daldal, N., & Önder, M. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7493770
Machine Learning‐Assisted Soft Sensors: From Material Design to Motion Recognition
Sun, H., Fu, S., Hou, Y., Liu, J., Yang, Y., Tao, C., Cheng, X., Huang, Y., & Wang, J. (2026) · Advanced Functional Materials · doi:10.1002/adfm.78483
Metadata
- Authors
- Chen, K., Zhang, D., Yao, L., Guo, B., Yu, Z., & Liu, Y.
- Affiliations
- Aalborg University (DK)
- UNSW Sydney (AU)
- Northwestern Polytechnical University (CN)
- Michigan State University (US)
- Venue
- ACM Computing Surveys
- Published
- 2021-05-24
- Volume
- 54
- Issue
- 4
- Pages
- 1-40
- ISSN
- 0360-0300 (ISSN-L)
- 1557-7341
- OpenAlex
- W3164845984
- Cited by
- 690
- References
- 144
- Citations by year
- 2026: 83 · 2025: 137 · 2024: 172 · 2023: 149 · 2022: 101 · 2021: 45
- FWCI
- 41.94
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- ACM Computing Surveys (publishedVersion, public-domain)
- VBN Forskningsportal (Aalborg Universitet) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Physical Sciences › Engineering › Biomedical Engineering › Gait Recognition and Analysis
- Keywords
- human activity recognition, sensor-based activity recognition, deep learning, Internet of Things, sensor fusion, activity recognition dataset, wearable sensors
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.