A tutorial on human activity recognition using body-worn inertial sensors
The DOI matches its Crossref record; 131 of the 132 references it cites were found in a registry or an index.
Abstract
The last 20 years have seen ever-increasing research activity in the field of human activity recognition. With activity recognition having considerably matured, so has the number of challenges in designing, implementing, and evaluating activity recognition systems. This tutorial aims to provide a comprehensive hands-on introduction for newcomers to the field of human activity recognition. It specifically focuses on activity recognition using on-body inertial sensors. We first discuss the key research challenges that human activity recognition shares with general pattern recognition and identify those challenges that are specific to human activity recognition. We then describe the concept of an Activity Recognition Chain (ARC) as a general-purpose framework for designing and evaluating activity recognition systems. We detail each component of the framework, provide references to related research, and introduce the best practice methods developed by the activity recognition research community. We conclude with the educational example problem of recognizing different hand gestures from inertial sensors attached to the upper and lower arm. We illustrate how each component of this framework can be implemented for this specific activity recognition problem and demonstrate how different implementations compare and how they impact overall recognition performance.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The promising results delivered in these works without the use of labels have resulted in a general direction of integrating unsupervised learning as-is into conventional activity recognition chains (ARC) [8] in the feature extraction step.”
From Contrastive Predictive Coding for Human Activity Recognition (2021) · cites this paper as [8]
In the citing paper
“The comparative studies using our evaluation framework are novel, because such an analysis has in particular been missing in most survey papers of sensor-based HAR [1,2,3].”
From Comparison of Feature Learning Methods for Human Activity Recognition Using Wearable Sensors (2018) · cites this paper as [3]
In the citing paper
“In [3], the authors carried out extensive experiments to assess the effectiveness of several approaches, but only considered simple hand-crafted features and classifiers using a small self-produced dataset.”
From Comparison of Feature Learning Methods for Human Activity Recognition Using Wearable Sensors (2018) · cites this paper as [3]
In the citing paper
“While the HAR problem seeks to find a perfect segmentation of activities, this task is difficult to achieve since humans perform activities in a fluent way and the segmentation of time series into intervals that represent single activities is not straightforward (Bulling et al. 2014).”
From Advanced Sensing and Human Activity Recognition in Early Intervention and Rehabilitation of Elderly People (2020) · cites this paper as Bulling et al. 2014
In the citing paper
“The second aspect that goes along with this is that human activity is highly diverse and the recognition of specific activities therefore requires an accurate selection of sensors and their placement (Bulling et al. 2014).”
From Advanced Sensing and Human Activity Recognition in Early Intervention and Rehabilitation of Elderly People (2020) · cites this paper as Bulling et al. 2014
2 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
A tutorial on hidden Markov models and selected applications in speech recognition
Rabiner, L. R. (1989) · Proceedings of the IEEE · doi:10.1109/5.18626 · Cited by 22,743 · cited 2×
“Given the training data T and an initial guess of the parameters ., a separate model is trained for each class using expectation maximization [Rabiner 1989; Fink 2008].”
MethodAn introduction to ROC analysis
Fawcett, T. (2006) · Pattern Recognition Letters · doi:10.1016/j.patrec.2005.10.010 · Cited by 21,879 · cited 1×
“Therefore, a common strategy is to sweep the threshold on the score for each individual class (one vs. all) and analyze the behavior in so-called Receiver Operating Characteristic (ROC) or Precision-Recall (PR) curves [Fawcett 2006].”
MethodPeng, H., Long, F., & Ding, C. (2005) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2005.159 · Cited by 10,544 · cited 2×
“These can be categorized into wrapper [Kohavi and John 1997], filter [Peng et al. 2005], or hybrid [Somol et al.”
BackgroundMethodWrappers for feature subset selection
Kohavi, R., & John, G. H. (1997) · Artificial Intelligence · doi:10.1016/s0004-3702(97)00043-x · Cited by 8,947 · cited 2×
“A large number of researchers investigated machine recognition of gestures and activities from still images and video in constrained environments or stationary settings (see Mitra and Acharya [2007], Turaga et al.”
BackgroundAn introduction to variable and feature selection
GuyonIsabelle, & ElisseeffAndré (2003) · Journal of Machine Learning Research · doi:10.5555/944919.944968 · Cited by 7,715
The Pascal Visual Object Classes Challenge: A Retrospective
Everingham, M., Eslami, S. M. A., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-014-0733-5 · Cited by 7,139
Friedman, J., Hastie, T., & Tibshirani, R. (2000) · The Annals of Statistics · doi:10.1214/aos/1016218223 · Cited by 6,884 · cited 2×
“Introduced in machine learning and computer vision [Friedman et al. 2000; Torralba et al. 2007], boosting as a variant of decision fusion has been successfully applied to activity recognition as well [Lester et al. 2005; Blanke and Schiele 2009; Zinnen et al. 2009b].”
Method2011 Compendium of Physical Activities: A Second Update of Codes and MET Values
AINSWORTH, B. E., HASKELL, W. L., HERRMANN, S. D., MECKES, N., BASSETT, D. R., TUDOR-LOCKE, C., GREER, J. L., VEZINA, J., WHITT-GLOVER, M. C., & LEON, A. S. (2011) · Medicine & Science in Sports & Exercise · doi:10.1249/mss.0b013e31821ece12 · Cited by 6,317 · cited 2×
“Other resources include the comprehensive compendium of physical activity [Ainsworth et al. 2011].”
BackgroundProceedings of the 19th International Joint Conference on Artificial Intelligence
Pujol, J. M., Delgado, J., Ramon Sangüesa, A., & Flache, A. (2005) · International Joint Conference on Artificial Intelligence · Cited by 5,771
Kittler, J., Hatef, M., Duin, R. P. W., & Matas, J. (1998) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.667881 · Cited by 5,290 · cited 3×
“J. Kittler et al. 1998.”
Background
Research works printed but not linked: 7
- 10.5555/2015035.2015320 · doi:10.5555/2015035.2015320
- 10.5555/1642293.1642416 · doi:10.5555/1642293.1642416
- 10.5555/1642293.1642417 · doi:10.5555/1642293.1642417
- 10.5555/2283516.2283683 · doi:10.5555/2283516.2283683
- 10.5555/1460232.1460248 · doi:10.5555/1460232.1460248
- 10.5555/2021975.2022004 · doi:10.5555/2021975.2022004
- 10.5555/862896.881089 · doi:10.5555/862896.881089
Cited by
Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition
Ordóñez, F., & Roggen, D. (2016) · Sensors · doi:10.3390/s16010115 · Cited by 2,761
“Sliding window approaches are commonly used for static and periodic activities, while sporadic activities lend themselves to template matching approaches or hidden Markov modelling [8,11].”
BackgroundDeep learning for sensor-based activity recognition: A survey
Wang, J., Chen, Y., Hao, S., Peng, X., & Hu, L. (2019) · Pattern Recognition Letters · doi:10.1016/j.patrec.2018.02.010 · Cited by 2,014
“HAR aims to understand human behaviors which enable the computing systems to proactively assist users based on their requirement (Bulling et al., 2014).”
Background(2015) · Cited by 989
A survey of unsupervised deep domain adaptation
Wilson, G., & Cook, D. J. (2020) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3400066 · Cited by 987
Nweke, H. F., Teh, Y. W., Al-garadi, M. A., & Alo, U. R. (2018) · Expert Systems with Applications · doi:10.1016/j.eswa.2018.03.056 · Cited by 930
Multi-sensor fusion in body sensor networks: State-of-the-art and research challenges
Gravina, R., Alinia, P., Ghasemzadeh, H., & Fortino, G. (2017) · Information Fusion · doi:10.1016/j.inffus.2016.09.005 · Cited by 899
Transition-Aware Human Activity Recognition Using Smartphones
Reyes-Ortiz, J.-L., Oneto, L., Samà, A., Parra, X., & Anguita, D. (2016) · Neurocomputing · doi:10.1016/j.neucom.2015.07.085 · Cited by 740
“A second issue considers that the activities carried out by people are, in real-life situations, more than the ones learned by any HAR system [31].”
BackgroundStisen, A., Blunck, H., Bhattacharya, S., Prentow, T. S., Kjærgaard, M. B., Dey, A., Sonne, T., & Jensen, M. M. (2015) · Proceedings of the 13th ACM Conference on Embedded Networked Sensor Systems · doi:10.1145/2809695.2809718 · Cited by 714
“A typical ARC for HAR is composed of five stages: (i) data collection, (ii) pre-processing, (iii) segmentation, (iv) feature extraction and (v) classification.”
BackgroundInfluentialDeep Learning for Sensor-based Human Activity Recognition
Chen, K., Zhang, D., Yao, L., Guo, B., Yu, Z., & Liu, Y. (2022) · ACM Computing Surveys · doi:10.1145/3447744 · Cited by 690
“vious works adopted machine learning methods in human activity recognition [79]. They highly rely on feature extraction techniques including time-frequency transformation [62], statistical approaches [21] and symbolic representation [88]. However, the features extracted are carefully engineered and heuristic. There we…”
BackgroundQiu, S., Zhao, H., Jiang, N., Wang, Z., Liu, L., An, Y., Zhao, H., Miao, X., Liu, R., & Fortino, G. (2022) · Information Fusion · doi:10.1016/j.inffus.2021.11.006 · Cited by 671
Deep Learning in Human Activity Recognition with Wearable Sensors: A Review on Advances
Zhang, S., Li, Y., Zhang, S., Shahabi, F., Xia, S., Deng, Y., & Alshurafa, N. (2022) · Sensors · doi:10.3390/s22041476 · Cited by 625
“Extensive experiments are performed on Opportunity [59], Skoda [316], and Hand-Gesture [49] datasets.”
BackgroundMethodDeep, Convolutional, and Recurrent Models for Human Activity Recognition Using Wearables
Hammerla, N. Y., Halloran, S., & Ploetz, T. (2016) · arXiv · doi:10.48550/arxiv.1604.08880 · Cited by 606
Zero-Effort Cross-Domain Gesture Recognition with Wi-Fi
Zheng, Y., Zhang, Y., Qian, K., Zhang, G., Liu, Y., Wu, C., & Yang, Z. (2019) · Proceedings of the 17th Annual International Conference on Mobile Systems, Applications, and Services · doi:10.1145/3307334.3326081 · Cited by 557
“Traditional approaches use cameras [16, 24, 42], wearable devices and phones [8, 17, 36] or sonar [22, 29, 48] as the sensing module.”
MethodEnsembles of Deep LSTM Learners for Activity Recognition using Wearables
Guan, Y., & Plötz, T. (2017) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3090076 · Cited by 511
“The majority of related work in HAR utilises (variants of) the well established sliding-window, frame-based analysis approach [Bulling et al. 2014].”
BackgroundMethodInfluentialA Survey of Online Activity Recognition Using Mobile Phones
Shoaib, M., Bosch, S., Incel, O., Scholten, H., & Havinga, P. (2015) · Sensors · doi:10.3390/s150102059 · Cited by 489
“The choice of a specific performance metric or a combination of different performance metrics depends on the type of application and its performance requirements [28,66].”
BackgroundMethodInfluentialFrom action to activity: Sensor-based activity recognition
Liu, Y., Nie, L., Liu, L., & Rosenblum, D. S. (2016) · Neurocomputing · doi:10.1016/j.neucom.2015.08.096 · Cited by 486
Human Behavior Analysis by Means of Multimodal Context Mining
Banos, O., Villalonga, C., Bang, J., Hur, T., Kang, D., Park, S., Huynh-The, T., Le-Ba, V., Amin, M., Razzaq, M., et al. (2016) · Sensors · doi:10.3390/s16081264 · Cited by 450
“Transitions among the contexts of interest are left out of the study since a null-class rejection schema has not been explicitly implemented [55].”
BackgroundA deep learning approach to on-node sensor data analytics for mobile or wearable devices
Ravi, D., Wong, C., Lo, B., & Yang, G.-Z. (2017) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2016.2633287 · Cited by 439
Complex Human Activity Recognition Using Smartphone and Wrist-Worn Motion Sensors
Shoaib, M., Bosch, S., Incel, O., Scholten, H., & Havinga, P. (2016) · Sensors · doi:10.3390/s16040426 · Cited by 423
“Activity recognition using body-worn motion sensors in general [19] and especially using smartphone sensors has been studied in recent years [1,2], and it is still being studied extensively [1,2,19–21].”
BackgroundPrivacy‐preserving federated learning based on multi‐key homomorphic encryption
Ma, J., Naas, S., Sigg, S., & Lyu, X. (2022) · International Journal of Intelligent Systems · doi:10.1002/int.22818 · Cited by 421
“Exploiting such massive data for activity recognition and situational awareness has been an active field of research in recent years [2].”
BackgroundA survey on wearable sensor modality centred human activity recognition in health care
Wang, Y., Cang, S., & Yu, H. (2019) · Expert Systems with Applications · doi:10.1016/j.eswa.2019.04.057 · Cited by 409
UP-Fall Detection Dataset: A Multimodal Approach
Martínez-Villaseñor, L., Ponce, H., Brieva, J., Moya-Albor, E., Núñez-Martínez, J., & Peñafort-Asturiano, C. (2019) · Sensors · doi:10.3390/s19091988 · Cited by 403
“In this example we compare the performance of four well-known methods typically use in fall detection and human activity recognition systems [9,13,27,28,34,43]: Random Forest (RF).”
MethodNweke, H. F., Teh, Y. W., Mujtaba, G., & Al-garadi, M. A. (2019) · Information Fusion · doi:10.1016/j.inffus.2018.06.002 · Cited by 394
A Semisupervised Recurrent Convolutional Attention Model for Human Activity Recognition
Chen, K., Yao, L., Zhang, D., Wang, X., Chang, X., & Nie, F. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2927224 · Cited by 368
“The first is insufficient labeled observations [2].”
BackgroundA survey of depth and inertial sensor fusion for human action recognition
Chen, C., Jafari, R., & Kehtarnavaz, N. (2017) · Multimedia Tools and Applications · doi:10.1007/s11042-015-3177-1 · Cited by 356
“A tutorial was presented in [8] covering a comprehensive discussion of designing and evaluating activity recognition systems using body-worn inertial sensors.”
Using Inertial Sensors for Position and Orientation Estimation
Kok, M., Hol, J. D., & Schön, T. B. (2017) · Foundations and Trends® in Signal Processing · doi:10.1561/2000000094 · Cited by 347
“A separate but related field of research is that of activity recognition using inertial sensors, where the aim is to determine the type of motion that is performed by a person wearing or carrying a device containing inertial sensors, see [19, 4] for an overview.”
LSTM Networks Using Smartphone Data for Sensor-Based Human Activity Recognition in Smart Homes
Mekruksavanich, S., & Jitpattanakul, A. (2021) · Sensors · doi:10.3390/s21051636 · Cited by 335
“Not only Area Under the Curve (AUC) indicates the overall efficiency of classifiers, but it also describes the likelihood that positive cases selected will be ranked higher than negative cases [34].”
BackgroundInnoHAR: A Deep Neural Network for Complex Human Activity Recognition
Xu, C., Chai, D., He, J., Zhang, X., & Duan, S. (2019) · IEEE Access · doi:10.1109/access.2018.2890675 · Cited by 333
“The defacto standard workflow for activity recognition in ubiquitous and wearable computing [30] treats individual frames of sensor data as statistically independent, that is, isolated portions of data are converted into feature vectors that are then presented to a classifier without further temporal context.”
BackgroundMethodCust, E. E., Sweeting, A. J., Ball, K., & Robertson, S. (2019) · Journal of Sports Sciences · doi:10.1080/02640414.2018.1521769 · Cited by 324
“Recognition or classification of movements involves further interpretations and labelled predictions of the identified instance (Bulling et al., 2014; Bux, Angelov, & Habib, 2017), i.e., differentiating tennis strokes as a forehand or backhand.”
BackgroundBanos, O., Villalonga, C., Garcia, R., Saez, A., Damas, M., Holgado-Terriza, J. A., Lee, S., Pomares, H., & Rojas, I. (2015) · BioMedical Engineering OnLine · doi:10.1186/1475-925x-14-s2-s6 · Cited by 321
Deep Learning for Sensor-based Human Activity Recognition: Overview, Challenges and Opportunities
Chen, K., Zhang, D., Yao, L., Guo, B., Yu, Z., & Liu, Y. (2020) · arXiv · doi:10.48550/arxiv.2001.07416 · Cited by 314
“vious works adopted machine learning methods in human activity recognition [79]. They highly rely on feature extraction techniques including time-frequency transformation [62], statistical approaches [21] and symbolic representation [88]. However, the features extracted are carefully engineered and heuristic. There we…”
BackgroundWidar3.0: Zero-Effort Cross-Domain Gesture Recognition with Wi-Fi
Zhang, Y., Zheng, Y., Qian, K., Zhang, G., Liu, Y., Wu, C., & Yang, Z. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3105387 · Cited by 312
“Traditional approaches use cameras [1], [2], [3], wearable devices and phones [4], [5], [6] or sonar [7], [8], [9] as the sensing module.”
Imaging and fusing time series for wearable sensor-based human activity recognition
Qin, Z., Zhang, Y., Meng, S., Qin, Z., & Choo, K.-K. R. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.06.014 · Cited by 309
“…also eased human activity recognition using sensor data from BSN, and example applications include health-care [6,7] , heart-rate-based emotion reactions [8,9] , activity monitoring [10,11] , and commercial applications such as fitness tracking [12] and signal processing in node environment [13] .”
BackgroundComparison of Feature Learning Methods for Human Activity Recognition Using Wearable Sensors
Li, F., Shirahama, K., Nisar, M., Köping, L., & Grzegorzek, M. (2018) · Sensors · doi:10.3390/s18020679 · Cited by 302
“The previous experiments were carried out in a setup fixing some hyper-parameters related to the first stages of the ARC (depicted in Figure 1).”
BackgroundInfluentialRecent trends in machine learning for human activity recognition—A survey
Ramasamy Ramamurthy, S., & Roy, N. (2018) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1254 · Cited by 299
“The various scenarios where transfer learning can be applied are explained comprehensively by Cook et al. (2013). The authors have elaborated on the different modalities, data labeling process and the taxonomy of type of knowledge transferred in transfer learning-based AR. Khan and Roy (2017) have investigated transfe…”
BackgroundA Survey of Deep Learning-Based Human Activity Recognition in Radar
Li, X., He, Y., & Jing, X. (2019) · Remote Sensing · doi:10.3390/rs11091068 · Cited by 290
“The purpose of HAR is to identify a user’s behavior so as to allow computing systems to proactively provide assistance for the user [6].”
BackgroundMulti-representation adaptation network for cross-domain image classification
Zhu, Y., Zhuang, F., Wang, J., Chen, J., Shi, Z., Wu, W., & He, Q. (2019) · Neural Networks · doi:10.1016/j.neunet.2019.07.010 · Cited by 286
“Feature Visualization: We visualize the network representations of task A → W learned by MRAN and DAN using t-SNE embeddings (Donahue et al., 2014) in Figs.”
MethodDeep recurrent neural network for mobile human activity recognition with high throughput
Inoue, M., Inoue, S., & Nishida, T. (2018) · Artificial Life and Robotics · doi:10.1007/s10015-017-0422-x · Cited by 275
“Recently, many studies on the mobile activity recognition are carried out [5, 20].”
BackgroundA Survey on Deep Learning for Human Activity Recognition
Gu, F., Chung, M.-H., Chignell, M., Valaee, S., Zhou, B., & Liu, X. (2022) · ACM Computing Surveys · doi:10.1145/3472290 · Cited by 272
“Bulling et al. [27] provided a tutorial on HAR with conventional machine learning methods based on wearable inertial sensors.”
MethodRavi, D., Wong, C., Lo, B., & Yang, G.-Z. (2016) · 2016 IEEE 13th International Conference on Wearable and Implantable Body Sensor Networks (BSN) · doi:10.1109/bsn.2016.7516235 · Cited by 269
“Features, such as mean [1], Fourier transforms [2], and symbols [3], are typically extracted from segments of data and then trained using classification methods [4]–[8].”
BackgroundMethodNi, Q., García Hernando, A., & De la Cruz, I. (2015) · Sensors · doi:10.3390/s150511312 · Cited by 264
“[14] give a comprehensive introduction on human activity recognition, but similarly, they focus on a specific type of sensors (in this case body-worn inertial sensors).”
BackgroundHuman Activity Recognition Using Inertial Sensors in a Smartphone: An Overview
Sousa Lima, W., Souto, E., El-Khatib, K., Jalali, R., & Gama, J. (2019) · Sensors · doi:10.3390/s19143213 · Cited by 256
“Smartphones are included in the scope of wearable computing [23,24], and these devices are considered part of mobile computing-based HAR systems.”
BackgroundYang, Q., Zhang, Y., Dai, W., & Pan, S. J. (2020) · Cambridge University Press eBooks · doi:10.1017/9781139061773 · Cited by 254
Stratified Transfer Learning for Cross-domain Activity Recognition
Wang, J., Chen, Y., Hu, L., Peng, X., & Yu, P. S. (2018) · 2018 IEEE International Conference on Pervasive Computing and Communications (PerCom) · doi:10.1109/percom.2018.8444572 · Cited by 253
Sleep Quality Prediction From Wearable Data Using Deep Learning
Sathyanarayana, A., Joty, S., Fernandez-Luque, L., Ofli, F., Srivastava, J., Elmagarmid, A., Arora, T., & Taheri, S. (2016) · JMIR mHealth and uHealth · doi:10.2196/mhealth.6562 · Cited by 242
“In this sub-section, we limit our deep learning review to studies that employ techniques to analyses of physical activity data, and refer the reader to Chen et al. [27] and Bulling et al. [28] for more comprehensive reviews of sensor-based human activity recognition literature.”
MethodOn the use of ensemble of classifiers for accelerometer-based activity recognition
Catal, C., Tufekci, S., Pirmit, E., & Kocabag, G. (2015) · Applied Soft Computing · doi:10.1016/j.asoc.2015.01.025 · Cited by 240
Towards unsupervised physical activity recognition using smartphone accelerometers
Lu, Y., Wei, Y., Liu, L., Zhong, J., Sun, L., & Liu, Y. (2017) · Multimedia Tools and Applications · doi:10.1007/s11042-015-3188-y · Cited by 225
“The motion data recorded from an accelerometer or gyroscope are often more difficult to interpret than data from cameras [6].”
BackgroundWearable Sensor-Based Human Activity Recognition in the Smart Healthcare System
Serpush, F., Menhaj, M. B., Masoumi, B., & Karasfi, B. (2022) · Computational Intelligence and Neuroscience · doi:10.1155/2022/1391906 · Cited by 221
“…the CNN approach is mainly used for image classification, it performs well in time series signals received from wearable sensors and generates high-value features [61]. deep architecture of this approach can extract special features from these signals representing a high level of abstraction [94].”
BackgroundWearable-Based Affect Recognition—A Review
Schmidt, P., Reiss, A., Dürichen, R., & Laerhoven, K. V. (2019) · Sensors · doi:10.3390/s19194079 · Cited by 218
“Following the classical time series analysis pipeline, presented by Bulling et al. [135], the raw data is first synchronised, filtered, segmented, features are computed, and finally feature-based classifiers are employed.”
MethodIslam, M. M., Nooruddin, S., Karray, F., & Muhammad, G. (2022) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2022.106060 · Cited by 217
“[44] 2015 OPPORTUNITY [92], hand gesture dataset [112] 2-D CNN Accuracy = 96, F1 Score = 95.”
BackgroundDanHAR: Dual Attention Network for multimodal human activity recognition using wearable sensors
Gao, W., Zhang, L., Teng, Q., He, J., & Wu, H. (2021) · Applied Soft Computing · doi:10.1016/j.asoc.2021.107728 · Cited by 214
Forkan, A. R. M., Khalil, I., Tari, Z., Foufou, S., & Bouras, A. (2015) · Pattern Recognition · doi:10.1016/j.patcog.2014.07.007 · Cited by 205
Multi-source information fusion: Progress and future
LI, X., DUNKIN, F., & DEZERT, J. (2024) · Chinese Journal of Aeronautics · doi:10.1016/j.cja.2023.12.009 · Cited by 204
“87 AI based information fusion methods have gained significant importance in practical engineering applications.”
BackgroundSemi-supervised near-miss fall detection for ironworkers with a wearable inertial measurement unit
Yang, K., Ahn, C. R., Vuran, M. C., & Aria, S. S. (2016) · Automation in Construction · doi:10.1016/j.autcon.2016.04.007 · Cited by 202
Recognizing Complex Activities by a Probabilistic Interval-Based Model
Liu, L., Cheng, L., Liu, Y., Jia, Y., & Rosenblum, D. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10155 · Cited by 202
“Current techniques are becoming mature to recognize basic actions and movements from cameras or other sensors (Frank, Mannor, and Precup 2010; Gupta and Mooney 2010; Lara and Labrador 2013; Bulling, Blanke, and Schiele 2014; Y¨ur¨uten, Zhang, and Pu 2014).”
BackgroundMethodComputational Analysis of Behavior
Egnor, S. E. R., & Branson, K. (2016) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-070815-013845 · Cited by 201
“…we discuss trajectory-based features ( Figure 3 a , c )—features extracted from the tracked positions of the animals and/or their parts over time (Bulling et al. 2014, Burgos-Artizzu et al. 2012, Eyjolfsdottir et al. 2014, Jhuang et al. 2010, Kabra et al. 2013, Mirat et al. 2013, Ramanan &…”
BackgroundZhang, H., Xiao, Z., Wang, J., Li, F., & Szczerbicki, E. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2019.2949715 · Cited by 197
“there are additional challenges, such as intraclass variability, interclass similarity, the NULL-class dominance, and complexness and diversity of physical activities [15], [16].”
BackgroundMethodConvolutional Neural Networks for Human Activity Recognition Using Body-Worn Sensors
Moya Rueda, F., Grzeszick, R., Fink, G. A., Feldhorst, S., & Ten Hompel, M. (2018) · Informatics · doi:10.3390/informatics5020026 · Cited by 196
“A traditional HAR pipeline segments sequences using a sliding-window approach, extracts relevant hand-crafted features from the segmented sequences, and it trains a classifier for assigning certain action labels to the sequences [6–8].”
BackgroundMethodInfluentialGholamiangonabadi, D., Kiselov, N., & Grolinger, K. (2020) · IEEE Access · doi:10.1109/access.2020.3010715 · Cited by 196
Dehghani, A., Sarbishei, O., Glatard, T., & Shihab, E. (2019) · Sensors · doi:10.3390/s19225026 · Cited by 198
“, window size, heavily impacts the performance of the model [4,15].”
BackgroundMethodInfluentialAttention induced multi-head convolutional neural network for human activity recognition
Khan, Z. N., & Ahmad, J. (2021) · Applied Soft Computing · doi:10.1016/j.asoc.2021.107671 · Cited by 187
Zeng, M., Gao, H., Yu, T., Mengshoel, O. J., Langseth, H., Lane, I., & Liu, X. (2018) · Proceedings of the 2018 ACM International Symposium on Wearable Computers · doi:10.1145/3267242.3267286 · Cited by 185
“INTRODUCTION Wearable-based human activity recognition (HAR) systems are an indispensable component in mobile ubiquitous computing and human-computer interaction [5].”
BackgroundTan, B., Song, Y., Zhong, E., & Yang, Q. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2783295 · Cited by 184
A Systematic Study of Unsupervised Domain Adaptation for Robust Human-Activity Recognition
Chang, Y., Mathur, A., Isopoussu, A., Song, J., & Kawsar, F. (2020) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3380985 · Cited by 138
“These advancements can be attributed to the years of research in the area of human-activity recognition (HAR) [4, 8, 15, 31] as well as the advances in computational models that process raw inertial data to infer human activities.”
BackgroundDeep Learning for Inertial Positioning: A Survey
Chen, C., & Pan, X. (2024) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2024.3381161 · Cited by 185
“…role in enabling completely egocentric motion tracking and positioning, making them essential for a range of applications. stations) [1], supporting security and safety services (e.g., aiding first-responders) [2], enabling smart city/infrastructure, and facilitating human-device interaction [3].”
BackgroundFEMO: A Platform for Free-weight Exercise Monitoring with RFIDs
Ding, H., Shangguan, L., Yang, Z., Han, J., Zhou, Z., Yang, P., Xi, W., & Zhao, J. (2015) · Proceedings of the 13th ACM Conference on Embedded Networked Sensor Systems · doi:10.1145/2809695.2809708 · Cited by 183
“We evaluate the activity segmentation scheme based on six metrics [8]: insertion rate, deletion rate, fragmentation rate, merge rate, underfill rate and accuracy.”
BackgroundMethodDivide and Conquer-Based 1D CNN Human Activity Recognition Using Test Data Sharpening
Cho, H., & Yoon, S. (2018) · Sensors · doi:10.3390/s18041055 · Cited by 183
“A typical activity recognition process comprises the stages for data acquisition, signal preprocessing and segmentation, feature extraction and selection, training, and classification [37].”
Deep Transfer Learning for Cross-domain Activity Recognition
Wang, J., Zheng, V. W., Chen, Y., & Huang, M. (2018) · Proceedings of the 3rd International Conference on Crowd Science and Engineering · doi:10.1145/3265689.3265705 · Cited by 142
“Human Activity recognition has been a popular research topic in pervasive computing [6] for its competence in learning profound high-level knowledge about human activity from raw sensor inputs.”
BackgroundWang, L., Gjoreski, H., Ciliberto, M., Mekki, S., Valentin, S., & Roggen, D. (2019) · IEEE Access · doi:10.1109/access.2019.2890793 · Cited by 180
“User-independent activity recognition aims to design recognition systems that will generalize well to new users [57].”
BackgroundPD Disease State Assessment in Naturalistic Environments Using Deep Learning
Hammerla, N., Fisher, J., Andras, P., Rochester, L., Walker, R., & Ploetz, T. (2015) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v29i1.9484 · Cited by 179
“Our system comprises a typical analysis pipeline common for activity recognition in ubiquitous computing (Bulling, Blanke, and Schiele 2014).”
The Cambridge Handbook of Computing Education Research
(2019) · Cambridge University Press eBooks · doi:10.1017/9781108654555 · Cited by 126
Novice Programmers and Introductory Programming
Robins, A. V. (2019) · The Cambridge Handbook of Computing Education Research · doi:10.1017/9781108654555.013 · Cited by 175
Cruciani, F., Vafeiadis, A., Nugent, C., Cleland, I., McCullagh, P., Votis, K., Giakoumis, D., Tzovaras, D., Chen, L., & Hamzaoui, R. (2020) · CCF Transactions on Pervasive Computing and Interaction · doi:10.1007/s42486-020-00026-2 · Cited by 142
“On the one hand, it simplifies the ARC by automating a step that typically requires significant domain-specific expertise to identify a suitable feature set (LeCun et al. 2015).”
BackgroundMethodInfluentialUsing Smart City Technology to Make Healthcare Smarter
Cook, D. J., Duncan, G., Sprint, G., & Fritz, R. L. (2018) · Proceedings of the IEEE · doi:10.1109/jproc.2017.2787688 · Cited by 173
“Additionally, machine learning algorithms can be used to map the vector of features onto activity labels [24], [25].”
Learning Human Identity From Motion Patterns
Neverova, N., Wolf, C., Lacey, G., Fridman, L., Chandra, D., Barbello, B., & Taylor, G. (2016) · IEEE Access · doi:10.1109/access.2016.2557846 · Cited by 172
Automated Action Recognition Using an Accelerometer-Embedded Wristband-Type Activity Tracker
Ryu, J., Seo, J., Jebelli, H., & Lee, S. H. (2018) · Journal of Construction Engineering and Management · doi:10.1061/(asce)co.1943-7862.0001579 · Cited by 173
Optimising sampling rates for accelerometer-based human activity recognition
Khan, A., Hammerla, N., Mellor, S., & Plötz, T. (2016) · Pattern Recognition Letters · doi:10.1016/j.patrec.2016.01.001 · Cited by 171
“This was achieved by using an unsupervised statistical approach followed by a standard HAR approach for the classification of human activities in 5 benchmark datasets.”
MethodInfluentialZheng, X., Wang, M., & Ordieres-Meré, J. (2018) · Sensors · doi:10.3390/s18072146 · Cited by 146
“There are two evaluation schemes for the activity recognition model, which are a person-dependent method and a person-independent, leave-one-out method [17].”
MethodHuman Activity Recognition Using Federated Learning
Sozinov, K., Vlassov, V., & Girdzijauskas, S. (2018) · 2018 IEEE Intl Conf on Parallel & Distributed Processing with Applications, Ubiquitous Computing & Communications, Big Data & Cloud Computing, Social Computing & Networking, Sustainable Computing & Communications (ISPA/IUCC/BDCloud/SocialCom/SustainCom) · doi:10.1109/bdcloud.2018.00164 · Cited by 167
“Beyond accelerometer and gyroscope magnetic field sensors can be used, but they are less common [7].”
BackgroundSelf-supervised learning for human activity recognition using 700,000 person-days of wearable data
Yuan, H., Chan, S., Creagh, A. P., Tong, C., Acquah, A., Clifton, D. A., & Doherty, A. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01062-3 · Cited by 166
“HAR models are usually built following the Activity Recognition Chain (Bulling et al., 2014), which describes the pre-processing and segmentation of raw time series into fixed-length frames, from which features are extracted for classification.”
InfluentialAmbient Sensors for Elderly Care and Independent Living: A Survey
Uddin, M. Z., Khaksar, W., & Torresen, J. (2018) · Sensors · doi:10.3390/s18072027 · Cited by 166
“[16] gave a detailed outline of human activity recognition but, similarly, focused on inertial sensors.”
BackgroundA Hybrid CNN–LSTM Network for the Classification of Human Activities Based on Micro-Doppler Radar
Zhu, J., Chen, H., & Ye, W. (2020) · IEEE Access · doi:10.1109/access.2020.2971064 · Cited by 166
“INTRODUCTION Human activity recognition (HAR) provides excellent potential for various applications, including personal health systems (PHS), human-computer interaction (HCI), and antiterrorism monitoring [1]–[3].”
BackgroundModeling and Understanding Human Routine Behavior
Banovic, N., Buzali, T., Chevalier, F., Mankoff, J., & Dey, A. K. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858557 · Cited by 166
Panwar, M., Biswas, D., Bajaj, H., Jobges, M., Turk, R., Maharatna, K., & Acharyya, A. (2019) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2019.2899927 · Cited by 158
“nition is to identify human behavior in real time scenario to provide proactive assistance to users even outside the conventional clinical setting [10].”
BackgroundA Deep Learning Approach for Human Activities Recognition From Multimodal Sensing Devices
Ihianle, I. K., Nwajana, A. O., Ebenuwa, S. H., Otuka, R. I., Owa, K., & Orisatoki, M. O. (2020) · IEEE Access · doi:10.1109/access.2020.3027979 · Cited by 158
“These efforts so far have focused on the use of video [1], wearable sensors and wireless sensor networks [2], [3] to capture simple human activities.”
BackgroundOn attention models for human activity recognition
Murahari, V. S., & Plötz, T. (2018) · Proceedings of the 2018 ACM International Symposium on Wearable Computers · doi:10.1145/3267242.3267287 · Cited by 157
“This is even more pertinent for HAR problems as there is typically only little, if any, relation between current and distant past activities [2].”
BackgroundMethodHuang, W., Zhang, L., Wu, H., Min, F., & Song, A. (2022) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2022.3174816 · Cited by 155
“Following recent research literatures [15], [21], [39], [40], [41], we build a 3-layer CNN as our baseline, which could provide very close results comparing to representative convolutional networks.”
MethodContrastive Predictive Coding for Human Activity Recognition
Haresamudram, H., Essa, I., & Plötz, T. (2021) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3463506 · Cited by 150
“This pre-training step is integrated into an overarching human activity recognition framework, that is based on the standard Activity Recognition Chain (ARC) [8].”
BackgroundMethodInfluentialBody Sensor Networks: In the Era of Big Data and Beyond
Poon, C. C. Y., Lo, B. P. L., Yuce, M. R., Alomainy, A., & Hao, Y. (2015) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2015.2427254 · Cited by 150
Márquez, E. Q. M., Hare, J. S., & Niranjan, M. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2805098 · Cited by 149
Ehatisham-ul-Haq, M., Awais Azam, M., Naeem, U., Amin, Y., & Loo, J. (2018) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2018.02.020 · Cited by 138
HealthyOffice: Mood recognition at work using smartphones and wearable sensors
Zenonos, A., Khan, A., Kalogridis, G., Vatsikas, S., Lewis, T., & Sooriyabandara, M. (2016) · IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops) · doi:10.1109/percomw.2016.7457166 · Cited by 148
“Feature extraction for the heart-rate variability was performed using the HRVAS toolbox [23] and acceleration features were calculated based on the standard Human Activity Recognition (HAR) features detailed in [24] and [25].”
MethodPosition-aware activity recognition with wearable devices
Sztyler, T., Stuckenschmidt, H., & Petrich, W. H. (2017) · Pervasive and Mobile Computing · doi:10.1016/j.pmcj.2017.01.008 · Cited by 148
“Focusing on different cross-subjects approaches, most of the existing works examined leave-one-subject-out [6,7], i.”
BackgroundMasked reconstruction based self-supervision for human activity recognition
Haresamudram, H., Beedu, A., Agrawal, V., Grady, P. L., Essa, I., Hoffman, J., & Plötz, T. (2020) · Proceedings of the 2020 International Symposium on Wearable Computers · doi:10.1145/3410531.3414306 · Cited by 146
“Our work is focused on deriving effective data representations to be used as part of the standard activity recognition chain (ARC) [3].”
BackgroundMethodInfluentialA Comprehensive Study of Activity Recognition Using Accelerometers
Twomey, N., Diethe, T., Fafoutis, X., Elsts, A., McConville, R., Flach, P., & Craddock, I. (2018) · Informatics · doi:10.3390/informatics5020027 · Cited by 143
“7) These issues are shared with many other settings where Machine Learning (ML) is applied to Digital Signal Processing (DSP), and as such this is a fairly mature research area [10,11].”
BackgroundA Comprehensive Study of Activity Recognition Using Accelerometers
Twomey, N., Diethe, T., Fafoutis, X., Elsts, A., McConville, R., Flach, P., & Craddock, I. (2018) · doi:10.20944/preprints201803.0147.v1 · Cited by 143
OneFi: One-Shot Recognition for Unseen Gesture via COTS WiFi
Xiao, R., Liu, J., Han, J., & Ren, K. (2021) · ACM Conference on Embedded Networked Sensor Systems (SenSys) · doi:10.1145/3485730.3485936 · Cited by 141
“Traditional HGR solutions are mainly camera-based [12, 24, 43] or device-based [2, 14, 35].”
BackgroundHuman Activity Recognition Based on Wearable Sensor Using Hierarchical Deep LSTM Networks
Wang, L., & Liu, R. (2019) · Circuits Systems and Signal Processing · doi:10.1007/s00034-019-01116-y · Cited by 141
Triple Cross-Domain Attention on Human Activity Recognition Using Wearable Sensors
Tang, Y., Zhang, L., Teng, Q., Min, F., & Song, A. (2022) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2021.3136642 · Cited by 139
“require feature extraction from the data, which heavily depends on expert knowledge from specific domain [6].”
BackgroundShoulder physiotherapy exercise recognition: machine learning the inertial signals from a smartwatch
Burns, D. M., Leung, N., Hardisty, M., Whyne, C. M., Henry, P., & McLachlin, S. (2018) · Physiological Measurement · doi:10.1088/1361-6579/aacfd9 · Cited by 103
“Although other authors (Gonzalez et al. 2015; Bulling et al. 2014) have utilized inertial sensor fusion for HAR, only raw sensor data is utilized in this study.”
BackgroundMethodInfluential
Metadata
- Authors
- Bulling, A., Blanke, U., & Schiele, B.
- Affiliations
- Max Planck Institute for Informatics (DE)
- ETH Zurich (CH)
- Venue
- ACM Computing Surveys
- Published
- 2014-01
- Volume
- 46
- Issue
- 3
- Pages
- 1-33
- ISSN
- 0360-0300 (ISSN-L)
- 1557-7341
- OpenAlex
- W2023302299
- Cited by
- 1,668
- References
- 132
- Citations by year
- 2026: 114 · 2025: 105 · 2024: 104 · 2023: 125 · 2022: 158 · 2021: 153 · 2020: 201 · 2019: 213 · 2018: 164 · 2017: 118 · 2016: 115 · 2015: 75 · 2014: 22
- FWCI
- 98.29
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- ACM Computing Surveys (publishedVersion)
- Max Planck Digital Library (submittedVersion)
- MPG.PuRe (Max Planck Society) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Physical Sciences › Engineering › Biomedical Engineering › Gait Recognition and Analysis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Keywords
- human activity recognition, wearable sensors, hand gesture recognition
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.