The DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.
Understanding and improving recurrent networks for human activity recognition by continuous attention
Abstract
Deep neural networks, including recurrent networks, have been successfully applied to human activity recognition. Unfortunately, the final representation learned by recurrent networks might encode some noise (irrelevant signal components, unimportant sensor modalities, etc.). Besides, it is difficult to interpret the recurrent networks to gain insight into the models' behavior. To address these issues, we propose two attention models for human activity recognition: temporal attention and sensor attention. These two mechanisms adaptively focus on important signals and sensor modalities. To further improve the understandability and mean Fl score, we add continuity constraints, considering that continuous sensor signals are more robust than discrete ones. We evaluate the approaches on three datasets and obtain state-of-the-art results. Furthermore, qualitative analysis shows that the attention learned by the models agree well with human intuition.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Temporal attention was applied by Murahari et al. [138] whereas continuous sensor and temporal attention were evaluated by Zeng et al. [214], both leading to increased performance.”
From Past, Present, and Future of Sensor-based Human Activity Recognition Using Wearables: A Surveying Tutorial on a Still Challenging Task (2025) · cites this paper as [214]
The cited source says what the claim attributes to it.
In the citing paper
“[226] applies temporal attention and sensor attention into LSTM to improve the overall activity recognition accuracy by adaptively focusing on important time windows and sensor modalities.”
From Deep Learning in Human Activity Recognition with Wearable Sensors: A Review on Advances (2022) · cites this paper as [226]
In the citing paper
“The squeeze-and-excitation network (SENet) [15] is one of the representative methods, which brings clear performance gain across various deep CNN architectures via learning channel attention for each convolution block.”
From DanHAR: Dual Attention Network for multimodal human activity recognition using wearable sensors (2021) · cites this paper as [15]
In the citing paper
“SENet [15] for the first time proposes an effective mechanism to learn channel attention and achieves promising performance.”
From DanHAR: Dual Attention Network for multimodal human activity recognition using wearable sensors (2021) · cites this paper as [15]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 8×
“Models PAMAP2 DG Skoda4 LSTM baseline ([12]) (LSTM without Attention) 0.”
BackgroundMethodResultInfluentialNeural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213 · cited 4×
“The attention approach is proposed to mitigate this problem for speech and natural language processing (NLP) [3, 14], and it offers means to explicate the inner workings of neural networks.”
BackgroundInfluentialApproximation by superpositions of a sigmoidal function
Cybenko, G. (1989) · Mathematics of Control, Signals, and Systems · doi:10.1007/bf02551274 · Cited by 13,883
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.03044 · Cited by 8,248
Activity Recognition from User-Annotated Acceleration Data
Bao, L., & Intille, S. S. (2004) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-24646-6_1 · Cited by 3,147 · cited 4×
“RECOGNITION We frame HAR as a sequence classification problem [4].”
BackgroundMethodInfluentialDeep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition
Ordóñez, F., & Roggen, D. (2016) · Sensors · doi:10.3390/s16010115 · Cited by 2,761 · cited 4×
“The baseline methods include standard LSTM [12], DeepConvLSTM [19], and LSTM-S (LSTM + sample-by-sample analysis) [11].”
BackgroundMethodInfluentialAttention-Based Models for Speech Recognition
Chorowski, J., Bahdanau, D., Serdyuk, D., Cho, K., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1506.07503 · Cited by 2,021 · cited 1×
“Deep neural networks (DNNs) has had revolutionary impact in speech recognition [7], image classification [24], and machine translation [3].”
BackgroundA tutorial on human activity recognition using body-worn inertial sensors
Bulling, A., Blanke, U., & Schiele, B. (2014) · ACM Computing Surveys · doi:10.1145/2499621 · Cited by 1,668 · cited 2×
“INTRODUCTION Wearable-based human activity recognition (HAR) systems are an indispensable component in mobile ubiquitous computing and human-computer interaction [5].”
BackgroundIntroducing a New Benchmarked Dataset for Activity Monitoring
Reiss, A., & Stricker, D. (2012) · 2012 16th International Symposium on Wearable Computers · doi:10.1109/iswc.2012.13 · Cited by 1,456 · cited 1×
“The first is the PAMAP2 dataset (Physical Activity Monitoring for Aging People 2) [21].”
Background(2015) · Cited by 989
Research works printed but not linked: 1
- Nils Y Hammerla , Shane Halloran , and Thomas Ploetz . 2016. Deep , convolutional, and recurrent models for human activity recognition using wearables. IJCAI ( 2016 ). Nils Y Hammerla, Shane Halloran, and Thomas Ploetz. 2016. Deep, convolutional, and recurrent models for human activity recognition using wearables. IJCAI (2016).
Cited by
Deep Learning in Human Activity Recognition with Wearable Sensors: A Review on Advances
Zhang, S., Li, Y., Zhang, S., Shahabi, F., Xia, S., Deng, Y., & Alshurafa, N. (2022) · Sensors · doi:10.3390/s22041476 · Cited by 625
“Their method has outperformed regular CNN considerably on a number of public datasets such as PAMAP2 [228], WISDM [155], UNIMIB SHAR [188], and Opportunity [59].”
BackgroundMethodInfluentialDeep Learning for Sensor-based Human Activity Recognition
Chen, K., Zhang, D., Yao, L., Guo, B., Yu, Z., & Liu, Y. (2022) · ACM Computing Surveys · doi:10.1145/3447744 · Cited by 690
Multi-input CNN-GRU based human activity recognition using wearable sensors
Dua, N., Singh, S. N., & Semwal, V. B. (2021) · Computing · doi:10.1007/s00607-021-00928-8 · Cited by 432
“The proposed model is compared with the models presented in [17, 20–22, 36, 37, 49–52].”
MethodDeep PPG: Large-Scale Heart Rate Estimation with Convolutional Neural Networks
Reiss, A., Indlekofer, I., Schmidt, P., & Van Laerhoven, K. (2019) · Sensors · doi:10.3390/s19143079 · Cited by 377
“Deep learning has already been applied to various time-series signals, such as for human activity recognition [20–22] or gait parameter extraction [23].”
MethodDeep Learning for Sensor-based Human Activity Recognition: Overview, Challenges and Opportunities
Chen, K., Zhang, D., Yao, L., Guo, B., Yu, Z., & Liu, Y. (2020) · arXiv · doi:10.48550/arxiv.2001.07416 · Cited by 314
Deep Learning for Time Series Classification and Extrinsic Regression: A Current Survey
Mohammadi Foumani, N., Miller, L., Tan, C. W., Webb, G. I., Forestier, G., & Salehi, M. (2024) · ACM Computing Surveys · doi:10.1145/3649448 · Cited by 268
“Table 4 shows a list of the self-supervised models reviewed in this article.”
BackgroundMultiscale Deep Feature Learning for Human Activity Recognition Using Wearable Sensors
Tang, Y., Zhang, L., Min, F., & He, J. (2023) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2022.3161812 · Cited by 224
“Zeng et al. [18] built a new network using LSTM as backbone, which incorporates two attention ideas to highlight more important components of sensor time series and sensor modalities.”
MethodDanHAR: Dual Attention Network for multimodal human activity recognition using wearable sensors
Gao, W., Zhang, L., Teng, Q., He, J., & Wu, H. (2021) · Applied Soft Computing · doi:10.1016/j.asoc.2021.107728 · Cited by 214
“Recently, two attention mechanisms are proposed via combining with a Gated Recurrent Units (GRU) network [12] and a Long Short-Term Memory (LSTM) network [13] respectively, which can capture the dependencies”
MethodInfluentialAttention induced multi-head convolutional neural network for human activity recognition
Khan, Z. N., & Ahmad, J. (2021) · Applied Soft Computing · doi:10.1016/j.asoc.2021.107671 · Cited by 187
Teng, Q., Wang, K., Zhang, L., & He, J. (2020) · IEEE Sensors Journal · doi:10.1109/jsen.2020.2978772 · Cited by 183
“For comparisons, we subsampled the sensor signals in order to match the frequency used in other related literatures [33], [34], [36], [37] on the dataset, which is still sufficient for sensing body motion.”
MethodAl-qaness, M. A. A., Dahou, A., Elaziz, M. A., & Helmi, A. M. (2023) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2022.3165875 · Cited by 175
“Zeng et al. [19] presented a HAR model based on two attention schemes to address critical challenges of traditional methods.”
MethodContrastive Predictive Coding for Human Activity Recognition
Haresamudram, H., Essa, I., & Plötz, T. (2021) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3463506 · Cited by 150
“Leveraging the temporal correlations in wearable sensor data at modeling level, (variants of) recurrent neural networks (RNNs) have been applied previously [27, 29, 80].”
MethodInception inspired CNN-GRU hybrid network for human activity recognition
Dua, N., Singh, S. N., Semwal, V. B., & Challa, S. K. (2022) · Multimedia Tools and Applications · doi:10.1007/s11042-021-11885-x · Cited by 160
Triple Cross-Domain Attention on Human Activity Recognition Using Wearable Sensors
Tang, Y., Zhang, L., Teng, Q., Min, F., & Song, A. (2022) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2021.3136642 · Cited by 139
“Zeng et al. [25] highlighted the important part of different time series and sensor modalities by designing temporal attention and sensor attention with Long Short Term Memory(LSTM).”
MethodHuman activity recognition using wearable sensors by heterogeneous convolutional neural networks
Han, C., Zhang, L., Tang, Y., Huang, W., Min, F., & He, J. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.116764 · Cited by 130
Shallow Convolutional Neural Networks for Human Activity Recognition Using Wearable Sensors
Huang, W., Zhang, L., Gao, W., Min, F., & He, J. (2021) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2021.3091990 · Cited by 129
“Zeng et al. [16] and Ma et al. [17] adopted attention to focus on these channels which have more contribution to activity recognition.”
BackgroundMethodInfluentialA Comparative Analysis of Hybrid Deep Learning Models for Human Activity Recognition
Abbaspour, S., Fotouhi-Ghazvini, F., Sedaghatbaf, A., Fotouhi, H., Vahabi, M., & Lindén, M. (2020) · Sensors · doi:10.3390/s20195707 · Cited by 142
“LSTMs [44] Temporal and sensor attentions are added to LSTMs to improve their performance for HAR.”
BackgroundHuang, W., Zhang, L., Wu, H., Min, F., & Song, A. (2022) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2022.3174816 · Cited by 154
“Concerning recent literatures on PAMAP2 dataset, [15] and [21] propose a combination of CNN and different LSTM architectures, where attention mechanism and multi-modal fusion are utilized to deal with sensor-based HAR .”
BackgroundMethodInfluentialSuh, S., Rey, V. F., & Lukowicz, P. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2022.110143 · Cited by 107
Deep Neural Networks for Sensor-Based Human Activity Recognition Using Selective Kernel Convolution
Gao, W., Zhang, L., Huang, W., Min, F., He, J., & Song, A. (2021) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2021.3102735 · Cited by 135
“Zeng et al. [31] embedded two continuous attention modules into recurrent networks along temporal and sensor axes, respectively, to improve the understandability of HAR.”
BackgroundHuman Activity Recognition for Elderly People Using Machine and Deep Learning Approaches
Hayat, A., Morgado‐Dias, F., Bhuyan, B. P., & Tomar, R. (2022) · Information · doi:10.3390/info13060275 · Cited by 107
“The attention-based mechanism could be used in HAR to highlight the most important and differentiable modalities (Zeng et al. [15] and Chen et al. [16]).”
MethodLi, B., & Sano, A. (2020) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3397318 · Cited by 116
“, saliency maps and attention mechanism) [86, 107], it is still very difficult to associate those deep outcomes with a standardized and human-understandable description, especially for sequential sensor data within the health and wellbeing context.”
BackgroundOn the role of features in human activity recognition
Haresamudram, H., Anderson, D. V., & Plötz, T. (2019) · Proceedings of the 23rd International Symposium on Wearable Computers · doi:10.1145/3341163.3347727 · Cited by 90
“The application of recurrent neural networks for HAR is extended in [45], where continuous attention mechanisms over the sensory channels as well as time are developed to improve the performance.”
BackgroundContrastive Self-supervised Learning for Sensor-based Human Activity Recognition
Khaertdinov, B., Ghaleb, E., & Asteriadis, S. (2021) · 2021 IEEE International Joint Conference on Biometrics (IJCB) · doi:10.1109/ijcb52358.2021.9484410 · Cited by 95
“Recent works on sensor-based Human Activity Recognition exploit attention mechanisms in order to adaptively focus on the most important parts of input signals [13, 26].”
BackgroundMethodCheng, X., Zhang, L., Tang, Y., Liu, Y., Wu, H., & He, J. (2022) · IEEE Sensors Journal · doi:10.1109/jsen.2022.3149337 · Cited by 99
“ith n=1) 89.97% 31.57M CondConv(with n=2) 91.80% 31.81M CondConv(with n=4) 92.70% 32.31M CondConv(with n=8) 93.79% 33.25M CondConv(with n=16) 94.01% 35.17M Yang et al.2018 [31] 91.40% Zeng et al.2018 [32] 89.96% Khan et al.2016 [33] 86.00% 6 3) The UNIMIB-SHAR dataset [16]: UNIMiB-SHAR is a new dataset including 11771…”
BackgroundInfluentialHuman activity recognition using marine predators algorithm with deep learning
Helmi, A. M., Al‐qaness, M. A. A., Dahou, A., & Elaziz, M. A. (2023) · Future Generation Computer Systems · doi:10.1016/j.future.2023.01.006 · Cited by 85
Barut, O., Zhou, L., & Luo, Y. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.2996578 · Cited by 70
“Zeng et al. [22] implemented continuous sensor and temporal attention mechanisms that are combined with a baseline LSTM model to evaluate their approach with PAMAP2, Daphnet Gait, and Skoda data sets.”
MethodExplainable Activity Recognition for Smart Home Systems
Das, D., Nishimura, Y., Vivek, R. P., Takeda, N., Fish, S. T., Plötz, T., & Chernova, S. (2023) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3561533 · Cited by 70
“Prior work within the field of activity recognition has shown the importance of incorporating temporal duration information to improve activity recognition performance [42, 77, 78], and we hypothesize that humans would similarly benefit from temporal information.”
BackgroundXu, S., Zhang, L., Tang, Y., Han, C., Wu, H., & Song, A. (2023) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2023.3277839 · Cited by 75
“In order to obtain a better interpretability into deep models’ behavior, Zeng et al. [27] implement two attention models over a LSTM backbone, which can adaptively concentrate on more important sensor modalities and time intervals simultaneously.”
BackgroundDual-Branch Interactive Networks on Multichannel Time Series for Human Activity Recognition
Tang, Y., Zhang, L., Wu, H., He, J., & Song, A. (2022) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2022.3193148 · Cited by 64
“[13] improved standard LSTM network by incorporating dual attention modules ( i.e., temporal attention and sensor attention), which can effectively capture long-range dependencies across a long sequence by placing an emphasis on more important time steps or sensor modalities.”
BackgroundMethodInfluentialA Survey of Deep Learning Based Models for Human Activity Recognition
Khan, N. S., & Ghani, M. S. (2021) · Wireless Personal Communications · doi:10.1007/s11277-021-08525-w · Cited by 69
“For such cases, other measures are used such as precision, recall [61], f-measure [80, 85, 86, 88, 92, 94–96], and specificity [61] which are more appropriate.”
MethodMulti-modal recognition of worker activity for human-centered intelligent manufacturing
Tao, W., Leu, M. C., & Yin, Z. (2020) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2020.103868 · Cited by 63
“Accuracy Hammerla et al. (2016) [33] 93.70 Murahari et al. (2018) [34] 87.50 Zeng et al. (2018) [35] 89.96 Xi et al. (2018) [36] 93.50 Xu et al. (2019) [37] 93.50 Our model 94.16 Table 6: Performance (”
Influential(2020) · doi:10.1145/3380999 · Cited by 57
“We compare GlobalFusion to both the state-of-the-art non-attentional DeepSense [40] framework, and attentional frameworks, including SADeepSense [42], BANet [34], and attnLSTM [47].”
BackgroundMethodInfluentialDeep Residual Network for Smartwatch-Based User Identification through Complex Hand Movements
Mekruksavanich, S., & Jitpattanakul, A. (2022) · Sensors · doi:10.3390/s22083094 · Cited by 62
“[55] presented a long short-term memory (LSTM) model based on continuous attention that emphasizes relevant sensor modalities and significant sections of the sensor data during TSC analysis.”
BackgroundCheng, D., Zhang, L., Bu, C., Wu, H., & Song, A. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.110789 · Cited by 55
Hoai Thu, N. T., & Han, D. S. (2021) · IEEE Access · doi:10.1109/access.2021.3122298 · Cited by 54
“[21] proposed a continuous attention-based long short-term memory (LSTM) model which pays more attention to important sensor modalities and salient parts of the sensor signal in HAR.”
BackgroundHuang, W., Zhang, L., Teng, Q., Song, C., & He, J. (2021) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2021.3092396 · Cited by 56
“Therefore, Zeng et al. [28] and Ma et al. [29] used attention in HAR to focus on the channels with more contribution for classification.”
BackgroundMethodResultInfluentialUnsupervised Human Activity Representation Learning with Multi-task Deep Clustering
Ma, H., Zhang, Z., Li, W., & Lu, S. (2021) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3448074 · Cited by 48
“Zeng et al. [68] proposed LSTM with two attention models called temporal attention and sensor attention for human activity recognition.”
BackgroundMethodA Survey on Human Activity Recognition Using Deep Learning Techniques and Wearable Sensor Data
Dua, N., Singh, S. N., Challa, S. K., Semwal, V. B., & Kumar, M. L. S. S. (2022) · Communications in computer and information science · doi:10.1007/978-3-031-24352-3_5 · Cited by 56
IF-ConvTransformer: A Framework for Human Activity Recognition Using IMU Fusion and ConvTransformer
Zhang, Y., Wang, L., Chen, H., Tian, A., Zhou, S., & Guo, Y. (2022) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3534584 · Cited by 84
“In [55], a model of similar structure with AttnSense was also proposed.”
BackgroundMethodA human activity recognition method using wearable sensors based on convtransformer model
Zhang, Z., Wang, W., An, A., Qin, Y., & Yang, F. (2023) · Evolving Systems · doi:10.1007/s12530-022-09480-y · Cited by 48
Recent use of deep learning techniques in clinical applications based on gait: a survey
Matsushita, Y., Tran, D. T., Yamazoe, H., & Lee, J. (2021) · Journal of Computational Design and Engineering · doi:10.1093/jcde/qwab054 · Cited by 58
Agac, S., & İncel, Ö. D. (2023) · Diagnostics · doi:10.3390/diagnostics13111861 · Cited by 58
“These datasets are extensively used in the sensor-based HAR domain [3], particularly in studies focusing on attention [19,27,29,34–36].”
BackgroundMethodInfluentialDeep Triplet Networks with Attention for Sensor-based Human Activity Recognition
Khaertdinov, B., Ghaleb, E., & Asteriadis, S. (2021) · IEEE International Conference on Pervasive Computing and Communications · doi:10.1109/percom50583.2021.9439116 · Cited by 52
“In some works, modern frameworks such as multimodal fusion ([11], [12]) and attention mechanisms [13] are adapted to sensor-based HAR.”
BackgroundMethodInfluentialLeveraging Activity Recognition to Enable Protective Behavior Detection in Continuous Data
Wang, C., Gao, Y., Mathur, A., Williams, A. C. D. C., Lane, N. D., & Bianchi‐Berthouze, N. (2021) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3463508 · Cited by 51
“For instance, several data representations consider the relative relationships between sensors/joints [41-43], with network architectures designed to enable local processing of movement dynamics [12, 13, 44-46, 64].”
BackgroundDeep Ensemble Learning for Human Activity Recognition Using Wearable Sensors via Filter Activation
Huang, W., Zhang, L., Wang, S., Wu, H., & Song, A. (2022) · ACM Transactions on Embedded Computing Systems · doi:10.1145/3551486 · Cited by 53
“Setting A to be 0.4 and c to be 100, we select six networks to evaluate the filter 15:13 [15] 75.54 [22] 76.04 [17] 89.3 [33] 97.82 [50] 93.79 [54] 95.18 [18] 76.83 [14] 72.8 [46] 89.96 [14] 98.81 [51] 91.7 [55] 96.63 [20] 79.32 [44] 75.66 [47] 85.4 [36] 97.52 [52] activation.”
BackgroundMethodInfluentialAdversarial Multi-view Networks for Activity Recognition
Bai, L., Yao, L., Wang, X., Kanhere, S. S., Guo, B., & Yu, Z. (2020) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3397323 · Cited by 40
“Most recent studies [19, 35, 38–40, 43, 49] deploy subject-independent settings in the evaluation to avoid the information leakage problem and evaluate the models’ generalization ability on new subjects.”
BackgroundMethodResultInfluentialLiang, W., Wang, F., Fan, A., Zhao, W., Yao, W., & Yang, P. (2023) · Sensors · doi:10.3390/s23094229 · Cited by 57
“Minimizing the number of IMUs while maintaining sufficient accuracy is becoming essential.”
BackgroundInfluentialGad, G., & Fadlullah, Z. M. (2022) · Sensors · doi:10.3390/s23010006 · Cited by 46
“[38] uses temporal and sensor attention combined with Long Short-Term Memory Units (LSTMs).”
Enhancing Human Activity Recognition in Smart Homes with Self-Supervised Learning and Self-Attention
Chen, H., Gouin-Vallerand, C., Bouchard, K., Gaboury, S., Couture, M., Bier, N., & Giroux, S. (2024) · Sensors · doi:10.3390/s24030884 · Cited by 48
“Zeng et al. [42] proposed temporal and sensor attention for HAR, focusing on important signals and sensor modalities.”
BackgroundConvolutional Neural Network With Multihead Attention for Human Activity Recognition
Tan, T.-H., Chang, Y.-L., Wu, J.-R., Chen, Y.-F., & Alkhaleefah, M. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3294421 · Cited by 41
“The work of [45] has presented two attention models for HAR, which are temporal attention and sensor attention.”
MethodRobust Inertial Motion Tracking through Deep Sensor Fusion across Smart Earbuds and Smartphone
Gong, J., Zhang, X., Huang, Y., Ren, J., & Zhang, Y. (2021) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3463517 · Cited by 39
“[47] proposed an LSTM based multi-sensor fusion architecture to recognize human actions from continuous data observations.”
BackgroundRepHAR: Decoupling Networks With Accuracy-Speed Tradeoff for Sensor-Based Human Activity Recognition
Teng, Q., Tang, Y., & Hu, G. (2023) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2023.3240198 · Cited by 39
Khaertdinov, B., Asteriadis, S., & Ghaleb, E. (2022) · IEEE Transactions on Biometrics Behavior and Identity Science · doi:10.1109/tbiom.2022.3180591 · Cited by 40
“In [31] and [32], attention mechanisms were added on top of the LSTM networks which were trained in an end-to-end manner using classical cross-entropy loss and triplet loss, respectively.”
MethodHaresamudram, H., Tang, C. I., Suh, S., Lukowicz, P., & Plötz, T. (2025) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3729467 · Cited by 33
“Temporal attention was applied by Murahari et al. [130] whereas continuous sensor and temporal attention were evaluated by Zeng et al. [207], both leading to increased performance.”
Ahmed, N., Numan, M. O. A., Kabir, R., Islam, M. R., & Watanobe, Y. (2024) · Sensors · doi:10.3390/s24134343 · Cited by 34
“Imen et al. [38] CWT-CNN 2D 54.4 Imen et al.[38] Random Forest 62.2 Zeng et al. [62] LSTM with Continuous Temporal Attention 89.9 Xi et al. [63] Deep Dilated Convolutional Networks 93.2 Qian et al. [64] Distribution-Embedded Deep NN (DDNN) 93.40 Sanchez et al. [65] CNN for Image Representation…”
BackgroundMethodInfluentialLearning Temporal and Bodily Attention in Protective Movement Behavior Detection
Wang, C., Peng, M., Olugbade, T., Lane, N. D., Williams, A. C. D. C., & Bianchi‐Berthouze, N. (2019) · International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW) · doi:10.1109/aciiw.2019.8925084 · Cited by 39
“On the other hand, except for that the BANet-compat is only a representative of the architectures used in [19][20][21], the results imply that encoding the importance of body joints at a single timestep is not valuable to the detection of protective behavior, but should be delayed to a higher-level processing given a…”
MethodResultWang, X., Zhang, L., Huang, W., Wang, S., Wu, H., He, J., & Song, A. (2021) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2021.3132088 · Cited by 42
“In HAR scenario, Zeng et al.[9] at the earliest time exploited CNN to automatically extract the local dependency and scale invariant features from raw acceleration time series.”
BackgroundLiang, J., Zhang, L., Bu, C., Cheng, D., Wu, H., & Song, A. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.122180 · Cited by 35
Yoshimura, N., Maekawa, T., Hara, T., Wada, A., & Namioka, Y. (2022) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3534572 · Cited by 38
“Some methods use recurrent models, such as LSTM and gated recurrent units (GRU) [19, 60], and others employ one-dimensional convolutional neural networks (CNNs) [19, 31, 33].”
MethodWeakly Supervised Multi-Task Representation Learning for Human Activity Analysis Using Wearables
Sheng, T., & Huber, M. (2020) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3397330 · Cited by 28
“We replicate previous work [14, 40] to downsample the data from 100Hz to 33.”
MethodQuan, Z., Chen, Q., Zhang, M., Hu, W., Zhao, Q., Hou, J., Li, Y., & Liu, Z. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3255832 · Cited by 29
“Zeng et al. [37] combined a recurrent neural network and an attention mechanism to propose a temporal attention and sensor attention model, which can more adaptively give attention to the most important sensor information.”
BackgroundFatigue assessment using ECG and actigraphy sensors
Bai, Y., Guan, Y., & Ng, W. (2020) · International Symposium on Wearable Computers · doi:10.1145/3410531.3414308 · Cited by 33
“For continuous signals, temporal attention regularisation was normally used which encourages the continuity [32].”
Contrastive Learning with Cross-Modal Knowledge Mining for Multimodal Human Activity Recognition
Brinzea, R., Khaertdinov, B., & Asteriadis, S. (2022) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn55064.2022.9892522 · Cited by 30
“based on attention mechanisms [16], [17], and deep metric learning, which attempt to learn robust feature embeddings relying on various contrastive loss functions, such as triplet loss [18], in a supervised manner.”
Ankalaki, S., & Thippeswamy, M. N. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-15766-3 · Cited by 29
PatchHAR: A MLP-Like Architecture for Efficient Activity Recognition Using Wearables
Wang, S., Zhang, L., Wang, X., Huang, W., Wu, H., & Song, A. (2024) · IEEE Transactions on Biometrics Behavior and Identity Science · doi:10.1109/tbiom.2024.3354261 · Cited by 33
“In our work, following the same setting [23], [24], we still vertically stacked all axes of sensor signals to form 2D matrices.”
MethodCentaur: Robust Multimodal Fusion for Human Activity Recognition
Xaviar, S., Yang, X., & Ardakanian, O. (2024) · IEEE Sensors Journal · doi:10.1109/jsen.2024.3388893 · Cited by 44
“We follow the preprocessing steps described in [47], [48], and [49] to segment sensor readings using a 5.12 s sliding window with 1 s overlap.”
InfluentialTransformer-Based Recognition of Activities of Daily Living from Wearable Sensor Data
Augustinov, G., Nisar, M. A., Li, F., Tabatabaei, A., Grzegorzek, M., Sohrabi, K., & Fudickar, S. (2022) · International Workshop on Sensor-based Activity Recognition and Interaction (iWOAR) · doi:10.1145/3558884.3558895 · Cited by 26
“The authors in [26] used a combination of attention mechanisms and LSTM layers and showed an improvement over baseline LSTM models with no attention on multiple HAR datasets.”
MethodEasy Ensemble: Simple Deep Ensemble Learning for Sensor-Based Human Activity Recognition
Hasegawa, T., & Kondo, K. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2022.3222221 · Cited by 28
“AlexNet has multiple CNN paths for parallel processing of multiple convolutions on multiple graphics processing units.”
Yao, M., Zhang, L., Cheng, D., Qin, L., Liu, X., Fu, Z., Wu, H., & Song, A. (2024) · IEEE Sensors Journal · doi:10.1109/jsen.2024.3371462 · Cited by 31
“To facilitate comparison [19], we use a sliding window cut dataset of length 171 with 78% overlap.”
MethodSelf-citationHuman Complex Activity Recognition With Sensor Data Using Multiple Features
Huan, R., Jiang, C., Ge, L., Shu, J., Zhan, Z., Chen, P., Chi, K., & Liang, R. (2021) · IEEE Sensors Journal · doi:10.1109/jsen.2021.3130913 · Cited by 25
Han, C., Zhang, L., Tang, Y., Xu, S., Min, F., Wu, H., & Song, A. (2022) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2022.3191653 · Cited by 26
Hierarchical multi-view aggregation network for sensor-based human activity recognition
Zhang, X., Wong, Y., Kankanhalli, M., & Geng, W. (2019) · PLoS ONE · doi:10.1371/journal.pone.0221390 · Cited by 27
An Efficient Diverse-Branch Convolution Scheme for Sensor-Based Human Activity Recognition
Han, C., Zhang, L., Xu, S., Wang, X., Wu, H., & Song, A. (2023) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2023.3265128 · Cited by 29
“Zeng et al. [10] at the earliest time applied CNN to automatically extract activity features from acceleration time series data, which is used to mitigate the problem of handcrafted feature design in the HAR scenario.”
BackgroundMethodData Augmentation Strategies for Human Activity Data Using Generative Adversarial Neural Networks
Hoelzemann, A., Sorathiya, N., & Laerhoven, K. V. (2021) · IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops) · doi:10.1109/percomworkshops51409.2021.9431046 · Cited by 26
“Not choosing the correct metrics with respect to the dataset attributes. other papers, for instance [26], [27] that worked with similar architectures and datasets.”
BackgroundLiang, J., Zhang, L., Han, C., Bu, C., Wu, H., & Song, A. (2023) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2023.3271306 · Cited by 26
“To remove the impact of different sensing modalities, following previous literatures including Yang et al. [7], Zeng et al. [10], and Ronao et al. [6], we only perform one-dimensional (1D) convolutions along temporal dimension of multivariate time series sensor signals, where all the feature maps…”
BackgroundMethodInfluentialXia, H., Tang, J., Qiao, J., Zhang, J., & Yu, W. (2022) · Neural Computing and Applications · doi:10.1007/s00521-021-06809-7 · Cited by 27
“…representation interpretation for the hidden layer [166, 167], sensitivity analysis [166–171], model imitation [172–175], and multimodal interpretation [176, 177], and the other type focuses on recurrent neural networks, such as its visualization [178–180] and neural attention mechanism [181–184].”
BackgroundResource-Efficient Continual Learning for Sensor-Based Human Activity Recognition
Leite, C. F. S., & Xiao, Y. (2022) · ACM Transactions on Embedded Computing Systems · doi:10.1145/3530910 · Cited by 29
Temporal Feature Alignment in Contrastive Self-Supervised Learning for Human Activity Recognition
Khaertdinov, B., & Asteriadis, S. (2022) · IEEE International Joint Conference on Biometrics (IJCB) · doi:10.1109/ijcb54206.2022.10007984 · Cited by 28
“Moreover, attention mechanisms have been exploited in various forms, such as sensor attention, temporal attention and transformer self-attention [40, 26, 20].”
BackgroundHuman Activity Recognition Based on Two-Channel Residual–GRU–ECA Module with Two Types of Sensors
Wang, X., & Shang, J. (2023) · Electronics · doi:10.3390/electronics12071622 · Cited by 20
Siamese Networks for Weakly Supervised Human Activity Recognition
Sheng, T., & Huber, M. (2019) · 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC) · doi:10.1109/smc.2019.8914045 · Cited by 21
“The results in Table II show that the proposed weakly supervised method outperforms the state-of-the-art, LSTM with continuous temporal and sensor attention [28] , on the DG dataset.”
BackgroundMethodRecognition of Fine-Grained Walking Patterns Using a Smartwatch with Deep Attentive Neural Networks
Kim, H., Kim, H.-J., Park, J., Ryu, J.-K., & Kim, S.-C. (2021) · Sensors · doi:10.3390/s21196393 · Cited by 20
“A similar approach was used in recent studies [48,49].”
ResultYang, X., Xu, Z., Liu, H., Shull, P. B., Redmond, S. J., Liu, G., & Wang, C. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3400022 · Cited by 21
GraphConvLSTM: Spatiotemporal Learning for Activity Recognition with Wearable Sensors
Han, J., He, Y., Liu, J., Zhang, Q., & Jing, X. (2019) · Globecom · doi:10.1109/globecom38437.2019.9013934 · Cited by 30
“Recently the authors of [10] adopt sensor attention to adaptively select important sensor signals and assign different weights to them.”
BackgroundMethodImproving Cross-Subject Activity Recognition via Adversarial Learning
Leite, C. F. S., & Xiao, Y. (2020) · IEEE Access · doi:10.1109/access.2020.2993818 · Cited by 21
“This is a common choice of subjects in the literature [7], [8], [26], [37], [40].”
BackgroundMethodRegional Attention Network (RAN) for Head Pose and Fine-Grained Gesture Recognition
Behera, A., Wharton, Z., Liu, Y., Ghahremani, M., Kumar, S., & Bessis, N. (2020) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2020.3031841 · Cited by 21
“visual question answering (VQA) [10], image captioning [11], [12], [13], human activities recognition [14], [15], [16], and other applications.”
BackgroundSensor-based Detection and Classification of Soccer Goalkeeper Training Exercises
Haladjian, J., Schlabbers, D., Taheri, S., Tharr, M., & Bruegge, B. (2020) · ACM Transactions on Internet of Things · doi:10.1145/3372342 · Cited by 18
“…Long-Short-Term-Memory (LSTM) - have been adapted to exploit the time-dependency of the sensor signals, attaining the highest recognition results so far on large public benchmark datasets [Li et al. 2018; Murahari and Plötz 2018; Neverova et al. 2016; Ordóñez and Roggen 2016; Zeng et al. 2018].”
BackgroundLi, X., Xu, H., Wang, Y., Zeng, J., Li, Y., Li, X., Ai, W., Zheng, H., & Duan, Y. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3491362 · Cited by 19
“Zeng et al. [22] incorporated two attention mechanisms into an LSTM-based network to emphasize more prominent information about sensor time series and sensor modalities.”
Bu, C., Zhang, L., Cui, H., Cheng, D., Wu, H., & Song, A. (2024) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2024.3351260 · Cited by 16
“For example, Zeng et al. [15] and Ronao and Cho [16] at the earliest time have adopted deep convolutional networks to extract activity features from multivariate time series sensor signals.”
BackgroundTeng, Q., Tang, Y., & Hu, G. (2024) · IEEE Sensors Journal · doi:10.1109/jsen.2024.3364187 · Cited by 18
“Zeng et al. [36] proposed dual attention based on recurrent networks (i.e., temporal attention and sensor attention) to simultaneously highlight salient time windows and key sensor modalities.”
BackgroundHaladjian, J. (2019) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3369813 · Cited by 19
“recognition applications with wearables [41, 60].”
MethodHierarchical Self Attention Based Autoencoder for Open-Set Human Activity Recognition
Tonmoy, M. T. H., Mahmud, S., Rahman, A. K. M. M., Amin, M. A., & Ali, A. A. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-75768-7_28 · Cited by 22
Robust Human Activity Recognition via Wearable Sensors Using Dynamic Gaussian Kernel Learning
Wang, S., Sun, Y., Sha, Y., Yang, G., Cheng, D., Zhang, L., Wu, H., & Song, A. (2024) · IEEE Sensors Journal · doi:10.1109/jsen.2024.3355704 · Cited by 17
“Zeng et al. [30] proposed an initial CNN-based method for feature extraction, which effectively captures local dependencies and scale invariance properties of accelerometer data.”
MethodResultA hybrid CNN and BLSTM network for human complex activity recognition with multi-feature fusion
Huan, R., Zhan, Z., Ge, L., Chi, K., Chen, P., & Liang, R. (2021) · Multimedia Tools and Applications · doi:10.1007/s11042-021-11363-4 · Cited by 18
“Zeng et al. [41] applied LSTM to automatically highlight the important parts of the time series signals and applied sensor attention to reweight the sensor modality importance.”
BackgroundInfluentialKumar, P., & Chauhan, S. (2021) · CCF Transactions on Pervasive Computing and Interaction · doi:10.1007/s42486-021-00063-5 · Cited by 13
RetractedHuman Activity Recognition: A Dynamic Inductive Bias Selection Perspective
Hamidi, M., & Osmani, A. (2021) · Sensors · doi:10.3390/s21217278 · Cited by 17
“These are widely used in natural language processing [137] and also in human activity recognition where, for example, the authors in [138] leveraged both temporal and sensor attention layers in order to help the recognition model to focus on more informative time steps of the inputs as well as on more important sensin…”
BackgroundLiu, J., Zhang, L., Liu, X., Yao, M., Wang, L., Wu, H., & Song, A. (2025) · IEEE Sensors Journal · doi:10.1109/jsen.2025.3565646 · Cited by 13
“For consistency, the raw data were sampled at 100 Hz and later subsampled to 33.3 Hz for analysis [44].”
InfluentialSelf-citationMachine learning models for wearable-based human activity recognition: A comparative study
Ciortuz, G., Pour, H. H., Irshad, M. T., Nisar, M. A., Huang, X., & Fudickar, S. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.130911 · Cited by 18
Yu, J., Zhang, L., Cheng, D., Bu, C., Wu, H., & Song, A. (2024) · IEEE Sensors Journal · doi:10.1109/jsen.2024.3412736 · Cited by 15
“For consistency, we follow the same protocol for partitioning the data into training, validation, and test sets [42].”
MethodInfluentialDynamic instance-aware layer-bit-select network on human activity recognition using wearable sensors
Ye, N., Zhang, L., Cheng, D., Bu, C., Sun, S., Wu, H., & Song, A. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.109260 · Cited by 12
Metadata
- Authors
- Zeng, M., Gao, H., Yu, T., Mengshoel, O. J., Langseth, H., Lane, I., & Liu, X.
- Affiliations
- Carnegie Mellon University (US)
- The Norwegian University of Science and Technology
- Google (United States) (US)
- Venue
- Proceedings of the 2018 ACM International Symposium on Wearable Computers
- Publisher
- ACM
- Published
- 2018-10-08
- Pages
- 56-63
- OpenAlex
- W2894702700
- Cited by
- 185
- References
- 30
- Citations by year
- 2026: 6 · 2025: 18 · 2024: 32 · 2023: 25 · 2022: 35 · 2021: 37 · 2020: 19 · 2019: 13
- FWCI
- 5.20
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM International Symposium on Wearable Computers (ISWC) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Physical Sciences › Engineering › Biomedical Engineering › Gait Recognition and Analysis
- Keywords
- human activity recognition, recurrent neural networks, attention mechanism, temporal attention, interpretability, noise robustness, continuity constraints
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.