The DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.
Understanding and improving recurrent networks for human activity recognition by continuous attention
Abstract
Deep neural networks, including recurrent networks, have been successfully applied to human activity recognition. Unfortunately, the final representation learned by recurrent networks might encode some noise (irrelevant signal components, unimportant sensor modalities, etc.). Besides, it is difficult to interpret the recurrent networks to gain insight into the models' behavior. To address these issues, we propose two attention models for human activity recognition: temporal attention and sensor attention. These two mechanisms adaptively focus on important signals and sensor modalities. To further improve the understandability and mean Fl score, we add continuity constraints, considering that continuous sensor signals are more robust than discrete ones. We evaluate the approaches on three datasets and obtain state-of-the-art results. Furthermore, qualitative analysis shows that the attention learned by the models agree well with human intuition.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Temporal attention was applied by Murahari et al. [138] whereas continuous sensor and temporal attention were evaluated by Zeng et al. [214], both leading to increased performance.”
From Past, Present, and Future of Sensor-based Human Activity Recognition Using Wearables: A Surveying Tutorial on a Still Challenging Task (2025) · cites this paper as [214]
The cited source says what the claim attributes to it.
In the citing paper
“[226] applies temporal attention and sensor attention into LSTM to improve the overall activity recognition accuracy by adaptively focusing on important time windows and sensor modalities.”
From Deep Learning in Human Activity Recognition with Wearable Sensors: A Review on Advances (2022) · cites this paper as [226]
In the citing paper
“The squeeze-and-excitation network (SENet) [15] is one of the representative methods, which brings clear performance gain across various deep CNN architectures via learning channel attention for each convolution block.”
From DanHAR: Dual Attention Network for multimodal human activity recognition using wearable sensors (2021) · cites this paper as [15]
In the citing paper
“SENet [15] for the first time proposes an effective mechanism to learn channel attention and achieves promising performance.”
From DanHAR: Dual Attention Network for multimodal human activity recognition using wearable sensors (2021) · cites this paper as [15]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 8×
“Models PAMAP2 DG Skoda4 LSTM baseline ([12]) (LSTM without Attention) 0.”
BackgroundMethodResultInfluentialNeural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213 · cited 4×
“The attention approach is proposed to mitigate this problem for speech and natural language processing (NLP) [3, 14], and it offers means to explicate the inner workings of neural networks.”
BackgroundInfluentialApproximation by superpositions of a sigmoidal function
Cybenko, G. (1989) · Mathematics of Control, Signals, and Systems · doi:10.1007/bf02551274 · Cited by 13,883
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.03044 · Cited by 8,248
Activity Recognition from User-Annotated Acceleration Data
Bao, L., & Intille, S. S. (2004) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-24646-6_1 · Cited by 3,147 · cited 4×
“RECOGNITION We frame HAR as a sequence classification problem [4].”
BackgroundMethodInfluentialDeep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition
Ordóñez, F., & Roggen, D. (2016) · Sensors · doi:10.3390/s16010115 · Cited by 2,761 · cited 4×
“The baseline methods include standard LSTM [12], DeepConvLSTM [19], and LSTM-S (LSTM + sample-by-sample analysis) [11].”
BackgroundMethodInfluentialAttention-Based Models for Speech Recognition
Chorowski, J., Bahdanau, D., Serdyuk, D., Cho, K., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1506.07503 · Cited by 2,021 · cited 1×
“Deep neural networks (DNNs) has had revolutionary impact in speech recognition [7], image classification [24], and machine translation [3].”
BackgroundA tutorial on human activity recognition using body-worn inertial sensors
Bulling, A., Blanke, U., & Schiele, B. (2014) · ACM Computing Surveys · doi:10.1145/2499621 · Cited by 1,668 · cited 2×
“INTRODUCTION Wearable-based human activity recognition (HAR) systems are an indispensable component in mobile ubiquitous computing and human-computer interaction [5].”
BackgroundIntroducing a New Benchmarked Dataset for Activity Monitoring
Reiss, A., & Stricker, D. (2012) · 2012 16th International Symposium on Wearable Computers · doi:10.1109/iswc.2012.13 · Cited by 1,456 · cited 1×
“The first is the PAMAP2 dataset (Physical Activity Monitoring for Aging People 2) [21].”
Background(2015) · Cited by 989
Convolutional Neural Networks for Human Activity Recognition using Mobile Sensors
Zeng, M., Nguyen, L. T., Yu, B., Mengshoel, O. J., Zhu, J., Wu, P., & Zhang, J. (2014) · Proceedings of the 6th International Conference on Mobile Computing, Applications and Services · doi:10.4108/icst.mobicase.2014.257786 · Cited by 876 · cited 1×
“The 1D convolutional neural netowrks (CNN) employ more effective signal processing units, such as convolution (capturing local dependency), pooling (capturing time invariant features), and it also makes use of the available label information in feature extraction [28].”
MethodSelf-citationDeepSense: A Unified Deep Learning Framework for Time-Series Mobile Sensing Data Processing
Yao, S., Hu, S., Zhao, Y., Zhang, A., & Abdelzaher, T. (2017) · Proceedings of the 26th International Conference on World Wide Web · doi:10.1145/3038912.3052577 · Cited by 667 · cited 1×
“(ii) Different sensor modalities play different roles for recognizing different activities [29, 26].”
BackgroundEnsembles of Deep LSTM Learners for Activity Recognition using Wearables
Guan, Y., & Plötz, T. (2017) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3090076 · Cited by 511 · cited 8×
“[8] used ensemble LSTM model to capture diverse data during the training and significantly improved the recognition performance and robustness.”
BackgroundMethodResultInfluentialAnalyzing features for activity recognition
Huynh, T., & Schiele, B. (2005) · Proceedings of the 2005 joint conference on Smart objects and ambient intelligence: innovative context-aware services: usages and technologies · doi:10.1145/1107548.1107591 · Cited by 339
Feature Learning for Activity Recognition in Ubiquitous Computing
Plötz, T., Hammerla, N., & Olivier, P. (2011) · Newcastle University · doi:10.25405/ncl.34175319 · Cited by 337
Activity Recognition from On-Body Sensors: Accuracy-Power Trade-Off by Dynamic Sensor Selection
Zappi, P., Lombriser, C., Stiefmeier, T., Farella, E., Roggen, D., Benini, L., & Tröster, G. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-77690-1_2 · Cited by 329
CNN-based sensor fusion techniques for multimodal human activity recognition
Münzner, S., Schmidt, P., Reiss, A., Hanselmann, M., Stiefelhagen, R., & Dürichen, R. (2017) · Proceedings of the 2017 ACM International Symposium on Wearable Computers · doi:10.1145/3123021.3123046 · Cited by 235 · cited 1×
“To fuse different sensor modalities [25, 17], the 2D CNN regards the set of signals as an image.”
MethodPottie, G. J. (2002) · 1998 Information Theory Workshop (Cat. No.98EX131) · doi:10.1109/itw.1998.706478 · Cited by 234
Neural Machine Translation and Sequence-to-sequence Models: A TutorialPreprint
Neubig, G. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.01619 · Cited by 213
PD Disease State Assessment in Naturalistic Environments Using Deep Learning
Hammerla, N., Fisher, J., Andras, P., Rochester, L., Walker, R., & Ploetz, T. (2015) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v29i1.9484 · Cited by 179 · cited 1×
“DNNs also provided promising results in HAR domain ( e.g., [20, 22, 10]).”
BackgroundDeep Convolutional Neural Networks for Human Activity Recognition with Smartphone Sensors
Ronao, C. A., & Cho, S.-B. (2015) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-26561-2_6 · Cited by 176 · cited 1×
“DNNs also provided promising results in HAR domain ( e.g., [20, 22, 10]).”
BackgroundMorales, F. J. O., & Roggen, D. (2016) · Proceedings of the 2016 ACM International Symposium on Wearable Computers · doi:10.1145/2971763.2971764 · Cited by 136 · cited 1×
“[16] demonstrate the use of transfer learning to reduce the influence on randomly initialized CNN weights.”
BackgroundDeep Activity Recognition Models with Triaxial Accelerometers
Alsheikh, M. A., Selim, A., Niyato, D., Doyle, L., Lin, S., & Tan, H.-P. (2015) · arXiv · doi:10.48550/arxiv.1511.04664 · Cited by 126 · cited 1×
“Similar to speech recognition, the combination of traditional sequence models (HMMs) and DNNs is also applied in HAR [31, 1].”
MethodBächlin, M., Roggen, D., Tröster, G., Plotnik, M., Inbar, N., Meidan, I., Herman, T., Brozgol, M., Shaviv, E., Giladi, N., et al. (2009) · International Symposium on Wearable Computers · doi:10.1109/iswc.2009.14 · Cited by 100 · cited 3×
“And the sensor attached to the leg is more important than that on the trunk [2].”
BackgroundMethodInfluentialSemi-supervised convolutional neural networks for human activity recognition
Zeng, M., Yu, T., Wang, X., Nguyen, L. T., Mengshoel, O. J., & Lane, I. (2017) · 2017 IEEE International Conference on Big Data (Big Data) · doi:10.1109/bigdata.2017.8257967 · Cited by 85 · cited 1×
“[30] propose the CNN encoder-decoder and CNN ladder for HAR in the semi-supervised setting.”
MethodSelf-citationHuman activity recognition with HMM-DNN model
Zhang, L., Wu, X. H., & Luo, D. (2015) · IEEE International Conference on Cognitive Informatics & Cognitive Computing (ICCI*CC) · doi:10.1109/icci-cc.2015.7259385 · Cited by 62 · cited 1×
“Similar to speech recognition, the combination of traditional sequence models (HMMs) and DNNs is also applied in HAR [31, 1].”
MethodMobile Lifelogger – Recording, Indexing, and Understanding a Mobile User’s Life
Chennuru, S. K., Chen, P.-W., Zhu, J., & Zhang, J. Y. (2012) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-642-29336-8_15 · Cited by 41 · cited 1×
“), HAR systems are used in a large number of context aware applications such as daily life logging [6], and cross-device user recognition [23].”
BackgroundXRec: Behavior-Based User Recognition Across Mobile Devices
Wang, X., Yu, T., Zeng, M., & Tague, P. (2017) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3130975 · Cited by 19 · cited 1×
“), HAR systems are used in a large number of context aware applications such as daily life logging [6], and cross-device user recognition [23].”
BackgroundSelf-citationKim, S., & Lane, I. R. (2017) · Interspeech (USB) · doi:10.21437/interspeech.2017-1536 · Cited by 17 · cited 3×
“The attention approach is proposed to mitigate this problem for speech and natural language processing (NLP) [3, 14], and it offers means to explicate the inner workings of neural networks.”
BackgroundSelf-citationAdaptive Activity Recognition with Dynamic Heterogeneous Sensor Fusion
Zeng, M., Wang, X., Nguyen, L. T., Wu, P., Mengshoel, O. J., & Zhang, J. (2014) · Mobile Computing, Applications, and Services · doi:10.4108/icst.mobicase.2014.257787 · Cited by 15 · cited 1×
“(ii) Different sensor modalities play different roles for recognizing different activities [29, 26].”
BackgroundSelf-citationInfluence-Directed Explanations for Deep Convolutional NetworksPreprint
Leino, K. R. M., Sen, S., Datta, A., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.03788 · Cited by 6 · cited 1×
“Although LSTM is able to forget the unimportant signals and remember the important signals, to quantitatively evaluate the impact from the previous signals and determine signal importance, In future work, we need to use other approach, such as analyzing the gradients during the back propagation [15].”
Method
Research works printed but not linked: 1
- Nils Y Hammerla , Shane Halloran , and Thomas Ploetz . 2016. Deep , convolutional, and recurrent models for human activity recognition using wearables. IJCAI ( 2016 ). Nils Y Hammerla, Shane Halloran, and Thomas Ploetz. 2016. Deep, convolutional, and recurrent models for human activity recognition using wearables. IJCAI (2016).
Cited by
Deep Learning in Human Activity Recognition with Wearable Sensors: A Review on Advances
Zhang, S., Li, Y., Zhang, S., Shahabi, F., Xia, S., Deng, Y., & Alshurafa, N. (2022) · Sensors · doi:10.3390/s22041476 · Cited by 625
“Their method has outperformed regular CNN considerably on a number of public datasets such as PAMAP2 [228], WISDM [155], UNIMIB SHAR [188], and Opportunity [59].”
BackgroundMethodInfluentialDeep Learning for Sensor-based Human Activity Recognition
Chen, K., Zhang, D., Yao, L., Guo, B., Yu, Z., & Liu, Y. (2022) · ACM Computing Surveys · doi:10.1145/3447744 · Cited by 690
Multi-input CNN-GRU based human activity recognition using wearable sensors
Dua, N., Singh, S. N., & Semwal, V. B. (2021) · Computing · doi:10.1007/s00607-021-00928-8 · Cited by 432
“The proposed model is compared with the models presented in [17, 20–22, 36, 37, 49–52].”
MethodDeep PPG: Large-Scale Heart Rate Estimation with Convolutional Neural Networks
Reiss, A., Indlekofer, I., Schmidt, P., & Van Laerhoven, K. (2019) · Sensors · doi:10.3390/s19143079 · Cited by 377
“Deep learning has already been applied to various time-series signals, such as for human activity recognition [20–22] or gait parameter extraction [23].”
MethodDeep Learning for Sensor-based Human Activity Recognition: Overview, Challenges and Opportunities
Chen, K., Zhang, D., Yao, L., Guo, B., Yu, Z., & Liu, Y. (2020) · arXiv · doi:10.48550/arxiv.2001.07416 · Cited by 314
Metadata
- Authors
- Zeng, M., Gao, H., Yu, T., Mengshoel, O. J., Langseth, H., Lane, I., & Liu, X.
- Affiliations
- Carnegie Mellon University (US)
- The Norwegian University of Science and Technology
- Google (United States) (US)
- Venue
- Proceedings of the 2018 ACM International Symposium on Wearable Computers
- Publisher
- ACM
- Published
- 2018-10-08
- Pages
- 56-63
- OpenAlex
- W2894702700
- Cited by
- 185
- References
- 30
- Citations by year
- 2026: 6 · 2025: 18 · 2024: 32 · 2023: 25 · 2022: 35 · 2021: 37 · 2020: 19 · 2019: 13
- FWCI
- 5.20
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM International Symposium on Wearable Computers (ISWC) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Physical Sciences › Engineering › Biomedical Engineering › Gait Recognition and Analysis
- Keywords
- human activity recognition, recurrent neural networks, attention mechanism, temporal attention, interpretability, noise robustness, continuity constraints
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.