Sequential Weakly Labeled Multiactivity Localization and Recognition on Wearable Sensors Using Recurrent Attention Networks
Open accessThe DOI matches its Crossref record; 17 of the 17 references it cites were found in a registry or an index.
Abstract
With the popularity and development of the wearable devices such as smartphones, human activity recognition (HAR) based on sensors has become as a key research area in human computer interaction and ubiquitous computing. The emergence of deep learning leads to a recent shift in the research of HAR, which requires massive strictly labeled data in supervised learning scenario. In comparison with video data, activity data recorded from accelerometer or gyroscope are often more difficult to interpret and segment. Recently, several attention mechanisms are proposed to handle the weakly labeled human activity data, which do not require accurate data annotation. However, these attention-based models can only handle the weakly labeled dataset whose sample includes one target activity, as a result it limits efficiency and practicality. In the article, we propose a recurrent attention networks (RAN) to handle sequential weakly labeled multiactivity recognition and location tasks. The model can repeatedly perform steps of attention on multiple activities of one sample and each step is corresponding to the current focused activity. The effectiveness of the RAN model is validated on a collected sequential weakly labeled multiactivity dataset and the other two public datasets. The experiment results show that our RAN model can simultaneously infer multiactivity types from the coarse-grained sequential weak labels and determine specific locations of every target activity with only knowledge of which types of activities contained in the long sequence. It will greatly reduce the burden of manual labeling.1
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Activity Recognition from User-Annotated Acceleration Data
Bao, L., & Intille, S. S. (2004) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-24646-6_1 · Cited by 3,147
Activity recognition using cell phone accelerometers
Kwapisz, J. R., Weiss, G. M., & Moore, S. A. (2011) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1964897.1964918 · Cited by 2,912 · cited 2×
“e@nuist.edu.cn). performed by people in their daily lives. The traditional machine learning approaches such as Support Vector Machine and Hidden Markov Model, based on hand-crafted features [7], [8], [9], [10], have been extensively used in the HAR fields. Due to the emergence of deep learning, there is a recent shift…”
MethodDeep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition
Ordóñez, F., & Roggen, D. (2016) · Sensors · doi:10.3390/s16010115 · Cited by 2,761 · cited 2×
“tation technique and discrete cosine transformation (DCT), then fed the 2D signal image into a two layer 2D CNN to classify its signal image equaling to the desired activity recognition. Ordez et al. [16] proposed an architecture comprised of CNN and LSTM recurrent units (DeepConvLSTM), which outperforms CNN. However,…”
BackgroundMethodDeep learning for sensor-based activity recognition: A survey
Wang, J., Chen, Y., Hao, S., Peng, X., & Hu, L. (2019) · Pattern Recognition Letters · doi:10.1016/j.patrec.2018.02.010 · Cited by 2,014
A tutorial on human activity recognition using body-worn inertial sensors
Bulling, A., Blanke, U., & Schiele, B. (2014) · ACM Computing Surveys · doi:10.1145/2499621 · Cited by 1,668 · cited 1×
“H UMAN activity is unique, as the information inferred from raw activity data has been proved to be very critical in human activity recognition (HAR) [1], health support [2], and smart homes [3] to name a few.”
BackgroundConvolutional Neural Networks for Human Activity Recognition using Mobile Sensors
Zeng, M., Nguyen, L. T., Yu, B., Mengshoel, O. J., Zhu, J., Wu, P., & Zhang, J. (2014) · Proceedings of the 6th International Conference on Mobile Computing, Applications and Services · doi:10.4108/icst.mobicase.2014.257786 · Cited by 876 · cited 1×
“Deep learning approaches such as Convolutional Neural Networks (CNN) [11], [12], [13], [14] and Recurrent Neural Networks (RNN) [15], [16], [17] have proven to be more effective than the shallow learning techniques in discovering, learning, and inferring complex activity from data.”
BackgroundHuman Activity Recognition Using Wearable Sensors by Deep Convolutional Neural Networks
Jiang, W., & Yin, Z. (2015) · Proceedings of the 23rd ACM international conference on Multimedia · doi:10.1145/2733373.2806333 · Cited by 676 · cited 2×
“Furthermore, Jiang and Yin [13] converted the raw sensor signal into 2D signal image by utilizing a specific permutation technique and discrete cosine transformation (DCT), then fed the 2D signal image into a two layer 2D CNN to classify the signal image equaling to its desired activity recognition.”
BackgroundMethodDeep, Convolutional, and Recurrent Models for Human Activity Recognition Using Wearables
Hammerla, N. Y., Halloran, S., & Ploetz, T. (2016) · arXiv · doi:10.48550/arxiv.1604.08880 · Cited by 606
A Survey of Online Activity Recognition Using Mobile Phones
Shoaib, M., Bosch, S., Incel, O., Scholten, H., & Havinga, P. (2015) · Sensors · doi:10.3390/s150102059 · Cited by 489 · cited 1×
“However, these methods that belong to supervised learning [26] requires massive data with perfect ground-truth to train models.”
BackgroundKeeping the Resident in the Loop: Adapting the Smart Home to the User
Rashidi, P., & Cook, D. J. (2009) · IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans · doi:10.1109/tsmca.2009.2025137 · Cited by 443 · cited 1×
“H UMAN activity is unique, as the information inferred from raw activity data has been proved to be very critical in human activity recognition (HAR) [1], health support [2], and smart homes [3] to name a few.”
BackgroundJetley, S., Lord, N. A., Lee, N., & Torr, P. H. S. (2018) · arXiv · doi:10.48550/arxiv.1804.02391 · Cited by 383 · cited 2×
“However, compared with the attention mechanism proposed in [19], [28], the soft attention only adds the weights to the features extracted at the end of the CNN pipeline, which potentially weakens the specific features selection capability of attention mechanism.”
BackgroundA Deep Learning Approach to Human Activity Recognition Based on Single Accelerometer
Chen, Y., & Xue, Y. (2015) · 2015 IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc.2015.263 · Cited by 378 · cited 2×
“For instance, Chen and Xue [12] fed raw signal into a sophisticated CNN, which had an architecture composed of three convolutional layers and three max-pooling layers.”
BackgroundComparison of Feature Learning Methods for Human Activity Recognition Using Wearable Sensors
Li, F., Shirahama, K., Nisar, M., Köping, L., & Grzegorzek, M. (2018) · Sensors · doi:10.3390/s18020679 · Cited by 302
Deep Convolutional Neural Networks for Human Activity Recognition with Smartphone Sensors
Ronao, C. A., & Cho, S.-B. (2015) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-26561-2_6 · Cited by 176
Semi-supervised convolutional neural networks for human activity recognition
Zeng, M., Yu, T., Wang, X., Nguyen, L. T., Mengshoel, O. J., & Lane, I. (2017) · 2017 IEEE International Conference on Big Data (Big Data) · doi:10.1109/bigdata.2017.8257967 · Cited by 85 · cited 1×
“Zeng et al. [27] presented the semi-supervised methods based on CNN that can learn from both labeled and unlabeled data.”
Background
Research works printed but not linked: 16
- pascanu, 0, How to construct deep recurrent neural networks, Proc 2nd Int Conf Learn Representations
- 10.1145/1959826.1959853 · DOI · Paper page
- yang, 2015, Deep convolutional neural networks on multichannel time series for human activity recognition, Proc Int Conf Artif Intell, 3995
- 10.1109/TFUZZ.2018.2870590 · DOI
- 10.1016/j.displa.2018.08.001 · DOI
- 10.1109/JSEN.2019.2917225 · DOI · Paper page
- xu, 2015, Show, attend and tell: Neural image caption generation with visual attention, Proc Int Conf Mach Learn, 2048
- witten, 2016, Data Mining Practical Machine Learning Tools and Techniques
- 10.1109/JSEN.2018.2885796 · DOI · Paper page
- 10.3390/s100201154 · DOI · Paper page
- ilse, 2018, Attention-based deep multiple instance learning, Proc Int Conf Mach Learn, 2127
- 10.3390/s130201402 · DOI · Paper page
- 10.1007/978-3-540-75160-1_4 · DOI · Paper page
- 10.1109/TSMC.2013.2283661 · DOI
- 10.1007/978-3-642-35395-6_30 · DOI · Paper page
- 10.1109/JSEN.2020.2978772 · DOI · Paper page
Cited by
Ramanujam, E., Perumal, T., & Padmavathi, S. (2021) · IEEE Sensors Journal · doi:10.1109/jsen.2021.3069927 · Cited by 455
Multiscale Deep Feature Learning for Human Activity Recognition Using Wearable Sensors
Tang, Y., Zhang, L., Min, F., & He, J. (2023) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2022.3161812 · Cited by 224
“Comparing to Wang et al.’s attention mechanism [23], the HS-ResNet is still able to provide a performance gain of 0.55%, which refreshes their result to a higher score.”
BackgroundMethodInfluentialSelf-citationHuman activity recognition using wearable sensors by heterogeneous convolutional neural networks
Han, C., Zhang, L., Tang, Y., Huang, W., Min, F., & He, J. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.116764 · Cited by 130
Self-citation
Metadata
- Authors
- Wang, K., He, J., & Zhang, L.
- Venue
- IEEE Transactions on Human-Machine Systems
- Published
- 2021-08
- Volume
- 51
- Issue
- 4
- Pages
- 355-364
- ISSN
- 2168-2291 (ISSN-L)
- 2168-2305
- OpenAlex
- W3049146908
- Cited by
- 49
- References
- 17
- FWCI
- 3.62
- Type
- journal article
- Language
- en
- Open access
- green
- Licence
- Funders
- National Natural Science Foundation of China (61203237)
- Natural Science Foundation of Jiangsu Province (BK20191371)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.