Deep Dilation on Multimodality Time Series for Human Activity Recognition
Open accessThe DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
Abstract
Providing accurate information on people's activities and behaviors plays an important role in innumerable applications, such as medical, security, and entertainment. In recent years, deep learning has been applied in human activity recognition, and achieved a better performance. However, if the spatial dependency of inter-sensors is considered, it is possible to enhance the discriminative ability. In this paper, we present a novel deep learning framework for human activity recognition problems. First, on the basis of our previous work, we utilize dilated convolutional neural network to extract features of inter-sensors and intra-sensors. Since the extracted features are local and short-temporal, it is necessary to utilize RNN to model the long-temporal dependencies. However, duo to that LSTM and GRU often rely on the completely previous computations, it will result in slow inference and hard convergence. Hence, inspired by the idea of dilation operation, we present a novel recurrent model to learn the temporal dependencies at different time scales. Then, at the topmost layer, a fully-connected layer with softmax function is utilized to generate a class probability distribution, and the predicted activity is obtained. Eventually, we evaluate the proposed framework in two open human activity datasets, OPPORTUNITY and PAMAP2. Results demonstrate that the proposed framework achieves a higher classification performance than the state-of-the-art methods. Moreover, it takes the least time to recognize an activity. Besides, it also performs faster and easier to converge in the training stage.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Activity recognition using cell phone accelerometers
Kwapisz, J. R., Weiss, G. M., & Moore, S. A. (2011) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1964897.1964918 · Cited by 2,912 · cited 1×
“Subsequently, a shallow supervised machine learning algorithm is applied to recognize activities, and the most popular learning algorithms include decision tree [8], [19], KNN [6], [20], SVM [3], [7].”
MethodDeep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition
Ordóñez, F., & Roggen, D. (2016) · Sensors · doi:10.3390/s16010115 · Cited by 2,761 · cited 2×
“The results of [25] reveal that a combination of CNN and recurrent dense layers can achieve a better performance, the reason being that CNN can capture spatial relationship while RNNmodels temporal relationship.”
BackgroundDeep learning for sensor-based activity recognition: A survey
Wang, J., Chen, Y., Hao, S., Peng, X., & Hu, L. (2019) · Pattern Recognition Letters · doi:10.1016/j.patrec.2018.02.010 · Cited by 2,014
A tutorial on human activity recognition using body-worn inertial sensors
Bulling, A., Blanke, U., & Schiele, B. (2014) · ACM Computing Surveys · doi:10.1145/2499621 · Cited by 1,668 · cited 1×
“Besides, they also ignore intra-class variability and similarity that are implicit features required to recognize complex activities [15].”
BackgroundIntroducing a New Benchmarked Dataset for Activity Monitoring
Reiss, A., & Stricker, D. (2012) · 2012 16th International Symposium on Wearable Computers · doi:10.1109/iswc.2012.13 · Cited by 1,456 · cited 8×
“The PAMAP2 Dataset [34] consisting of 12 lifestyle activities (as TABLE 1 lists), includes both household activities and exercise activities such as Nordic walking, playing soccer, et al.”
MethodInfluentialConvolutional Neural Networks for Human Activity Recognition using Mobile Sensors
Zeng, M., Nguyen, L. T., Yu, B., Mengshoel, O. J., Zhu, J., Wu, P., & Zhang, J. (2014) · Proceedings of the 6th International Conference on Mobile Computing, Applications and Services · doi:10.4108/icst.mobicase.2014.257786 · Cited by 876 · cited 7×
“The contributions of this paper are as follows: • For HAR, according to the characteristics of intersensors and intra-sensors, we propose a new method to automatically extract feature representations using dilated convolution, exponentially expands receptive fields without losing resolution or coverage and aggregates…”
MethodInfluentialCollecting complex activity datasets in highly rich networked sensor environments
Roggen, D., Calatroni, A., Rossi, M., Holleczek, T., Forster, K., Troster, G., Lukowicz, P., Bannach, D., Pirkl, G., Ferscha, A., et al. (2010) · 2010 Seventh International Conference on Networked Sensing Systems (INSS) · doi:10.1109/inss.2010.5573462 · Cited by 788
Human Activity Recognition Using Wearable Sensors by Deep Convolutional Neural Networks
Jiang, W., & Yin, Z. (2015) · Proceedings of the 23rd ACM international conference on Multimedia · doi:10.1145/2733373.2806333 · Cited by 676 · cited 1×
“Therefore, many models [5], [16], [17] have been used to recognize human activities, such as convolutional neural network(CNN ), recurrent neural network(RNN ).”
USC-HAD: a daily activity dataset for ubiquitous activity recognition using wearable sensors
Zhang, M., & Sawchuk, A. A. (2012) · Proceedings of the 2012 ACM Conference on Ubiquitous Computing · doi:10.1145/2370216.2370438 · Cited by 554 · cited 1×
“With the diversity and popularity of wearables and on-body sensing devices, unobtrusive and automatic human activity recognition (HAR) has drawn extensive attention, and plays an important role in various areas, such as humancomputer interaction (HCI) [1], human behavior analysis [2], mobile and ubiquitous computing […”
Background
Research works printed but not linked: 26
- hammerla, 2016, Deep convolutional and recurrent models for human activity recognition using wearables
- lecun, 1998, Efficient backprop, Neural Networks, 9
- 10.1109/ICCVW.2017.37 · DOI
- van den oord, 2016, WaveNet A Generative Model for Raw Audio
- lei, 2017, Training RNNs as fast as CNNs
- abadi, 2016, Tensorflow Large-scale machine learning on heterogeneous distributed systems
- xi, 2018, Deep dilated convolution on multimodality time series for human activity recognition, Proc Int Joint Conf Neural Netw
- xu, 2012, Robust human activity and sensor location corecognition via sparse signal representation, IEEE Trans Biomed Eng, 59, 3169, 10.1109/TBME.2012.2211355 · DOI · Paper page
- 10.1145/1107548.1107591 · DOI · Paper page
- xu, 2016, Wearable sensor-based human activity recognition method with multi-features extracted from Hilbert–Huang transform, SENSORS, 16, 2048, 10.3390/s16122048 · DOI
- 10.1145/882085.882086 · doi:10.1145/882085.882086 · Paper page
- lin, 2013, Network in Network
- 10.1109/CVPR.2015.7298965 · DOI · Paper page
- 10.1109/78.157290 · DOI · Paper page
- anguita, 2013, A public domain dataset for human activity recognition using smartphones, Proc ESANN
- yu, 2015, Multi-scale context aggregation by dilated convolutions
- yang, 2015, Deep convolutional neural networks on multichannel time series for human activity recognition, Proc IJCAI, 3995
- 10.1145/1409635.1409637 · DOI · Paper page
- 10.1145/2030112.2030172 · DOI
- 10.1007/978-3-540-72348-6_3 · DOI · Paper page
- altun, 2010, Human activity recognition using inertial/magnetic sensor units, Proc Int Workshop Human Behav Understand, 38, 10.1007/978-3-642-14715-9_5 · DOI · Paper page
- 10.1016/j.cviu.2006.10.019 · DOI · Paper page
- hasan, 2016, Human activity recognition using smartphone sensors with context filtering, Proc 9th Int Conf Adv Comput -Hum Interact (ACHI), 69
- 10.1007/s10015-017-0422-x · DOI · Paper page
- almaslukh, 2017, An effective deep autoencoder approach for Online smartphone-based human activity recognition, Int J Comput Sci Netw Secur (IJCSNS), 17, 160
- 10.1007/978-3-642-75988-8_28 · DOI · Paper page
Cited by
Deep Learning for Time Series Forecasting: A Survey
Torres, J. F., Hadjout, D., Sebaa, A., Martínez-Álvarez, F., & Troncoso, A. (2021) · Big Data · doi:10.1089/big.2020.0159 · Cited by 818
Deep Learning for Sensor-based Human Activity Recognition
Chen, K., Zhang, D., Yao, L., Guo, B., Yu, Z., & Liu, Y. (2022) · ACM Computing Surveys · doi:10.1145/3447744 · Cited by 690
“N to extract elaborated correlations of individual sensing axes. Dilated convolution is another solution to exploiting nonadjacent modalities without information loss and extra computational expenses [161]. In addition to wearable sensors, RFID-based activity recognition requires the fusion of multiple RFID signals as…”
BackgroundDeep Learning for Sensor-based Human Activity Recognition: Overview, Challenges and Opportunities
Chen, K., Zhang, D., Yao, L., Guo, B., Yu, Z., & Liu, Y. (2020) · arXiv · doi:10.48550/arxiv.2001.07416 · Cited by 314
“N to extract elaborated correlations of individual sensing axes. Dilated convolution is another solution to exploiting nonadjacent modalities without information loss and extra computational expenses [161]. In addition to wearable sensors, RFID-based activity recognition requires the fusion of multiple RFID signals as…”
BackgroundA Human Activity Recognition Algorithm Based on Stacking Denoising Autoencoder and LightGBM
Gao, X., Luo, H., Wang, Q., Zhao, F., Ye, L., & Zhang, Y. (2019) · Sensors · doi:10.3390/s19040947 · Cited by 92
Metadata
- Authors
- Xi, R., Li, M., Hou, M., Fu, M., Qu, H., Liu, D., & Haruna, C. R.
- Venue
- IEEE Access
- Published
- 2018
- Volume
- 6
- Pages
- 53381-53396
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W2892279525
- Cited by
- 35
- References
- 13
- FWCI
- 2.87
- Type
- journal article
- Language
- en
- Open access
- gold
- Funders
- NSF China Projects (61472067)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.