Efficient dense labelling of human activity sequences from wearables using fully convolutional networks
The DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.
Abstract
Abstract Recognising human activities in sequential data from sensors is a challenging research area. A significant problem arises from the need to determine fixed sequence partitions (windows) to overcome the inability of a single sample to provide adequate information about an activity; commonly overcome by using a fixed size sliding window over consecutive samples to extract information—either handcrafted or learned features—and predicting a single label for all the samples in the window. Two key issues arise from this approach: (i) the samples in one window may not always share the same label, a problem more significant for short duration activities such as gestures. We refer to this as the multi-class windows problem. (ii) the inferencing phase is constrained by the window size selected during training while the best window size is difficult to tune in practice. We propose an efficient method for predicting the label of each sample, which we call dense labelling, in a sequence of activity data of arbitrary length based on a fully convolutional network (FCN) design. In particular, our approach overcomes the problems posed by multi-class windows and fixed size sequence partitions imposed during training. Further, our network learns both features and the classifier automatically. We conduct extensive experiments and demonstrate that our proposed approach is able to outperform the state-of-the-arts in terms of sample-based classification and activity-based label misalignment measures on three challenging datasets: Opportunity, Hand Gesture, and our new dataset—an activity dataset we release based on a wearable sensor worn by hospitalised patients.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition
Ordóñez, F., & Roggen, D. (2016) · Sensors · doi:10.3390/s16010115 · Cited by 2,761
A tutorial on human activity recognition using body-worn inertial sensors
Bulling, A., Blanke, U., & Schiele, B. (2014) · ACM Computing Surveys · doi:10.1145/2499621 · Cited by 1,668
Convolutional Neural Networks for Human Activity Recognition using Mobile Sensors
Zeng, M., Nguyen, L. T., Yu, B., Mengshoel, O. J., Zhu, J., Wu, P., & Zhang, J. (2014) · Proceedings of the 6th International Conference on Mobile Computing, Applications and Services · doi:10.4108/icst.mobicase.2014.257786 · Cited by 876
Collecting complex activity datasets in highly rich networked sensor environments
Roggen, D., Calatroni, A., Rossi, M., Holleczek, T., Forster, K., Troster, G., Lukowicz, P., Bannach, D., Pirkl, G., Ferscha, A., et al. (2010) · 2010 Seventh International Conference on Networked Sensing Systems (INSS) · doi:10.1109/inss.2010.5573462 · Cited by 788
The Opportunity challenge: A benchmark database for on-body sensor-based activity recognition
Chavarriaga, R., Sagha, H., Calatroni, A., Digumarti, S. T., Tröster, G., Millán, J. D. R., & Roggen, D. (2013) · Pattern Recognition Letters · doi:10.1016/j.patrec.2012.12.014 · Cited by 742
Deep, Convolutional, and Recurrent Models for Human Activity Recognition Using Wearables
Hammerla, N. Y., Halloran, S., & Ploetz, T. (2016) · arXiv · doi:10.48550/arxiv.1604.08880 · Cited by 606
Ensembles of Deep LSTM Learners for Activity Recognition using Wearables
Guan, Y., & Plötz, T. (2017) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3090076 · Cited by 511
Feature Learning for Activity Recognition in Ubiquitous Computing
Plötz, T., Hammerla, N., & Olivier, P. (2011) · Newcastle University · doi:10.25405/ncl.34175319 · Cited by 337
Layered representations for human activity recognition
Oliver, N., Horvitz, E., & Garg, A. (2003) · Proceedings. Fourth IEEE International Conference on Multimodal Interfaces · doi:10.1109/icmi.2002.1166960 · Cited by 312
Performance metrics for activity recognition
Ward, J. A., Lukowicz, P., & Gellersen, H. W. (2011) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/1889681.1889687 · Cited by 182
Research works printed but not linked: 12
- Ngo, 2015, Similar gait action recognition using an inertial sensor, Pattern Recognit., 48, 1289, 10.1016/j.patcog.2014.10.012 · DOI
- Wickramasinghe, 2017, Sequence learning with passive RFID sensors for real time bed-egress recognition in older people, IEEE J. Biomed. Health Inform., 21, 917, 10.1109/JBHI.2016.2576285 · DOI · Paper page
- K. Simonyan, A. Zisserman, Very deep convolutional networks for large-scale image recognition, CoRR arXiv:1409.1556 (2014). · arXiv
- Neverova, 2016, Learning human identity from motion patterns, IEEE Access, 4, 1810, 10.1109/ACCESS.2016.2557846 · DOI · Paper page
- Ijjina, 2016, Human action recognition using genetic algorithms and convolutional neural networks, Pattern Recognit., 59, 199, 10.1016/j.patcog.2016.01.012 · DOI · Paper page
- Junker, 2008, Gesture spotting with body-worn inertial sensors to detect user activities, Pattern Recognit., 41, 2010, 10.1016/j.patcog.2007.11.016 · DOI · Paper page
- Bulling, 2012, Multimodal recognition of reading activity in transit using body-worn sensors, ACM Trans. Appl. Percept., 9, 2, 10.1145/2134203.2134205 · DOI · Paper page
- Shi, 2011, Human action segmentation and recognition using discriminative semi-Markov models, Int. J. Comput. Vis., 93, 22, 10.1007/s11263-010-0384-0 · DOI
- Bhattacharya, 2014, Using unlabeled data in a sparse-coding framework for human activity recognition, Pervasive Mob. Comput., 15, 242, 10.1016/j.pmcj.2014.05.006 · DOI
- Vollmer, 2013, Learning features for activity recognition with shift-invariant sparse coding, Lect. Notes Comput. Sci., 8131, 367, 10.1007/978-3-642-40728-4_46 · DOI
- Wang, 2017, Embedding structured contour and location prior in siamesed fully convolutional networks for road detection, IEEE Trans. Intell. Transp. Syst., PP, 1
- Chang, 2011, Libsvm: a library for support vector machines, ACM Trans. Intel. Syst. Technol. (TIST), 2, 27
Cited by
Sensor-based and vision-based human activity recognition: A comprehensive survey
Minh Dang, L., Min, K., Wang, H., Jalil Piran, M., Hee Lee, C., & Moon, H. (2020) · Pattern Recognition · doi:10.1016/j.patcog.2020.107561 · Cited by 730
Deep Learning for Sensor-based Human Activity Recognition
Chen, K., Zhang, D., Yao, L., Guo, B., Yu, Z., & Liu, Y. (2022) · ACM Computing Surveys · doi:10.1145/3447744 · Cited by 690
“f the classifier is less than a threshold. Different from the hierarchical framework, some researchers explored to directly assign a label for each time-step instead of predicting a window as a whole [168, 190]. Inspired by semantic segmentation in the computer vision community, the authors employed fully connected ne…”
MethodDeep Learning for Sensor-based Human Activity Recognition: Overview, Challenges and Opportunities
Chen, K., Zhang, D., Yao, L., Guo, B., Yu, Z., & Liu, Y. (2020) · arXiv · doi:10.48550/arxiv.2001.07416 · Cited by 314
“f the classifier is less than a threshold. Different from the hierarchical framework, some researchers explored to directly assign a label for each time-step instead of predicting a window as a whole [168, 190]. Inspired by semantic segmentation in the computer vision community, the authors employed fully connected ne…”
MethodDeep Learning for Time Series Classification and Extrinsic Regression: A Current Survey
Mohammadi Foumani, N., Miller, L., Tan, C. W., Webb, G. I., Forestier, G., & Salehi, M. (2024) · ACM Computing Surveys · doi:10.1145/3649448 · Cited by 268
Convolutional Neural Networks for Human Activity Recognition Using Body-Worn Sensors
Moya Rueda, F., Grzeszick, R., Fink, G. A., Feldhorst, S., & Ten Hompel, M. (2018) · Informatics · doi:10.3390/informatics5020026 · Cited by 196
Metadata
- Authors
- Yao, R., Lin, G., Shi, Q., & Ranasinghe, D. C.
- Venue
- Pattern Recognition
- Publisher
- Elsevier BV
- Published
- 2018-06
- Volume
- 78
- Pages
- 252-266
- ISSN
- 0031-3203 (ISSN-L)
- OpenAlex
- W2592284297
- Cited by
- 105
- References
- 12
- FWCI
- 5.98
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.