Transition-Aware Detection of Modes of Locomotion and Transportation Through Hierarchical Segmentation
Abstract
Recognizing human daily activities with motion sensors data, specifically, modes of locomotion and transportation provides important contextual information that enhances the effectiveness of mobile applications. For instance, in assisted living or sports monitoring it is essential to log driving or running episodes. Previous studies in this field have utilized a fixed-size windowing technique for segmenting the sequential data of sensors. While segmenting signals into larger windows provides more information about the signal for classifiers, it increases misclassification rate when a transition occurs between the activities (i.e., multi-class windows). This will lead to inaccurate detection and logging of the activities of interest. To identify the exact time of transition from one to another activity as precisely as possible, this article proposes a fast and efficient hierarchical search algorithm that finds the exact sample at which transition occurs. This search algorithm can be applied to any activity recognition model with various lengths of segmentation window. To further improve the performance, we propose a new structure of 2D signal inputs to be used with 2D convolutional neural networks (CNN), which improves the ability of the CNN in capturing patterns underlying in inter-axes correlations. Experimental results show that the proposed transition detection method improves the F1-score by 28% compared to using fixed-size windowing approach for multi-class windows. In addition, the proposed method is 20 times faster than the naïve search.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Works using frequency-domain inputs generally use 2D-CNN kernels.
“Works that use frequency-domain inputs (e.g., spectrogram), which have an additional frequency dimension, will generally use 2D-CNN kernels [164].”
From Deep Learning in Human Activity Recognition with Wearable Sensors: A Review on Advances (2022) · cites this paper as [164]
Cites
(2014) · Cited by 33,419 · see citing works
Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition
Ordóñez, F., & Roggen, D. (2016) · Sensors · doi:10.3390/s16010115 · Cited by 2,761 · cited 2× · see citing works
“Another interesting conclusion from this figure is that hand-crafted features perform better than automated features of CNN which is in contrast to [21].”
MethodA survey of mobile phone sensing
Lane, N., Miluzzo, E., Lu, H., Peebles, D., Choudhury, T., & Campbell, A. (2010) · IEEE Communications Magazine · doi:10.1109/mcom.2010.5560598 · Cited by 2,292 · see citing works
(2015) · Cited by 989 · see citing works
Convolutional Neural Networks for Human Activity Recognition using Mobile Sensors
Zeng, M., Nguyen, L. T., Yu, B., Mengshoel, O. J., Zhu, J., Wu, P., & Zhang, J. (2014) · Proceedings of the 6th International Conference on Mobile Computing, Applications and Services · doi:10.4108/icst.mobicase.2014.257786 · Cited by 876 · see citing works
Human Activity Recognition Using Wearable Sensors by Deep Convolutional Neural Networks
Jiang, W., & Yin, Z. (2015) · Proceedings of the 23rd ACM international conference on Multimedia · doi:10.1145/2733373.2806333 · Cited by 676 · see citing works
Window Size Impact in Human Activity Recognition
Baños, O., Gálvez, J. M., Damas, M., Pomares, H., & Rojas, I. (2014) · Sensors · doi:10.3390/s140406474 · Cited by 656 · see citing works
Accelerometer-based transportation mode detection on smartphones
Hemminki, S., Nurmi, P., & Tarkoma, S. (2013) · ACM Conference on Embedded Networked Sensor Systems (SenSys) · doi:10.1145/2517351.2517367 · Cited by 436 · see citing works
Walk detection and step counting on unconstrained smartphones
Brajdic, A., & Harle, R. K. (2013) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/2493432.2493449 · Cited by 376 · see citing works
Wu, J., Sun, L., & Jafari, R. (2016) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2016.2598302 · Cited by 279 · see citing works
Self-citationGjoreski, H., Ciliberto, M., Wang, L., Morales, F. J. O., Mekki, S., Valentin, S., & Roggen, D. (2018) · IEEE Access · doi:10.1109/access.2018.2858933 · Cited by 277 · see citing works
Smartphone sensing methods for studying behavior in everyday life
Harari, G. M., Müller, S. R., Aung, M. H., & Rentfrow, P. J. (2017) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2017.07.018 · Cited by 248 · see citing works
Performance metrics for activity recognition
Ward, J. A., Lukowicz, P., & Gellersen, H. W. (2011) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/1889681.1889687 · Cited by 182 · see citing works
When and Why Are Deep Networks Better Than Shallow Ones?
Mhaskar, H. N., Liao, Q., & Poggio, T. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10913 · Cited by 178 · see citing works
Noor, M. H. M., Salcic, Z. A., & Wang, K. I. (2016) · Pervasive and Mobile Computing · doi:10.1016/j.pmcj.2016.09.009 · Cited by 145 · see citing works
Real-Time Change Point Detection with Application to Smart Home Time Series Data
Aminikhanghahi, S., Wang, T., & Cook, D. J. (2019) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2018.2850347 · Cited by 123 · see citing works
Yao, R., Lin, G., Shi, Q., & Ranasinghe, D. C. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2017.12.024 · Cited by 108 · see citing works
Summary of the Sussex-Huawei Locomotion-Transportation Recognition Challenge
Wang, L., Gjoreski, H., Murao, K., et al. (2018) · doi:10.1145/3267305.3267519 · Cited by 90 · see citing works
Guo, H., Chen, L., Peng, L., & Chen, G. (2016) · Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing · doi:10.1145/2971648.2971708 · Cited by 83 · see citing works
Orientation Independent Activity/Gesture Recognition Using Wearable Motion Sensors
Wu, J., & Jafari, R. (2018) · IEEE Internet of Things Journal · doi:10.1109/jiot.2018.2856119 · Cited by 76 · see citing works
Self-citationEnhancing activity recognition using CPD-based activity segmentation
Aminikhanghahi, S., & Cook, D. J. (2019) · Pervasive and Mobile Computing · doi:10.1016/j.pmcj.2019.01.004 · Cited by 75 · see citing works
Human activity recognition based on time series analysis using U-NetPreprint
Zhang, Y., Zhang, Y., Zhang, Z., Bao, J., & Song, Y. (2018) · arXiv.org · doi:10.48550/arxiv.1809.08113 · Cited by 41 · see citing works
Benchmarking the SHL Recognition Challenge with Classical and Deep-Learning Pipelines
Wang, L., Gjoreski, H., Ciliberto, M., et al. (2018) · doi:10.1145/3267305.3267531 · Cited by 40 · see citing works
Distributed Continuous Action Recognition Using a Hidden Markov Model in Body Sensor Networks
Guenterberg, E., Ghasemzadeh, H. G., Loseu, V., et al. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-02085-8_11 · Cited by 35 · see citing works
Evaluation of Algorithms for Orientation Invariant Inertial Gait Matching
Subramanian, R., & Sarkar, S. (2018) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2018.2850032 · Cited by 35 · see citing works
Hierarchical Signal Segmentation and Classification for Accurate Activity Recognition
Akbari, A., Wu, J., Grimsley, R., & Jafari, R. (2018) · UbiComp/ISWC Adjunct · doi:10.1145/3267305.3267528 · Cited by 35 · see citing works
Self-citationMulti-label Discriminative Weakly-Supervised Human Activity Recognition and Localization
Adeli, E., Cabral, R., la Torre, F. D., & Fathy, M. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-16814-2_16 · Cited by 31 · see citing works
Top in the Lab, Flop in the Field?
Widhalm, P., Leodolter, M., & Brändle, N. (2018) · doi:10.1145/3267305.3267514 · Cited by 26 · see citing works
Mohamed, R., Zainudin, M. N. S., Sulaiman, M. N., et al. (2018) · Journal of Information and Communication Technology · doi:10.32890/jict2018.17.2.8252 · Cited by 25 · see citing works
Applying Multiple Knowledge to Sussex-Huawei Locomotion Challenge
Gjoreski, M., Janko, V., Reščič, N., et al. (2018) · doi:10.1145/3267305.3267515 · Cited by 24 · see citing works
Mohamed, R., Zainudin, M. N. S., Sulaiman, M. N., et al. (2018) · Journal of Information and Communication Technology · doi:10.32890/jict2018.17.2.3 · Cited by 20 · see citing works
An adaptive time window method for human activity recognition
Sheng, Z., Hailong, C., Jiang, C., et al. (2015) · IEEE Canadian Conference on Electrical and Computer Engineering (CCECE) · doi:10.1109/ccece.2015.7129445 · Cited by 19 · see citing works
Human Action Recognition Based on Non-linear SVM Decision Tree
Zhao, H., & Liu, Z. (2011) · Cited by 13 · see citing works
Wu, J., Akbari, A., Grimsley, R., et al. (2018) · doi:10.1145/3267305.3267525 · Cited by 12 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Deep Learning in Human Activity Recognition with Wearable Sensors: A Review on Advances
Zhang, S., Li, Y., Zhang, S., Shahabi, F., Xia, S., Deng, Y., & Alshurafa, N. (2022) · Sensors · doi:10.3390/s22041476 · Cited by 625 · see citing works
“frequency dimension, will generally use 2D-CNN kernels [11].”
BackgroundMethod
- Authors
- Akbari, A., & Jafari, R.
- Affiliations
- Texas A&M University (US)
- Venue
- IEEE Sensors Journal
- Published
- 2021-02-01
- Volume
- 21
- Issue
- 3
- Pages
- 3301-3313
- ISSN
- 1530-437X (ISSN-L)
- 1558-1748
- 2379-9153
- OpenAlex
- W3085136140
- Cited by
- 8
- References
- 34
- Citations by year
- 2026: 2 · 2024: 2 · 2023: 1 · 2022: 3
- FWCI
- 0.39
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Sensors Journal (publishedVersion)
- Funders
- National Science Foundation (ECCS-1509063)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Social Sciences › Transportation › Human Mobility and Location-Based Analysis
- Physical Sciences › Engineering › Biomedical Engineering › Gait Recognition and Analysis
- Keywords
- activity recognition, transportation mode detection, hierarchical segmentation, motion sensor data, 2D convolutional neural network, assisted living, sports monitoring
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
