Deep Activity Recognition Models with Triaxial Accelerometers
Open accessAbstract
Despite the widespread installation of accelerometers in almost all mobile phones and wearable devices, activity recognition using accelerometers is still immature due to the poor recognition accuracy of existing recognition methods and the scarcity of labeled training data. We consider the problem of human activity recognition using triaxial accelerometers and deep learning paradigms. This paper shows that deep activity recognition models (a) provide better recognition accuracy of human activities, (b) avoid the expensive design of handcrafted features in existing systems, and (c) utilize the massive unlabeled acceleration samples for unsupervised feature extraction. Moreover, a hybrid approach of deep learning and hidden Markov models (DL-HMM) is presented for sequential activity recognition. This hybrid approach integrates the hierarchical representations of deep activity recognition models with the stochastic modeling of temporal sequences in the hidden Markov models. We show substantial recognition improvement on real world datasets over state-of-the-art methods of human activity recognition using triaxial accelerometers.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
DeepConvLSTM outperforms other deep non-recurrent approaches on the Skoda dataset, improving the best reported result by 6%.
“From the results in Table 9, we can see that DeepConvLSTM outperforms other deep non-recurrent approaches on the Skoda dataset, improving the best reported result by 6%.”
From Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition (2016) · cites this paper as [43]
The Temporal Modeling section (p. 6) reports a deep-model result of 89.38% on the Skoda dataset, providing a result the citing paper could use for comparison. It does not report DeepConvLSTM results or establish that DeepConvLSTM improves on the best prior result by 6%; that comparison is attributed to the citing paper’s Table 9. (LLM, conf=0.82)
Experimental — accuracy not yet measuredThe Skoda dataset has been employed to evaluate decision fusion techniques in sensor networks and deep learning techniques, which makes it a suitable dataset to evaluate the proposed solution.
“The Skoda dataset has been employed to evaluate decision fusion techniques in sensor networks [38] and deep learning techniques [23,43], which makes it a suitable dataset to evaluate our proposed solution.”
From Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition (2016) · cites this paper as [43]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Training Products of Experts by Minimizing Contrastive Divergence
Hinton, G. E. (2002) · Neural Computation · doi:10.1162/089976602760128018 · Cited by 4,879 · cited 1× · see citing works
“An RBM can be trained using the contrastive divergence approximation (Hinton 2002) as follows:”
MethodGreedy layer-wise training of deep networks
Bengio, Y., Lamblin, P., Popovici, D., & Larochelle, H. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0024 · Cited by 4,841 · see citing works
Activity recognition using cell phone accelerometers
Kwapisz, J. R., Weiss, G. M., & Moore, S. A. (2011) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1964897.1964918 · Cited by 2,912 · cited 4× · see citing works
“These three testbeds are described as follows: • WISDM Actitracker dataset (Kwapisz, Weiss, and Moore 2011): This dataset contains 1, 098, 213 samples of one triaxial accelerometer that is programmed to sample at a rate of 20 Hz.”
BackgroundMethodInfluentialWearable Assistant for Parkinson’s Disease Patients With the Freezing of Gait Symptom
Bachlin, M., Plotnik, M., Roggen, D., Maidan, I., Hausdorff, J. M., Giladi, N., & Troster, G. (2010) · IEEE Transactions on Information Technology in Biomedicine · doi:10.1109/titb.2009.2036165 · Cited by 693 · cited 6× · see citing works
“• Daphnet freezing of gait dataset (Bächlin et al. 2010): We used this dataset to demonstrate the healthcare applications of deep activity recognition models.”
BackgroundMethodInfluentialHammerla, N. Y., Kirkham, R., Andras, P., & Ploetz, T. (2013) · Proceedings of the 2013 International Symposium on Wearable Computers · doi:10.1145/2493988.2494353 · Cited by 138 · cited 1× · see citing works
“[28] C4.”
Flagged first, then by citations
Research works printed but not linked: 19
- [Altun and Barshan 2010] Altun, K., and Barshan, B. 2010. Human activity recognition using inertial/magnetic sensor units. In Human Behavior Understanding . Springer. 38-51.
- [Berchtold et al. 2010] Berchtold, M.; Budde, M.; Gordon, D.; Schmidtke, H. R.; and Beigl, M. 2010. ActiServ: Activity recognition service for mobile phones. In Proceedings of the International Symposium on Wearable Computers , 1-8. IEEE.
- [Chen et al. 2012] Chen, L.; Hoey, J.; Nugent, C. D.; Cook, D. J.; and Yu, Z. 2012. Sensor-based activity recognition. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews 42(6):790-808.
- [Dahl et al. 2012] Dahl, G. E.; Yu, D.; Deng, L.; and Acero, A. 2012. Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition. IEEE Transactions on Audio, Speech, and Language Processing 20(1):30-42.
- [Erhan et al. 2010] Erhan, D.; Bengio, Y.; Courville, A.; Manzagol, P.-A.; Vincent, P.; and Bengio, S. 2010. Why does unsupervised pre-training help deep learning? The Journal of Machine Learning Research 11:625-660.
- [Fender et al. 2008] Fender, A.; MacPherson, W. N.; Maier, R.; Barton, J. S.; George, D. S.; Howden, R. I.; Smith, G. W.; Jones, B.; McCulloch, S.; Chen, X.; et al. 2008. Twoaxis temperature-insensitive accelerometer based on multicore fiber Bragg gratings. IEEE sensors journal 7(8):12921298.
- [Hinton, Osindero, and Teh 2006] Hinton, G. E.; Osindero, S.; and Teh, Y.-W. 2006. A fast learning algorithm for deep belief nets. Neural computation 18(7):1527-1554.
- [Hinton 2012] Hinton, G. E. 2012. A practical guide to training restricted Boltzmann machines. In Neural Networks: Tricks of the Trade . Springer. 599-619.
- [Khan et al. 2010] Khan, A. M.; Lee, Y.-K.; Lee, S. Y.; and Kim, T.-S. 2010. A triaxial accelerometer-based physicalactivity recognition via augmented-signal features and a hierarchical recognizer. IEEE Transactions on Information Technology in Biomedicine 14(5):1166-1172.
- [Lara and Labrador 2013] Lara, O. D., and Labrador, M. A. 2013. A survey on human activity recognition using wearable sensors. IEEE Communications Surveys & Tutorials 15(3):1192-1209.
- [Mohamed, Dahl, and Hinton 2012] Mohamed, A.-R.; Dahl, G. E.; and Hinton, G. 2012. Acoustic modeling using deep belief networks. IEEE Transactions on Audio, Speech, and Language Processing 20(1):14-22.
- [Parkka et al. 2006] Parkka, J.; Ermes, M.; Korpipaa, P.; Mantyjarvi, J.; Peltola, J.; and Korhonen, I. 2006. Activity classification using realistic data from wearable sensors. IEEE Transactions on Information Technology in Biomedicine 10(1):119-128.
- [Pl¨ otz, Hammerla, and Olivier 2011] Pl¨ otz, T.; Hammerla, N. Y.; and Olivier, P. 2011. Feature learning for activity recognition in ubiquitous computing. In IJCAI ProceedingsInternational Joint Conference on Artificial Intelligence , volume 22, 1729.
- [Rabiner and Juang 1986] Rabiner, L. R., and Juang, B.-H. 1986. An introduction to hidden markov models. IEEE ASSP Magazine 3(1):4-16.
- [Rumelhart, Hinton, and Williams 1986] Rumelhart, D. E.; Hinton, G. E.; and Williams, R. J. 1986. Learning representations by back-propagating errors. Nature 323(6088):533536.
- [Salakhutdinov 2015] Salakhutdinov, R. 2015. Learning deep generative models. Annual Review of Statistics and Its Application 2(1):361-385.
- [Xu et al. 2012] Xu, W.; Zhang, M.; Sawchuk, A. A.; and Sarrafzadeh, M. 2012. Robust human activity and sensor location corecognition via sparse signal representation. IEEE Transactions on Biomedical Engineering 59(11):3169-3176.
- [Zappi et al. 2008] Zappi, P.; Lombriser, C.; Stiefmeier, T.; Farella, E.; Roggen, D.; Benini, L.; and Tr¨ oster, G. 2008. Activity recognition from on-body sensors: Accuracy-power trade-off by dynamic sensor selection. In Wireless sensor networks . Springer. 17-33.
- [Zheng, Zhang, and Song 2001] Zheng, F.; Zhang, G.; and Song, Z. 2001. Comparison of different implementations of MFCC. Journal of Computer Science and Technology 16(6):582-589.
Cited by
Sorted by citations
Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition
Ordóñez, F., & Roggen, D. (2016) · Sensors · doi:10.3390/s16010115 · Cited by 2,761 · see citing works
“CNN [43] Results reported by Alsheikh et.”
MethodDeep learning for sensor-based activity recognition: A survey
Wang, J., Chen, Y., Hao, S., Peng, X., & Hu, L. (2019) · Pattern Recognition Letters · doi:10.1016/j.patrec.2018.02.010 · Cited by 2,014 · see citing works
A deep learning approach to on-node sensor data analytics for mobile or wearable devices
Ravi, D., Wong, C., Lo, B., & Yang, G.-Z. (2017) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2016.2633287 · Cited by 439 · see citing works
Demrozi, F., Pravadelli, G., Bihorac, A., & Rashidi, P. (2020) · IEEE Access · doi:10.1109/access.2020.3037715 · Cited by 368 · see citing works
Deep Learning for Sensor-based Human Activity Recognition: Overview, Challenges and Opportunities
Chen, K., Zhang, D., Yao, L., Guo, B., Yu, Z., & Liu, Y. (2020) · arXiv · doi:10.48550/arxiv.2001.07416 · Cited by 314 · see citing works
“enges and Opportunities 111:17 Table 2. Deep Learning Works for Annotation Scarcity in Activity Recognition Training Fashion Learning Approach References Unsupervised Pretraining Alsheikh et al. 2016 [6]; Hammerla et al. 2015 [56]; Plötz et al. 2011 [111]; Chikhaoui and Gouineau [33]; Gu et al. 2018 [47]; Zeng et al.…”
BackgroundMethodKanjo, E., Younis, E. M. G., & Ang, C. S. (2019) · Information Fusion · doi:10.1016/j.inffus.2018.09.001 · Cited by 293 · see citing works
Deep Learning-Based Gait Recognition Using Smartphones in the Wild
Zou, Q., Wang, Y., Wang, Q., Zhao, Y., & Li, Q. (2020) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2020.2985628 · Cited by 305 · see citing works
Blockchain-based Personal Health Data Sharing System Using Cloud Storage
Zheng, X., Mukkamala, R. R., Vatrapu, R., & Ordieres-Mere, J. (2018) · 2018 IEEE 20th International Conference on e-Health Networking, Applications and Services (Healthcom) · doi:10.1109/healthcom.2018.8531125 · Cited by 281 · see citing works
DanHAR: Dual Attention Network for multimodal human activity recognition using wearable sensors
Gao, W., Zhang, L., Teng, Q., He, J., & Wu, H. (2021) · Applied Soft Computing · doi:10.1016/j.asoc.2021.107728 · Cited by 214 · see citing works
Zeng, M., Gao, H., Yu, T., Mengshoel, O. J., Langseth, H., Lane, I., & Liu, X. (2018) · Proceedings of the 2018 ACM International Symposium on Wearable Computers · doi:10.1145/3267242.3267286 · Cited by 185 · see citing works
“Similar to speech recognition, the combination of traditional sequence models (HMMs) and DNNs is also applied in HAR [31, 1].”
MethodA Comprehensive Study of Activity Recognition Using Accelerometers
Twomey, N., Diethe, T., Fafoutis, X., Elsts, A., McConville, R., Flach, P., & Craddock, I. (2018) · Informatics · doi:10.3390/informatics5020027 · Cited by 143 · see citing works
Increasing Robustness in the Detection of Freezing of Gait in Parkinson’s Disease
San-Segundo, R., Navarro-Hellín, H., Torres-Sánchez, R., Hodgins, J., & De la Torre, F. (2019) · Electronics · doi:10.3390/electronics8020119 · Cited by 86 · see citing works
Mahmud, T., Sazzad Sayyed, A. Q. M., Fattah, S. A., & Kung, S.-Y. (2021) · IEEE Sensors Journal · doi:10.1109/jsen.2020.3015781 · Cited by 58 · see citing works
Deep Auto-Set: A Deep Auto-Encoder-Set Network for Activity Recognition Using Wearables
Varamin, A. A., Abbasnejad, E., Shi, Q., Ranasinghe, D. C., & Rezatofighi, H. (2018) · Proceedings of the 15th EAI International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services · doi:10.1145/3286978.3287024 · Cited by 44 · see citing works
Self-supervised transfer learning of physiological representations from free-living wearable data
Spathis, D., Perez-Pozuelo, I., Brage, S., Wareham, N. J., & Mascolo, C. (2021) · Proceedings of the Conference on Health, Inference, and Learning · doi:10.1145/3450439.3451863 · Cited by 40 · see citing works
Sensor-data augmentation for human activity recognition with time-warping and data masking
Jeong, C. Y., Shin, H. C., & Kim, M. (2021) · Multimedia Tools and Applications · doi:10.1007/s11042-021-10600-0 · Cited by 38 · see citing works
Towards deep clustering of human activities from wearables
Abedin, A., Motlagh, F., Shi, Q., Rezatofighi, H., & Ranasinghe, D. (2020) · Proceedings of the 2020 International Symposium on Wearable Computers · doi:10.1145/3410531.3414312 · Cited by 27 · see citing works
Siamese Networks for Weakly Supervised Human Activity Recognition
Sheng, T., & Huber, M. (2019) · 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC) · doi:10.1109/smc.2019.8914045 · Cited by 21 · see citing works
- Authors
- Alsheikh, M. A., Selim, A., Niyato, D., Doyle, L., Lin, S., & Tan, H.-P.
- Affiliations
- Nanyang Technological University (SG)
- Trinity College Dublin (IE)
- Institute for Infocomm Research (SG)
- Singapore Management University (SG)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2015
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1511.04664
- OpenAlex
- W2962882167
- Cited by
- 126
- References
- 5
- Citations by year
- 2024: 1 · 2023: 1 · 2021: 36 · 2020: 26 · 2019: 26 · 2018: 23 · 2017: 8 · 2016: 4 · 2015: 1
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Social Sciences › Transportation › Human Mobility and Location-Based Analysis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Keywords
- human activity recognition, triaxial accelerometer, wearable sensors, deep learning, unsupervised feature extraction, hidden Markov models, temporal modeling, mobile devices
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review