CoChat
Sign Up
Cited by 36Context-Aware Activity Recognition SystemsGait Recognition and AnalysisHuman Pose and Action Recognition

Deep Transfer Learning with Graph Neural Network for Sensor-Based Human Activity Recognition

Open access
Liao, Tianzheng, Zhao, Jinjin, Liu, Yushi, Ivanov, Kamen, Xiong, Jing, Yan, Yan·2022·2022 IEEE International Conference on Bioinformatics and Biomedicine (BIBM)
DOI 10.1109/bibm55620.2022.9995660
Open this paper in CoChat — free accountOpen PDF

Abstract

The sensor-based human activity recognition (HAR) in mobile application scenarios is often confronted with variation in sensing modalities and deficiencies in annotated samples. To address these two challenging problems, we devised a graph-inspired deep learning approach that uses data from human-body mounted wearable sensors. As a step toward a complete HAR solution, the proposed method was further used to build a deep transfer learning model. Specifically, we present a multi-layer residual structure involving graph convolutional neural network (ResGCNN) toward the sensor-based HAR tasks, namely the HAR-ResGCNN approach. Experimental results on the PAMAP2 and mHealth data sets demonstrate that our ResGCNN is effective at capturing the characteristics of actions with comparable results compared to other sensor-based HAR models (with an average accuracy of 98.18% and 99.07%, respectively). More importantly, the parameter-based transfer learning experiments using the ResGCNN model show excellent transferability and small sample learning ability, which is a promising solution in sensor-based HAR applications.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/bibm55620.2022.9995660

Cites

29 of 29 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 32

  1. [1][1] J. Liu, A. Shahroudy, M. Perez, G. Wang, L.-Y. Duan, and A. C. Kot, 'Ntu rgb+d 120: A large-scale benchmark for 3d human activity understanding,' IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 42, no. 10, pp. 2684-2701, 2020. -
  2. [2][2] J. Ye, G.-J. Qi, N. Zhuang, H. Hu, and K. A. Hua, 'Learning compact features for human activity recognition via probabilistic first-take-all,' IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 42, no. 1, pp. 126-139, 2020. -
  3. [3][3] W. Ding, X. Guo, and G. Wang, 'Radar-based human activity recognition using hybrid neural network model with multidomain fusion,' IEEE Transactions on Aerospace and Electronic Systems , vol. 57, no. 5, pp. 2889-2898, 2021. -
  4. [4][4] J. Huang, B. Liu, C. Miao, Y. Lu, Q. Zheng, Y. Wu, J. Liu, L. Su, and C. W. Chen, 'Phaseanti: an anti-interference wifi-based activity recognition system using interference-independent phase component,' IEEE Transactions on Mobile Computing , pp. 1-1, 2021. -
  5. [6][6] A. Jalal, S. Kamal, and D. Kim, 'A depth video sensor-based life-logging human activity recognition system for elderly care in smart indoor environments,' Sensors , vol. 14, no. 7, pp. 11 73511 759, 2014. -
  6. [7][7] S. Ji, W. Xu, M. Yang, and K. Yu, '3d convolutional neural networks for human action recognition,' IEEE transactions on pattern analysis and machine intelligence , vol. 35, no. 1, pp. 221-231, 2012. -
  7. [8][8] A. Jalal, Y.-H. Kim, Y.-J. Kim, S. Kamal, and D. Kim, 'Robust human activity recognition from depth video using spatiotemporal multi-fused features,' Pattern recognition , vol. 61, pp. 295-308, 2017. -
  8. [9][9] K. Simonyan and A. Zisserman, 'Two-stream convolutional networks for action recognition,' in Proceedings of the Neural Information Processing Systems (NIPS) , 2015. -
  9. [11][11] M. Shoaib, S. Bosch, O. D. Incel, H. Scholten, and P. J. Havinga, 'Complex human activity recognition using smartphone and wrist-worn motion sensors,' Sensors , vol. 16, no. 4, p. 426, 2016. -
  10. [12][12] J. Guo, X. Zhou, Y. Sun, G. Ping, G. Zhao, and Z. Li, 'Smartphonebased patients' activity recognition by using a self-learning scheme for medical monitoring,' Journal of medical systems , vol. 40, no. 6, pp. 1-14, 2016. -
  11. [13][13] L. Fan, Z. Wang, and H. Wang, 'Human activity recognition model based on decision tree,' in 2013 International Conference on Advanced Cloud and Big Data . IEEE, 2013, pp. 64-68. -
  12. [14][14] C. Hu, Y. Chen, L. Hu, and X. Peng, 'A novel random forests based class incremental learning method for activity recognition,' Pattern Recognition , vol. 78, pp. 277-290, 2018. -
  13. [15][15] K. M. Chathuramali and R. Rodrigo, 'Faster human activity recognition with svm,' in International Conference on Advances in ICT for Emerging Regions (ICTer2012) . IEEE, 2012, pp. 197-203. -
  14. [16][16] Q. Xiao and R. Song, 'Action recognition based on hierarchical dynamic bayesian network,' Multimedia Tools and Applications , vol. 77, no. 6, pp. 6955-6968, 2018. -
  15. [17][17] P. Sok, T. Xiao, Y. Azeze, A. Jayaraman, and M. V. Albert, 'Activity recognition for incomplete spinal cord injury subjects using hidden markov models,' IEEE Sensors Journal , vol. 18, no. 15, pp. 6369-6374, 2018. -
  16. [18][18] A. M. Khan, A. Tufail, A. M. Khattak, and T. H. Laine, 'Activity recognition on smartphones via sensor-fusion and kda-based svms,' International Journal of Distributed Sensor Networks , vol. 10, no. 5, p. 503291, 2014. -
  17. [19][19] M. Munoz-Organero, 'Outlier detection in wearable sensor data for human activity recognition (har) based on drnns,' IEEE Access , vol. 7, pp. 74 422-74 436, 2019. -
  18. [20][20] J. Wang, Y. Chen, S. Hao, X. Peng, and L. Hu, 'Deep learning for sensor-based activity recognition: A survey,' Pattern Recognition Letters , vol. 119, pp. 3-11, 2019. -
  19. [22][22] C. A. Ronao and S.-B. Cho, 'Deep convolutional neural networks for human activity recognition with smartphone sensors,' in International Conference on Neural Information Processing . Springer, 2015, pp. 46-53. -
  20. [23][23] T. Zebin, P. J. Scully, and K. B. Ozanyan, 'Human activity recognition with inertial sensors using a deep learning approach,' in 2016 IEEE SENSORS . IEEE, 2016, pp. 1-3. -
  21. [24][24] J. Yang, M. N. Nguyen, P. P. San, X. L. Li, and S. Krishnaswamy, 'Deep convolutional neural networks on multichannel time series for human activity recognition,' in Twenty-fourth international joint conference on artificial intelligence , 2015. -
  22. [25][25] A. Ignatov, 'Real-time human activity recognition from accelerometer data using convolutional neural networks,' Applied Soft Computing , vol. 62, pp. 915-922, 2018. -
  23. [29][29] F. Scarselli, M. Gori, A. C. Tsoi, M. Hagenbuchner, and G. Monfardini, 'The graph neural network model,' IEEE Transactions on Neural Networks , vol. 20, no. 1, pp. 61-80, 2009. -
  24. [31][31] B. Yu, H. Yin, and Z. Zhu, 'Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting,' in Proceedings of the 27th International Joint Conference on Artificial Intelligence , ser. IJCAI'18. AAAI Press, 2018, p. 3634-3640. -
  25. [32][32] S. Kearnes, K. McCloskey, M. Berndl, V. Pande, and P. Riley, 'Molecular graph convolutions: moving beyond fingerprints,' Journal of Computer-Aided Molecular Design , vol. 30, no. 8, pp. 595-608, 2016. [Online]. Available: https://doi.org/10.1007/ s10822-016-9938-8 -
  26. [34][34] H. Al-Mubaid and S. Umair, 'A new text categorization technique using distributional clustering and learning logic,' IEEE Transactions on Knowledge and Data Engineering , vol. 18, no. 9, pp. 11561165, 2006. -
  27. [35][35] U. K. F. A. H. J. I. M. O.-Y. S. Muazzam Maqsood, Faria Nazir, 'Transfer learning assisted classification and detection of alzheimer's disease stages using 3d mri scans,' Sensors , vol. 19, no. 11, p. 2645, 2019. -
  28. [36][36] S. L. Y. M. G. M. H. A. C. T. Scarselli, Franco and M. Maggini, 'Graph neural networks for ranking web pages,' in The 2005 IEEE/WIC/ACM International Conference on Web Intelligence (WI'05) . IEEE, Sep. 2005, pp. 666-672. -
  29. [37][37] M. G. A. C. T. M. H. Scarselli, Franco and G. Monfardini, 'The graph neural network model,' IEEE Transactions on Neural Networks , vol. 20, no. 1, pp. 61-80, 2008. -
  30. [38][38] M. Defferrard, X. Bresson, and P. Vandergheynst, 'Convolutional neural networks on graphs with fast localized spectral filtering,' in Proceedings of the 30th International Conference on Neural Information Processing Systems , ser. NIPS'16. Red Hook, NY, USA: Curran Associates Inc., 2016, p. 3844-3852. -
  31. [39][39] C. Tan, F. Sun, T. Kong, W. Zhang, C. Yang, and C. Liu, 'A survey on deep transfer learning,' in Artificial Neural Networks and Machine Learning - ICANN 2018 , V. K˚ urkov´ a, Y. Manolopoulos, B. Hammer, L. Iliadis, and I. Maglogiannis, Eds. Cham: Springer International Publishing, 2018, pp. 270-279. -
  32. [40][40] T. Cai, S. Luo, K. Xu, D. He, T.-Y. Liu, and L. Wang, 'Graphnorm: A principled approach to accelerating graph neural network training,' in Proceedings of the 38th International Conference on Machine Learning , ser. Proceedings of Machine Learning Research, M. Meila and T. Zhang, Eds., vol. 139. PMLR, 18-24 Jul 2021, pp. 1204-1215. [Online]. Available: https://proceedings.mlr.press/v139/cai21e.html -

Cited by

Sorted by citations

Related

Authors
Liao, T., Zhao, J., Liu, Y., Ivanov, K., Xiong, J., & Yan, Y.
Affiliations
  • Chinese Academy of Sciences (CN)
  • Southeast University (CN)
  • Shenzhen Institutes of Advanced Technology,Chinese Academy of Sciences,Shenzhen,China
  • Bulgarian Academy of Sciences (BG)
ORCID
Venue
2022 IEEE International Conference on Bioinformatics and Biomedicine (BIBM)
Publisher
IEEE
Published
2022-12-06
Pages
2445-2452
DOI
10.1109/bibm55620.2022.9995660
OpenAlex
W4225528848
Cited by
36
References
29
Citations by year
2026: 5 · 2025: 13 · 2024: 9 · 2023: 6 · 2022: 3
FWCI
2.81
Type
conference paper
Language
en
Open access
Licence
Locations
  • IEEE International Conference on Bioinformatics and Biomedicine (BIBM) (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
Research and Development
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
  • Physical Sciences › Engineering › Biomedical Engineering › Gait Recognition and Analysis
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
Keywords
human activity recognition, wearable sensors, graph convolutional networks, transfer learning, sensor modalities, small sample learning, PAMAP2, mobile health

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review