The DOI matches its Crossref record; 53 of the 53 references it cites were found in a registry or an index.
Towards a Dynamic Inter-Sensor Correlations Learning Framework for Multi-Sensor-Based Wearable Human Activity Recognition
Abstract
Multi-sensor-based wearable human activity recognition (WHAR) is a research hotspot in the field of ubiquitous computing. Extracting effective features from multi-sensor data is essential to improve the performance of activity recognition. Despite the excellent achievements of previous works, the challenge remains for modelling the dynamic correlations between sensors. In this paper, we propose a lightweight yet efficient GCN-based dynamic inter-sensor correlations learning framework called DynamicWHAR for automatically learning the dynamic correlations between sensors. DynamicWHAR is mainly composed of two modules: Initial Feature Extraction and Dynamic Information Interaction. Firstly, Initial Feature Extraction module performs data-to-feature transformation to extract the initial features of each sensor. Subsequently, Dynamic Information Interaction module explicitly models the specific interaction intensity between any two sensors, and performs dynamic information aggregation between sensors by the learned interaction intensity. Extensive experiments on four diverse WHAR datasets and two different resource-constrained devices validate that DynamicWHAR outperforms the SOTA models in both recognition performance and computational complexity.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Realtime Multi-person 2D Pose Estimation Using Part Affinity Fields
Cao, Z., Simon, T., Wei, S.-E., & Sheikh, Y. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.143 · Cited by 7,539 · cited 1×
“In skeleton-based HAR, the time series of 3D coordinates of multiple skeleton joints are estimated either from 2D images by pose estimation algorithms [6] or directly collected by depth cameras ( e.g., Kinetics) [58].”
MethodSpatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition
Yan, S., Xiong, Y., & Lin, D. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12328 · Cited by 5,410 · cited 2×
“In recent years, skeleton-based HAR models have been developed from CNN-RNN-based models [24, 25, 56] to GCN-based models [8, 29, 41, 42, 51, 55, 57].”
MethodNTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis
Shahroudy, A., Liu, J., Ng, T.-T., & Wang, G. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.115 · Cited by 3,148
Activity Recognition from User-Annotated Acceleration Data
Bao, L., & Intille, S. S. (2004) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-24646-6_1 · Cited by 3,147
Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition
Ordóñez, F., & Roggen, D. (2016) · Sensors · doi:10.3390/s16010115 · Cited by 2,761 · cited 15×
“To further investigate the efficiency of the proposed framework, we implement DynamicWHAR and the SOTA models on two embedded devices, i.e. , Raspberry Pi 4 Model B and OnePlus 8.”
BackgroundMethodInfluentialProceedings of the International Joint Conference on Artificial Intelligence 2007
Bundy, A. (2007) · International Joint Conference on Artificial Intelligence · Cited by 2,761
Microsoft Kinect Sensor and Its Effect
Zhang, Z. (2012) · IEEE Multimedia · doi:10.1109/mmul.2012.24 · Cited by 2,487
Two-Stream Adaptive Graph Convolutional Networks for Skeleton-Based Action Recognition
Shi, L., Zhang, Y., Cheng, J., & Lu, H. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01230 · Cited by 2,071
NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding
Liu, J., Shahroudy, A., Perez, M., Wang, G., Duan, L.-Y., & Kot, A. C. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2916873 · Cited by 1,818
Wang, R., Chen, F., Chen, Z., Li, T., Harari, G., Tignor, S., Zhou, X., Ben-Zeev, D., & Campbell, A. T. (2014) · Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing · doi:10.1145/2632048.2632054 · Cited by 1,116
Channel-wise Topology Refinement Graph Convolution for Skeleton-Based Action Recognition
Chen, Y., Zhang, Z., Yuan, C., Li, B., Deng, Y., & Hu, W. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01311 · Cited by 1,039
Collecting complex activity datasets in highly rich networked sensor environments
Roggen, D., Calatroni, A., Rossi, M., Holleczek, T., Forster, K., Troster, G., Lukowicz, P., Bannach, D., Pirkl, G., Ferscha, A., et al. (2010) · 2010 Seventh International Conference on Networked Sensing Systems (INSS) · doi:10.1109/inss.2010.5573462 · Cited by 788
Eye Movement Analysis for Activity Recognition Using Electrooculography
Bulling, A., Ward, J. A., Gellersen, H., & Tröster, G. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.86 · Cited by 718
Stisen, A., Blunck, H., Bhattacharya, S., Prentow, T. S., Kjærgaard, M. B., Dey, A., Sonne, T., & Jensen, M. M. (2015) · Proceedings of the 13th ACM Conference on Embedded Networked Sensor Systems · doi:10.1145/2809695.2809718 · Cited by 714
Zhang, P., Lan, C., Xing, J., Zeng, W., Xue, J., & Zheng, N. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.233 · Cited by 691 · cited 1×
“In recent years, skeleton-based HAR models have been developed from CNN-RNN-based models [24, 25, 56] to GCN-based models [8, 29, 41, 42, 51, 55, 57].”
MethodDeepSense: A Unified Deep Learning Framework for Time-Series Mobile Sensing Data Processing
Yao, S., Hu, S., Zhao, Y., Zhang, A., & Abdelzaher, T. (2017) · Proceedings of the 26th International Conference on World Wide Web · doi:10.1145/3038912.3052577 · Cited by 667 · cited 4×
“Thanks to CNN’s high performance in modelling the multi-modal features and temporal features [34, 53], we choose it for initial feature extraction.”
BackgroundInfluentialPreprocessing techniques for context recognition from accelerometer data
Figo, D., Diniz, P. C., Ferreira, D. R., & Cardoso, J. M. P. (2010) · Personal and Ubiquitous Computing · doi:10.1007/s00779-010-0293-9 · Cited by 578
Ensembles of Deep LSTM Learners for Activity Recognition using Wearables
Guan, Y., & Plötz, T. (2017) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3090076 · Cited by 511 · cited 2×
“Following [1, 13], Skoda dataset are split into training, validation, and test data by the ratio of 0.8:0.1:0.1.”
MethodInfluentialOn-body localization of wearable devices: An investigation of position-aware activity recognition
Sztyler, T., & Stuckenschmidt, H. (2016) · 2016 IEEE International Conference on Pervasive Computing and Communications (PerCom) · doi:10.1109/percom.2016.7456521 · Cited by 369
Wearable Activity Tracking in Car Manufacturing
Stiefmeier, T., Roggen, D., Ogris, G., Lukowicz, P., & Tr, G. (2008) · IEEE Pervasive Computing · doi:10.1109/mprv.2008.40 · Cited by 367
Song, Y.-F., Zhang, Z., Shan, C., & Wang, L. (2020) · Proceedings of the 28th ACM International Conference on Multimedia · doi:10.1145/3394171.3413802 · Cited by 350
Dynamic GCN: Context-enriched Topology Learning for Skeleton-based Action Recognition
Ye, F., Pu, S., Zhong, Q., Li, C., Xie, D., & Tang, H. (2020) · Proceedings of the 28th ACM International Conference on Multimedia · doi:10.1145/3394171.3413941 · Cited by 341
Proceedings of the 2012 ACM Conference on Ubiquitous Computing
(2012) · doi:10.1145/2370216 · Cited by 331
Elderly activities recognition and classification for applications in assisted living
Chernbumroong, S., Cang, S., Atkins, A., & Yu, H. (2013) · Expert Systems with Applications · doi:10.1016/j.eswa.2012.09.004 · Cited by 317
Ha, S., & Choi, S. (2016) · 2016 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2016.7727224 · Cited by 307
AttnSense: Multi-level Attention Mechanism For Multimodal Human Activity Recognition
Ma, H., Li, W., Zhang, X., Gao, S., & Lu, S. (2019) · Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/431 · Cited by 266 · cited 3×
“[31] introduced the framework of combining attention mechanism with a CNN-RNN network to capture the dependencies of sensing signals in both spatial and temporal domains.”
MethodLearning to Drop: Robust Graph Neural Network via Topological Denoising
Luo, D., Cheng, W., Yu, W., Zong, B., Ni, J., Chen, H., & Zhang, X. (2021) · Proceedings of the 14th ACM International Conference on Web Search and Data Mining · doi:10.1145/3437963.3441734 · Cited by 248
Graph Edge Convolutional Neural Networks for Skeleton-Based Action Recognition
Zhang, X., Xu, C., Tian, X., & Tao, D. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2935173 · Cited by 205
LSTM Networks for Mobile Human Activity Recognition
Chen, Y., Zhong, K., Zhang, J., Sun, Q., & Zhao, X. (2016) · Proceedings of the 2016 International Conference on Artificial Intelligence: Technologies and Applications · doi:10.2991/icaita-16.2016.13 · Cited by 197
Deep Convolutional Neural Networks for Human Activity Recognition with Smartphone Sensors
Ronao, C. A., & Cho, S.-B. (2015) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-26561-2_6 · Cited by 176
On attention models for human activity recognition
Murahari, V. S., & Plötz, T. (2018) · Proceedings of the 2018 ACM International Symposium on Wearable Computers · doi:10.1145/3267242.3267287 · Cited by 157 · cited 2×
“Since the data recorded from two arms are not synchronized in time, following [1, 13, 31, 33], we only use the data from the right arm.”
MethodKwon, H., Tong, C., Haresamudram, H., Gao, Y., Abowd, G. D., Lane, N. D., & Plötz, T. (2020) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3411841 · Cited by 152
Contrastive Predictive Coding for Human Activity Recognition
Haresamudram, H., Essa, I., & Plötz, T. (2021) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3463506 · Cited by 150 · cited 2×
“Tang et al. [46] and Haresamudram et al. [16] explored the self-supervised learning in WHAR.”
BackgroundSelfHAR: Improving Human Activity Recognition through Self-training with Unlabeled Data
Tang, C. I., Perez-Pozuelo, I., Spathis, D., Brage, S., Wareham, N., & Mascolo, C. (2021) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3448112 · Cited by 140
Binarized-BLSTM-RNN based Human Activity Recognition
Edel, M., & Köppe, E. (2016) · International Conference on Indoor Positioning and Indoor Navigation · doi:10.1109/ipin.2016.7743581 · Cited by 134
Understanding the telework experience of people with disabilities
Tang, J. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3449104 · Cited by 109
Abedin, A., Ehsanpour, M., Shi, Q., Rezatofighi, H., & Ranasinghe, D. C. (2021) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3448083 · Cited by 107
PMC: A Privacy-preserving Deep Learning Model Customization Framework for Edge Computing
Liu, B., Li, Y., Liu, Y., Guo, Y., & Chen, X. (2020) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3432208 · Cited by 95
Graph-Temporal LSTM Networks for Skeleton-Based Action Recognition
Li, H., Zhu, G., Zhang, L., Song, J., & Shen, P. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-60639-8_40 · Cited by 82
Attention-Based Deep Learning Framework for Human Activity Recognition With User Adaptation
Buffelli, D., & Vandin, F. (2021) · IEEE Sensors Journal · doi:10.1109/jsen.2021.3067690 · Cited by 81
Deep Dilated Convolution on Multimodality Time Series for Human Activity Recognition
Xi, R., Hou, M., Fu, M., Qu, H., & Liu, D. (2018) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2018.8489540 · Cited by 65
(2020) · doi:10.1145/3380999 · Cited by 57
Yao, S., Zhao, Y., Shao, H., Liu, D., Liu, S., Hao, Y., Piao, A., Hu, S., Lu, S., & Abdelzaher, T. F. (2019) · IEEE INFOCOM 2019 - IEEE Conference on Computer Communications · doi:10.1109/infocom.2019.8737500 · Cited by 52
Leveraging Activity Recognition to Enable Protective Behavior Detection in Continuous Data
Wang, C., Gao, Y., Mathur, A., Williams, A. C. D. C., Lane, N. D., & Bianchi‐Berthouze, N. (2021) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3463508 · Cited by 51
Xia, Q., Wada, A., Korpela, J., Maekawa, T., & Namioka, Y. (2019) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3328931 · Cited by 48
Unsupervised Human Activity Representation Learning with Multi-task Deep Clustering
Ma, H., Zhang, Z., Li, W., & Lu, S. (2021) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3448074 · Cited by 48 · cited 2×
“Ma et al. [32] proposed a multi-task deep clustering framework for unsupervised WHAR.”
BackgroundApproaching the Real-World: Supporting Activity Recognition Training with Virtual IMU Data
Kwon, H., Wang, B., Abowd, G. D., & Plötz, T. (2021) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3478096 · Cited by 48
Towards reading trackers in the wild
Ishimaru, S., Hoshika, K., Kunze, K. S., Kise, K., & Dengel, A. R. (2017) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/3123024.3129271 · Cited by 46
GraphConvLSTM: Spatiotemporal Learning for Activity Recognition with Wearable Sensors
Han, J., He, Y., Liu, J., Zhang, Q., & Jing, X. (2019) · Globecom · doi:10.1109/globecom38437.2019.9013934 · Cited by 30
Chen, L., Zhang, Y., Miao, S., Zhu, S., Hu, R. J., Peng, L., & Lv, M. (2022) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2022.3171312 · Cited by 23
Self-citationSensor-Based Activity Recognition via Learning From Distributions
Qian, H., Pan, S. J., & Miao, C. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12078 · Cited by 20
Liu, S., & He, T. (2020) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3432228 · Cited by 9
Detecting Expressive Writing in Online Health Communities by Modeling Aggregated Empirical Data
Ma, H., Parawala, S., & Yarosh, S. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3449136 · Cited by 8
Research works printed but not linked: 9
- Will Kay Joao Carreira Karen Simonyan Brian Zhang Chloe Hillier Sudheendra Vijayanarasimhan Fabio Viola Tim Green Trevor Back Paul Natsev et al. 2017. The kinetics human action video dataset. arXiv preprint arXiv:1705.06950 (2017). · arXiv
- Diederik P Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In Proceedings of the International Conference on Learning Representations. 1--15.
- Song-Mi Lee, Sang Min Yoon, and Heeryon Cho. 2017. Human activity recognition from accelerometer data using convolutional neural network. In Proceedings of the IEEE International Conference on Dig Data and Smart Computing. 131--134.
- Chenglin Li, Carrie Lu Tong, Di Niu, Bei Jiang, Xiao Zuo, Lei Cheng, Jian Xiong, and Jianming Yang. 2021. Similarity embedding networks for robust human activity recognition. ACM Transactions on Knowledge Discovery from Data 15, 6 (2021), 1--17.
- Chao Li, Qiaoyong Zhong, Di Xie, and Shiliang Pu. 2017. Skeleton-based action recognition with convolutional neural networks. In Proceedings of the IEEE International Conference on Multimedia & Expo Workshops. 597--600.
- Chao Li, Qiaoyong Zhong, Di Xie, and Shiliang Pu. 2018. Co-occurrence feature learning from skeleton data for action recognition and detection with hierarchical aggregation. arXiv preprint arXiv:1804.06055 (2018). · arXiv
- 10.1145/3432208 · DOI · Paper page
- Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. 2019. PyTorch: An imperative style, high-performance deep learning library. In Proceedings of the Advances in Neural Information Processing Systems. 8026--8037.
- Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of the Advances in Neural Information Processing Systems. 5998--6008.
Cited by
Suh, S., Rey, V. F., & Lukowicz, P. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2022.110143 · Cited by 107
Yin, Y., Xie, L., Jiang, Z., Xiao, F., Cao, J., & Lu, S. (2024) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2024.3357591 · Cited by 108
“…especially for fine-grained activity and multi-level activity recognition, e.g., in-air hand gesture recognition [119], finger touch identification [79], tongue-jaw movement recognition [61], personal cough detection [89], sleep apnea detection [93], and daily activity recognition [97], [98], [103].”
Context-aware mutual learning for semi-supervised human activity recognition using wearable sensors
Qu, Y., Tang, Y., Yang, X., Wen, Y., & Zhang, W. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.119679 · Cited by 34
Chen, W., Cheng, J., Wang, L., Zhao, W., & Matusik, W. (2024) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3699747 · Cited by 27
Spatial-Temporal Masked Autoencoder for Multi-Device Wearable Human Activity Recognition
Miao, S., Chen, L., & Hu, R. (2023) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3631415 · Cited by 23
Self-citation
Metadata
- Authors
- Miao, S., Chen, L., Hu, R., & Luo, Y.
- Affiliations
- Zhejiang University (CN)
- Venue
- Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
- Published
- 2022-09-06
- Volume
- 6
- Issue
- 3
- Pages
- 1-25
- ISSN
- 2474-9567 (ISSN-L)
- OpenAlex
- W4294898655
- Cited by
- 36
- References
- 53
- Citations by year
- 2026: 8 · 2025: 9 · 2024: 12 · 2023: 6 · 2022: 1
- FWCI
- 2.29
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Physical Sciences › Engineering › Biomedical Engineering › Gait Recognition and Analysis
- Physical Sciences › Engineering › Biomedical Engineering › Advanced Sensor and Energy Harvesting Materials
- Keywords
- multi-sensor data, graph convolutional networks, feature extraction, resource-constrained devices, lightweight models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.