Unsupervised Human Activity Representation Learning with Multi-task Deep Clustering
The DOI matches its Crossref record; 57 of the 57 references it cites were found in a registry or an index.
Abstract
Human activity recognition (HAR) based on sensing data from wearable and mobile devices has become an active research area in ubiquitous computing, and it envisions a wide range of application scenarios in mobile social networking, environmental context sensing, health and well-being monitoring, etc. However, activity recognition based on manually annotated sensing data is manpower-expensive, time-consuming, and privacy-sensitive, which prevents HAR systems from being really deployed in scale. In this paper, we address the problem of unsupervised human activity recognition, which infers activities from unlabeled datasets without the need of domain knowledge. We propose an end-to-end multi-task deep clustering framework to solve the problem. Taking the unlabeled multi-dimensional sensing signals as input, we firstly apply a CNN-BiLSTM autoencoder to form a compressed latent feature representation. Then we apply a K-means clustering algorithm based on the extracted features to partition the dataset into different groups, which produces pseudo labels for the instances. We further train a deep neural network (DNN) with the latent features and pseudo labels for activity recognition. The tasks of feature representation, clustering, and classification are integrated into a uniform multi-task learning framework and optimized jointly to achieve unsupervised activity classification. We conduct extensive experiments based on three public datasets. It is shown that the proposed approach…
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Multi-task deep clustering algorithm uses an autoencoder based on CNN-BiLSTM architecture to form a latent feature representation.The K-means clustering algorithm is then applied to divide the dataset into different groups for generating pseudo-labels.Finally, a deep neural network (DNN) is further trained for human activity recognition.”
From TS2ACT: Few-Shot Human Activity Sensing with Cross-Modal Co-Learning (2023) · cites this paper as [28]
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
(2008) · Journal of Machine Learning Research · Cited by 33,051
Activity Recognition from User-Annotated Acceleration Data
Bao, L., & Intille, S. S. (2004) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-24646-6_1 · Cited by 3,147
Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition
Ordóñez, F., & Roggen, D. (2016) · Sensors · doi:10.3390/s16010115 · Cited by 2,761
Deep Clustering for Unsupervised Learning of Visual Features
Caron, M., Bojanowski, P., Joulin, A., & Douze, M. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01264-9_9 · Cited by 2,674
A Survey on Human Activity Recognition using Wearable Sensors
Lara, O. D., & Labrador, M. A. (2013) · IEEE Communications Surveys & Tutorials · doi:10.1109/surv.2012.110112.00192 · Cited by 2,662 · cited 2×
“In the past decade, human activity recognition (HAR) based on embedded sensing data has become a very active research field in ubiquitous computing [33, 61].”
BackgroundMethodInternational Conference on Machine Learning (ICML)-2005
津田, 宏. (2006) · 人工知能学会誌 = Journal of Japanese Society for Artificial Intelligence · Cited by 2,558
A Comprehensive Survey of Clustering Algorithms
Xu, D., & Tian, Y. (2015) · Annals of Data Science · doi:10.1007/s40745-015-0040-1 · Cited by 1,939
Doherty, A., Jackson, D., Hammerla, N., Plötz, T., Olivier, P., Granat, M. H., White, T., van Hees, V. T., Trenell, M. I., Owen, C. G., et al. (2017) · PLOS ONE · doi:10.1371/journal.pone.0169649 · Cited by 1,250
Research works printed but not linked: 17
- Margareta Ackerman and Sanjoy Dasgupta. 2014. Incremental clustering: The case for extra clusters. In Advances in Neural Information Processing Systems. 307--315. Margareta Ackerman and Sanjoy Dasgupta. 2014. Incremental clustering: The case for extra clusters. In Advances in Neural Information Processing Systems. 307--315.
- Mohammad Abu Alsheikh , Ahmed Selim , Dusit Niyato , Linda Doyle , Shaowei Lin , and Hwee-Pink Tan . 2016 . Deep activity recognition models with triaxial accelerometers . In Workshops at the Thirtieth AAAI Conference on Artificial Intelligence. Mohammad Abu Alsheikh, Ahmed Selim, Dusit Niyato, Linda Doyle, Shaowei Lin, and Hwee-Pink Tan. 2016. Deep activity recognition models with triaxial accelerometers. In Workshops at the Thirtieth AAAI Conference on Artificial Intelligence.
- Rich Caruana . 1997. Multitask learning. Machine learning 28, 1 ( 1997 ), 41--75. Rich Caruana. 1997. Multitask learning. Machine learning 28, 1 (1997), 41--75.
- Sojeong Ha , Jeong-Min Yun , and Seungjin Choi . 2015 . Multi-modal Convolutional Neural Networks for Activity Recognition. 2015 IEEE International Conference on Systems, Man, and Cybernetics, 3017--3022 . Sojeong Ha, Jeong-Min Yun, and Seungjin Choi. 2015. Multi-modal Convolutional Neural Networks for Activity Recognition. 2015 IEEE International Conference on Systems, Man, and Cybernetics, 3017--3022.
- 10.5555/2886521.2886562 · doi:10.5555/2886521.2886562
- 10.5555/3060832.3060835 · doi:10.5555/3060832.3060835
- Md Rezaul Karim , Oya Beyan , Achille Zappa , Ivan G Costa , Dietrich Rebholz-Schuhmann , Michael Cochez , and Stefan Decker . 2020. Deep learning-based clustering approaches for bioinformatics. Briefings in Bioinformatics ( 2020 ). Md Rezaul Karim, Oya Beyan, Achille Zappa, Ivan G Costa, Dietrich Rebholz-Schuhmann, Michael Cochez, and Stefan Decker. 2020. Deep learning-based clustering approaches for bioinformatics. Briefings in Bioinformatics (2020).
- Yongmou Li , Dianxi Shi , Bo Ding , and Dongbo Liu . 2014. Unsupervised feature learning for human activity recognition using smartphone sensors . In Mining intelligence and knowledge exploration . Springer , 99--107. Yongmou Li, Dianxi Shi, Bo Ding, and Dongbo Liu. 2014. Unsupervised feature learning for human activity recognition using smartphone sensors. In Mining intelligence and knowledge exploration. Springer, 99--107.
- Haojie Ma , Wenzhong Li , X. Zhang , S. Gao , and S. Lu . 2019 . AttnSense: Multi-level Attention Mechanism For Multimodal Human Activity Recognition. In the 28th International Joint Conference on Artificial Intelligence (IJCAI'19) . Haojie Ma, Wenzhong Li, X. Zhang, S. Gao, and S. Lu. 2019. AttnSense: Multi-level Attention Mechanism For Multimodal Human Activity Recognition. In the 28th International Joint Conference on Artificial Intelligence (IJCAI'19).
- Naveen Sai Madiraju , Seid M Sadat , Dimitry Fisher , and Homa Karimabadi . 2018. Deep temporal clustering: Fully unsupervised learning of time-domain features. arXiv preprint arXiv:1802.01059 ( 2018 ). Naveen Sai Madiraju, Seid M Sadat, Dimitry Fisher, and Homa Karimabadi. 2018. Deep temporal clustering: Fully unsupervised learning of time-domain features. arXiv preprint arXiv:1802.01059 (2018). · arXiv
- D. Minnen , T. Starner , I. Essa , and C. Isbell . 2006. Discovering Characteristic Actions from On-Body Sensor Data . In IEEE 10th International Symposium on Wearable Computers. 11--18 . https://doi.org/10.1109/ISWC. 2006 .286337 D. Minnen, T. Starner, I. Essa, and C. Isbell. 2006. Discovering Characteristic Actions from On-Body Sensor Data. In IEEE 10th International Symposium on Wearable Computers. 11--18. https://doi.org/10.1109/ISWC.2006.286337 · doi:10.1109/iswc
- Hangwei Qian , Sinno Jialin Pan , and C. Miao . 2018. Sensor-Based Activity Recognition via Learning From Distributions . In Thirty-Second AAAI Conference on Artificial Intelligence. Hangwei Qian, Sinno Jialin Pan, and C. Miao. 2018. Sensor-Based Activity Recognition via Learning From Distributions. In Thirty-Second AAAI Conference on Artificial Intelligence.
- Neda Tavakoli , Sima Siami-Namini , Mahdi Adl Khanghah , Fahimeh Mirza Soltani, and Akbar Siami Namin. 2020 . Clustering Time Series Data through Autoencoder-based Deep Learning Models . arXiv preprint arXiv:2004.07296 (2020). Neda Tavakoli, Sima Siami-Namini, Mahdi Adl Khanghah, Fahimeh Mirza Soltani, and Akbar Siami Namin. 2020. Clustering Time Series Data through Autoencoder-based Deep Learning Models. arXiv preprint arXiv:2004.07296 (2020). · arXiv
- Jindong Wang , Yiqiang Chen , Shuji Hao , Xiaohui Peng , and Lisha Hu. 2018. Deep learning for sensor-based activity recognition: A survey. Pattern Recognition Letters ( 2018 ). Jindong Wang, Yiqiang Chen, Shuji Hao, Xiaohui Peng, and Lisha Hu. 2018. Deep learning for sensor-based activity recognition: A survey. Pattern Recognition Letters (2018).
- Bo Yang , Xiao Fu , Nicholas D Sidiropoulos , and Mingyi Hong . 2017 . Towards k-means-friendly spaces: Simultaneous deep learning and clustering . In international conference on machine learning (ICML'17) . PMLR, 3861--3870. Bo Yang, Xiao Fu, Nicholas D Sidiropoulos, and Mingyi Hong. 2017. Towards k-means-friendly spaces: Simultaneous deep learning and clustering. In international conference on machine learning (ICML'17). PMLR, 3861--3870.
- Jianbo Yang , Minh Nhut Nguyen , Phyo Phyo San, Xiaoli Li, and Shonali Krishnaswamy. 2015 . Deep Convolutional Neural Networks on Multichannel Time Series for Human Activity Recognition. In IJCAI. Jianbo Yang, Minh Nhut Nguyen, Phyo Phyo San, Xiaoli Li, and Shonali Krishnaswamy. 2015. Deep Convolutional Neural Networks on Multichannel Time Series for Human Activity Recognition. In IJCAI.
- Hongyuan Zha Xiaofeng He Chris Ding Ming Gu and Horst D Simon. 2002. Spectral relaxation for k-means clustering. In Advances in neural information processing systems. 1057--1064. Hongyuan Zha Xiaofeng He Chris Ding Ming Gu and Horst D Simon. 2002. Spectral relaxation for k-means clustering. In Advances in neural information processing systems. 1057--1064.
Cited by
Qiu, S., Zhao, H., Jiang, N., Wang, Z., Liu, L., An, Y., Zhao, H., Miao, X., Liu, R., & Fortino, G. (2022) · Information Fusion · doi:10.1016/j.inffus.2021.11.006 · Cited by 671
Yin, Y., Xie, L., Jiang, Z., Xiao, F., Cao, J., & Lu, S. (2024) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2024.3357591 · Cited by 108
“When combining multi-modal sensor data, different recognition approaches were also used [135], e.g., using image processing to track fingers in 2D space, while using phase changes of ultrasound to track the depth of fingers in 3D space, to provide a depth-aware tapping scheme.”
Self-citationSingh, J., & Singh, D. (2024) · Advanced Engineering Informatics · doi:10.1016/j.aei.2024.102799 · Cited by 105
A survey on unsupervised learning for wearable sensor-based activity recognition
Ige, A. O., & Noor, M. H. M. (2022) · Applied Soft Computing · doi:10.1016/j.asoc.2022.109363 · Cited by 99
Li, Y., Wang, X., & An, L. (2023) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3580795 · Cited by 82
“With the rapid development of deep learning techniques, a large number of works have applied deep learning methods for HAR[5][7][28], but these works generally require the collection of user data for centralized training, which undoubtedly leaks the privacy of users (e.g., health data, locations). unit (IMU), radar, a…”
Method
Metadata
- Authors
- Ma, H., Zhang, Z., Li, W., & Lu, S.
- Affiliations
- Nanjing University (CN)
- Venue
- Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
- Published
- 2021-03-19
- Volume
- 5
- Issue
- 1
- Pages
- 1-25
- ISSN
- 2474-9567 (ISSN-L)
- OpenAlex
- W3152187098
- Cited by
- 48
- References
- 56
- Citations by year
- 2026: 5 · 2025: 7 · 2024: 18 · 2023: 9 · 2022: 8 · 2021: 1
- FWCI
- 3.24
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Social Sciences › Transportation › Human Mobility and Location-Based Analysis
- Physical Sciences › Computer Science › Signal Processing › Time Series Analysis and Forecasting
- Keywords
- pseudo-labeling, K-means clustering, latent representations, activity classification, annotation cost reduction, mobile health monitoring
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Unsupervised Human Activity Representation Learning with Multi-task Deep Clustering. https://app.cochat.ai/papers/10.1145/3448074
- The paper
- Ma, H., Zhang, Z., Li, W., & Lu, S. (2021). Unsupervised Human Activity Representation Learning with Multi-task Deep Clustering. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies. https://doi.org/10.1145/3448074
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.