Multi-task Self-Supervised Learning for Human Activity Detection
Open accessThe DOI matches its Crossref record; 85 of the 85 references it cites were found in a registry or an index.
Abstract
Deep learning methods are successfully used in applications pertaining to ubiquitous computing, pervasive intelligence, health, and well-being. Specifically, the area of human activity recognition (HAR) is primarily transformed by the convolutional and recurrent neural networks, thanks to their ability to learn semantic representations directly from raw input. However, in order to extract generalizable features massive amounts of well-curated data are required, which is a notoriously challenging task; hindered by privacy issues and annotation costs. Therefore, unsupervised representation learning (i.e., learning without manually labeling the instances) is of prime importance to leverage the vast amount of unlabeled data produced by smart devices. In this work, we propose a novel self-supervised technique for feature learning from sensory data that does not require access to any form of semantic labels, i.e., activity classes. We learn a multi-task temporal convolutional network to recognize transformations applied on an input signal. By exploiting these transformations, we demonstrate that simple auxiliary tasks of the binary classification result in a strong supervisory signal for extracting useful features for the down-stream task. We extensively evaluate the proposed approach on several publicly available datasets for smartphone-based HAR in unsupervised, semi-supervised and transfer learning settings. Our method achieves performance levels superior to or comparable with fully-supervised networks trained directly with activity labels, and it performs significantly better than unsupervised learning through autoencoders. Notably, for the semi-supervised case, the self-supervised features substantially boost the detection rate by attaining a kappa score between 0.7 - 0.8 with only 10 labeled examples per class. We get similar impressive performance even if the features are transferred from a different data source. Self-supervision drastically reduces the requirement of labeled activity data, effectively narrowing the gap between supervised and unsupervised techniques for learning meaningful representations. While this paper focuses on HAR as the application domain, the proposed approach is general and could be applied to a wide variety of problems in other areas.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“In contrast, previous studies applied self-supervised training and fine-tuning only on the same data sources24,27,40, making it necessary to pre-train on every new dataset in practical applications.”
From Self-supervised learning for human activity recognition using 700,000 person-days of wearable data (2024) · cites this paper as [27]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“For windows of sensor data, Saeed et al. [166] apply each transformation with a probability of 50%, and passed the probabilistically transformed windows through a common convolutional encoder.”
From Past, Present, and Future of Sensor-based Human Activity Recognition Using Wearables: A Surveying Tutorial on a Still Challenging Task (2025) · cites this paper as Saeed et al. [166]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Haresamudram et al. [81] show that the SimCLR framework [184] and multi-task self-supervised learning [166], outperform other approaches, including BYOL, SimSiam, masked reconstruction, CPC and autoencoders for HAR.”
From Past, Present, and Future of Sensor-based Human Activity Recognition Using Wearables: A Surveying Tutorial on a Still Challenging Task (2025) · cites this paper as [166]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The established multi-task self-sup. Ref. [15] framework is ill-suited for discrete representation learning, with significant reductions in performance throughout, consistently performing worse by approx. 10% for most datasets and peaking at around 29% for Mobiact.”
From Towards Learning Discrete Representations via Self-Supervision for Wearables-Based Human Activity Recognition (2024) · cites this paper as [15]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“We show that our method outperforms existing self-supervised methods in HAR for UCI HAR, MotionSense, and HAPT datasets.”
From Self-supervised Human Activity Recognition by Learning to Predict Cross-Dimensional Motion (2021) · cites this paper as [19]
Experimental — accuracy not yet measured
19 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 1×
“These techniques have achieved indisputable empirical success across a broad spectrum of problems [4, 21, 28, 36, 43, 56, 65, 66] thanks to the increasing dataset sizes and computing power availability.”
BackgroundAdam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 3×
“We use ReLU as non-linearity in all the layers (except the output) and train a network with Adam optimizer [26] for a maximum of 30 epochs with a learning rate of 0.”
MethodInfluentialLeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
(2008) · Journal of Machine Learning Research · Cited by 33,051
Pan, S. J., & Yang, Q. (2010) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2009.191 · Cited by 23,387 · cited 1×
“owledge transfer se−ings (such as features, instances, weights) from existing labeled data of some related task or domain. For a detailed review of transfer learning, we refer an interested reader to [53]. 2.4 Semi-supervised Learning Semi-supervised learning provides a compelling framework for leveraging unlabeled da…”
BackgroundInfluentialSequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766 · cited 1×
“These techniques have achieved indisputable empirical success across a broad spectrum of problems [4, 21, 28, 36, 43, 56, 65, 66] thanks to the increasing dataset sizes and computing power availability.”
Background(2010) · International Conference on Machine Learning · Cited by 13,995
Representation Learning: A Review and New Perspectives
Bengio, Y., Courville, A., & Vincent, P. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.50 · Cited by 13,001 · cited 3×
“is inherent limitation of the fully-supervised learning paradigm emphasizes the importance of unsupervised learning to leverage a large amount of unlabeled data for representation learning [8] that can be easily acquired in a real-world seing.”
BackgroundInfluentialCaruana, R. (1997) · Machine Learning · doi:10.1023/a:1007379606734 · Cited by 6,636 · cited 2×
“e goal of multi-task learning (MTL) is to enhance the learning eciency and accuracy through simultaneously optimizing multiple objectives based on shared representations and exploiting relations among the tasks [10].”
BackgroundDeepFace: Closing the Gap to Human-Level Performance in Face Verification
Taigman, Y., Yang, M., Ranzato, M., & Wolf, L. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.220 · Cited by 6,542 · cited 1×
“ese techniques have achieved indisputable empirical success across a broad spectrum of problems [4, 21, 28, 35, 43, 56, 65, 66] thanks to the increasing dataset sizes and computing power availability.”
Background
Research works printed but not linked: 7
- Baldi Pierre, 2012, Proceedings of ICML workshop on unsupervised and transfer learning. 37--49
- Chatzaki Charikleia, 2016, International Conference on Information and Communication Technologies for Ageing Well and e-Health. Springer, 100--118
- 10.5555/3104322.3104425 · doi:10.5555/3104322.3104425
- 10.5555/3305890.3305968 · doi:10.5555/3305890.3305968
- Plötz Thomas, 2011, IJCAI Proceedings-International Joint Conference on Artificial Intelligence, 22
- Razavian Narges, 2016, Machine Learning for Healthcare Conference. 73--100
- Wawrzyniak S., 2015 Federated Conference on Computer Science and Information Systems (FedCSIS). 411--416
Cited by
Items 101–200 of 345
ConvBoost: Boosting ConvNets for Sensor-based Activity Recognition
Shao, S., Guan, Y., Zhai, B., Missier, P., & Plötz, T. (2023) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3596234 · Cited by 30
“To alleviate this overfitting issue, several research directions were explored including data augmentation [19, 38, 45], self-supervised learning (SSL)[12, 13, 32, 35], or learning paradigm design (e.”
MethodA Self-Supervised Human Activity Recognition Approach via Body Sensor Networks in Smart City
Zhou, Y., Xie, C., Sun, S., Zhang, X., & Wang, Y. (2023) · IEEE Sensors Journal · doi:10.1109/jsen.2023.3282601 · Cited by 23
“Saeed et al. [29] proposed a new self-supervised technique capable of learning features of activity from sensory data without activity labels, while learning a multi-task temporal convolutional network to identify transformations applied to that input signal, extracting useful features for…”
BackgroundWearable-based behaviour interpolation for semi-supervised human activity recognition
Duan, H., Wang, S., Ojha, V., Wang, S., Huang, Y., Long, Y., Ranjan, R. K., & Zheng, Y. (2024) · Information Sciences · doi:10.1016/j.ins.2024.120393 · Cited by 29
“The final MixHAR achieved significant performance and showed the effectiveness of us-1 Both [5] and [10] are the methods that aim to take advantage of unlabelled data.”
BackgroundMethodInfluentialRobust Feature Representation Using Multi-Task Learning for Human Activity Recognition
Azadi, B., Haslgrübler, M., Anzengruber-Tánase, B., Sopidis, G., & Ferscha, A. (2024) · Sensors · doi:10.3390/s24020681 · Cited by 26
“Saeed et al. [30] highlighted the importance of unsupervised learning exploiting vast amounts of unlabeled data and proposed a self-supervised technique for representation learning from unlabeled sensory data.”
BackgroundPanagiotou, M., Zlatintsi, A., Filntisis, P. P., Roumeliotis, A. J., Efthymiou, N., & Maragos, P. A. (2022) · European Signal Processing Conference (EUSIPCO) · doi:10.23919/eusipco55093.2022.9909697 · Cited by 26
“, emotional wellbeing [4], sleep tracking [5], and physical activity detection [6]”
BackgroundBrand, Y. E., Kluge, F., Palmerini, L., Paraschiv-Ionescu, A., Becker, C., Cereatti, A., Maetzler, W., Sharrack, B., Vereijken, B., Yarnall, A. J., et al. (2024) · Scientific Reports · doi:10.1038/s41598-024-71491-3 · Cited by 19
TS2ACT: Few-Shot Human Activity Sensing with Cross-Modal Co-Learning
Xia, K., Li, W., Gan, S., & Lu, S. (2023) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3631445 · Cited by 19
Ray, S. J., Cherian, J., Liberty, A. M., Hammond, T. A., & Shireman, P. K. (2025) · Journal of Medical Internet Research · doi:10.2196/67373 · Cited by 18
Spatiotemporal Contrastive Learning of Facial Expressions in Videos
Roy, S., & Etemad, A. (2021) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii52823.2021.9597460 · Cited by 22
Deep Crash Detection From Vehicular Sensor Data With Multimodal Self-Supervision
Kubin, L., Bianconcini, T., de Andrade, D. C., Simoncini, M., Taccari, L., & Sambo, F. (2021) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2021.3114816 · Cited by 19
“Contrastive self-supervised learning techniques have been successfully applied as pretraining tasks in numerous deep learning field, from speech [15], time series [16], [17], structured language models [18]”
MethodRF-CM: Cross-Modal Framework for RF-enabled Few-Shot Human Activity Recognition
Wang, X., Liu, T., Feng, C., Fang, D., & Chen, X. (2023) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3580859 · Cited by 26
“Therefore, a fine-tuning strategy of transfer learning [34] is ineffective since the fine-tuning radar samples and the WiFi data have the same dimension.”
BackgroundModality Consistency-Guided Contrastive Learning for Wearable-Based Human Activity Recognition
Guo, C., Zhang, Y., Chen, Y., Xu, C., & Wang, Z. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3379019 · Cited by 24
“In the field of time series, many pretext tasks have been proposed according to the unique attributes of time series, including masked reconstruction [11], data transformation predictions [12], etc.”
BackgroundSun, Y., Xu, X., Tian, X., Zhou, L., & Li, Y. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.108705 · Cited by 26
Jaberi, M., & Ravanmehr, R. (2022) · Pervasive and Mobile Computing · doi:10.1016/j.pmcj.2022.101620 · Cited by 21
Ray, S. J., Cherian, J., Liberty, A. M., Hammond, T. A., & Shireman, P. K. (2024) · doi:10.2196/preprints.67373 · Cited by 18
If only we had more data!: Sensor-Based Human Activity Recognition in Challenging Scenarios
Plötz, T. (2023) · 2023 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops) · doi:10.1109/percomworkshops56833.2023.10150267 · Cited by 16
“For wearables, [40] first explored self-supervision via a multi-task setting by performing transformation discrimination of different augmentations applied separately and probabilistically to accelerometry.”
MethodGOAT: A Generalized Cross-Dataset Activity Recognition Framework with Natural Language Supervision
Miao, S., & Chen, L. (2024) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3699736 · Cited by 27
“Self-supervised methods [13, 30, 47, 61] have emerged as a powerful paradigm to exploit large-scale unlabeled data.”
DAPPER: Label-Free Performance Estimation after Personalization for Heterogeneous Mobile Sensing
Gong, T., Kim, Y., Orzikulova, A., Liu, Y., Hwang, S. J., Shin, J., & Lee, S.-J. (2023) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3596256 · Cited by 23
“For the backbone networks, we adopted 1D convolutional neural networks (CNN) [32] followed by fully-connected layers, which are commonly used in mobile sensing [6, 21, 30, 63].”
MethodSemi-supervised contrastive regression for pharmaceutical processes
Li, Y., Liao, Y., Sun, Z., & Liu, X. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.121974 · Cited by 18
Modality aware contrastive learning for multimodal human activity recognition
Dixon, S., Yao, L., & Davidson, R. (2024) · Concurrency and Computation Practice and Experience · doi:10.1002/cpe.8020 · Cited by 19
MESEN: Exploit Multimodal Data to Design Unimodal Human Activity Recognition with Few Labels
Xu, L., Gu, C., Tan, R., He, S., & Chen, J. (2023) · ACM Conference on Embedded Networked Sensor Systems (SenSys) · doi:10.1145/3625687.3625782 · Cited by 26
“Depending on the modalities involved in the model design and deployment phases, existing studies can be divided into the categories of single-modality [31, 33] and multi-modality [10, 14, 23, 26, 42].”
BackgroundMethodSheng, Y., Yan, J., & Piao, M. (2024) · Journal of Intelligent Manufacturing · doi:10.1007/s10845-024-02444-w · Cited by 16
“Recently, self-supervised learning has successfully applied to language representation (Lan et al., 2019), human action recognition (Saeed et al., 2019), semantic segmentation (Ziegler & Asano, 2022), object detection (Wang et al., 2021), image classification (Azizi et al., 2021), etc.”
Khowaja, S. A., Khuwaja, P., Dharejo, F. A., Raza, S., Lee, I. H., Naqvi, R. A., & Dev, K. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.102044 · Cited by 19
Haresamudram, H., Essa, I., & Plötz, T. (2024) · Sensors · doi:10.3390/s24041238 · Cited by 14
“[15] framework is ill-suited for discrete representation learning, with significant reductions in performance throughout, consistently performing worse by approx.”
BackgroundMethodInfluentialZou, C., Deng, Z., He, B., Yan, M., Wu, J., & Zhu, Z. (2024) · Cognitive Computation and Systems · doi:10.1049/ccs2.12107 · Cited by 20
Self-supervised Learning for Reading Activity Classification
Islam, M. R., Sakamoto, S., Yamada, Y., Vargo, A. W., Iwata, M., Iwamura, M., & Kise, K. (2021) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3478088 · Cited by 24
“In HAR, the eight signal transformations are proposed by Saeed et al. [46] for ACC and GYRO sensor data; noise addition, scale, rotation, vertical flip, horizontal flip, permutation, time-warp, and channel-shuffle.”
MethodUsing additional training sensors to improve single-sensor complex activity recognition
Lago, P., Matsuki, M., Adachi, K., & Inoue, S. (2021) · 2021 International Symposium on Wearable Computers · doi:10.1145/3460421.3480421 · Cited by 14
Kwon, H., Tong, C., Haresamudram, H., Gao, Y., Abowd, G. D., Lane, N. D., & Ploetz, T. (2020) · arXiv · doi:10.48550/arxiv.2006.05675 · Cited by 14
“et dataset is significantly smaller compared to the base dataset, transfer learning enables training a large target network without overfitting [94], and typically results in improved performance. In [77], the authors propose a self-supervision pretext task and demonstrate its effectiveness for unsupervised transfer l…”
BackgroundChen, X., Zhou, X., Sun, M., & Wang, H. (2025) · IEEE Sensors Journal · doi:10.1109/jsen.2024.3491933 · Cited by 13
“In [8], a multitask self-supervised model has been proposed to learn sensor representations and predict human activities.”
InfluentialLLM4HAR: Generalizable On-device Human Activity Recognition with Pretrained LLMs
Hong, Z., Song, Y., Li, Z., Yu, A., Zhong, S., Ding, Y. C., He, T., & Zhang, D. (2025) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3711896.3737226 · Cited by 34
“We evaluate models with UCI [41], Shoaib [46], Motion [34], and HHAR [47], as they are widely used in previous HAR studies [5, 37, 44, 60, 61, 67].”
Wei, B., Yi, C., Zhang, Q., Zhu, H., Zhu, J., & Jiang, F. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3314150 · Cited by 16
“D convolutional layers to extract features along the temporal dimension [36].”
Background1D Convolutional Neural Network with Long Short-Term Memory for Human Activity Recognition
Goh, J. X., Lim, K. M., & Lee, C. P. (2021) · IEEE International Conference on Artificial Intelligence in Engineering and Technology (IICAIET) · doi:10.1109/iicaiet51634.2021.9573979 · Cited by 14
Sign Language Recognition With Self-Learning Fusion Model
Vu, H. N., Hoang, T., Tran, C., & Pham, C. (2023) · IEEE Sensors Journal · doi:10.1109/jsen.2023.3314728 · Cited by 13
“To solve this critical problem, self-learning-based methods have been proposed to perform automatic feature learning from unlabeled data [17], [18], [19].”
BackgroundMethodSemiHAR: Improving Semisupervised Human Activity Recognition via Multitask Learning
Wei, C., Wang, Z., Yuan, J., Wang, X., Liu, H., & Zhao, Q. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3330879 · Cited by 14
Federated Learning for Privacy-aware Cognitive Workload Estimation
Fenoglio, D., Josifovski, D., Gobbetti, A., Formo, M., Gjoreski, H., Gjoreski, M., & Langheinrich, M. (2023) · International Conference on Mobile and Ubiquitous Multimedia (MUM) · doi:10.1145/3626705.3627783 · Cited by 16
“Specifically, the plot illustrates the mean values along with the standard deviation (± 𝜎 ) observed during the 4-fold cross-validation.”
MethodMTS-LOF: Medical Time-Series Representation Learning via Occlusion-Invariant Features
Li, H., Carreon-Rascon, A. S., Chen, X., Yuan, G., & Li, A. (2024) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2024.3373439 · Cited by 21
“Index Terms —Medical time series, self-supervised learning, health monitoring, masked autoencoder,”
Tian, Y., Hettiarachchi, D., Yu, H. S., & Kamijo, S. (2024) · IEEE Sensors Journal · doi:10.1109/jsen.2024.3404777 · Cited by 15
“Our method was compared with 8 state-of-the-art methods, including: ABLSTM [26], TPN [27], DCL [28], PEN [25], MSRLSTM [29], EResVit [31], DEBONAIR [42], InnoHAR [43] based HAR algorithms, and demonstrated superior performance.”
MethodLin, Z., Jiang, X., Zhang, K., Fan, C., & Liu, Y. (2025) · Future Generation Computer Systems · doi:10.1016/j.future.2025.107724 · Cited by 17
Tian, Y., Hettiarachchi, D., & Kamijo, S. (2022) · Sensors · doi:10.3390/s22176453 · Cited by 12
“Saeed et al. proposed a self-supervised deep network, named the Transformation Prediction Network (TPN) [19], to recognize transportation modes, and is composed of several convolutional layers, using self-supervised pre-training to improve the recognition performance.”
MethodGupta, E., Gupta, V., Chopra, M., Chhipa, P. C., & Liwicki, M. (2023) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn54540.2023.10191796 · Cited by 11
“A potential solution is to train models on unlabeled images using self-supervised learning [9] [10].”
BackgroundContrastSense: Domain-invariant Contrastive Learning for In-the-Wild Wearable Sensing
Dai, G., Xu, H., Yoon, H., Li, M., Tan, R., & Lee, S.-J. (2024) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3699744 · Cited by 27
“The problem is further challenged by the scarcity of class labels, as annotating wearable sensing data could be expensive and time-consuming [49, 53, 62].”
MuJo: Multimodal Joint Feature Space Learning for Human Activity Recognition
Fritsch, S. G., Oguz, C., Rey, V. F., Ray, L. S. S., Kiefer-Emmanouilidis, M., & Lukowicz, P. (2025) · IEEE International Conference on Pervasive Computing and Communications · doi:10.1109/percom64205.2025.00020 · Cited by 16
“Moreover, we compare it with three other commonly used pre-training methods, namely Multi-task [38], Auto-encoder [39], and simCLR [40], which we also pre-trained on our FiMAD dataset.”
InfluentialClustering of Human Activities from Wearables by Adopting Nearest Neighbors
Ahmed, A., Haresamudram, H., & Plötz, T. (2022) · ACM International Symposium on Wearable Computers (ISWC) · doi:10.1145/3544794.3558477 · Cited by 14
“A more recent work builds on multi-task self-supervision [27] by adding self-training, which distills the knowledge of both unlabeled and labeled datasets [29].”
BackgroundMethodOrdered physical human activity recognition based on ordinal classification
DAŞ, D. B., & Birant, D. (2021) · TURKISH JOURNAL OF ELECTRICAL ENGINEERING & COMPUTER SCIENCES · doi:10.3906/elk-2010-75 · Cited by 11
Self-Supervised Learning via Multi-Transformation Classification for Action Recognition
Vu, D.-Q., Le, N., & Wang, J. (2024) · IEEE International Conference on Multimedia and Expo workshops · doi:10.1109/icmew63481.2024.10645477 · Cited by 13
“The success of multi-task self-supervised learning inspires our proposed method ( [14]–[17]).”
MethodA New Deep Learning Method with Self-Supervised Learning for Delineation of the Electrocardiogram
Wu, W., Huang, Y., & Wu, X. (2022) · Entropy · doi:10.3390/e24121828 · Cited by 10
“Referring to [12], different signal transformations were applied to segmented heartbeats, respectively.”
MethodWang, J., Cui, W., Zhu, T., Ning, H., & Liu, Z. (2024) · IEEE Sensors Journal · doi:10.1109/jsen.2024.3390755 · Cited by 13
“These tasks include augmentation method discrimination [19], contrastive learning [20], [21], [22], [23], data reconstruction [24], and masked reconstruction [25], [26].”
MethodYu, J., Zhang, L., Cheng, D., Huang, W., Wu, H., & Song, A. (2024) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2024.3457830 · Cited by 14
“Saeed et al. [45] present a novel self-supervised feature learning approach for sensory data with no need of semantic labels.”
BackgroundResultQin, Y., Luo, H., Zhao, F., Wang, C., & Fang, Y. (2021) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2021.3060761 · Cited by 11
“TNP [40]: A self-supervised deep network named Transformation Prediction Network (TPN) is used for transportation mode recognition.”
BackgroundLeveraging Self-Supervised Learning for Human Activity Recognition with Ambient Sensors
Chen, H., Gouin-Vallerand, C., Bouchard, K., Gaboury, S., Couture, M., Bier, N., & Giroux, S. (2023) · International Conference on Information Technology for Social Good (GoodIT) · doi:10.1145/3582515.3609551 · Cited by 11
“The augmentation module applied seven selected augmentation techniques [28, 29] to the ambient”
MethodYoshimura, N., Maekawa, T., & Hara, T. (2021) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9533888 · Cited by 12
“Saeed et al. [17] introduced self-supervised learning to reduce the amount of labeled data required for network training.”
BackgroundYang, L., Li, X., Ma, Z., Li, L., Xiong, N. N., & Ma, J. (2023) · ACM Transactions on Internet Technology · doi:10.1145/3594538 · Cited by 12
“The benchmark studies used to compare the performance with our proposed method are SPs[47], [29], [38], [35], and DeepConvLSTM[33].”
MethodResultHow Much Unlabeled Data is Really Needed for Effective Self-Supervised Human Activity Recognition?
Dhekane, S. G., Haresamudram, H., Thukral, M., & Plötz, T. (2023) · Proceedings of the 2023 International Symposium on Wearable Computers · doi:10.1145/3594738.3611366 · Cited by 9
“For sensor-based HAR, it was introduced in Multi-task self-supervised learning [29], which comprises identifying whether data transformations have been applied or not in a multi-task setting.”
Artificial Intelligence-Enhanced Flexible Sensors for Human Motion and Posture Sensing
Jiang, Y., & He, T. (2026) · Sensors · doi:10.3390/s26051562 · Cited by 10
“Furthermore, while self-supervised learning (SSL) has gained prominence for addressing label scarcity, it fundamentally requires a substantial volume of unlabeled data for pre-text tasks to learn robust representations before fine-tuning on small labeled subsets [96,97].”
STAR: A Scalable Self-taught Learning Framework for Older Adults’ Activity Recognition
Ramamurthy, S. R., Ghosh, I., Gangopadhyay, A., Galik, E. M., & Roy, N. (2021) · IEEE International Conference on Smart Computing · doi:10.1109/smartcomp52413.2021.00037 · Cited by 9
“The key difference between [20] and STAR lies in the pre-training phase where we use a generative energy based model while compared to a learning a focused variation or augmentation version of the data.”
BackgroundMethodInfluentialFedMPT: Federated Learning for Multiple Personalized Tasks Over Mobile Computing
Zhang, X., Ou, Z., & Yang, Z. (2023) · IEEE Transactions on Network Science and Engineering · doi:10.1109/tnse.2023.3246463 · Cited by 9
“the mobile computing environment also shows some promising results [33], [34].”
BackgroundSelf-Supervised PPG Representation Learning Shows High Inter-Subject Variability
Ghorbani, R., Reinders, M. J. T., & Tax, D. M. J. (2023) · International Conference on Machine Learning Technologies (ICMLT) · doi:10.1145/3589883.3589902 · Cited by 15
“[19] used the task of identifying which signal transformation”
MethodMovement Representation Learning for Pain Level Classification
Olugbade, T., Williams, A. C. D. C., Gold, N., & Bianchi‐Berthouze, N. (2023) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2023.3334522 · Cited by 9
“…CNN (based on convolution of short-term fast Fourier transforms at different timescales) that led to better performance than the vanilla CNN. Unlike in Saeed et al. [14], individual transformations performed Authorized licensed use limited to the terms of the applicable license agreement with IEEE.”
BackgroundMethodResultInfluentialMobHAR: Source-free Knowledge Transfer for Human Activity Recognition on Mobile Devices
Xue, M., Zhu, Y., Xie, W., Wang, Z., Chen, Y., Jiang, K., & Zhang, Q. (2025) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3712620 · Cited by 14
“Existing HAR models such as [17, 19, 21, 27, 40, 45, 46, 56, 57] commonly assume that data adheres to the concept of independent and identically distributed (IID) [50].”
TFC: Time–frequency contrasting network for wearable-based human activity recognition
Huang, Z., Deng, J., Wang, S., Tang, C., & Xiao, S. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113373 · Cited by 9
Enhancing Sensor-Based Human Activity Recognition With Multiresolution Wavelet-Attention
Wieland, C., & Pankratius, V. (2025) · IEEE Sensors Journal · doi:10.1109/jsen.2025.3571572 · Cited by 14
“Common pretraining objectives include signal reconstruction [36], masked signal modeling [37], signal forecasting [38], transformation detection [39], and contrastive learning [40].”
AutoLife: Automatic Life Journaling with Smartphones and LLMs
Xu, H., Zeng, Z., Tong, P., Li, M., & Srivastava, M. B. (2025) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3770683 · Cited by 26
“There are extensive HAR studies and wearable-based solutions [23, 30, 32, 39, 57, 59, 73, 74, 85] can be implemented on off-the-shelf smart devices and are more ubiquitous compared with vision-based [56, 68, 81] or wireless-based [35, 77, 83] solutions.”
Deep Transfer Learning Using Class Augmentation for Sensor-Based Human Activity Recognition
Kondo, K., & Hasegawa, T. (2022) · IEEE Sensors Letters · doi:10.1109/lsens.2022.3206472 · Cited by 8
Multi-task Learning for Stress Recognition
Pogliaghi, A., Lascio, E. D., Gashi, S., Piciucco, E., Santini, S., & Gjoreski, M. (2022) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/3544793.3563404 · Cited by 9
“MTL for jointly recognizing users and performing other classification tasks has been previously studied in other domains, with activity recognition from wearable accelerometers being the closest one to our work [9, 24].”
BackgroundHuman Activity Recognition via Temporal Fusion Contrastive Learning
Kim, I., Lim, J., & Lee, J. (2024) · IEEE Access · doi:10.1109/access.2024.3357143 · Cited by 10
Time-series representation learning via Time-Frequency Fusion Contrasting
Zhao, W., & Fan, L. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1414352 · Cited by 13
“In response to the distinctive characteristics of time series data, several pretext tasks have emerged, such as masked reconstruction (Haresamudram et al., 2020) and data transformation predictions (Saeed et al., 2019).”
BackgroundTemporal Convolutional Network with Wavelet Transform for Fall Detection
Lu, X., Ling, Y., & Liu, S. (2022) · Journal of Sensors · doi:10.1155/2022/7267099 · Cited by 10
Teikari, P., & Pietrusz, A. (2021) · SportRxiv (OSF Preprints) · doi:10.31236/osf.io/w734a · Cited by 16
Deep Learning: Predicting Environments From Short-Time Observations of Postural Balance
Phan, V., & Lee, H. (2022) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2022.3170850 · Cited by 8
SelfVis: Self-Supervised Learning for Human Activity Recognition Based on Area Charts
Jiang, A., & Ye, J. (2024) · IEEE Transactions on Emerging Topics in Computing · doi:10.1109/tetc.2024.3392850 · Cited by 7
A Review Work: Human Action Recognition in Video Surveillance Using Deep Learning Techniques
Gupta, N. S., Ramya, K., & Karnati, R. (2024) · Informatics and Automation · doi:10.15622/ia.23.2.5 · Cited by 7
Shuqair, M., Shahed, J. J., & Ghoraani, B. (2024) · Bioengineering · doi:10.3390/bioengineering11070689 · Cited by 10
“In our work, to improve the monitoring of PD patients, we propose an innovative M-SSL methodology (see Figure 1B), building upon the previously introduced multi-task SSL [35].”
MethodLearning Disentangled Representation for Multi-Modal Time-Series Sensing Signals
Cai, R., Jiang, Z., Zheng, K., Li, Z., Chen, W., XueXin, C., Shen, Y., Chen, G., Hao, Z., & Zhang, K. (2025) · International World Wide Web Conference (WWW) · doi:10.1145/3696410.3714931 · Cited by 23
Self-Learning Pipeline for Low-Energy Resource-Constrained Devices
Sakr, F., Berta, R., Doyle, J. C., Gloria, A. D., & Bellotti, F. (2021) · Energies · doi:10.3390/en14206636 · Cited by 9
“Human activity detection with self-supervised learning is introduced in [16].”
BackgroundCogAx: Early Assessment of Cognitive and Functional Impairment from Accelerometry
Ramamurthy, S. R., Chatterjee, S., Galik, E. M., Gangopadhyay, A., Roy, N., Mitra, B., & Chakraborty, S. (2022) · IEEE International Conference on Pervasive Computing and Communications · doi:10.1109/percom53586.2022.9762401 · Cited by 11
VALERIAN: Invariant Feature Learning for IMU Sensor-based Human Activity Recognition in the Wild
Hao, Y., Wang, B., & Zheng, R. (2023) · IEEE/ACM International Conference on Internet-of-Things Design and Implementation (IoTDI) · doi:10.1145/3576842.3582390 · Cited by 8
A Survey on Federated Learning PoisoningAttacks and DefensesPreprint
Liang, J., & Rong, W. (2022) · Research Square · doi:10.21203/rs.3.rs-1900743/v1 · Cited by 17
Guo, P., & Nakayama, M. (2025) · Sensors · doi:10.3390/s25226988 · Cited by 6
“Self-supervised pretraining (e.g., Multi-task SSL [33]) learns transformation-aware representations through auxiliary tasks.”
Chen, Y., Hu, C., Kimura, T., Li, Q., Liu, S., Wu, F., & Chen, G. (2024) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3699779 · Cited by 9
Self-Supervised Learning for Activity Recognition Based on Datasets With Imbalanced Classes
Hamad, R. A., Woo, W. L., Wei, B. W., & Yang, L. (2025) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2025.3526504 · Cited by 11
Forensic detection of heterogeneous activity in data using deep learning methods
Nyarko, B. N. E., Wu, B., Zhou, J., Odoom, J., Danso, S. A., & Gyarteng, E. S. A. (2023) · Intelligent Systems with Applications · doi:10.1016/j.iswa.2023.200303 · Cited by 6
Zlatintsi, A., Filntisis, P. P., Efthymiou, N., Garoufis, C., Retsinas, G., Sounapoglou, T., Maglogiannis, I. G., Tsanakas, P. D., Smyrnis, N., & Maragos, P. A. (2024) · IEEE Open Journal of Signal Processing · doi:10.1109/ojsp.2024.3376300 · Cited by 8
“This abundance of sensory data has facilitated the development of numerous applications for general user and health monitoring, as well as predictive analytic tasks such as emotional well-being [2], [3], sleep tracking [4], eating [5], agitation [6], and physical activity detection [7].”
BackgroundPellatt, L., & Roggen, D. (2021) · International Symposium on Wearable Computers · doi:10.1145/3460421.3478813 · Cited by 7
“[45] IMU, Altitude HHAR [47], UniMiB, UCI HAR [2], MobiAct [48], WISDM [50], MSense [30] CNN, Pool, MLP 5 —”
MethodSheng, T., & Huber, M. (2025) · Sensors · doi:10.3390/s25134032 · Cited by 5
Lee, J., Ham, I., Kim, Y., & Ko, H. (2024) · Sensors · doi:10.3390/s24247932 · Cited by 6
Nshimyimana, D., Rey, V. F., Suh, S., Zhou, B., & Lukowicz, P. (2025) · International Conference on Activity and Behavior Computing (ABC) · doi:10.1109/abc64332.2025.11118365 · Cited by 7
“One popular SSL approach in HAR is multi-task learning with data augmentations, where transformations like Gaussian noise addition, scaling, rotation, and time-warping are applied to the data [10], [12].”
InfluentialWatchGuardian: Enabling User-Defined Personalized Just-in-Time Intervention on Smartwatch
Lei, Y., Cao, Y., Wang, W. K., Dong, Y., Yin, C., Cao, W., Zhang, P., Yang, J., Yao, B., Peng, Y., et al. (2026) · ACM Transactions on Computing for Healthcare · doi:10.1145/3788689 · Cited by 10
Orientation Estimation Piloted by Deep Reinforcement Learning
Liu, M., Yang, S., Rathee, A., & Du, W. (2024) · IEEE/ACM International Conference on Internet-of-Things Design and Implementation (IoTDI) · doi:10.1109/iotdi61053.2024.00016 · Cited by 6
“Self-supervised learning has recently been used to extract contextual features from unlabeled IMU data for different application tasks, aiming at building "foundation" models, e.g., TPN [32] and LIMU-BERT [30]; however, they still require labeled data for new scenarios.”
MethodSelf-Supervised Federated Learning for Personalized Human Activity Recognition
Deng, S., Teng, D., Guo, Z., Chen, J., Chen, D., Jia, T., & Wang, H. (2024) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme57554.2024.10687970 · Cited by 6
“For example, eight signal transformations are constructed as multiple proxy tasks on the data [10].”
BackgroundMethodStudy on temperature $$(\tau )$$ variation for SimCLR-based activity recognition
Kumar, P., & Chauhan, S. (2022) · Signal Image and Video Processing · doi:10.1007/s11760-021-02122-x · Cited by 7
Contrastive Sensor Transformer for Predictive Maintenance of Industrial Assets
Bukhsh, Z. A. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9746728 · Cited by 5
Augmentation Robust Self-Supervised Learning for Human Activity Recognition
Xu, C., Li, Y., Lee, D., Park, D. H., Mao, H., Do, H., Chung, J., & Nair, D. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10096151 · Cited by 9
“Based on the sequential characteristics of the wearable sensor data, the pretext task can be defined in many ways, such as data reconstruction [6], data transformation predictions [7], and contrastive comparison [8].”
MethodDon't freeze: Finetune encoders for better Self-Supervised HAR
Rey, V. F., Nshimyimana, D., & Lukowicz, P. (2023) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/3594739.3610790 · Cited by 10
“This varies across pretexts and datasets, with cpc or Multi-task providing the most improvement.”
BackgroundMethodInfluentialSelfAct: Personalized Activity Recognition Based on Self-Supervised and Active Learning
Arrotta, L., Civitarese, G., & Bettini, C. (2024) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-031-63989-0_19 · Cited by 6
Quantum contrastive learning for human activity recognition
Ren, Y., Wang, D., An, L., Mao, S., & Wang, X. (2025) · Smart Health · doi:10.1016/j.smhl.2025.100574 · Cited by 9
Transfer Learning Model Knowledge Across Multi-Sensors Locations Over Body Sensor Network
Qian, X., Chen, H., Cai, Y. M., Chu, K., Xu, W., & Huang, M.-C. (2022) · IEEE Sensors Journal · doi:10.1109/jsen.2022.3166187 · Cited by 5
“Saeed et al. [12] applied multi-task learning to find the representative relations of accelerometer signals.”
BackgroundRamamurthy, S. R., Ghosh, I., Gangopadhyay, A., Galik, E. M., & Roy, N. (2022) · Pervasive and Mobile Computing · doi:10.1016/j.pmcj.2022.101698 · Cited by 9
Evaluating Contrastive Learning for Fine-grained Reading Detection
Islam, M. R., Vargo, A. W., Iwata, M., Iwamura, M., & Kise, K. (2022) · IIAI International Congress on Advanced Applied Informatics (IIAI-AAI) · doi:10.1109/iiaiaai55812.2022.00091 · Cited by 7
“Researchers have employed different techniques, including self-supervised learning [3], [4], to tackle the lack of large-sized labeled data issues.”
MethodLIMU-BERT: Unleashing the Potential of Unlabeled Data for IMU Sensing Applications
Xu, H., Zhou, P., Tan, R., Li, M., & Shen, G. (2022) · GetMobile Mobile Computing and Communications · doi:10.1145/3568113.3568124 · Cited by 7
“, 20 Hz) compared with the existing works [1][2], and accordingly decrease the length of the input IMU sequences and reduce model size.”
BackgroundSelf-supervised Wearable-based Activity Recognition by Learning to Forecast MotionPreprint
Taghanaki, S. R., & Etemad, A. (2020) · arXiv (Cornell University) · Cited by 8
Metadata
- Authors
- Saeed, A., Ozcelebi, T., & Lukkien, J.
- Affiliations
- Eindhoven University of Technology (NL)
- Eindhoven University of Technology, Eindhoven, The Netherlands
- ORCID
- Saeed, Aaqib
- Venue
- Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
- Published
- 2019-06-21
- Volume
- 3
- Issue
- 2
- Pages
- 1-30
- ISSN
- 2474-9567 (ISSN-L)
- OpenAlex
- W3099025572
- Cited by
- 346
- References
- 85
- Citations by year
- 2026: 20 · 2025: 65 · 2024: 82 · 2023: 71 · 2022: 49 · 2021: 42 · 2020: 17
- FWCI
- 11.94
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies (publishedVersion, public-domain)
- TU/e Research Portal (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- TU/e Research Portal (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Funders
- Horizon 2020 Framework Programme (737422)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Signal Processing › Time Series Analysis and Forecasting
- Keywords
- human activity recognition, self-supervised learning, unsupervised feature learning, multi-task learning, temporal convolutional network, smartphone sensing, semi-supervised learning, transfer learning, unlabeled data
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.