The DOI matches its Crossref record; 85 of the 85 references it cites were found in a registry or an index.
Multi-task Self-Supervised Learning for Human Activity Detection
Open accessAbstract
Deep learning methods are successfully used in applications pertaining to ubiquitous computing, pervasive intelligence, health, and well-being. Specifically, the area of human activity recognition (HAR) is primarily transformed by the convolutional and recurrent neural networks, thanks to their ability to learn semantic representations directly from raw input. However, in order to extract generalizable features massive amounts of well-curated data are required, which is a notoriously challenging task; hindered by privacy issues and annotation costs. Therefore, unsupervised representation learning (i.e., learning without manually labeling the instances) is of prime importance to leverage the vast amount of unlabeled data produced by smart devices. In this work, we propose a novel self-supervised technique for feature learning from sensory data that does not require access to any form of semantic labels, i.e., activity classes. We learn a multi-task temporal convolutional network to recognize transformations applied on an input signal. By exploiting these transformations, we demonstrate that simple auxiliary tasks of the binary classification result in a strong supervisory signal for extracting useful features for the down-stream task. We extensively evaluate the proposed approach on several publicly available datasets for smartphone-based HAR in unsupervised, semi-supervised and transfer learning settings. Our method achieves performance levels superior to or comparable with fully-supervised networks trained directly with activity labels, and it performs significantly better than unsupervised learning through autoencoders. Notably, for the semi-supervised case, the self-supervised features substantially boost the detection rate by attaining a kappa score between 0.7 - 0.8 with only 10 labeled examples per class. We get similar impressive performance even if the features are transferred from a different data source. Self-supervision drastically reduces the requirement of labeled activity data, effectively narrowing the gap between supervised and unsupervised techniques for learning meaningful representations. While this paper focuses on HAR as the application domain, the proposed approach is general and could be applied to a wide variety of problems in other areas.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“In contrast, previous studies applied self-supervised training and fine-tuning only on the same data sources24,27,40, making it necessary to pre-train on every new dataset in practical applications.”
From Self-supervised learning for human activity recognition using 700,000 person-days of wearable data (2024) · cites this paper as [27]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“For windows of sensor data, Saeed et al. [166] apply each transformation with a probability of 50%, and passed the probabilistically transformed windows through a common convolutional encoder.”
From Past, Present, and Future of Sensor-based Human Activity Recognition Using Wearables: A Surveying Tutorial on a Still Challenging Task (2025) · cites this paper as Saeed et al. [166]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Haresamudram et al. [81] show that the SimCLR framework [184] and multi-task self-supervised learning [166], outperform other approaches, including BYOL, SimSiam, masked reconstruction, CPC and autoencoders for HAR.”
From Past, Present, and Future of Sensor-based Human Activity Recognition Using Wearables: A Surveying Tutorial on a Still Challenging Task (2025) · cites this paper as [166]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The established multi-task self-sup. Ref. [15] framework is ill-suited for discrete representation learning, with significant reductions in performance throughout, consistently performing worse by approx. 10% for most datasets and peaking at around 29% for Mobiact.”
From Towards Learning Discrete Representations via Self-Supervision for Wearables-Based Human Activity Recognition (2024) · cites this paper as [15]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“We show that our method outperforms existing self-supervised methods in HAR for UCI HAR, MotionSense, and HAPT datasets.”
From Self-supervised Human Activity Recognition by Learning to Predict Cross-Dimensional Motion (2021) · cites this paper as [19]
Experimental — accuracy not yet measured
19 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 1×
“These techniques have achieved indisputable empirical success across a broad spectrum of problems [4, 21, 28, 36, 43, 56, 65, 66] thanks to the increasing dataset sizes and computing power availability.”
BackgroundAdam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 3×
“We use ReLU as non-linearity in all the layers (except the output) and train a network with Adam optimizer [26] for a maximum of 30 epochs with a learning rate of 0.”
MethodInfluentialLeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
(2008) · Journal of Machine Learning Research · Cited by 33,051
Pan, S. J., & Yang, Q. (2010) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2009.191 · Cited by 23,387 · cited 1×
“owledge transfer se−ings (such as features, instances, weights) from existing labeled data of some related task or domain. For a detailed review of transfer learning, we refer an interested reader to [53]. 2.4 Semi-supervised Learning Semi-supervised learning provides a compelling framework for leveraging unlabeled da…”
BackgroundInfluentialSequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766 · cited 1×
“These techniques have achieved indisputable empirical success across a broad spectrum of problems [4, 21, 28, 36, 43, 56, 65, 66] thanks to the increasing dataset sizes and computing power availability.”
Background(2010) · International Conference on Machine Learning · Cited by 13,995
Representation Learning: A Review and New Perspectives
Bengio, Y., Courville, A., & Vincent, P. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.50 · Cited by 13,001 · cited 3×
“is inherent limitation of the fully-supervised learning paradigm emphasizes the importance of unsupervised learning to leverage a large amount of unlabeled data for representation learning [8] that can be easily acquired in a real-world seing.”
BackgroundInfluentialCaruana, R. (1997) · Machine Learning · doi:10.1023/a:1007379606734 · Cited by 6,636 · cited 2×
“e goal of multi-task learning (MTL) is to enhance the learning eciency and accuracy through simultaneously optimizing multiple objectives based on shared representations and exploiting relations among the tasks [10].”
BackgroundDeepFace: Closing the Gap to Human-Level Performance in Face Verification
Taigman, Y., Yang, M., Ranzato, M., & Wolf, L. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.220 · Cited by 6,542 · cited 1×
“ese techniques have achieved indisputable empirical success across a broad spectrum of problems [4, 21, 28, 35, 43, 56, 65, 66] thanks to the increasing dataset sizes and computing power availability.”
BackgroundDeep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
Simonyan, K., Vedaldi, A., & Zisserman, A. (2013) · arXiv · doi:10.48550/arxiv.1312.6034 · Cited by 5,370 · cited 4×
“To further understand the predictions produced by both models, we visualize saliency maps [63] for the highest-scoring class on randomly selected instances from the MotionSense dataset.”
BackgroundMethodInfluentialAn Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling
Bai, S., Kolter, J. Z., & Koltun, V. (2018) · arXiv · doi:10.48550/arxiv.1803.01271 · Cited by 4,655 · cited 1×
“In particular, CNNs are becoming increasingly popular in sequence (or time-series) modeling due to their ability of weight sharing, translation invariance, scale separation and localization of lters in space and time [5, 32].”
BackgroundCNN Features Off-the-Shelf: An Astounding Baseline for Recognition
Razavian, A. S., Azizpour, H., Sullivan, J., & Carlsson, S. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2014.131 · Cited by 4,451 · cited 1×
“In recent years, it has shown remarkable improvement in performance on several very challenging problems, especially in areas, where lile-labeled data are available such as in natural language understanding, object recognition, and activity recognition [23, 44, 62].”
BackgroundUniversal Language Model Fine-tuning for Text Classification
Howard, J., & Ruder, S. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1031 · Cited by 4,100 · cited 1×
“In recent years, it has shown remarkable improvement in performance on several very challenging problems, especially in areas, where lile-labeled data are available such as in natural language understanding, object recognition, and activity recognition [23, 44, 62].”
(1989) · Cited by 3,214
Multi-task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics
Cipolla, R., Gal, Y., & Kendall, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00781 · Cited by 3,151
Hannun, A. Y., Rajpurkar, P., Haghpanahi, M., Tison, G. H., Bourn, C., Turakhia, M. P., & Ng, A. Y. (2019) · Nature Medicine · doi:10.1038/s41591-018-0268-3 · Cited by 3,148 · cited 3×
“5 RELATED WORK Deep learning methods have been successfully used in several applications of ubiquitous computing, pervasive intelligence, health, and well-being [17, 21, 38, 56, 60, 73] and eliminate the need of hand-craed feature engineering.”
BackgroundMethodActivity recognition using cell phone accelerometers
Kwapisz, J. R., Weiss, G. M., & Moore, S. A. (2011) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1964897.1964918 · Cited by 2,912 · cited 1×
“e dataset from the Wireless Sensor and Data Mining (WISDM) project [29] was collected in a controlled study from 29 volunteers, who carried the cell phone in their pockets.”
MethodUnsupervised Learning of Visual Representations by Solving Jigsaw Puzzles
Noroozi, M., & Favaro, P. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46466-4_5 · Cited by 2,775 · cited 2×
“Most prominent among them are: colorization of grayscale images [30, 74], predicting image rotations [18], solving jigsaw puzzles [47], predicting the direction of video playback [71], temporal order verication [42], odd sequence detection [15], audio-visual correspondence [3, 52], and curiosity-driven agents [54].”
BackgroundUnsupervised Visual Representation Learning by Context Prediction
Doersch, C., Gupta, A., & Efros, A. A. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.167 · Cited by 2,629 · cited 1×
“Another emerging line of research for unsupervised feature learning (also studied in this work), which has shown promising results and does not require manual annotations, is self-supervised learning [1, 14, 58].”
BackgroundUnsupervised Representation Learning by Predicting Image Rotations
Gidaris, S., Singh, P., & Komodakis, N. (2018) · arXiv · doi:10.48550/arxiv.1803.07728 · Cited by 2,477 · cited 3×
“Similarly, colorization of gray-scale images [30, 74], rotation classication [18], odd sequence detection [15], frame order prediction [42], learning the arrow of time [71], audio-visual correspondence [3, 52] and synchronization [27, 51] are some of the recently explored directions of self-supervised techniques.”
BackgroundLee, H., Grosse, R., Ranganath, R., & Ng, A. Y. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553453 · Cited by 2,412 · cited 1×
“e classical approaches for unsupervised learning include autoencoders [6], restricted Boltzmann machines [46], and convolutional deep belief networks [33].”
BackgroundDeep learning for sensor-based activity recognition: A survey
Wang, J., Chen, Y., Hao, S., Peng, X., & Hu, L. (2019) · Pattern Recognition Letters · doi:10.1016/j.patrec.2018.02.010 · Cited by 2,014 · cited 3×
“In fact, (1D) temporal CNNs are now widely used in the area of HAR (see [68] for a detailed review), but the prior works are mostly concerned with supervised learning approaches.”
BackgroundMethod(2013) · RECERCAT (Consorci de Serveis Universitaris de Catalunya) · Cited by 1,959
Curiosity-Driven Exploration by Self-Supervised Prediction
Pathak, D., Agrawal, P., Efros, A. A., & Darrell, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.70 · Cited by 1,815 · cited 2×
“Most prominent among them are: colorization of grayscale images [30, 74], predicting image rotations [18], solving jigsaw puzzles [47], predicting the direction of video playback [71], temporal order verication [42], odd sequence detection [15], audio-visual correspondence [3, 52], and curiosity-driven agents [54].”
BackgroundAcoustic Modeling Using Deep Belief Networks
Mohamed, A.-R., Dahl, G. E., & Hinton, G. (2012) · IEEE Transactions on Audio, Speech, and Language Processing · doi:10.1109/tasl.2011.2109382 · Cited by 1,727 · cited 1×
“ese techniques have achieved indisputable empirical success across a broad spectrum of problems [4, 21, 28, 35, 43, 56, 65, 66] thanks to the increasing dataset sizes and computing power availability.”
BackgroundSelf-taught learning: transfer learning from unlabeled data
Raina, R., Battle, A., Lee, H., Packer, B., & Ng, A. Y. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273592 · Cited by 1,510
(2011) · Cited by 1,258
(2015) · Cited by 989
Shuffle and Learn: Unsupervised Learning Using Temporal Order Verification
Misra, I., Zitnick, C. L., & Hebert, M. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46448-0_32 · Cited by 779 · cited 3×
“Self-supervision has been actively studied recently in the vision domain, and several surrogate tasks have been proposed for learning representations from static images, videos, sound, and in robotics [3, 14, 15, 18, 19, 24, 30, 34, 42, 47, 51, 52, 54, 74].”
BackgroundInfluentialRealistic Evaluation of Deep Semi-Supervised Learning Algorithms
Oliver, A., Odena, A., Raffel, C., Cubuk, E. D., & Goodfellow, I. J. (2018) · arXiv · doi:10.48550/arxiv.1804.09170 · Cited by 772
Um, T. T., Pfister, F. M. J., Pichler, D., Endo, S., Lang, M., Hirche, S., Fietzek, U., & Kulić, D. (2017) · Proceedings of the 19th ACM International Conference on Multimodal Interaction · doi:10.1145/3136755.3136817 · Cited by 729 · cited 4×
“In [67] task-specific transformations (such as added noise and rotation) are applied to wearable sensor data to augment and improve the performance of Parkinson’s disease monitoring systems.”
MethodInfluentialStisen, A., Blunck, H., Bhattacharya, S., Prentow, T. S., Kjærgaard, M. B., Dey, A., Sonne, T., & Jensen, M. M. (2015) · Proceedings of the 13th ACM Conference on Embedded Networked Sensor Systems · doi:10.1145/2809695.2809718 · Cited by 714 · cited 1×
“The Heterogeneity Human Activity Recognition (HHAR) dataset [64] contains signals from two sensors (accelerometer and gyroscope) of smartphones and smartwatches for 6 different activities, i.”
BackgroundAudio-Visual Scene Analysis with Self-Supervised Multisensory Features
Owens, A., & Efros, A. A. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01231-1_39 · Cited by 704
Split-Brain Autoencoders: Unsupervised Learning by Cross-Channel Prediction
Zhang, R., Isola, P., & Efros, A. A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.76 · Cited by 616
Deep, Convolutional, and Recurrent Models for Human Activity Recognition Using Wearables
Hammerla, N. Y., Halloran, S., & Ploetz, T. (2016) · arXiv · doi:10.48550/arxiv.1604.08880 · Cited by 606 · cited 3×
“Deep learning has been increasingly used for end-to-end HAR with far superior performance that can be achieved through traditional machine learning methods [20, 44, 56, 61, 72].”
BackgroundUm, T. T. (n.d.) · mediaTUM – the media and publications repository of the Technical University Munich (Technical University Munich) · Cited by 602
Multi-task Self-Supervised Visual Learning
Doersch, C., & Zisserman, A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.226 · Cited by 598 · cited 4×
“e availability of strong supervisory signals from the surrogate tasks enables us to leverage objective functions as utilized in a standard supervised learning seing [13].”
BackgroundMethodInfluentialUniMiB SHAR: A Dataset for Human Activity Recognition Using Acceleration Data from Smartphones
Micucci, D., Mobilio, M., & Napoletano, P. (2017) · Applied Sciences · doi:10.3390/app7101101 · Cited by 596
UniMiB SHAR: A Dataset for Human Activity Recognition Using Acceleration Data from Smartphones
Micucci, D., Mobilio, M., & Napoletano, P. (2017) · doi:10.20944/preprints201706.0033.v2 · Cited by 596
UniMiB SHAR: A Dataset for Human Activity Recognition Using Acceleration Data from Smartphones
Micucci, D., Mobilio, M., & Napoletano, P. (2017) · doi:10.20944/preprints201706.0033.v1 · Cited by 596 · cited 1×
“is dataset [41] contains triaxial accelerometer signals collected from a Samsung Galaxy Nexus smartphone at 50Hz.”
MethodThe Building Blocks of Interpretability
Olah, C., Satyanarayan, A., Johnson, I., Carter, S., Schubert, L., Ye, K., & Mordvintsev, A. (2018) · Distill · doi:10.23915/distill.00010 · Cited by 591 · cited 1×
“The interpretability of the neural networks and deciphering of the learned representations have recently gained significant attention, especially, for images (see [49] for an excellent review).”
BackgroundPreprocessing techniques for context recognition from accelerometer data
Figo, D., Diniz, P. C., Ferreira, D. R., & Cardoso, J. M. P. (2010) · Personal and Ubiquitous Computing · doi:10.1007/s00779-010-0293-9 · Cited by 578 · cited 1×
“Before the era of end-to-end learning, manual feature design strategies [16] such as those that employ statistical measures have been used with clustering algorithms to discover a latent group of activities [70].”
MethodA Joint Many-Task Model: Growing a Neural Network for Multiple NLP Tasks
Hashimoto, K., xiong, C., Tsuruoka, Y., & Socher, R. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1206 · Cited by 535 · cited 1×
“ectives based on shared representations and exploiting relations among the tasks [10]. It is widely utilized in several application domains within machine learning such as natural language processing [22], computer vision [25], audio sensing [17], and well-being [60]. In this learning se−ing,T supervised tasks, each w…”
BackgroundUnsupervised Representation Learning by Sorting Sequences
Lee, H.-Y., Huang, J.-B., Singh, M., & Yang, M.-H. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.79 · Cited by 534 · cited 1×
“elf-supervision has been actively studied recently in the vision domain, and several surrogate tasks have been proposed for learning representations from static images, videos, sound, and in robotics [3, 13, 15, 18, 19, 24, 30, 34, 42, 47, 51, 52, 54, 74]. For example, in images and videos, spatial and temporal contex…”
BackgroundMeasuring the Intrinsic Dimension of Objective LandscapesPreprint
Li, C., Farkhoor, H., Liu, R., & Yosinski, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.08838 · Cited by 462 · cited 1×
“However, there is plenty of evidence that high-performing deep networks do not utilize the entire dimensionality of a layer [31, 36, 45].”
BackgroundColorization as a Proxy Task for Visual Understanding
Larsson, G., Maire, M., & Shakhnarovich, G. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.96 · Cited by 458
Multi-Scale Convolutional Neural Networks for Time Series Classification
Cui, Z., Chen, W., & Chen, Y. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.06995 · Cited by 455 · cited 1×
“[12] used transformed signals as extra channels to the model for learning multiscale features.”
MethodSelf-Supervised Video Representation Learning with Odd-One-Out Networks
Fernando, B., Bilen, H., Gavves, E., & Gould, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.607 · Cited by 421 · cited 2×
“elf-supervision has been actively studied recently in the vision domain, and several surrogate tasks have been proposed for learning representations from static images, videos, sound, and in robotics [3, 13, 15, 18, 19, 24, 30, 34, 42, 47, 51, 52, 54, 74]. For example, in images and videos, spatial and temporal contex…”
BackgroundMethodArandjelović, R., & Zisserman, A. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01246-5_27 · Cited by 403
Agrawal, P., Carreira, J., & Malik, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.13 · Cited by 395 · cited 1×
“Another emerging line of research for unsupervised feature learning (also studied in this work), which has shown promising results and does not require manual annotations, is self-supervised learning [1, 14, 58].”
BackgroundAmbient Sound Provides Supervision for Visual Learning
Owens, A., Wu, J., McDermott, J. H., Freeman, W. T., & Torralba, A. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46448-0_48 · Cited by 373 · cited 3×
“Similarly, colorization of gray-scale images [30, 74], rotation classication [18], odd sequence detection [15], frame order prediction [42], learning the arrow of time [71], audio-visual correspondence [3, 52] and synchronization [27, 51] are some of the recently explored directions of self-supervised techniques.”
BackgroundFeature Learning for Activity Recognition in Ubiquitous Computing
Plötz, T., Hammerla, N., & Olivier, P. (2011) · Newcastle University · doi:10.25405/ncl.34175319 · Cited by 337
Raghu, M., Gilmer, J., Yosinski, J., & Sohl-Dickstein, J. (2017) · arXiv · doi:10.48550/arxiv.1706.05806 · Cited by 335
A Complexity-Invariant Distance Measure for Time Series
Batista, G. E. A. P. A., Wang, X., & Keogh, E. J. (2011) · Proceedings of the 2011 SIAM International Conference on Data Mining · doi:10.1137/1.9781611972818.60 · Cited by 329 · cited 4×
“We show that when training a CNN to predict generally known (time-series) transformations [7, 67] as a surrogate task, the model can learn features that are on a par with a fully-supervised network and far beer than unsupervised pre-training with an autoencoder.”
MethodInfluentialLearning and Using the Arrow of Time
Wei, D., Lim, J., Zisserman, A., & Freeman, W. T. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00840 · Cited by 315 · cited 2×
“supervision to learn features. Similarly, colorization of gray-scale images [30, 74], rotation classi•cation [18], odd sequence detection [15], frame order prediction [42], learning the arrow of time [71], audio-visual correspondence [3, 52] and synchronization [27, 51] are some of the recently explored directions of…”
BackgroundOn the importance of single directions for generalization
Morcos, A. S., Barrett, D. G. T., Rabinowitz, N. C., & Botvinick, M. (2018) · arXiv · doi:10.48550/arxiv.1803.06959 · Cited by 305 · cited 1×
“However, there is plenty of evidence that high-performing deep networks do not utilize the entire dimensionality of a layer [32, 35, 45].”
BackgroundCooperative Learning of Audio and Video Models from Self-Supervised Synchronization
Korbar, B., Tran, D., & Torresani, L. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.00230 · Cited by 299 · cited 1×
“Similarly, colorization of gray-scale images [30, 74], rotation classication [18], odd sequence detection [15], frame order prediction [42], learning the arrow of time [71], audio-visual correspondence [3, 52] and synchronization [27, 51] are some of the recently explored directions of self-supervised techniques.”
BackgroundDeepFood: Deep Learning-Based Food Image Recognition for Computer-Aided Dietary Assessment
Liu, C., Cao, Y., Luo, Y., Chen, G., Vokkarane, V., & Ma, Y. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-39601-9_4 · Cited by 289 · cited 2×
“5 RELATED WORK Deep learning methods have been successfully used in several applications of ubiquitous computing, pervasive intelligence, health, and well-being [17, 21, 38, 56, 60, 73] and eliminate the need of hand-craed feature engineering.”
BackgroundMethodMultimodal Deep Learning for Activity and Context Recognition
Radu, V., Tong, C., Bhattacharya, S., Lane, N. D., Mascolo, C., Marina, M. K., & Kawsar, F. (2018) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3161174 · Cited by 281 · cited 4×
“Deep learning has been increasingly used for end-to-end HAR with far superior performance that can be achieved through traditional machine learning methods [20, 44, 56, 61, 72].”
BackgroundMethodInfluentialSoundNet: Learning Sound Representations from Unlabeled Video
Aytar, Y., Vondrick, C., & Torralba, A. B. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.09001 · Cited by 222 · cited 1×
“ese techniques have achieved indisputable empirical success across a broad spectrum of problems [4, 21, 28, 35, 43, 56, 65, 66] thanks to the increasing dataset sizes and computing power availability.”
BackgroundDeep Transfer Learning for Cross-domain Activity Recognition
Wang, J., Zheng, V. W., Chen, Y., & Huang, M. (2018) · International Conference on Crowd Science and Engineering (ICCSE) · doi:10.1145/3265689.3265705 · Cited by 183 · cited 1×
“In terms of transfer learning, our approach also differs significantly from some earlier attempts [44, 69] that were concerned with features transferability from a fully-supervised model learned from inertial measurement units data, as our approach utilizes widely available smartphones and does not require labeled dat…”
ResultProtecting sensory data against sensitive inferences
Malekzadeh, M., Clegg, R. G., Cavallaro, A., & Haddadi, H. (2018) · Spiral (Imperial College London) · Cited by 166
Protecting Sensory Data against Sensitive Inferences
Malekzadeh, M., Clegg, R. G., Cavallaro, A., & Haddadi, H. (2018) · Proceedings of the 1st Workshop on Privacy by Design in Distributed Systems · doi:10.1145/3195258.3195260 · Cited by 154 · cited 1×
“e MotionSense dataset [40] comprises an accelerometer, gyroscope, and altitude data from 24 participants of varying age, gender, weight, and height.”
BackgroundMorales, F. J. O., & Roggen, D. (2016) · Proceedings of the 2016 ACM International Symposium on Wearable Computers · doi:10.1145/2971763.2971764 · Cited by 136 · cited 4×
“Deep learning has been increasingly used for end-to-end HAR with far superior performance that can be achieved through traditional machine learning methods [20, 44, 56, 61, 72].”
BackgroundInfluentialUsing unlabeled data in a sparse-coding framework for human activity recognition
Bhattacharya, S., Nurmi, P., Hammerla, N., & Plötz, T. (2014) · Pervasive and Mobile Computing · doi:10.1016/j.pmcj.2014.05.006 · Cited by 121
Self-Supervised Feature Learning by Learning to Spot Artifacts
Jenni, S., & Favaro, P. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00289 · Cited by 115
Self-Supervised Learning of Visual Features through Embedding Images into Text Topic Spaces
Gómez, L., Patel, Y., Rusiñol, M., Karatzas, D., & Jawahar, C. V. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.218 · Cited by 109 · cited 1×
“Self-supervision has been actively studied recently in the vision domain, and several surrogate tasks have been proposed for learning representations from static images, videos, sound, and in robotics [3, 13, 15, 18, 19, 24, 30, 34, 42, 47, 51, 52, 54, 74].”
BackgroundMulti-task Prediction of Disease Onsets from Longitudinal Laboratory Tests
Razavian, N., Marcus, J., & Sontag, D. A. (2016) · Cited by 107
Deep Supervision with Shape Concepts for Occlusion-Aware 3D Object Parsing
Li, C., Zia, M. Z., Tran, Q.-H., Yu, X., Hager, G. D., & Chandraker, M. K. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.49 · Cited by 96 · cited 1×
“ese techniques have achieved indisputable empirical success across a broad spectrum of problems [4, 21, 28, 35, 43, 56, 65, 66] thanks to the increasing dataset sizes and computing power availability.”
BackgroundUnsupervised Feature Learning for Human Activity Recognition Using Smartphone Sensors
Li, Y., Shi, D., Ding, B., & Liu, D. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-13817-6_11 · Cited by 93 · cited 1×
“tilizes widely available smartphones and does not require labeled data. Finally, the proposed technique is also di‡erent from previously studied unsupervised pre-training methods such as autoencoders [37], restricted Boltzmann machines [55] and sparse coding [9] as we employ an end-to-end (self) supervised learning pa…”
MethodHuman Daily Activity and Fall Recognition Using a Smartphone’s Acceleration Sensor
Chatzaki, C., Pediaditis, M., Vavoulas, G., & Tsiknakis, M. (2017) · Communications in Computer and Information Science · doi:10.1007/978-3-319-62704-5_7 · Cited by 91 · cited 1×
“The MobiAct4 dataset [11] contains signals from a smartphone’s inertial sensors (accelerometer, gyroscope, and orientation) for 11 different activities of daily living and 4 types of falls.”
Colorization as a Proxy Task for Visual UnderstandingPreprint
Larsson, G., Maire, M., & Shakhnarovich, G. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.04044 · Cited by 81
Georgiev, P., Bhattacharya, S., Lane, N. D., & Mascolo, C. (2017) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3131895 · Cited by 67 · cited 3×
“5 RELATED WORK Deep learning methods have been successfully used in several applications of ubiquitous computing, pervasive intelligence, health, and well-being [17, 21, 38, 56, 60, 73] and eliminate the need of hand-craed feature engineering.”
BackgroundMethodAccurately Estimating Sherd 3D Surface Geometry with Application to Pot Reconstruction
Willis, A., Orriols, X., & Cooper, D. B. (2003) · IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2003.10014 · Cited by 59
SenseGAN: Enabling Deep Learning for Internet of Things with a Semi-Supervised Framework
Yao, S., Zhao, Y., Shao, H., Zhang, C., Zhang, A., Hu, S., Liu, D., Liu, S., Su, L., & Abdelzaher, T. (2018) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3264954 · Cited by 54
Objects that SoundPreprint
Arandjelović, R., & Zisserman, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.06651 · Cited by 50 · cited 3×
“Similarly, colorization of gray-scale images [30, 74], rotation classication [18], odd sequence detection [15], frame order prediction [42], learning the arrow of time [71], audio-visual correspondence [3, 52] and synchronization [27, 51] are some of the recently explored directions of self-supervised techniques.”
BackgroundLearning to Exploit Invariances in Clinical Time-Series Data using Sequence Transformer Networks
Oh, J., Wang, J., & Wiens, J. (2018) · PubMed · doi:10.48550/arxiv.1808.06725 · Cited by 42 · cited 1×
“[48] focused on learning invariances directly from clinical time-series data with specialized neural network architecture.”
BackgroundPersonalized Driver Stress Detection with Multi-task Neural Networks using Physiological SignalsPreprint
Saeed, A., & Trajanovski, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.06116 · Cited by 32 · cited 4×
“Convolutional and recurrent neural networks have shown dominant performance in solving numerous high-level recognition tasks from temporal data such as activity detection and stress recognition [20, 60, 68].”
BackgroundMethodInfluentialSelf-citationSynthesizing and Reconstructing Missing Sensory Modalities in Behavioral Context Recognition
Saeed, A., Özçelebi, T., & Lukkien, J. J. (2018) · Sensors · doi:10.3390/s18092967 · Cited by 25 · cited 2×
“Deep learning has been increasingly used for end-to-end HAR with far superior performance that can be achieved through traditional machine learning methods [20, 44, 56, 61, 72].”
BackgroundMethodSelf-citationSelf-supervised learning of visual features through embedding images into text topic spacesPreprint
Gómez, L., Patel, Y., Rusiñol, M., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.08631 · Cited by 18 · cited 1×
“Self-supervision has been actively studied recently in the vision domain, and several surrogate tasks have been proposed for learning representations from static images, videos, sound, and in robotics [3, 13, 15, 18, 19, 24, 30, 34, 42, 47, 51, 52, 54, 74].”
BackgroundClustering Approach to the Problem of Human Activity Recognition using Motion Data
Wawrzyniak, S., & Niemiro, W. (2015) · Annals of Computer Science and Information Systems · doi:10.15439/2015f424 · Cited by 13 · cited 1×
“the era of end-to-end learning, manual feature design strategies [16] such as those that employ statistical measures have been used with clustering algorithms to discover a latent group of activities [70]. Although deep learning techniques have almost entirely replaced hand-cra›ed feature extraction with directly lear…”
MethodSelf-Supervised Feature Learning by Learning to Spot ArtifactsPreprint
Jenni, S., & Favaro, P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.05024 · Cited by 11 · cited 1×
“Self-supervision has been actively studied recently in the vision domain, and several surrogate tasks have been proposed for learning representations from static images, videos, sound, and in robotics [3, 13, 15, 18, 19, 24, 30, 34, 42, 47, 51, 52, 54, 74].”
BackgroundDeep Supervision with Shape Concepts for Occlusion-Aware 3D Object ParsingPreprint
Li, C., Zia, M. Z., Tran, Q.-H., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.02699 · Cited by 9 · cited 1×
“ese techniques have achieved indisputable empirical success across a broad spectrum of problems [4, 21, 28, 35, 43, 56, 65, 66] thanks to the increasing dataset sizes and computing power availability.”
BackgroundMining Intelligence and Knowledge Exploration
Kadry, S., & Prasath, R. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-44084-7 · Cited by 5
Research works printed but not linked: 7
- Baldi Pierre, 2012, Proceedings of ICML workshop on unsupervised and transfer learning. 37--49
- Chatzaki Charikleia, 2016, International Conference on Information and Communication Technologies for Ageing Well and e-Health. Springer, 100--118
- 10.5555/3104322.3104425 · doi:10.5555/3104322.3104425
- 10.5555/3305890.3305968 · doi:10.5555/3305890.3305968
- Plötz Thomas, 2011, IJCAI Proceedings-International Joint Conference on Artificial Intelligence, 22
- Razavian Narges, 2016, Machine Learning for Healthcare Conference. 73--100
- Wawrzyniak S., 2015 Federated Conference on Computer Science and Information Systems (FedCSIS). 411--416
Cited by
Time-Series Representation Learning via Temporal and Contextual Contrasting
Eldele, E., Ragab, M., Chen, Z., Wu, M., Kwoh, C. K., Li, X., & Guan, C. (2021) · Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/324 · Cited by 696
Self-Supervised ECG Representation Learning for Emotion Recognition
Sarkar, P., & Etemad, A. (2022) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2020.3014842 · Cited by 383
“A similar approach was used in [19] for human activity recognition.”
BackgroundMethodInfluentialDeep Gait Recognition: A Survey
Sepas-Moghaddam, A., & Etemad, A. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3151865 · Cited by 307
“eral loss functions [203], can provide the network with more representative features [204], [205], [206].”
BackgroundSelf-Supervised Contrastive Representation Learning for Semi-Supervised Time-Series Classification
Eldele, E., Ragab, M., Chen, Z., Wu, M., Kwoh, C.-K., Li, X., & Guan, C. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3308189 · Cited by 224
“The first approach is to apply generic regularizations, e.g., ensembling and reconstruction.”
Self-supervised learning for human activity recognition using 700,000 person-days of wearable data
Yuan, H., Chan, S., Creagh, A. P., Tong, C., Acquah, A., Clifton, D. A., & Doherty, A. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01062-3 · Cited by 166
“Recent studies explored the utility of SSL for HAR (Saeed et al., 2019; Tang et al., 2020), but these still relied on small-scale laboratory-style datasets, hence the full potential of SSL-HAR remains unknown.”
Influential
Metadata
- Authors
- Saeed, A., Ozcelebi, T., & Lukkien, J.
- Affiliations
- Eindhoven University of Technology (NL)
- Eindhoven University of Technology, Eindhoven, The Netherlands
- ORCID
- Saeed, Aaqib
- Venue
- Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
- Published
- 2019-06-21
- Volume
- 3
- Issue
- 2
- Pages
- 1-30
- ISSN
- 2474-9567 (ISSN-L)
- OpenAlex
- W3099025572
- Cited by
- 346
- References
- 85
- Citations by year
- 2026: 20 · 2025: 65 · 2024: 82 · 2023: 71 · 2022: 49 · 2021: 42 · 2020: 17
- FWCI
- 11.94
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies (publishedVersion, public-domain)
- TU/e Research Portal (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- TU/e Research Portal (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Funders
- Horizon 2020 Framework Programme (737422)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Signal Processing › Time Series Analysis and Forecasting
- Keywords
- human activity recognition, self-supervised learning, unsupervised feature learning, multi-task learning, temporal convolutional network, smartphone sensing, semi-supervised learning, transfer learning, unlabeled data
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.