The DOI matches its Crossref record; 116 of the 116 references it cites were found in a registry or an index; 20 of the 41 sources its claims cite were checked for support.
Contrastive Predictive Coding for Human Activity Recognition
Open accessAbstract
Feature extraction is crucial for human activity recognition (HAR) using body-worn movement sensors. Recently, learned representations have been used successfully, offering promising alternatives to manually engineered features. Our work focuses on effective use of small amounts of labeled data and the opportunistic exploitation of unlabeled data that are straightforward to collect in mobile and ubiquitous computing scenarios. We hypothesize and demonstrate that explicitly considering the temporality of sensor data at representation level plays an important role for effective HAR in challenging scenarios. We introduce the Contrastive Predictive Coding (CPC) framework to human activity recognition, which captures the temporal structure of sensor data streams. Through a range of experimental evaluations on real-life recognition tasks, we demonstrate its effectiveness for improved HAR. CPC-based pre-training is self-supervised, and the resulting learned representations can be integrated into standard activity chains. It leads to significantly improved recognition performance when only small amounts of labeled training data are available, thereby demonstrating the practical value of our approach. Through a series of experiments, we also develop guidelines to help practitioners adapt and modify the framework towards other mobile and ubiquitous computing scenarios.
Overview
CoChat Summary
TL;DR
The paper introduces Contrastive Predictive Coding (CPC) as self-supervised pre-training for wearable human activity recognition, learning representations by predicting multiple future timesteps in latent space. Across four benchmark datasets, CPC-based frozen features outperform prior unsupervised methods on three datasets and improve recognition when only small amounts of labeled data are available.
Key findings
- CPC unsupervised mean F1-scores were Mobiact 80.97, Motionsense 89.05, UCI-HAR 81.65, and USC-HAD 52.01; it outperformed all state-of-the-art unsupervised baselines on Mobiact, Motionsense, and USC-HAD, and was comparable to masked reconstruction on UCI-HAR (81.89).
- Relative to supervised DeepConvLSTM, CPC improved performance by approximately 4% on Motionsense and over 7% on USC-HAD; on Mobiact and UCI-HAR, CPC scored below DeepConvLSTM in Table 2 (80.97 vs 82.40; 81.65 vs 82.83).
- With limited labeled data, CPC improved over DeepConvLSTM on all datasets: Mobiact was consistently higher by over 5% and over 15% above random-initialized features when more than ten samples per class were available; Motionsense reached a maximum improvement of approximately 10% at 50 samples; UCI-HAR peaked at over 10% at ten samples; USC-HAD improved even with one sample per class.
- For pre-training choices, k=12 future steps was best for Mobiact, Motionsense, and UCI-HAR (about 400 ms), while k=8 was best for USC-HAD (about 270 ms); kernel size 9 was around 13% lower than kernel size 3, recurrent encoders were around 42% lower, and using only the first two encoder layers gave Mobiact 85.22% and UCI-HAR 82.58%.
Methods
- Self-supervised CPC pre-training on unlabeled sliding windows: an encoder \(g_{enc}\) maps sensor windows to latent representations, a GRU autoregressive network \(g_{ar}\) produces context representations, and linear predictors \(W_k\) predict k future timesteps using the InfoNCE/NCE loss. Frozen learned weights are used as a feature extractor with a three-layer MLP classifier (256, 128, softmax; batch normalization, ReLU, dropout p=0.2).
- Four benchmark datasets were used: Mobiact (61 users, 11 activities), Motionsense (24 users, 6 activities), UCI-HAR (30 users, 6 activities), and USC-HAD (14 users, 12 activities). Raw accelerometer and gyroscope data were downsampled to 30 Hz, normalized, and segmented into 1 s windows with 50% overlap; splits were participant-based, with 20% test and 20% of remaining data for validation, except USC-HAD used participants 1–10 for training, 11–12 for validation, and 13–14 for testing.
- Pre-training ran for 150 epochs with Adam, learning rates tuned over {1e-3, 5e-4}, and k over {2, 4, 8, 12, 16}; the classifier was trained for 150 epochs with cross-entropy, learning rates over {5e-4, 1e-4} decayed by 0.8 every 25 epochs. The primary metric was test set mean F1-score.
Limitations
- The paper does not state a dedicated limitations section. Evident from the text: evaluation is limited to four benchmark datasets and primarily locomotion-style activities, and the authors note open challenges for sensor placements different from the source dataset and domain shift when using virtual movement data.
- Evident from the text: CPC pre-training uses the entire unlabeled dataset for each benchmark, and the paper does not report cross-dataset transfer or deployment on new sensor placements; it frames these as future research agenda.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
14 Valid · 7 Not valid · 7 Not checked · 84 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“We introduce the Contrastive Predictive Coding (CPC) framework to human activity recognition, which captures the long-term temporal structure of sensor data streams.”
What the sources say
- Partly supportedcites [58] [58] Aaron van den Oord, Yazhe Li, and Oriol Vinyals. 2018. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748 (2018). sourceExperimental — accuracy not yet measured
Section 2.4 (p. 4) says CPC extracts features that retain information across long time horizons, supporting the temporal-structure part of the claim. The paper tests audio, images, language, and reinforcement learning (Conclusion, p. 9), but does not establish that CPC captures long-term structure in the sensor data streams used for human activity recognition. (LLM, conf=0.84)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“Therefore, it is applicable to a wider range of scenarios and environments.”
What the sources say
- Could not verifycites [69] [69] Aaqib Saeed, Tanir Ozcelebi, and Johan Lukkien. 2019. Multi-task Self-Supervised Learning for Human Activity Detection. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 3, 2 (2019), 1-30. - sourceExperimental — accuracy not yet measured
Section 4.1 (p. 8) says the source evaluated its method across datasets covering different devices and environments. The paper also says its approach could apply to a wide variety of problems (p. 1). Neither passage establishes whether the citing paper's work applies to a wider range of scenarios and environments than this method. (LLM, conf=0.76)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
Every source the paper cites for this claim was checked and supports it.
In the paper
“This results in representations that encode the high-level information between temporally separated parts of the time-series signal [58], thereby leading to improved downstream recognition performance.”
What the sources say
- Supportedcites [58] [58] Aaron van den Oord, Yazhe Li, and Oriol Vinyals. 2018. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748 (2018). source
Section 2.1 (p. 2) says CPC learns representations of information shared between different parts of a signal while discarding local detail; predicting further into the future requires the model to infer more global structure. Section 2.4 (p. 4) describes features spanning long time horizons, supporting the paraphrase about high-level information in temporally separated parts of a time series. (LLM, conf=0.93)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“While [32] utilizes both accelerometer and gyroscope data, it delivers mixed results relative to DeepConvLSTM [59] when fine-tuning the pre-trained weights using limited labeled data.”
What the sources say
- Not matchedcites [32] [32] Harish Haresamudram, Apoorva Beedu, Varun Agrawal, Patrick L Grady, Irfan Essa, Judy Hoffman, and Thomas Plötz. 2020. Masked reconstruction based self-supervision for human activity recognition. In Proceedings of the 2020 International Symposium on Wearable Computers . 45-49. -
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Supportedcites [59] [59] Francisco Javier Ordóñez and Daniel Roggen. 2016. Deep convolutional and lstm recurrent neural networks for multimodal wearable activity recognition. Sensors 16, 1 (2016), 115. - source
The citation to [59] identifies DeepConvLSTM as the comparison model; it does not attribute the Transformer fine-tuning results to this source. Section 3 (Architecture) introduces DeepConvLSTM, confirming that identification. (LLM, conf=0.92)
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“While they result in effective representations, most of these approaches do not specifically target a characteristic inherent to body-worn sensor data - temporality.”
What the sources say
- Partly supportedcites [31] [31] Harish Haresamudram, David V Anderson, and Thomas Plötz. 2019. On the role of features in human activity recognition. In Proceedings of the 23rd International Symposium on Wearable Computers . 78-88. - sourceExperimental — accuracy not yet measured
Section 2 (pp. 1–2) describes body-worn sensor data as time series and says movement-data frames are usually treated as statistically independent. It also discusses pre-trained networks, but does not establish that most unsupervised pre-training approaches fail to target temporality; its account of their results is mixed rather than uniformly effective. (LLM, conf=0.78)
- Not checked — no full text of the sourcecites [74] [74] Alireza Abedin Varamin, Ehsan Abbasnejad, Qinfeng Shi, Damith C Ranasinghe, and Hamid Rezatofighi. 2018. Deep auto-set: A deep auto-encoder-set network for activity recognition using wearables. In Proceedings of the 15th EAI International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services . 246-253. - source
- Not matchedcites [32] [32] Harish Haresamudram, Apoorva Beedu, Varun Agrawal, Patrick L Grady, Irfan Essa, Judy Hoffman, and Thomas Plötz. 2020. Masked reconstruction based self-supervision for human activity recognition. In Proceedings of the 2020 International Symposium on Wearable Computers . 45-49. -
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Partly supportedcites [69] [69] Aaqib Saeed, Tanir Ozcelebi, and Johan Lukkien. 2019. Multi-task Self-Supervised Learning for Human Activity Detection. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 3, 2 (2019), 1-30. - sourceExperimental — accuracy not yet measured
The Related Work section (pp. 22–23) treats sensor activity data as temporal and distinguishes the authors’ transformation-based method from earlier unsupervised pre-training methods. It does not say that most such approaches fail to target temporality; that comparative claim is unsupported by our copy. (LLM, conf=0.78)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
77 more claims · 2 with their cited sources checked
Create a free account to see all 112 claims and to verify the 4 references still open.
- Probably not a substantial claim. 21
- It may state two separate claims as one. 15
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“Leave-one-group-out cross-validation approach is used for performance evaluation follows the previous works [26,34] .”
From Spatial-Temporal Masked Autoencoder for Multi-Device Wearable Human Activity Recognition (2023) · cites this paper as [26]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Both methods were designed to leverage unlabeled data by pre-training on unannotated signals and subsequently fine-tuning on labeled data.”
From Towards User-Generalizable Wearable-Sensor-Based Human Activity Recognition: A Multi-Task Contrastive Learning Approach (2025) · cites this paper as [30]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, the contrastive loss function of the above work is InfoNCE [39] or NT-Xent [40]; both are essentially the instance discrimination task that will regard the same-class samples as negative examples.”
From Negative Selection by Clustering for Contrastive Learning in Human Activity Recognition (2023) · cites this paper as [39]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In contrast, previous studies applied self-supervised training and fine-tuning only on the same data sources24,27,40, making it necessary to pre-train on every new dataset in practical applications.”
From Self-supervised learning for human activity recognition using 700,000 person-days of wearable data (2024) · cites this paper as [40]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Recently a number of contributions have been made that introduced self-supervised learning into the field of HAR, including, Multi-task self-supervision, Masked Reconstruction, CPC, and SimCLR, to name but a few.”
From Assessing the State of Self-Supervised Human Activity Recognition Using Wearables (2022) · cites this paper as [27]
Experimental — accuracy not yet measured
12 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 6.1 Analyzing Contrastive Predictive Coding · p. 16
“As mentioned in [58], either 𝑐 𝑡 (the output of 𝑔 𝑒𝑛𝑐 ) or 𝑧 𝑡 (the output of 𝑔 𝑎𝑟 ) can be utilized as the representation.”
§ 3.1 Contrastive Predictive Coding · p. 6
“a non-linear encoder 𝑔 𝑒𝑛𝑐 is utilized to map a window of sensor data 𝑥 𝑡 to a sequence of latent representations given by 𝑧 𝑡 = 𝑔 𝑒𝑛𝑐 ( 𝑥 𝑡 ) . Subsequently, an autoregressive model 𝑔 𝑎𝑟 is used to summarize all 𝑧 ≤ 𝑡 into a latent space, providing a context latent representation 𝑐 𝑡 = 𝑔 𝑎𝑟 ( 𝑧 ≤ 𝑡 ) .”
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 1×
“The availability of large-scale annotated datasets has resulted in astonishing improvements in performance due to the application of deep learning to computer vision [34, 42], speech recognition [3, 26] and natural language tasks [18, 52].”
BackgroundVaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 1×
“The availability of large-scale annotated datasets has resulted in astonishing improvements in performance due to the application of deep learning to computer vision [34, 42], speech recognition [3, 26] and natural language tasks [18, 52].”
BackgroundHochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 1×
“[59] where 2D convolutional features are fed to a long short-term memory [36] network to perform activity recognition.”
BackgroundAdam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071 · cited 1×
“In the case of natural language processing, context-based self-supervised learning has been applied to obtain word and sentence level embeddings for approaches such as Word2Vec [52], GloVe [62] and universal sentence encoder [9].”
Method(2014) · Cited by 33,419
PyTorch: An Imperative Style, High-Performance Deep Learning Library
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al. (2019) · arXiv · doi:10.48550/arxiv.1912.01703 · Cited by 31,473 · cited 1×
“We implemented our framework using PyTorch [60].”
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533 · cited 2×
“The availability of large-scale annotated datasets has resulted in astonishing improvements in performance due to the application of deep learning to computer vision [34, 42], speech recognition [3, 26] and natural language tasks [18, 52].”
BackgroundMethodBatch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S. (2024) · arXiv (Cornell University) · doi:10.57702/o9raffed · Cited by 14,214
(2010) · International Conference on Machine Learning · Cited by 13,995
Momentum Contrast for Unsupervised Visual Representation Learning
He, K., Fan, H., Wu, Y., Xie, S., & Girshick, R. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00975 · Cited by 13,347 · cited 1×
“This framework has resulted in excellent representation learning performance across domains such as computer vision [12, 33, 35], natural language [49], speech recognition [6, 58] and time-series data [39].”
BackgroundEmpirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1412.3555 · Cited by 11,008 · cited 1×
“For the autoregressive network gar , we utilize a two-layer gated recurrent units (GRU) network [14] containing 256 units and dropout with p=0.”
MethodA simple framework for contrastive learning of visual representations
Chen, T., Kornblith, S., Norouzi, M., & Hinton, G. (2020) · arXiv · doi:10.48550/arxiv.2002.05709 · Cited by 10,857 · cited 1×
“This framework has resulted in excellent representation learning performance across domains such as computer vision [12, 33, 35], natural language [49], speech recognition [6, 58] and time-series data [39].”
BackgroundSpeech recognition with deep recurrent neural networks
Graves, A., Mohamed, A.-R., & Hinton, G. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6638947 · Cited by 8,893 · cited 1×
“The availability of large-scale annotated datasets has resulted in astonishing improvements in performance due to the application of deep learning to computer vision [34, 42], speech recognition [3, 26] and natural language tasks [18, 52].”
BackgroundRepresentation Learning with Contrastive Predictive Coding
Oord, A. V. D., Li, Y., & Vinyals, O. (2018) · arXiv · doi:10.48550/arxiv.1807.03748 · Cited by 6,242 · cited 8×
“We adopt the Contrastive Predictive Coding (CPC) [58] framework for unsupervised representation learning of body-worn sensor data.”
BackgroundMethodInfluentialContext Encoders: Feature Learning by Inpainting
Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T., & Efros, A. A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.278 · Cited by 5,691 · cited 1×
“Other approaches such as in-painting [61] mask portions of the image, and predicting relative positions of patches [19] have also been shown to perform well for representation learning.”
BackgroundRao, R. P. N., & Ballard, D. H. (1999) · Nature Neuroscience · doi:10.1038/4580 · Cited by 5,483
Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation
Powers, D. M. W. (2020) · arXiv · doi:10.48550/arxiv.2010.16061 · Cited by 5,440 · cited 1×
“The datasets used in this study show substantial class imbalance and thus experiments require evaluation metrics that are less affected negatively by such biased class distributions [64].”
MethodZhang, R., Isola, P., & Efros, A. A. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46487-9_40 · Cited by 3,178 · cited 1×
“For example, colorization [86] involves a model that is trained to colorize grayscale images.”
BackgroundThe Theory of Event Coding (TEC): A framework for perception and action planning
Hommel, B., Müsseler, J., Aschersleben, G., & Prinz, W. (2001) · Behavioral and Brain Sciences · doi:10.1017/s0140525x01000103 · Cited by 3,091
Noise-contrastive estimation: A new estimation principle for unnormalized statistical models
Gutmann, M. U., & Hyvärinen, A. (2010) · Edinburgh Research Explorer (University of Edinburgh) · Cited by 2,992
Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition
Ordóñez, F., & Roggen, D. (2016) · Sensors · doi:10.3390/s16010115 · Cited by 2,761 · cited 6×
“Approach Method type Mobiact Motionsense UCI-HAR USC-HAD DeepConvLSTM† [59] Supervised 82.”
BackgroundMethodResultInfluentialUnsupervised Visual Representation Learning by Context Prediction
Doersch, C., Gupta, A., & Efros, A. A. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.167 · Cited by 2,629 · cited 1×
“Other approaches such as in-painting [61] mask portions of the image, and predicting relative positions of patches [19] have also been shown to perform well for representation learning.”
BackgroundUnsupervised Representation Learning by Predicting Image Rotations
Gidaris, S., Singh, P., & Komodakis, N. (2018) · arXiv · doi:10.48550/arxiv.1803.07728 · Cited by 2,477
wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations
Baevski, A., Zhou, H., Mohamed, A., & Auli, M. (2020) · arXiv · doi:10.48550/arxiv.2006.11477 · Cited by 2,369
Deep Speech 2: End-to-End Speech Recognition in English and Mandarin
Amodei, D., Anubhai, R., Battenberg, E., Case, C. T., Casper, J., Catanzaro, B., Chen, J., Chrzanowski, M., Coates, A., Diamos, G., et al. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1512.02595 · Cited by 2,110 · cited 1×
“The availability of large-scale annotated datasets has resulted in astonishing improvements in performance due to the application of deep learning to computer vision [34, 42], speech recognition [3, 26] and natural language tasks [18, 52].”
Background(2013) · RECERCAT (Consorci de Serveis Universitaris de Catalunya) · Cited by 1,959
A tutorial on human activity recognition using body-worn inertial sensors
Bulling, A., Blanke, U., & Schiele, B. (2014) · ACM Computing Surveys · doi:10.1145/2499621 · Cited by 1,668 · cited 5×
“This pre-training step is integrated into an overarching human activity recognition framework, that is based on the standard Activity Recognition Chain (ARC) [8].”
BackgroundMethodInfluentialIntroducing a New Benchmarked Dataset for Activity Monitoring
Reiss, A., & Stricker, D. (2012) · 2012 16th International Symposium on Wearable Computers · doi:10.1109/iswc.2012.13 · Cited by 1,456 · cited 1×
“Previous works such as [31, 63, 69] have demonstrated how unlabeled data can be utilized to learn useful representations for wide ranging tasks, including identifying kitchen activities [11], activity tracking in car manufacturing [71], classifying every day activities such as walking or running [4, 10, 51, 66], and m…”
BackgroundSlow Feature Analysis: Unsupervised Learning of Invariances
Wiskott, L., & Sejnowski, T. J. (2002) · Neural Computation · doi:10.1162/089976602317318938 · Cited by 1,350
Freezing of gait: moving forward on a mysterious clinical phenomenon
Nutt, J. G., Bloem, B. R., Giladi, N., Hallett, M., Horak, F. B., & Nieuwboer, A. (2011) · The Lancet Neurology · doi:10.1016/s1474-4422(11)70143-0 · Cited by 1,304 · cited 1×
“Previous works such as [31, 63, 69] have demonstrated how unlabeled data can be utilized to learn useful representations for wide ranging tasks, including identifying kitchen activities [11], activity tracking in car manufacturing [71], classifying every day activities such as walking or running [4, 10, 51, 66], and m…”
BackgroundCer, D., Yang, Y., Kong, S.-Y., Hua, N., Limtiaco, N., John, R. S., Constant, N., Guajardo-Cespedes, M., Yuan, S., Tar, C., et al. (2018) · arXiv · doi:10.48550/arxiv.1803.11175 · Cited by 1,267 · cited 1×
“In the case of natural language processing, context-based self-supervised learning has been applied to obtain word and sentence level embeddings for approaches such as Word2Vec [52], GloVe [62] and universal sentence encoder [9].”
MethodDoherty, A., Jackson, D., Hammerla, N., Plötz, T., Olivier, P., Granat, M. H., White, T., van Hees, V. T., Trenell, M. I., Owen, C. G., et al. (2017) · PLOS ONE · doi:10.1371/journal.pone.0169649 · Cited by 1,250 · cited 2×
“Instead, we can leverage massive movement datasets such as the UK Biobank [20] to first learn generic representations”
BackgroundMethodSelf-citationWang, R., Chen, F., Chen, Z., Li, T., Harari, G., Tignor, S., Zhou, X., Ben-Zeev, D., & Campbell, A. T. (2014) · Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing · doi:10.1145/2632048.2632054 · Cited by 1,116 · cited 1×
“1 INTRODUCTION Body-worn movement sensors, such as accelerometers or full-fledged inertial measurement units (IMU), have been extensively utilized for a wide range of applications in mobile and ubiquitous computing, including but not limited to novel interaction paradigms [67, 82, 84], gesture recognition [83], eating…”
BackgroundData-Efficient Image Recognition with Contrastive Predictive Coding
Hénaff, O. J., Srinivas, A., De Fauw, J., Razavi, A., Doersch, C., Eslami, S. M. A., & Oord, A. V. D. (2019) · arXiv · doi:10.48550/arxiv.1905.09272 · Cited by 1,014 · cited 1×
“This framework has resulted in excellent representation learning performance across domains such as computer vision [12, 33, 35], natural language [49], speech recognition [6, 58] and time-series data [39].”
Background(2015) · Cited by 989
Neural mechanisms for the recognition of biological movements
Giese, M. A., & Poggio, T. (2003) · Nature Reviews Neuroscience · doi:10.1038/nrn1057 · Cited by 955 · cited 1×
“[26] discuss a feedforward mechanism for visual recognition of complex movements.”
BackgroundConvolutional Neural Networks for Human Activity Recognition using Mobile Sensors
Zeng, M., Nguyen, L. T., Yu, B., Mengshoel, O. J., Zhu, J., Wu, P., & Zhang, J. (2014) · Proceedings of the 6th International Conference on Mobile Computing, Applications and Services · doi:10.4108/icst.mobicase.2014.257786 · Cited by 876 · cited 1×
“Convolutional networks have been utilized to classify multi-variate sensor data [29, 79, 81].”
MethodShuffle and Learn: Unsupervised Learning Using Temporal Order Verification
Misra, I., Zitnick, C. L., & Hebert, M. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46448-0_32 · Cited by 779 · cited 1×
“Identifying the odd or unrelated video sub-sequence [22], sequence verification of temporal order [53], and learning the arrow of time [78] have been used as pretext tasks.”
MethodThe Opportunity challenge: A benchmark database for on-body sensor-based activity recognition
Chavarriaga, R., Sagha, H., Calatroni, A., Digumarti, S. T., Tröster, G., Millán, J. D. R., & Roggen, D. (2013) · Pattern Recognition Letters · doi:10.1016/j.patrec.2012.12.014 · Cited by 742 · cited 1×
“Previous works such as [31, 63, 69] have demonstrated how unlabeled data can be utilized to learn useful representations for wide ranging tasks, including identifying kitchen activities [11], activity tracking in car manufacturing [71], classifying every day activities such as walking or running [4, 10, 51, 66], and m…”
BackgroundConscious intention and motor cognition
Haggard, P. (2005) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2005.04.012 · Cited by 698
Deep, Convolutional, and Recurrent Models for Human Activity Recognition Using Wearables
Hammerla, N. Y., Halloran, S., & Ploetz, T. (2016) · arXiv · doi:10.48550/arxiv.1604.08880 · Cited by 606 · cited 4×
“Leveraging the temporal correlations in wearable sensor data at modeling level, (variants of) recurrent neural networks (RNNs) have been applied previously [27, 29, 80].”
BackgroundMethodInfluentialPreprocessing techniques for context recognition from accelerometer data
Figo, D., Diniz, P. C., Ferreira, D. R., & Cardoso, J. M. P. (2010) · Personal and Ubiquitous Computing · doi:10.1007/s00779-010-0293-9 · Cited by 578 · cited 2×
“Earlier works utilized hand crafted features for activity recognition [23].”
BackgroundMethodUSC-HAD: a daily activity dataset for ubiquitous activity recognition using wearable sensors
Zhang, M., & Sawchuk, A. A. (2012) · Proceedings of the 2012 ACM Conference on Ubiquitous Computing · doi:10.1145/2370216.2370438 · Cited by 554 · cited 1×
“The USC-HAD dataset [85] was collected on the MotionNode sensing platform and consists of data from 14 subjects.”
MethodDebiased Contrastive LearningPreprint
Chuang, C.-Y., Robinson, J., Lin, Y.-C., Torralba, A. B., & Jegelka, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.00224 · Cited by 534
Ensembles of Deep LSTM Learners for Activity Recognition using Wearables
Guan, Y., & Plötz, T. (2017) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3090076 · Cited by 511 · cited 4×
“Leveraging the temporal correlations in wearable sensor data at modeling level, (variants of) recurrent neural networks (RNNs) have been applied previously [27, 29, 80].”
BackgroundMethodInfluentialSelf-citationAttend and Diagnose: Clinical Time Series Analysis Using Attention Models
Song, H., Rajan, D., Thiagarajan, J., & Spanias, A. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11635 · Cited by 468 · cited 1×
“General time-series forecasting also involves prediction of future samples, and is used, for example to predict product supply and demand, influenza prevalence [80], and patient mortality from clinical time-series data [72].”
BackgroundElias, P. (1955) · IEEE Transactions on Information Theory · doi:10.1109/tit.1955.1055126 · Cited by 428 · cited 1×
“Predicting one future timestep is an established approach in signal processing [5, 21] and exploits the local smoothness of signals.”
BackgroundAn efficient framework for learning sentence representations
Logeswaran, L., & Lee, H. (2018) · arXiv · doi:10.48550/arxiv.1803.02893 · Cited by 427 · cited 1×
“This framework has resulted in excellent representation learning performance across domains such as computer vision [13, 35, 37], natural language processing [49], speech recognition [7, 58], and general time-series data analysis [40].”
BackgroundSelf-Supervised Video Representation Learning with Odd-One-Out Networks
Fernando, B., Bilen, H., Gavves, E., & Gould, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.607 · Cited by 421
wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations
Baevski, A., Zhou, Y., Mohamed, A., & Auli, M. (2020) · neural information processing systems · Cited by 386
Wearable Activity Tracking in Car Manufacturing
Stiefmeier, T., Roggen, D., Ogris, G., Lukowicz, P., & Tr, G. (2008) · IEEE Pervasive Computing · doi:10.1109/mprv.2008.40 · Cited by 367 · cited 1×
“Previous works such as [31, 63, 69] have demonstrated how unlabeled data can be utilized to learn useful representations for wide ranging tasks, including identifying kitchen activities [11], activity tracking in car manufacturing [71], classifying every day activities such as walking or running [4, 10, 51, 66], and m…”
BackgroundAn Unsupervised Autoregressive Model for Speech Representation Learning
Chung, Y.-A., Hsu, W.-N., Tang, H., & Glass, J. (2019) · Interspeech 2019 · doi:10.21437/interspeech.2019-1473 · Cited by 357
Multi-task Self-Supervised Learning for Human Activity Detection
Saeed, A., Ozcelebi, T., & Lukkien, J. (2019) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3328932 · Cited by 346 · cited 7×
“Previous works such as [31, 63, 69] have demonstrated how unlabeled data can be utilized to learn useful representations for wide ranging tasks, including identifying kitchen activities [11], activity tracking in car manufacturing [71], classifying every day activities such as walking or running [4, 10, 51, 66], and m…”
BackgroundMethodInfluentialHuang, Y., Kaufmann, M., Aksan, E., Black, M. J., Hilliges, O., & Pons-Moll, G. (2018) · ACM Transactions on Graphics · doi:10.1145/3272127.3275108 · Cited by 342 · cited 1×
“Recent approaches addressing the lack of annotated datasets involve extraction of virtual movement data from modalities such as mocap [37, 72] and videos [44], which contain a large number of participants and diverse set of actions and environments.”
BackgroundAnalyzing features for activity recognition
Huynh, T., & Schiele, B. (2005) · Proceedings of the 2005 joint conference on Smart objects and ambient intelligence: innovative context-aware services: usages and technologies · doi:10.1145/1107548.1107591 · Cited by 339 · cited 1×
“However, they can broadly be broken down into three categories: (i) Statistical features, e.g., mean and standard deviation of raw data [38]; (ii) Distribution-based representations, including those based on empirical cumulative distribution functions of the raw data [30]; and (iii) Learned…”
BackgroundMockingjay: Unsupervised Speech Representation Learning with Deep Bidirectional Transformer Encoders
Liu, A. T., Yang, S.-W., Chi, P.-H., Hsu, P.-C., & Lee, H.-Y. (2020) · ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp40776.2020.9054458 · Cited by 337
Feature Learning for Activity Recognition in Ubiquitous Computing
Plötz, T., Hammerla, N., & Olivier, P. (2011) · Newcastle University · doi:10.25405/ncl.34175319 · Cited by 337
Self-citationLearning and Using the Arrow of Time
Wei, D., Lim, J., Zisserman, A., & Freeman, W. T. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00840 · Cited by 315 · cited 1×
“Identifying the odd or unrelated video sub-sequence [22], sequence verification of temporal order [53], and learning the arrow of time [78] have been used as pretext tasks.”
MethodTERA: Self-Supervised Learning of Transformer Encoder Representation for Speech
Liu, A. T., Li, S.-W., & Lee, H.-Y. (2021) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2021.3095662 · Cited by 312 · cited 1×
“[47] utilize reconstruction of time, channel and magnitude altered inputs for self-supervision (see also [48, 88].”
Predictive coding of speech signals and subjective error criteria
Atal, B., & Schroeder, M. (1979) · IEEE Transactions on Acoustics, Speech, and Signal Processing · doi:10.1109/tassp.1979.1163237 · Cited by 305
A practical approach for recognizing eating moments with wrist-mounted inertial sensing
Thomaz, E., Essa, I., & Abowd, G. D. (2015) · Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing · doi:10.1145/2750858.2807545 · Cited by 299 · cited 1×
“1 INTRODUCTION Body-worn movement sensors, such as accelerometers or full-fledged inertial measurement units (IMU), have been extensively utilized for a wide range of applications in mobile and ubiquitous computing, including but not limited to novel interaction paradigms [67, 82, 84], gesture recognition [83], eating…”
BackgroundSelf-citationThe mirror-neuron system: a Bayesian perspective
Kilner, J. M., Friston, K. J., & Frith, C. D. (2007) · NeuroReport · doi:10.1097/wnr.0b013e3281139ed0 · Cited by 286 · cited 1×
“[42] utilize predictive coding on the basis of empirical Bayesian inference in order to understand intentions of people by observing their movements.”
MethodAdaptive Predictive Coding of Speech Signals
Atal, B. S., & Schroeder, M. R. (1970) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1970.tb04297.x · Cited by 256 · cited 1×
“Predicting one future timestep is an established approach in signal processing [5, 21] and exploits the local smoothness of signals.”
BackgroundUnsupervised Visual Representation Learning by Context Prediction
Doersch, C., Gupta, A., & Efros, A. A. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.05192 · Cited by 242
Unsupervised Pretraining Transfers Well Across Languages
Riviere, M., Joulin, A., Mazare, P.-E., & Dupoux, E. (2020) · ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp40776.2020.9054548 · Cited by 186 · cited 1×
“As explored in [68], alternates to this setup include 1-layer Transformer encoders and recurrent networks.”
BackgroundZeng, M., Gao, H., Yu, T., Mengshoel, O. J., Langseth, H., Lane, I., & Liu, X. (2018) · Proceedings of the 2018 ACM International Symposium on Wearable Computers · doi:10.1145/3267242.3267286 · Cited by 185 · cited 2×
“Leveraging the temporal correlations in wearable sensor data at modeling level, (variants of) recurrent neural networks (RNNs) have been applied previously [27, 29, 80].”
MethodEarBit: Using Wearable Sensors to Detect Eating Episodes in Unconstrained Environments
Bedri, A., Li, R., Haynes, M., Kosaraju, R. P., Grover, I., Prioleau, T., Beh, M. Y., Goel, M., Starner, T., & Abowd, G. (2017) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3130902 · Cited by 185 · cited 1×
“1 INTRODUCTION Body-worn movement sensors, such as accelerometers or full-fledged inertial measurement units (IMU), have been extensively utilized for a wide range of applications in mobile and ubiquitous computing, including but not limited to novel interaction paradigms [67, 82, 84], gesture recognition [83], eating…”
BackgroundDetection of eating and drinking arm gestures using inertial body-worn sensors
Amft, O., Junker, H., & Troster, G. (2005) · Ninth IEEE International Symposium on Wearable Computers (ISWC'05) · doi:10.1109/iswc.2005.17 · Cited by 180 · cited 1×
“1 INTRODUCTION Body-worn movement sensors, such as accelerometers or full-fledged inertial measurement units (IMU), have been extensively utilized for a wide range of applications in mobile and ubiquitous computing, including but not limited to novel interaction paradigms [67, 82, 84], gesture recognition [83], eating…”
BackgroundUnsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICA
Hyvärinen, A., & Morioka, H. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.06336 · Cited by 168 · cited 1×
“This framework has resulted in excellent representation learning performance across domains such as computer vision [12, 33, 35], natural language [49], speech recognition [6, 58] and time-series data [39].”
BackgroundProtecting sensory data against sensitive inferences
Malekzadeh, M., Clegg, R. G., Cavallaro, A., & Haddadi, H. (2018) · Spiral (Imperial College London) · Cited by 166
On attention models for human activity recognition
Murahari, V. S., & Plötz, T. (2018) · Proceedings of the 2018 ACM International Symposium on Wearable Computers · doi:10.1145/3267242.3267287 · Cited by 157 · cited 1×
“Subsequent work on this architecture to improve performance includes temporal attention [55].”
BackgroundSelf-citationGenerative Pre-Training for Speech with Autoregressive Predictive Coding
Chung, Y.-A., & Glass, J. (2020) · ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp40776.2020.9054438 · Cited by 157 · cited 1×
“) Autoregressive predictive coding has also been explored for speech self-supervision in [15, 17] and [16].”
BackgroundSpeech Recognition with Deep Recurrent Neural NetworksPreprint
Graves, A., Mohamed, A., & Hinton, G. E. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1303.5778 · Cited by 156
Protecting Sensory Data against Sensitive Inferences
Malekzadeh, M., Clegg, R. G., Cavallaro, A., & Haddadi, H. (2018) · Proceedings of the 1st Workshop on Privacy by Design in Distributed Systems · doi:10.1145/3195258.3195260 · Cited by 154 · cited 2×
“The Motionsense dataset [51] consists of 24 participants of different ages, gender, weight and height.”
BackgroundKwon, H., Tong, C., Haresamudram, H., Gao, Y., Abowd, G. D., Lane, N. D., & Plötz, T. (2020) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3411841 · Cited by 152
Self-citationMasked reconstruction based self-supervision for human activity recognition
Haresamudram, H., Beedu, A., Agrawal, V., Grady, P. L., Essa, I., Hoffman, J., & Plötz, T. (2020) · Proceedings of the 2020 International Symposium on Wearable Computers · doi:10.1145/3410531.3414306 · Cited by 146 · cited 10×
“While [32] utilizes both accelerometer and gyroscope data, it delivers mixed results relative to DeepConvLSTM [59] when fine-tuning the pre-trained weights using limited labeled data.”
BackgroundMethodResultInfluentialSelf-citationHammerla, N. Y., Kirkham, R., Andras, P., & Ploetz, T. (2013) · Proceedings of the 2013 International Symposium on Wearable Computers · doi:10.1145/2493988.2494353 · Cited by 138 · cited 2×
“, mean and standard deviation of raw data [38]; (ii) Distribution-based representations, including those based on empirical cumulative distribution functions of the raw data [30]; and (iii) Learned features, that directly utilize the data itself to derive representations, involving supervised or unsupervised learning…”
BackgroundMethodSelf-citationHuman Activity Recognition from Wearable Sensor Data Using Self-AttentionPreprint
Mahmud, S., Tonmoy, M. T. H., Bhaumik, K. K., Rahman, A. K. M. M., Amin, M. A., Shoyaib, M., Khan, M. A. H., & Ali, A. (2020) · European Conference on Artificial Intelligence · doi:10.48550/arxiv.2003.09018 · Cited by 132 · cited 1×
“More recently, Transformer [75] encoder networks have been utilized for activity recognition [32, 50].”
MethodPrediction of Mood Instability with Passive Sensing
Morshed, M. B., Saha, K., Li, R. C., D’Mello, S. K., Choudhury, M. D., Abowd, G. D., & Plötz, T. (2019) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3351233 · Cited by 116
Self-citationNeckSense: A Multi-Sensor Necklace for Detecting Eating Activities in Free-Living Conditions
Zhang, S., Zhao, Y., Nguyen, D. T., Xu, R., Sen, S., Hester, J. D., & Alshurafa, N. I. (2020) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3397313 · Cited by 116 · cited 1×
“1 INTRODUCTION Body-worn movement sensors, such as accelerometers or full-fledged inertial measurement units (IMU), have been extensively utilized for a wide range of applications in mobile and ubiquitous computing, including but not limited to novel interaction paradigms [67, 82, 84], gesture recognition [83], eating…”
BackgroundWhat next, ubicomp? celebrating an intellectual disappearing act
Abowd, G. D. (2012) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/2370216.2370222 · Cited by 109 · cited 1×
“1 Application Scenarios In our work, we focus on studying self-supervised pre-training primarily for locomotion style activities such as walking, running, and sitting, as these broadly represent a significant portion of activities performed daily and are of significant interest for the mobile and ubiquitous computing…”
BackgroundFederated Self-Supervised Learning of Multisensor Representations for Embedded Intelligence
Saeed, A., Salim, F. D., Ozcelebi, T., & Lukkien, J. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3009358 · Cited by 106 · cited 1×
“[70], where the authors propose a signal correspondence learning approach based on wavelet transforms in order to learn useful representations from unlabeled data, for modalities including blood volume pulse, accelerometry, electroencephalography, and WiFi channel state information in a federated learning framework.”
BackgroundUnsupervised home monitoring of Parkinson's disease motor symptoms using body-worn accelerometers
Fisher, J. M., Hammerla, N., Plötz, T., András, P., Rochester, L., & Walker, R. W. (2016) · Parkinsonism & Related Disorders · doi:10.1016/j.parkreldis.2016.09.009 · Cited by 100 · cited 1×
“1 INTRODUCTION Body-worn movement sensors, such as accelerometers or full-fledged inertial measurement units (IMU), have been extensively utilized for a wide range of applications in mobile and ubiquitous computing, including but not limited to novel interaction paradigms [67, 82, 84], gesture recognition [83], eating…”
BackgroundSelf-citationEliaš, P. (1955) · IEEE Transactions on Information Theory · doi:10.1109/tit.1955.1055116 · Cited by 95
Human Activity Recognition from Wearable Sensor Data Using Self-Attention
Saif, M., Tanjid Hasan Tonmoy, M., Kishor, K. B., Mahbubur Rahman, A. K. M., Ashraful Amin, M., Mohammad, S., Muhammad, A. H. K., & Amin, A. A. (2020) · Frontiers in Artificial Intelligence and Applications · doi:10.3233/faia200236 · Cited by 93
Human Daily Activity and Fall Recognition Using a Smartphone’s Acceleration Sensor
Chatzaki, C., Pediaditis, M., Vavoulas, G., & Tsiknakis, M. (2017) · Communications in Computer and Information Science · doi:10.1007/978-3-319-62704-5_7 · Cited by 91 · cited 2×
“The dataset covers eleven activities of daily living and four types of falls [10].”
BackgroundOn the role of features in human activity recognition
Haresamudram, H., Anderson, D. V., & Plötz, T. (2019) · Proceedings of the 23rd International Symposium on Wearable Computers · doi:10.1145/3341163.3347727 · Cited by 90 · cited 10×
“There is not yet a consensus on the gold standard feature representation for HAR using wearables [31].”
BackgroundMethodInfluentialSelf-citationAction perception as hypothesis testing
Donnarumma, F., Costantini, M., Ambrosini, E., Friston, K., & Pezzulo, G. (2017) · Cortex · doi:10.1016/j.cortex.2017.01.016 · Cited by 90
Zhang, C., Bedri, A., Reyes, G., Bercik, B., Inan, O. T., Starner, T. E., & Abowd, G. D. (2016) · ACM International Conference on Interactive Surfaces and Spaces (ISS) · doi:10.1145/2992154.2992187 · Cited by 90 · cited 1×
“1 INTRODUCTION Body-worn movement sensors, such as accelerometers or full-fledged inertial measurement units (IMU), have been extensively utilized for a wide range of applications in mobile and ubiquitous computing, including but not limited to novel interaction paradigms [67, 82, 84], gesture recognition [83], eating…”
BackgroundUnsupervised Pre-Training of Bidirectional Speech Encoders via Masked Reconstruction
Wang, W., Tang, Q., & Livescu, K. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9053541 · Cited by 83 · cited 1×
“In [77], Wang et al.”
Zhang, C., Waghmare, A., Kundra, P., Pu, Y., Gilliland, S. M., Plötz, T., Starner, T. E., Inan, O. T., & Abowd, G. D. (2017) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3130985 · Cited by 82 · cited 1×
“1 INTRODUCTION Body-worn movement sensors, such as accelerometers or full-fledged inertial measurement units (IMU), have been extensively utilized for a wide range of applications in mobile and ubiquitous computing, including but not limited to novel interaction paradigms [67, 82, 84], gesture recognition [83], eating…”
BackgroundSelf-citationZhang, C., Xue, Q., Waghmare, A., et al. (2017) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3090095 · Cited by 63 · cited 1×
“1 INTRODUCTION Body-worn movement sensors, such as accelerometers or full-fledged inertial measurement units (IMU), have been extensively utilized for a wide range of applications in mobile and ubiquitous computing, including but not limited to novel interaction paradigms [67, 82, 84], gesture recognition [83], eating…”
BackgroundImproved Speech Representations with Multi-Target Autoregressive Predictive Coding
Chung, Y.-A., & Glass, J. R. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.213 · Cited by 60 · cited 1×
“More recently, autoregressive predictive coding has also been explored for speech self-supervision in [15, 17] and [16].”
BackgroundImproved Speech Representations with Multi-Target Autoregressive Predictive CodingPreprint
Chung, Y.-A., & Glass, J. R. (2020) · Annual Meeting of the Association for Computational Linguistics · doi:10.48550/arxiv.2004.05274 · Cited by 58
An Unsupervised Autoregressive Model for Speech Representation LearningPreprint
Chung, Y.-A., Hsu, W.-N., Tang, H., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.03240 · Cited by 45 · cited 1×
“) Autoregressive predictive coding has also been explored for speech self-supervision in [15, 17] and [16].”
BackgroundDeep Auto-Set: A Deep Auto-Encoder-Set Network for Activity Recognition Using Wearables
Varamin, A. A., Abbasnejad, E., Shi, Q., Ranasinghe, D. C., & Rezatofighi, H. (2018) · Proceedings of the 15th EAI International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services · doi:10.1145/3286978.3287024 · Cited by 44 · cited 3×
“[74] define HAR as a set prediction problems within the autoencoder setup.”
BackgroundMethodKwon, H., Abowd, G. D., & Plötz, T. (2018) · ACM International Symposium on Wearable Computers (ISWC) · doi:10.1145/3267242.3267258 · Cited by 40 · cited 1×
“Improvements to this representation technique include adding structure [43] and specializing window lengths for human activity recognition [46].”
Self-citationWhoosh: non-voice acoustics for low-cost, hands-free, and rapid input on smartwatches
Reyes, G., Zhang, D., Ghosh, S., Shah, P., Wu, J., Parnami, A., Bercik, B., Starner, T. E., Abowd, G. D., & Edwards, W. K. (2016) · ACM International Symposium on Wearable Computers (ISWC) · doi:10.1145/2971763.2971765 · Cited by 40 · cited 1×
“1 INTRODUCTION Body-worn movement sensors, such as accelerometers or full-fledged inertial measurement units (IMU), have been extensively utilized for a wide range of applications in mobile and ubiquitous computing, including but not limited to novel interaction paradigms [67, 82, 84], gesture recognition [83], eating…”
BackgroundBaños, O., Calatroni, A., Damas, M., Pomares, H., Rojas, I., Sagha, H., Millán, J. D. R., Tröster, G., Chavarriaga, R., & Roggen, D. (2012) · International Symposium on Wearable Computers · doi:10.1109/iswc.2012.17 · Cited by 35 · cited 1×
“[9] transform kinect data to IMU by utilizing MIMO mapping, which is learned by predicting from past input samples.”
MethodMusiCoder: A Universal Music-Acoustic Encoder Based on Transformer
Zhao, Y., & Guo, J. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-67832-6_34 · Cited by 30
Towards deep clustering of human activities from wearables
Abedin, A., Motlagh, F., Shi, Q., Rezatofighi, H., & Ranasinghe, D. (2020) · Proceedings of the 2020 International Symposium on Wearable Computers · doi:10.1145/3410531.3414312 · Cited by 27 · cited 1×
“[1] utilize future timestep prediction alongside reconstruction in order to learn effective representations for human activity recognition.”
MethodOn specialized window lengths and detector based human activity recognition
Li, H., Abowd, G. D., & Plötz, T. (2018) · Proceedings of the 2018 ACM International Symposium on Wearable Computers · doi:10.1145/3267242.3267246 · Cited by 19 · cited 1×
“Improvements to this representation technique include adding structure [43] and specializing window lengths for human activity recognition [46].”
Self-citationA Multi-Sensor Setting Activity Recognition Simulation Tool
Takeda, S., Okita, T., Lago, P., & Inoue, S. (2018) · Proceedings of the 2018 ACM International Joint Conference and 2018 International Symposium on Pervasive and Ubiquitous Computing and Wearable Computers · doi:10.1145/3267305.3267509 · Cited by 18 · cited 1×
“Recent approaches addressing the lack of annotated datasets involve extraction of virtual movement data from modalities such as mocap [37, 72] and videos [44], which contain a large number of participants and diverse set of actions and environments.”
BackgroundKwon, H., Tong, C., Haresamudram, H., Gao, Y., Abowd, G. D., Lane, N. D., & Ploetz, T. (2020) · arXiv · doi:10.48550/arxiv.2006.05675 · Cited by 14
Self-citationSelf-Supervised Video Representation Learning With Odd-One-Out NetworksPreprint
Fernando, B., Bilen, H., Gavves, E., & Gould, S. J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.06646 · Cited by 11
Generative Pre-Training for Speech with Autoregressive Predictive CodingPreprint
Chung, Y.-A., & Glass, J. R. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.12607 · Cited by 10
Doherty, A., Jackson, D., Hammerla, N., et al. (2017) · Newcastle University ePrints (Newcastle Univesity) · doi:10.25405/ncl.34308291
Ensembles of deep LSTM Learners for Activity Recognition using Wearables
Guan, Y., & Ploetz, T. (2017) · Newcastle University ePrints (Newcastle Univesity) · doi:10.25405/ncl.34321128
TERA: Self-Supervised Learning of Transformer Encoder Representation for Speech
Liu, A. T., Li, S.-W., & Lee, H.-Y. (2020) · arXiv · doi:10.48550/arxiv.2007.06028
MusiCoder: A Universal Music-Acoustic Encoder Based on Transformers
Zhao, Y., & Guo, J. (2020) · arXiv · doi:10.48550/arxiv.2008.00781
Untitled work
Srivastava (2014) · doi:10.5555/2627435.2670313
Research works printed but not linked: 5
- 10.5555/1460232.1460247 · doi:10.5555/1460232.1460247
- Cheng- I Lai . 2019. Contrastive predictive coding based feature for automatic speaker verification. arXiv preprint arXiv:1904.01575 ( 2019 ). Cheng-I Lai. 2019. Contrastive predictive coding based feature for automatic speaker verification. arXiv preprint arXiv:1904.01575 (2019). · arXiv
- William Lotter , Gabriel Kreiman , and David Cox . 2016. Deep predictive coding networks for video prediction and unsupervised learning. arXiv preprint arXiv:1605.08104 ( 2016 ). William Lotter, Gabriel Kreiman, and David Cox. 2016. Deep predictive coding networks for video prediction and unsupervised learning. arXiv preprint arXiv:1605.08104 (2016). · arXiv
- Aaqib Saeed , Victor Ungureanu , and Beat Gfeller . 2020. Sense and Learn: Self-Supervision for Omnipresent Sensors. arXiv preprint arXiv:2009.13233 ( 2020 ). Aaqib Saeed, Victor Ungureanu, and Beat Gfeller. 2020. Sense and Learn: Self-Supervision for Omnipresent Sensors. arXiv preprint arXiv:2009.13233 (2020). · arXiv
- Neo Wu , Bradley Green , Xue Ben , and Shawn O'Banion . 2020. Deep transformer models for time series forecasting: The influenza prevalence case. arXiv preprint arXiv:2001.08317 ( 2020 ). Neo Wu, Bradley Green, Xue Ben, and Shawn O'Banion. 2020. Deep transformer models for time series forecasting: The influenza prevalence case. arXiv preprint arXiv:2001.08317 (2020). · arXiv
Cited by
Self-supervised learning for human activity recognition using 700,000 person-days of wearable data
Yuan, H., Chan, S., Creagh, A. P., Tong, C., Acquah, A., Clifton, D. A., & Doherty, A. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01062-3 · Cited by 166
“, 2020a), contrastive learning (Haresamudram et al., 2020b; Tang et al., 2020), and multi-task SSL (Saeed et al.”
Human Activity Recognition: Review, Taxonomy and Open Challenges
Arshad, M. H., Bilal, M., & Gani, A. (2022) · Sensors · doi:10.3390/s22176463 · Cited by 178
“[27] Proposed a CPC framework based on CNN and LSTM for monitoring construction equipment activity.”
BackgroundMethodHuman activity recognition using wearable sensors by heterogeneous convolutional neural networks
Han, C., Zhang, L., Tang, Y., Huang, W., Min, F., & He, J. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.116764 · Cited by 130
ColloSSL: Collaborative Self-Supervised Learning for Human Activity Recognition
Jain, Y., Tang, C. I., Min, C., Kawsar, F., & Mathur, A. (2022) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3517246 · Cited by 113
“[22] also proposed Contrastive Predictive Coding which leverages the long-term temporal structure of sensor data streams for self-supervised learning.”
BackgroundAssessing the State of Self-Supervised Human Activity Recognition Using Wearables
Haresamudram, H., Essa, I., & Plötz, T. (2022) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3550299 · Cited by 107
“The solved tasks need to be replaced with more challenging and relevant ones, addressing discovered flaws in the established experimental setups [17].”
BackgroundMethodInfluentialSelf-citation
Metadata
- Authors
- Haresamudram, H., Essa, I., & Plötz, T.
- Affiliations
- Georgia Institute of Technology (US)
- Venue
- Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
- Published
- 2021-06-23
- Volume
- 5
- Issue
- 2
- Pages
- 1-26
- ISSN
- 2474-9567 (ISSN-L)
- OpenAlex
- W3111841649
- Cited by
- 150
- References
- 99
- Citations by year
- 2026: 15 · 2025: 42 · 2024: 39 · 2023: 27 · 2022: 21 · 2021: 6
- FWCI
- 8.35
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies (publishedVersion, public-domain)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Social Sciences › Transportation › Human Mobility and Location-Based Analysis
- Keywords
- human activity recognition, contrastive predictive coding, self-supervised learning, wearable sensors, motion sensors, learned representations, pre-training, limited labeled data, wearable computing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.