Past, Present, and Future of Sensor-based Human Activity Recognition Using Wearables: A Surveying Tutorial on a Still Challenging Task
Open accessThe DOI matches its Crossref record; 235 of the 236 references it cites were found in a registry or an index; 107 of the 168 sources its claims cite were checked for support.
Abstract
A comprehensive survey and tutorial on sensor-based human activity recognition from heuristic handcrafted feature engineering through deep learning, self-supervised learning, and multimodal foundation models, outlining benchmark datasets, placement challenges, and open problems.
Overview
CoChat Summary
TL;DR
This surveying tutorial reviews sensor-based wearable human activity recognition from handcrafted features through deep learning and self-supervised representation learning to emerging foundational-model approaches. It argues that HAR remains challenging due to signal ambivalence, input variability, information coding mismatch, and scarce labeled data, and provides a hands-on tutorial with code and Motionsense experiments.
Key findings
- The paper identifies four broad HAR challenges: signal ambivalence, variability of the input space, information coding, and lack of labeled training data.
- In the tutorial experiments on Motionsense, the ECDF-RF baseline achieved a test F1-score of 81.84%, the supervised Conv.classifier achieved 85.1%, and SimCLR with frozen pre-trained encoder weights achieved 83.17%.
- Class imbalance is substantial in wearable datasets: the null class comprises approx. 75% of the Opportunity benchmark, >35% of Mobiact is “Standing,” and transitionary activities such as “Sit to stand” comprise <5%.
- The paper concludes there is no “one-size-fits-all” HAR method, but self-supervised representation learning is currently the predominant approach and foundational models may drive a third paradigm shift.
Methods
- Design: a surveying tutorial that reviews the field from the Activity Recognition Chain and handcrafted features to end-to-end deep learning, self-supervised learning, data generation/augmentation, and foundational models.
- Data and tutorial experiments: the hands-on tutorial uses the Motionsense dataset, accelerometer data only, with six locomotion-style activities from 24 participants; splits are 15 train, 4 validation, and 5 test users.
- Main analysis: the tutorial compares distribution-based ECDF features with a Random Forest classifier, a supervised convolutional classifier, and SimCLR self-supervised learning, using mean/unweighted F1-score because it is more resistant to class imbalance.
Limitations
- The paper states that there is no universally applicable HAR method, that large-scale richly annotated datasets are lacking, and that plug-and-play sensor-language models remain unsolved; it also notes that many sensor-language works train and evaluate on different splits of the same dataset, leaving cross-distribution performance unclear.
- The paper states limitations for specific methods: GAN-based generation is dataset- and label-specific, requires substantial labeled data, and can suffer mode collapse; adversarial learning lacks a mechanism to measure generalization quality during training; diffusion models require considerable computational steps; and early simulation approaches often rely on MoCap and exclude gyroscope data.
- Evident from the text: the tutorial uses only accelerometer data to reduce training time and evaluates on Motionsense’s relatively simple locomotion activities; the ECDF-RF result shows 100% train performance versus 81.84% test F1, which the paper itself notes indicates potential overfitting.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures

![Fig. 2 An overview of the self-supervised learning pipeline. Reproduced with permission from [81].](https://figures.cochat.ai/paper-figures/b960558ff2124da488f7679a47132bd4/past-present-and-future-of-sensor-based-human-activity-recognition-using-figure-2.png)

![Fig. 3 Contrastive learning adaptations in human activity recognition. A comparison is drawn among SimCLR [184], CPC [80], BYOL [81] and SimSiam [81]. A high degree of commonality can be found among these frameworks, especially in the use of augmented anchors as positive samples for SimCLR, BYOL, and SimSiam. The CPC differs from the rest by using future samples instead of augmented views. SimCLR and CPC leverage time-misaligned samples as negatives, while BYOL and SimSiam leverage additional mechanisms to remove the requirement of negative samples.](https://figures.cochat.ai/paper-figures/5b18bcce22434c3b9d7b92809ed90bd2/past-present-and-future-of-sensor-based-human-activity-recognition-using-figure-3.png)
![Fig. 4 An overview of multi-device and multi-modal contrastive learning frameworks in human activity recognition. A comparison is drawn among ColloSSL [93], Learning from the Best (LftB) [55], COCOA [40] and SimCLR [184] (as a singledevice single-modality reference). Instead of relying on augmentations (as in SimCLR), these multi-device and multi-modal approaches leverage data from time-aligned data from different devices (ColloSSL and LftB), and different modalities (COCOA) for positive samples. Time-misaligned samples as negatives is a common feature among these approaches, with different frameworks imposing additional limitations on the sampling algorithm.](https://figures.cochat.ai/paper-figures/bf666a68bcd240de98433950d9703b89/past-present-and-future-of-sensor-based-human-activity-recognition-using-figure-4.png)



![Fig. 9 An overview of the SimCLR training pipeline for human activity recognition [184].](https://figures.cochat.ai/paper-figures/5267a8be66244545bd8513b169446e2a/past-present-and-future-of-sensor-based-human-activity-recognition-using-figure-9.png)



Verification
Claims the paper makes
62 Valid · 7 Not valid · 36 Not checked · 98 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“More recently, the self-attention mechanism introduced in the Transformer paper [194] has also been successfully applied for sensor-based HAR [20,57,58,129].”
§ Past, Present, and Future of Sensor-based Human Activity Recognition Using Wearables: A Surveying Tutorial on a Still Challenging Task > 3.2 End-to-End Learning Based Approaches
What the sources say
- Supportedcites [20] 20. Keyan Cao and Mingrui Wang. 2023. Human behavior recognition based on sparse transformer with channel attention mechanism. Frontiers in Physiology 14 (2023), 1239453. sourceExperimental — accuracy not yet measured
The Abstract describes a Transformer-based model using self-attention for sensor-based HAR and reports strong accuracy on three HAR datasets. Section 3.2 explains that the authors modified the Transformer’s self-attention mechanism for this application. (LLM, conf=0.93)
- Partly supportedcites [57] 57. Wenbin Gao, Lei Zhang, Qi Teng, Jun He, and Hao Wu. 2021. DanHAR: Dual attention network for multimodal human activity recognition using wearable sensors. Applied Soft Computing 111 (2021), 107728. sourceExperimental — accuracy not yet measured
The Conclusion (p. 10) reports improved sensor-based HAR performance using DanHAR’s dual attention mechanism. But the Model section (pp. 3–4) describes channel and temporal attention built with pooling and convolution, not Transformer-style self-attention; that specific part of the claim is unsupported. (LLM, conf=0.88)
- Supportedcites [58] 58. Ziqi Gao, Yuntao Wang, Jianguo Chen, Junliang Xing, Shwetak Patel, Xin Liu, and Yuanchun Shi. 2023. MMTSA: Multi-Modal Temporal Segment Attention Network for Efficient Human Activity Recognition. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 7, 3 (2023), 1-26. source
Section 4.3 (p. 14) links MMTSA’s attention mechanism to Transformer attention and describes its use in multimodal fusion. The Conclusion (p. 23) reports successful HAR results using RGB-camera and IMU sensor data, supporting the claim that Transformer-derived attention has been applied successfully to sensor-based HAR. (LLM, conf=0.88)
- Not checked — no full text of the sourcecites [129] 129. Saif Mahmud, M Tanjid Hasan Tonmoy, Kishor Kumar Bhaumik, AKM Mahbubur Rahman, M Ashraful Amin, Mohammad Shoyaib, Muhammad Asif Hossain Khan, and Amin Ahsan Ali. 2020. Human activity recognition from wearable sensor data using self-attention. In ECAI 2020. IOS Press, 1332-1339. source
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“The intuition here is that the pretext task captures core signal and sensor characteristics under different rotations and placements and levels of noise [166].”
§ Past, Present, and Future of Sensor-based Human Activity Recognition Using Wearables: A Surveying Tutorial on a Still Challenging Task > 4.1.2
What the sources say
- Supportedcites [166] 166. Aaqib Saeed, Tanir Ozcelebi, and Johan Lukkien. 2019. Multi-task self-supervised learning for human activity detection. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 3, 2 (2019), 1-30. source
Section 3.2 (p. 7) says the pretext tasks enable the network to capture core signal characteristics, including sensor behavior under different device placements and robustness against sensor noise. The same section describes a rotation transformation intended to provide sensor-orientation invariance. (LLM, conf=0.94)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Since the choice of augmentation function encodes the learning objective and invariance that the model should capture, the original SimCLR framework demonstrates significant performance differences among the choices of augmentation functions [28].”
§ Past, Present, and Future of Sensor-based Human Activity Recognition Using Wearables: A Surveying Tutorial on a Still Challenging Task > SimCLR.
What the sources say
- Supportedcites [28] 28. Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020. A simple framework for contrastive learning of visual representations. In International conference on machine learning. PMLR, 1597-1607. source
Section 3 (p. 3) says data augmentation defines predictive tasks, and Section 3.1 (pp. 4–5) compares different augmentation choices. Table 1 reports SimCLR top-1 accuracy ranging from 59.6% to 64.5% across the tested choices, supporting the claimed performance differences. (LLM, conf=0.93)
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“One such adaptation effort is the modification of the SimCLR framework for sensor time series by Tang et al. [184].”
§ Past, Present, and Future of Sensor-based Human Activity Recognition Using Wearables: A Surveying Tutorial on a Still Challenging Task > SimCLR.
What the sources say
- Supportedcites [184] 184. Chi Ian Tang, Ignacio Perez-Pozuelo, Dimitris Spathis, and Cecilia Mascolo. 2020. Exploring contrastive learning in human activity recognition for healthcare. arXiv preprint arXiv:2011.11542 (2020). source
The Introduction (p. 2) says the authors adapted SimCLR for wearable-sensor data by replacing image augmentations with transformations designed for sensor time series. The Method section (p. 2) also explicitly describes modifications for HAR. (LLM, conf=0.95)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Furthermore, investigations into enhancements of CPC for HAR, as presented by subsequent research [79,82], have yielded fully convolutional architectures that exhibit substantial improvements across diverse datasets and activity types.”
§ Past, Present, and Future of Sensor-based Human Activity Recognition Using Wearables: A Surveying Tutorial on a Still Challenging Task > Contrastive Predictive Coding.
What the sources say
- Supportedcites [79] 79. Harish Haresamudram, Irfan Essa, and Thomas Ploetz. 2024. Towards Learning Discrete Representations via Self-Supervision for Wearables-Based Human Activity Recognition. Sensors 24, 4 (2024), 1238. source
The Background (§2.3) says Enhanced CPC uses a fully convolutional architecture, substantially improves on the original CPC framework, and outperforms other self-supervised methods on four of six target datasets. The Results (§5.1) describe datasets spanning several sensor locations and activity types; the paper does not claim improvement on every dataset or activity. (LLM, conf=0.91)
- Supportedcites [82] 82. Harish Haresamudram, Irfan Essa, and Thomas Plötz. 2023. Investigating enhancements to contrastive predictive coding for human activity recognition. In 2023 IEEE International Conference on Pervasive Computing and Communications (PerCom). IEEE, 232-241. source
The paper’s opening summary says its CPC enhancements produce a fully convolutional architecture and substantial improvements across sensor positions and activities on four of six target datasets. The Conclusion also reports substantial improvements on four of six benchmark datasets. (LLM, conf=0.90)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
101 more claims · 26 with their cited sources checked
Create a free account to see all 203 claims and to verify the 2 references still open.
- It may state two separate claims as one. 47
- Probably not a substantial claim. 59
- The quote only partly states this claim. 1
- Borderline: one of the tests for a substantial claim is unclear. 1
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“A key challenge is that annotating IMU data is much more labor-intensive and costly than labeling images or text, as it often requires external reference signals and precise temporal alignment [25].”
From COMODO: Cross-Modal Video-to-IMU Distillation for Efficient Egocentric Human Activity Recognition (2026) · cites this paper as [25]
Experimental — accuracy not yet measuredIn the citing paper
“By leveraging unlabeled data samples through proxy objectives, SSL can learn transferable representations that approach and in some cases exceed supervised baselines while using substantially fewer labelled samples for downstream training1.”
From Siamese masked reconstruction with temporal rotation consistency for inertial-based human activity recognition (2026) · cites this paper as [1]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Past, Present, and Future of Sensor-based Human Activity Recognition Using Wearables: A Surveying Tutorial on a Still Challenging Task > B.2.1 Segmentation and Data Loading.
“We perform training for 50 epochs with the Adam optimizer.”
§ Past, Present, and Future of Sensor-based Human Activity Recognition Using Wearables: A Surveying Tutorial on a Still Challenging Task > B.4 Performance Evaluation
“10 optimizer = optim.AdamW(model.parameters(),lr=args['learning_rate'],”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Past, Present, and Future of Sensor-based Human Activity Recognition Using Wearables: A Surveying Tutorial on a Still Challenging Task > B.3 HAR with Self-Supervised Learning: SimCLR
“Each block has a 1D convolutional layer, followed by ReLU and dropout with 𝑝 = 0.2.”
§ Past, Present, and Future of Sensor-based Human Activity Recognition Using Wearables: A Surveying Tutorial on a Still Challenging Task > B.3 HAR with Self-Supervised Learning: SimCLR
“1 SimCLR( 5 (relu): ReLU() 6 (dropout): Dropout(p=0.1,inplace=False) 7 ) 8 (conv2)”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Past, Present, and Future of Sensor-based Human Activity Recognition Using Wearables: A Surveying Tutorial on a Still Challenging Task > B.2.1 Segmentation and Data Loading.
“Inside each block is a 1D convolutional layer, followed by ReLU activation, and dropout with p=0.2.”
§ Past, Present, and Future of Sensor-based Human Activity Recognition Using Wearables: A Surveying Tutorial on a Still Challenging Task > B.4 Performance Evaluation
“Figure: 245678 2 ( 4 ( 5 ( 6 ( 7 ) 8 ( conv): Conv1d(3, 32, kernel_size=(24,), stride=(1,)) relu): ReLU() dropout): Dropout(p=0.1,inplace=False)”
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 1×
“…[137] were introduced specifically eliminating the explicit feature design phase but rather utilizing proven feature learners such as convolutional blocks into the model architectures, complemented by explicit sequential modeling parts in form of LSTM blocks [83] or later ensembles thereof [65].”
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472
SMOTE synthetic minority over-sampling technique
Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002) · Journal of Artificial Intelligence Research · doi:10.1613/jair.953 · Cited by 32,865 · cited 1×
“Traditional oversampling techniques, such as the Synthetic Minority Over-sampling Technique (SMOTE) [22], SVM-SMOTE [132], and the Majority Weighted Minority Oversampling Technique (MWMOTE) [10], have been employed to mitigate data scarcity.”
PyTorch: An Imperative Style, High-Performance Deep Learning Library
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al. (2019) · arXiv · doi:10.48550/arxiv.1912.01703 · Cited by 31,473
High-Resolution Image Synthesis with Latent Diffusion Models
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01042 · Cited by 20,302
A survey on Image Data Augmentation for Deep Learning
Shorten, C., & Khoshgoftaar, T. M. (2019) · Journal of Big Data · doi:10.1186/s40537-019-0197-0 · Cited by 13,203 · cited 1×
“While traditional CV approaches employ simple affine transformations for data augmentation, such as translation, rotation, resizing, and shearing [121, 164], the unique characteristics of accelerometer signals in sensor-based HAR necessitate alternative strategies.”
A simple framework for contrastive learning of visual representations
Chen, T., Kornblith, S., Norouzi, M., & Hinton, G. (2020) · arXiv · doi:10.48550/arxiv.2002.05709 · Cited by 10,857
Masked Autoencoders Are Scalable Vision Learners
He, K., Chen, X., Xie, S., Li, Y., Dollar, P., & Girshick, R. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01553 · Cited by 8,489 · cited 1×
“Alternatively, Hong et al. [84], improve cross dataset performance via masked modeling in conjunction with contrastive regularization.”
Representation Learning with Contrastive Predictive Coding
Oord, A. V. D., Li, Y., & Vinyals, O. (2018) · arXiv · doi:10.48550/arxiv.1807.03748 · Cited by 6,242
Context Encoders: Feature Learning by Inpainting
Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T., & Efros, A. A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.278 · Cited by 5,691 · cited 1×
“An extension of the autoencoder setup involves masking portions of the input data, and training the network to reconstruct only the missing portion from available context, e.g., context encoders [139] and BERT [39].”
LLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602
Bootstrap Your Own Latent: A New Approach to Self-Supervised Learning
Grill, J.-B., Strub, F., Altché, F., Tallec, C., Richemond, P. H., Buchatskaya, E., Doersch, C., Pires, B. A., Guo, Z. D., Azar, M. G., et al. (2020) · arXiv · doi:10.48550/arxiv.2006.07733 · Cited by 3,846 · cited 2×
“Two primary categories of approaches address this issue: classic, and deep learning-based methods.”
Exploring Simple Siamese Representation Learning
Chen, X., & He, K. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01549 · Cited by 3,725 · cited 1×
“Additionally, it showed resilience to signal losses and required fewer labeled data for effective training.”
Diffusion Models Beat GANs on Image Synthesis
Dhariwal, P., & Nichol, A. (2021) · arXiv · doi:10.48550/arxiv.2105.05233 · Cited by 3,362
2009 International Joint Conference on Artificial Intelligence
(2009) · doi:10.1109/jcai15434.2009 · Cited by 2,800
Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition
Ordóñez, F., & Roggen, D. (2016) · Sensors · doi:10.3390/s16010115 · Cited by 2,761 · cited 2×
“A combination of convolutional and recurrent layers was proposed in DeepConvLSTM [137], which, even today, remains a strong baseline and is commonly utilized in contemporary HAR works [77, 181].”
A Survey on Human Activity Recognition using Wearable Sensors
Lara, O. D., & Labrador, M. A. (2013) · IEEE Communications Surveys & Tutorials · doi:10.1109/surv.2012.110112.00192 · Cited by 2,662 · cited 1×
“Lara & Labrador [107] provided an extensive review of early works in wearable-based HAR systems.”
Deep Unsupervised Learning using Nonequilibrium Thermodynamics
Sohl-Dickstein, J., Weiss, E. A., Maheswaranathan, N., & Ganguli, S. (2015) · arXiv · doi:10.48550/arxiv.1503.03585 · Cited by 2,637 · cited 1×
“One of the pioneering tools in this field is IMUSIM [203], which introduced a virtual IMU sensor system that simulates acceleration, angular velocity, and magnetic data from 3D motion sequences captured by motion capture (MoCap) equipment.”
Liu, H., Li, C., Wu, Q., & Lee, Y. J. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-1516 · Cited by 2,428
SIGNATURE VERIFICATION USING A “SIAMESE” TIME DELAY NEURAL NETWORK
BROMLEY, J., BENTZ, J. W., BOTTOU, L., GUYON, I., LECUN, Y., MOORE, C., SÄCKINGER, E., & SHAH, R. (1994) · Series in Machine Perception and Artificial Intelligence · doi:10.1142/9789812797926_0003 · Cited by 2,350
Improved Baselines with Momentum Contrastive Learning
Chen, X., Fan, H., Girshick, R., & He, K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.04297 · Cited by 1,956
Score-Based Generative Modeling through Stochastic Differential Equations
Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Ermon, S., & Poole, B. (2020) · arXiv · doi:10.48550/arxiv.2011.13456 · Cited by 1,953 · cited 1×
“Jiang et al. [93] utilized OpenSim, an open-source software system for biomechanical modeling, to simulate individuals with diverse physiological characteristics performing various movements to augment the IMU dataset.”
RePaint: Inpainting using Denoising Diffusion Probabilistic Models
Lugmayr, A., Danelljan, M., Romero, A., Yu, F., Timofte, R., & Van Gool, L. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01117 · Cited by 1,778 · cited 1×
“Using a 3D skinned multi-person linear model (SMPL), it simulated sensor data from arbitrary on-body positions and traind a CNN model to map imperfect 3D body pose estimations to IMU data.”
Improved Baselines with Visual Instruction Tuning
Liu, H., Li, C., Li, Y., & Lee, Y. J. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.02484 · Cited by 1,748 · cited 1×
“In abstract terms, the various approaches can be summarized as consisting of two steps: (i) creation of a representation of the activities for which we want to generate data for; and (ii) the translation of that representation into synthetic sensor data.”
Improved Baselines with Visual Instruction Tuning
Liu, H., Li, C., Li, Y., & Lee, Y. J. (2023) · arXiv · doi:10.48550/arxiv.2310.03744 · Cited by 1,733
A tutorial on human activity recognition using body-worn inertial sensors
Bulling, A., Blanke, U., & Schiele, B. (2014) · ACM Computing Surveys · doi:10.1145/2499621 · Cited by 1,668 · cited 5×
“The Activity Recognition Chain as summarized in [19].”
InfluentialImproved Denoising Diffusion Probabilistic Models
Nichol, A., & Dhariwal, P. (2021) · arXiv · doi:10.48550/arxiv.2102.09672 · Cited by 1,550 · cited 1×
“Simulation can reduce the need for extensive real-world data collection, thereby accelerating the development of motion recognition systems.”
A review: Data pre-processing and data augmentation techniques
Maharana, K., Mondal, S., & Nemade, B. (2022) · Global Transitions Proceedings · doi:10.1016/j.gltp.2022.04.020 · Cited by 1,337
Time Series FeatuRe Extraction on basis of Scalable Hypothesis tests (tsfresh – A Python package)
Christ, M., Braun, N., Neuffer, J., & Kempa-Liehr, A. W. (2018) · Neurocomputing · doi:10.1016/j.neucom.2018.03.067 · Cited by 1,257 · cited 1×
“Open-source libraries exist that facilitate for computing of hundreds of such heuristic features and thus lower the bar for practitioners entering the field (e.g., tsfresh [31]).”
Doherty, A., Jackson, D., Hammerla, N., Plötz, T., Olivier, P., Granat, M. H., White, T., van Hees, V. T., Trenell, M. I., Owen, C. G., et al. (2017) · PLOS ONE · doi:10.1371/journal.pone.0169649 · Cited by 1,250
Self-citationMWMOTE--Majority Weighted Minority Oversampling Technique for Imbalanced Data Set Learning
Barua, S., Islam, M. M., Yao, X., & Murase, K. (2014) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2012.232 · Cited by 1,150
Convolutional Neural Networks for Human Activity Recognition using Mobile Sensors
Zeng, M., Nguyen, L. T., Yu, B., Mengshoel, O. J., Zhu, J., Wu, P., & Zhang, J. (2014) · Proceedings of the 6th International Conference on Mobile Computing, Applications and Services · doi:10.4108/icst.mobicase.2014.257786 · Cited by 876 · cited 1×
“The wearables community has therefore moved on from studying deeper supervised neural networks, towards opportunistically leveraging available and more easily collected unlabeled data for activity recognition.”
ImageBind One Embedding Space to Bind Them All
Girdhar, R., El-Nouby, A., Liu, Z., Singh, M., Alwala, K. V., Joulin, A., & Misra, I. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01457 · Cited by 863 · cited 2×
“In this work, these are addressed by training the models on a collection of publicly available datasets, called Time Series Pile, with different techniques such as sub-sampling and padding to handle time series of different lengths.”
Collecting complex activity datasets in highly rich networked sensor environments
Roggen, D., Calatroni, A., Rossi, M., Holleczek, T., Forster, K., Troster, G., Lukowicz, P., Bannach, D., Pirkl, G., Ferscha, A., et al. (2010) · 2010 Seventh International Conference on Networked Sensing Systems (INSS) · doi:10.1109/inss.2010.5573462 · Cited by 788
Self-citationSMPL: A Skinned Multi-Person Linear Model
Loper, M., Mahmood, N., Romero, J., Pons-Moll, G., & Black, M. J. (2023) · Seminal Graphics Papers: Pushing the Boundaries, Volume 2 · doi:10.1145/3596711.3596800 · Cited by 768
The Opportunity challenge: A benchmark database for on-body sensor-based activity recognition
Chavarriaga, R., Sagha, H., Calatroni, A., Digumarti, S. T., Tröster, G., Millán, J. D. R., & Roggen, D. (2013) · Pattern Recognition Letters · doi:10.1016/j.patrec.2012.12.014 · Cited by 742 · cited 2×
“When tackling such non-trivial activity recognition problems it quickly becomes clear that HAR is still far from being a commodity with recognition performance on challenging benchmark tasks such as the Opportunity challenge [21] stagnating now for more than a decade.”
Sensor-based and vision-based human activity recognition: A comprehensive survey
Minh Dang, L., Min, K., Wang, H., Jalil Piran, M., Hee Lee, C., & Moon, H. (2020) · Pattern Recognition · doi:10.1016/j.patcog.2020.107561 · Cited by 730 · cited 1×
“Bulling et al. [19] compiled a seminal tutorial paper that explicitly captures the classical, pre-Deep-Learning era of HAR with wearable sensors.”
Um, T. T., Pfister, F. M. J., Pichler, D., Endo, S., Lang, M., Hirche, S., Fietzek, U., & Kulić, D. (2017) · Proceedings of the 19th ACM International Conference on Multimodal Interaction · doi:10.1145/3136755.3136817 · Cited by 729 · cited 3×
“It relies on a set of eight data transformations / augmentations introduced by Um et al. [184] for increasing sensor data diversity, including adding random Gaussian noise, scaling, rotations, negation, flipping channels, permuting sub-segments of windows, time-warping and channel shuffling.”
Vector Quantized Diffusion Model for Text-to-Image Synthesis
Gu, S., Chen, D., Bao, J., Wen, F., Zhang, B., Chen, D., Yuan, L., & Guo, B. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01043 · Cited by 720
Stisen, A., Blunck, H., Bhattacharya, S., Prentow, T. S., Kjærgaard, M. B., Dey, A., Sonne, T., & Jensen, M. M. (2015) · Proceedings of the 13th ACM Conference on Embedded Networked Sensor Systems · doi:10.1145/2809695.2809718 · Cited by 714 · cited 1×
“Sensor-based human activity recognition (HAR) corresponds to the automatic classification of sensor data into activities of interest (or the null class).”
Large language models for software engineering: survey and open problems
Fan, A., Gokkaya, B., Harman, M., Lyubarskiy, M., Sengupta, S., Yoo, S., & Zhang, J. M. (2023) · 2023 IEEE/ACM International Conference on Software Engineering: Future of Software Engineering (ICSE-FoSE) · doi:10.1109/icse-fose59343.2023.00008 · Cited by 650
Masked Autoencoders that Listen
Huang, P.-Y., Xu, H., Li, J., Baevski, A., Auli, M., Galuba, W., Metze, F., & Feichtenhofer, C. (2022) · neural information processing systems · doi:10.52202/068431-2081 · Cited by 632
Ego4D: Around the World in 3,000 Hours of Egocentric Video
Grauman, K., Westbury, A., Byrne, E., Chavis, Z., Furnari, A., Girdhar, R., Hamburger, J., Jiang, H., Liu, M., Liu, X., et al. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01842 · Cited by 630 · cited 2×
“…recognition is yet to be verified, these works provide a good vision of how foundational models for human activity recognition can be: that training with a well-selected set of datasets and modest model size could offer much more practical and effective solutions for this mobile sensing task.”
Deep, Convolutional, and Recurrent Models for Human Activity Recognition Using Wearables
Hammerla, N. Y., Halloran, S., & Ploetz, T. (2016) · arXiv · doi:10.48550/arxiv.1604.08880 · Cited by 606
Um, T. T. (n.d.) · mediaTUM – the media and publications repository of the Technical University Munich (Technical University Munich) · Cited by 602
Borderline over-sampling for imbalanced data classification
Nguyen, H. M., Cooper, E. W., & Kamei, K. (2011) · International Journal of Knowledge Engineering and Soft Data Paradigms · doi:10.1504/ijkesdp.2011.039875 · Cited by 592 · cited 1×
“Researchers have explored techniques like discretization optimization, non-Markovian processes, and partial sampling to accelerate the speed of diffusion models [43, 169].”
Preprocessing techniques for context recognition from accelerometer data
Figo, D., Diniz, P. C., Ferreira, D. R., & Cardoso, J. M. P. (2010) · Personal and Ubiquitous Computing · doi:10.1007/s00779-010-0293-9 · Cited by 578 · cited 1×
“Traditionally, the ARC employed handcrafted, statistical [52, 147] and distribution-based features [68, 102], or techniques such as Principle Component Analysis (PCA) [146].”
Revisiting Knowledge Distillation via Label Smoothing Regularization
Yuan, L., Tay, F. E., Li, G., Wang, T., & Feng, J. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00396 · Cited by 572 · cited 1×
“Their findings highlighted the effectiveness of rotations, particularly in capturing the variability of sensor placement, which significantly improved the performance of Convolutional Neural Network (CNN) models.”
DiffWave: A Versatile Diffusion Model for Audio Synthesis
Kong, Z., Ping, W., Huang, J., Zhao, K., & Catanzaro, B. (2020) · arXiv · doi:10.48550/arxiv.2009.09761 · Cited by 569
Wearable Sleep Technology in Clinical and Research Settings
DE ZAMBOTTI, M., CELLINI, N., GOLDSTONE, A., COLRAIN, I. M., & BAKER, F. C. (2019) · Medicine & Science in Sports & Exercise · doi:10.1249/mss.0000000000001947 · Cited by 564
Large Language Models for Software Engineering: Survey and Open Problems
Fan, A., Gokkaya, B., Harman, M., Lyubarskiy, M., Sengupta, S., Yoo, S., & Zhang, J. M. (2023) · arXiv · doi:10.48550/arxiv.2310.03533 · Cited by 523
Ensembles of Deep LSTM Learners for Activity Recognition using Wearables
Guan, Y., & Plötz, T. (2017) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3090076 · Cited by 511 · cited 1×
“…[137] were introduced specifically eliminating the explicit feature design phase but rather utilizing proven feature learners such as convolutional blocks into the model architectures, complemented by explicit sequential modeling parts in form of LSTM blocks [83] or later ensembles thereof [65].”
Self-citationSelf-organizing neural network that discovers surfaces in random-dot stereograms
Becker, S., & Hinton, G. E. (1992) · Nature · doi:10.1038/355161a0 · Cited by 495
Wu, Y., Chen, K., Zhang, T., Hui, Y., Berg-Kirkpatrick, T., & Dubnov, S. (2023) · ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp49357.2023.10095969 · Cited by 459 · cited 1×
“Looking beyond works in re-purposing language and vision foundation models for human activity recognition, significant efforts have been dedicated by researchers to developing foundational models for time-series data.”
Fine-Grained Activity Recognition by Aggregating Abstract Object Usage
Patterson, D. J., Fox, D., Kautz, H., & Philipose, M. (2005) · Ninth IEEE International Symposium on Wearable Computers (ISWC'05) · doi:10.1109/iswc.2005.22 · Cited by 452 · cited 1×
“…has been around since the early days of HAR research [15], initially in vision-based approaches [127], with later adoption in sensor-based work (e.g. [140]) that have shown how reasoning about interaction with objects recognized through wearable RFID can be used for reliable recognition).”
CLAP Learning Audio Concepts from Natural Language Supervision
Elizalde, B., Deshmukh, S., Ismail, M. A., & Wang, H. (2023) · ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp49357.2023.10095889 · Cited by 449 · cited 1×
“…through and with Foundational Models Another approach for incorporating knowledge from foundational models is via alignment between data from modalities, e.g., images/videos [120, 150, 194], speech and audio [48, 66, 189, 190], or sensor data [74, 192], with natural language descriptions the data.”
MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model
Zhang, M., Cai, Z., Pan, L., Hong, F., Guo, X., Yang, L., & Liu, Z. (2022) · arXiv · doi:10.48550/arxiv.2208.15001 · Cited by 437
Skeleton-based action recognition via spatial and temporal transformer networks
Plizzari, C., Cannici, M., & Matteucci, M. (2021) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2021.103219 · Cited by 409 · cited 1×
“TASKED [174] extended this with a transformer network [45, 142], accounting for sensor orientations and spatial-temporal features.”
Exploring Simple Siamese Representation Learning
Chen, X., & He, K. (2020) · arXiv · doi:10.48550/arxiv.2011.10566 · Cited by 381
(2019) · neural information processing systems · Cited by 350
VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text UnderstandingPreprint
Xu, H., Ghosh, G., Huang, P.-Y., Okhonko, D., Aghajanyan, A., Metze, F., Zettlemoyer, L., & Feichtenhofer, C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.14084 · Cited by 347
Multi-task Self-Supervised Learning for Human Activity Detection
Saeed, A., Ozcelebi, T., & Lukkien, J. (2019) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3328932 · Cited by 346 · cited 4×
“…to contrastive approaches in computer vision tasks, Haresamudram et al. [77] showed that the SimCLR framework [176] and multi-task self-supervised learning [158], outperformed other approaches, including BYOL, SimSiam, masked reconstruction, Contrastive Predictive Coding (CPC) and autoencoders.”
InfluentialJiang, A. Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D. S., Casas, D. D. L., Bressand, F., Lengyel, G., Lample, G., Saulnier, L., et al. (2023) · arXiv · doi:10.48550/arxiv.2310.06825 · Cited by 340 · cited 1×
“Data generation and augmentation have already been discussed in Section 4.”
Human activity recognition in artificial intelligence framework: a narrative review
Gupta, N., Gupta, S. K., Pathak, R. K., Jain, V., Rashidi, P., & Suri, J. S. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-021-10116-x · Cited by 337
Feature Learning for Activity Recognition in Ubiquitous Computing
Plötz, T., Hammerla, N., & Olivier, P. (2011) · Newcastle University · doi:10.25405/ncl.34175319 · Cited by 337
Self-citationAudioclip: Extending Clip to Image, Text and Audio
Guzhov, A., Raue, F., Hees, J., & Dengel, A. (2022) · ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp43922.2022.9747631 · Cited by 330 · cited 1×
“Looking beyond works in re-purposing language and vision foundation models for human activity recognition, significant efforts have been dedicated by researchers to developing foundational models for time-series data.”
Generating Human Motion from Textual Descriptions with Discrete Representations
Zhang, J., Zhang, Y., Cun, X., Zhang, Y., Zhao, H., Lu, H., Shen, X., & Ying, S. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01415 · Cited by 328 · cited 1×
“The utility of generated data for HAR is validated on a newly collected dataset, and a combination of real and synthetic data substantially outperforms using real pressure data only.”
Ho, J., Salimans, T., Gritsenko, A., Chan, W., Norouzi, M., & Fleet, D. J. (2022) · arXiv · doi:10.48550/arxiv.2204.03458 · Cited by 301
Movement, activity and action: the role of knowledge in the perception of motion
Bobick, A. F. (1997) · Philosophical Transactions of the Royal Society of London. Series B: Biological Sciences · doi:10.1098/rstb.1997.0108 · Cited by 294
Gemma: Open Models Based on Gemini Research and Technology
Gemma Team, Mesnard, T., Hardin, C., Dadashi, R., Bhupatiraju, S., Pathak, S., Sifre, L., Rivière, M., Kale, M. S., Love, J., et al. (2024) · arXiv · doi:10.48550/arxiv.2403.08295 · Cited by 284
Steven Eyobu, O., & Han, D. S. (2018) · Sensors · doi:10.3390/s18092892 · Cited by 259 · cited 1×
“Virtual IMU data generation has emerged as a viable solution to overcome data scarcity, with cross-modality transfer approaches [53, 103, 104, 153, 154] being instrumental in extracting virtual IMU data from 2D RGB videos of human activities.”
SLIP: Self-supervision Meets Language-Image Pre-training
Mu, N., Kirillov, A., Wagner, D., & Xie, S. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-19809-0_30 · Cited by 258 · cited 1×
“This is an inspiring example for the future development of foundation models for human activity recognition: a specialized foundation model could be developed at a fraction of the costs of training large language models.”
Ontology‐based activity recognition in intelligent pervasive environments
Chen, L., & Nugent, C. (2009) · International Journal of Web Information Systems · doi:10.1108/17440080911006199 · Cited by 253 · cited 1×
“The effort involved in the manual coding of semantic relationships has motivated the use of ontologies as source of information [17, 23].”
Willetts, M., Hollowell, S., Aslett, L., Holmes, C., & Doherty, A. (2018) · Scientific Reports · doi:10.1038/s41598-018-26174-1 · Cited by 250
Spatial Temporal Transformer Network for Skeleton-Based Action Recognition
Plizzari, C., Cannici, M., & Matteucci, M. (2021) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-68796-0_50 · Cited by 248
Germini, F., Noronha, N., Borg Debono, V., Abraham Philip, B., Pete, D., Navarro, T., Keepanasseril, A., Parpia, S., de Wit, K., & Iorio, A. (2022) · Journal of Medical Internet Research · doi:10.2196/30791 · Cited by 220
CogVLM: Visual Expert for Pretrained Language Models
Wang, W., Lv, Q., Yu, W., Hong, W., Qi, J., Wang, Y., Ji, J., Yang, Z., Zhao, L., Song, X., et al. (2023) · arXiv · doi:10.48550/arxiv.2311.03079 · Cited by 217
Chung, S., Lim, J., Noh, K. J., Kim, G., & Jeong, H. (2019) · Sensors · doi:10.3390/s19071716 · Cited by 215 · cited 2×
“However, leveraging the temporal nature of accelerometer signals allows for the application of augmentation methods grounded in signal processing [29, 32, 95, 125, 134, 171, 184].”
DanHAR: Dual Attention Network for multimodal human activity recognition using wearable sensors
Gao, W., Zhang, L., Teng, Q., He, J., & Wu, H. (2021) · Applied Soft Computing · doi:10.1016/j.asoc.2021.107728 · Cited by 214 · cited 1×
“More recently, the self-attention mechanism introduced in the Transformer paper [185] has also been successfully applied for sensor-based HAR [20, 55, 56, 122].”
Masked Autoencoders that Listen
Huang, P.-Y., Xu, H., Li, J., Baevski, A., Auli, M., Galuba, W., Metze, F., & Feichtenhofer, C. (2022) · arXiv · doi:10.48550/arxiv.2207.06405 · Cited by 206
The mobile fitness coach: Towards individualized skill assessment using personalized mobile devices
Kranz, M., Möller, A., Hammerla, N., Diewald, S., Plötz, T., Olivier, P., & Roalter, L. (2013) · Pervasive and Mobile Computing · doi:10.1016/j.pmcj.2012.06.002 · Cited by 199
Self-citationUnified Contrastive Learning in Image-Text-Label Space
Yang, J., Li, C., Zhang, P., Xiao, B., Liu, C., Yuan, L., & Gao, J. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01857 · Cited by 195 · cited 1×
“This is an inspiring example for the future development of foundation models for human activity recognition: a specialized foundation model could be developed at a fraction of the costs of training large language models.”
ReMoDiffuse: Retrieval-Augmented Motion Diffusion Model
Zhang, M., Guo, X., Pan, L., Cai, Z., Hong, F., Li, H., Yang, L., & Liu, Z. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00040 · Cited by 195 · cited 1×
“The utility of generated data for HAR is validated on a newly collected dataset, and a combination of real and synthetic data substantially outperforms using real pressure data only.”
Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond
Bai, J., Bai, S., Yang, S., Wang, S., Tan, S., Wang, P., Lin, J., Zhou, C., & Zhou, J. (2023) · arXiv · doi:10.48550/arxiv.2308.12966 · Cited by 193
Zeng, M., Gao, H., Yu, T., Mengshoel, O. J., Langseth, H., Lane, I., & Liu, X. (2018) · Proceedings of the 2018 ACM International Symposium on Wearable Computers · doi:10.1145/3267242.3267286 · Cited by 185 · cited 1×
“Temporal attention was applied by Murahari et al. [130] whereas continuous sensor and temporal attention were evaluated by Zeng et al. [207], both leading to increased performance.”
Mining models of human activities from the web
Perkowitz, M., Philipose, M., Fishkin, K., & Patterson, D. J. (2004) · Proceedings of the 13th international conference on World Wide Web · doi:10.1145/988672.988750 · Cited by 183 · cited 1×
“The general idea is not new: in [141] it has been proposed to mine activity descriptions from the web.”
GENIE: Higher-Order Denoising Diffusion Solvers
Dockhorn, T., Vahdat, A., & Kreis, K. (2022) · neural information processing systems · doi:10.52202/068431-2186 · Cited by 177
Human activity recognition: A review
Ann, O. C., & Theng, L. B. (2014) · 2014 IEEE International Conference on Control System, Computing and Engineering (ICCSCE 2014) · doi:10.1109/iccsce.2014.7072750 · Cited by 169 · cited 1×
“Key design issues were discussed in the survey, including energy consumption, sensor placement, and flexibility.”
Protecting sensory data against sensitive inferences
Malekzadeh, M., Clegg, R. G., Cavallaro, A., & Haddadi, H. (2018) · Spiral (Imperial College London) · Cited by 166
Self-supervised learning for human activity recognition using 700,000 person-days of wearable data
Yuan, H., Chan, S., Creagh, A. P., Tong, C., Acquah, A., Clifton, D. A., & Doherty, A. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01062-3 · Cited by 166
GENIE: Higher-Order Denoising Diffusion SolversPreprint
Dockhorn, T., Vahdat, A., & Kreis, K. (2022) · Neural Information Processing Systems · doi:10.48550/arxiv.2210.05475 · Cited by 162 · cited 1×
“Researchers have explored techniques like discretization optimization, non-Markovian processes, and partial sampling to accelerate the speed of diffusion models [43, 169].”
On attention models for human activity recognition
Murahari, V. S., & Plötz, T. (2018) · Proceedings of the 2018 ACM International Symposium on Wearable Computers · doi:10.1145/3267242.3267287 · Cited by 157 · cited 1×
“Temporal attention was applied by Murahari et al. [130] whereas continuous sensor and temporal attention were evaluated by Zeng et al. [207], both leading to increased performance.”
Self-citationProtecting Sensory Data against Sensitive Inferences
Malekzadeh, M., Clegg, R. G., Cavallaro, A., & Haddadi, H. (2018) · Proceedings of the 1st Workshop on Privacy by Design in Distributed Systems · doi:10.1145/3195258.3195260 · Cited by 154 · cited 1×
“Dataset: We utilize the Motionsense dataset [123] 2 .”
Kwon, H., Tong, C., Haresamudram, H., Gao, Y., Abowd, G. D., Lane, N. D., & Plötz, T. (2020) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3411841 · Cited by 152
Self-citationContrastive Predictive Coding for Human Activity Recognition
Haresamudram, H., Essa, I., & Plötz, T. (2021) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3463506 · Cited by 150 · cited 4×
“Alternatively, Hong et al. [84], improve cross dataset performance via masked modeling in conjunction with contrastive regularization.”
InfluentialSelf-citationXu, H., Zhou, P., Tan, R., Li, M., & Shen, G. (2021) · Proceedings of the 19th ACM Conference on Embedded Networked Sensor Systems · doi:10.1145/3485730.3485937 · Cited by 150 · cited 1×
“…recognition is yet to be verified, these works provide a good vision of how foundational models for human activity recognition can be: that training with a well-selected set of datasets and modest model size could offer much more practical and effective solutions for this mobile sensing task.”
MotionGPT: Human Motion as a Foreign Language
Jiang, B., Chen, X., Liu, W., Yu, J., Yu, G., & Chen, T. (2023) · arXiv · doi:10.48550/arxiv.2306.14795 · Cited by 149 · cited 1×
“The utility of generated data for HAR is validated on a newly collected dataset, and a combination of real and synthetic data substantially outperforms using real pressure data only.”
Sensing Fine-Grained Hand Activity with Smartwatches
Laput, G., & Harrison, C. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300568 · Cited by 147 · cited 1×
“The Activity Recognition Chain as summarized in [19].”
Soleimani, E., & Nazerfard, E. (2021) · Neurocomputing · doi:10.1016/j.neucom.2020.10.056 · Cited by 146 · cited 2×
“Studies [97, 168, 173, 174] highlight the impact of individual characteristics and behaviors on HAR performance, with significant degradation when models are applied to new users (Fig.”
Masked reconstruction based self-supervision for human activity recognition
Haresamudram, H., Beedu, A., Agrawal, V., Grady, P. L., Essa, I., Hoffman, J., & Plötz, T. (2020) · Proceedings of the 2020 International Symposium on Wearable Computers · doi:10.1145/3410531.3414306 · Cited by 146 · cited 1×
“For wearables, masked reconstruction [73] utilizes transformer encoder [185] layers and masks out 10% of the sensor data of an analysis windows at random timesteps.”
Self-citationTao, W., Lai, Z.-H., Leu, M. C., & Yin, Z. (2018) · Procedia Manufacturing · doi:10.1016/j.promfg.2018.07.152 · Cited by 145
SelfHAR: Improving Human Activity Recognition through Self-training with Unlabeled Data
Tang, C. I., Perez-Pozuelo, I., Spathis, D., Brage, S., Wareham, N., & Mascolo, C. (2021) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3448112 · Cited by 140
Self-citationHammerla, N. Y., Kirkham, R., Andras, P., & Ploetz, T. (2013) · Proceedings of the 2013 International Symposium on Wearable Computers · doi:10.1145/2493988.2494353 · Cited by 138 · cited 4×
“2, the Activity Recognition Chain (ARC) [19] was employed for traditional HAR, comprising of five steps: (i) data collection; (ii) pre-processing; (iii) segmentation; (iv) feature extraction; and (v) classification.”
InfluentialSelf-citationA Systematic Study of Unsupervised Domain Adaptation for Robust Human-Activity Recognition
Chang, Y., Mathur, A., Isopoussu, A., Song, J., & Kawsar, F. (2020) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3380985 · Cited by 138 · cited 1×
“…data from the stream of readings; (iv) feature extraction: where compact, meaningful representations–features–are extracted from aforementioned windows of (preprocessed) sensor data; and (v) classification: where features are classified into activities, typically employing machine learning methods.”
Morales, F. J. O., & Roggen, D. (2016) · Proceedings of the 2016 ACM International Symposium on Wearable Computers · doi:10.1145/2971763.2971764 · Cited by 136 · cited 1×
“…data from the stream of readings; (iv) feature extraction: where compact, meaningful representations–features–are extracted from aforementioned windows of (preprocessed) sensor data; and (v) classification: where features are classified into activities, typically employing machine learning methods.”
O’Connor, L., Brage, S., Griffin, S. J., Wareham, N. J., & Forouhi, N. G. (2015) · British Journal of Nutrition · doi:10.1017/s000711451500269x · Cited by 125 · cited 1×
“It was shown that such models–if trained properly–outperform conventional ML models, including neural networks, by substantial margins [101].”
MetaSense: few-shot adaptation to untrained conditions in deep mobile sensing
Gong, T., Kim, Y., Shin, J., & Lee, S.-J. (2019) · ACM Conference on Embedded Networked Sensor Systems (SenSys) · doi:10.1145/3356250.3360020 · Cited by 123 · cited 1×
“…recognition is yet to be verified, these works provide a good vision of how foundational models for human activity recognition can be: that training with a well-selected set of datasets and modest model size could offer much more practical and effective solutions for this mobile sensing task.”
ClimbAX: skill assessment for climbing enthusiasts
Ladha, C., Hammerla, N., Olivier, P. L., & Plötz, T. (2013) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/2493432.2493492 · Cited by 122 · cited 1×
“Yet, activity recognition goes beyond such “low hanging fruits” and the research community is now attempting more detailed activity assessments such as longitudinal health monitoring [160] including change detection, detailed sports tracking and coaching [100, 106, 124, 166, 212], or quality…”
Self-citationMotionGPT: Human Motion as a Foreign Language
Jiang, B., Chen, X., Wen, L. I. U., Yu, J., Yu, G., & Chen, T. (2023) · neural information processing systems · doi:10.52202/075280-0880 · Cited by 122
Tao, W.-J., Lai, Z.-H., Leu, M., et al. (2018) · EasyChair Preprints · doi:10.29007/bld3 · Cited by 120
Margarito, J., Helaoui, R., Bianchi, A., Sartor, F., & Bonomi, A. (2015) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2015.2471094 · Cited by 118 · cited 1×
“Yet, activity recognition goes beyond such “low hanging fruits” and the research community is now attempting more detailed activity assessments such as longitudinal health monitoring [160] including change detection, detailed sports tracking and coaching [100, 106, 124, 166, 212], or quality…”
ColloSSL: Collaborative Self-Supervised Learning for Human Activity Recognition
Jain, Y., Tang, C. I., Min, C., Kawsar, F., & Mathur, A. (2022) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3517246 · Cited by 113
Self-citationExpectation grammars: leveraging high-level expectations for activity recognition
Minnen, D., Essa, I., & Starner, T. (2003) · 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings. · doi:10.1109/cvpr.2003.1211525 · Cited by 110 · cited 1×
“…for the ambivalence of sensor-based information has been around since the early days of HAR research [15], initially in vision-based approaches [127], with later adoption in sensor-based work (e.g. [140]) that have shown how reasoning about interaction with objects recognized through…”
Assessing the State of Self-Supervised Human Activity Recognition Using Wearables
Haresamudram, H., Essa, I., & Plötz, T. (2022) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3550299 · Cited by 107 · cited 7×
“…recognition is yet to be verified, these works provide a good vision of how foundational models for human activity recognition can be: that training with a well-selected set of datasets and modest model size could offer much more practical and effective solutions for this mobile sensing task.”
InfluentialSelf-citationSuh, S., Rey, V. F., & Lukowicz, P. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2022.110143 · Cited by 107
Self-citationGLOBEM: Cross-Dataset Generalization of Longitudinal Human Behavior Modeling
Xu, X., Liu, X., Zhang, H., Wang, W., Nepal, S., Sefidgar, Y., Seo, W., Kuehn, K. S., Huckins, J. F., Morris, M. E., et al. (2022) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3569485 · Cited by 96
Patient health monitoring system using IoT
Bageshwar, M. R., & Demde, M. (2021) · Materials Today: Proceedings · doi:10.1016/j.matpr.2021.06.188 · Cited by 94
Human Activity Recognition from Wearable Sensor Data Using Self-Attention
Saif, M., Tanjid Hasan Tonmoy, M., Kishor, K. B., Mahbubur Rahman, A. K. M., Ashraful Amin, M., Mohammad, S., Muhammad, A. H. K., & Amin, A. A. (2020) · Frontiers in Artificial Intelligence and Applications · doi:10.3233/faia200236 · Cited by 93 · cited 1×
“The task cannot be too easy, lest the network learns nothing useful by solving it; similarly, the task cannot be excessively difficult to solve, making it too challenging to learn useful representations [76].”
Wang, J., Chen, Y., Gu, Y., Xiao, Y., & Pan, H. (2018) · 2018 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2018.8489106 · Cited by 92 · cited 1×
“By capturing the statistical properties of sensor data, the model generates synthetic sensor data closely resembling the characteristics of the original data.”
COCOA: Cross Modality Contrastive Learning for Sensor Data
Deldari, S., Xue, H., Saeed, A., Smith, D. V., & Salim, F. D. (2022) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3550316 · Cited by 92
Let's (not) stick together: pairwise similarity biases cross-validation in activity recognition
Hammerla, N. Y., & Plötz, T. (2015) · Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing · doi:10.1145/2750858.2807551 · Cited by 91
Self-citationDiffuSeq: Sequence to Sequence Text Generation with Diffusion Models
Gong, S., Li, M., Feng, J., Wu, Z., & Kong, L. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2210.08933 · Cited by 91
Human Daily Activity and Fall Recognition Using a Smartphone’s Acceleration Sensor
Chatzaki, C., Pediaditis, M., Vavoulas, G., & Tsiknakis, M. (2017) · Communications in Computer and Information Science · doi:10.1007/978-3-319-62704-5_7 · Cited by 91 · cited 1×
“…recognition is yet to be verified, these works provide a good vision of how foundational models for human activity recognition can be: that training with a well-selected set of datasets and modest model size could offer much more practical and effective solutions for this mobile sensing task.”
On the role of features in human activity recognition
Haresamudram, H., Anderson, D. V., & Plötz, T. (2019) · Proceedings of the 23rd International Symposium on Wearable Computers · doi:10.1145/3341163.3347727 · Cited by 90 · cited 4×
“…Features: These representations essentially resemble heuristics on capturing certain statistical aspects of the underlying sensor signals [72], including: (i) DC mean of the signal; (ii) its variance; (iii) the correlation (between channels); (iv) signal energy; and (v) frequency-domain…”
InfluentialSelf-citationAdversarial score matching and improved sampling for image generation
Jolicoeur-Martineau, A., Piché-Taillefer, R., Combes, R. T. D., & Mitliagkas, I. (2020) · arXiv · doi:10.48550/arxiv.2009.05475 · Cited by 88
Self-supervised Learning is More Robust to Dataset Imbalance
Liu, H., HaoChen, J. Z., Gaidon, A., & Ma, T. (2021) · arXiv · doi:10.48550/arxiv.2110.05025 · Cited by 84 · cited 1×
“…recognition is yet to be verified, these works provide a good vision of how foundational models for human activity recognition can be: that training with a well-selected set of datasets and modest model size could offer much more practical and effective solutions for this mobile sensing task.”
Transfer Learning in Human Activity Recognition: A SurveyPreprint
Dhekane, S. G., & Plötz, T. (2024) · arXiv.org · doi:10.48550/arxiv.2401.10185 · Cited by 83
Self-citationPersonalized Human Activity Recognition Based on Integrated Wearable Sensor and Transfer Learning
Fu, Z., He, X., Wang, E., Huo, J., Huang, J., & Wu, D. (2021) · Sensors · doi:10.3390/s21030885 · Cited by 79 · cited 1×
“…data from the stream of readings; (iv) feature extraction: where compact, meaningful representations–features–are extracted from aforementioned windows of (preprocessed) sensor data; and (v) classification: where features are classified into activities, typically employing machine learning methods.”
TinyHAR: A Lightweight Deep Learning Model Designed for Human Activity Recognition
Zhou, Y., Zhao, H., Huang, Y., Riedel, T., Hefenbrock, M., & Beigl, M. (2022) · Proceedings of the 2022 ACM International Symposium on Wearable Computers · doi:10.1145/3544794.3558467 · Cited by 78
Improving Deep Learning for HAR with Shallow LSTMs
Bock, M., Hölzemann, A., Moeller, M., & Van Laerhoven, K. (2021) · 2021 International Symposium on Wearable Computers · doi:10.1145/3460421.3480419 · Cited by 74 · cited 1×
“Bock et al. [16], however, found that using a shallower LSTM in the DeepConvLSTM setup is better for many HAR datasets.”
Improving CLIP Training with Language Rewrites
Fan, L., Krishnan, D., Isola, P., Katabi, D., & Tian, Y. (2023) · arXiv · doi:10.48550/arxiv.2305.20088 · Cited by 74
Akbari, A., & Jafari, R. (2019) · ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN) · doi:10.1145/3302506.3310391 · Cited by 73 · cited 1×
“…data from the stream of readings; (iv) feature extraction: where compact, meaningful representations–features–are extracted from aforementioned windows of (preprocessed) sensor data; and (v) classification: where features are classified into activities, typically employing machine learning methods.”
Ling, C. (2020) · doi:10.1145/3381012 · Cited by 72
Li, X., Luo, J., & Younes, R. (2020) · Adjunct Proceedings of the 2020 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2020 ACM International Symposium on Wearable Computers · doi:10.1145/3410530.3414367 · Cited by 71 · cited 2×
“ActivityGAN [113], utilized a convolutional GAN architecture for data augmentation in sensor-based HAR.”
Mathur, A., Zhang, T., Bhattacharya, S., Velickovic, P., Joffe, L., Lane, N. D., Kawsar, F., & Lio, P. (2018) · 2018 17th ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN) · doi:10.1109/ipsn.2018.00048 · Cited by 70 · cited 2×
“The obtained orientations enable the transformation of 3D joint positions into the IMU’s frame-of-reference, facilitating the computation of acceleration and angular velocity.”
Single Motion DiffusionPreprint
Raab, S., Leibovitch, I., Tevet, G., Arar, M., Bermano, A. H., & Cohen‐Or, D. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2302.05905 · Cited by 68
Exploring Contrastive Learning in Human Activity Recognition for Healthcare
Tang, C. I., Perez-Pozuelo, I., Spathis, D., & Mascolo, C. (2020) · arXiv · doi:10.48550/arxiv.2011.11542 · Cited by 67
Self-citationZero-Shot Learning for IMU-Based Activity Recognition Using Video Embeddings
Tong, C., Ge, J., & Lane, N. D. (2021) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3494995 · Cited by 61 · cited 1×
“…recognition is yet to be verified, these works provide a good vision of how foundational models for human activity recognition can be: that training with a well-selected set of datasets and modest model size could offer much more practical and effective solutions for this mobile sensing task.”
IMUGPT 2.0: Language-Based Cross Modality Transfer for Sensor-Based Human Activity Recognition
Leng, Z., Bhattacharjee, A., Rajasekhar, H., Zhang, L., Bruda, E., Kwon, H., & Plötz, T. (2024) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3678545 · Cited by 61
Self-citationChan, S., Hang, Y., Tong, C., Acquah, A., Schonfeldt, A., Gershuny, J., & Doherty, A. (2024) · Scientific Data · doi:10.1038/s41597-024-03960-3 · Cited by 61
Cheng, D., Zhang, L., Bu, C., Wu, H., & Song, A. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.110789 · Cited by 55
SenseGAN: Enabling Deep Learning for Internet of Things with a Semi-Supervised Framework
Yao, S., Zhao, Y., Shao, H., Zhang, C., Zhang, A., Hu, S., Liu, D., Liu, S., Su, L., & Abdelzaher, T. (2018) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3264954 · Cited by 54
A Deep Learning Method for Complex Human Activity Recognition Using Virtual Wearable Sensors
Xiao, F., Pei, L., Chu, L., Zou, D., Yu, W., Zhu, Y., & Li, T. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-69873-7_19 · Cited by 53 · cited 2×
“These works employ generative methods [53, 154] and trajectory-based approaches [104, 105, 193] to extract (virtual) IMU data from videos, expanding the applicability of sensor-based HAR beyond traditional IMU-equipped scenarios.”
Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Loss
HaoChen, J. Z., Wei, C., Gaidon, A., & Ma, T. (2021) · arXiv · doi:10.48550/arxiv.2106.04156 · Cited by 53 · cited 1×
“One such work looked at the use of contrastive predictive coding (CPC) to detect changes in web service traffic and mobile application usage, in addition to human activity recognition [37].”
Plotz, T., Chen, C., Hammerla, N. Y., & Abowd, G. D. (2012) · 2012 16th International Symposium on Wearable Computers · doi:10.1109/iswc.2012.15 · Cited by 51 · cited 1×
“…the chosen timesteps, processing is as follows: (i) with a probability of 80%, the data are set to zero (i.e., the masking is performed); (ii) the data are left unchanged with probability of 10%; and (iii) the data are replaced with a random timestep from within the frame with probability of 10%.”
Self-citationApproaching the Real-World: Supporting Activity Recognition Training with Virtual IMU Data
Kwon, H., Wang, B., Abowd, G. D., & Plötz, T. (2021) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3478096 · Cited by 48
Self-citationMyoGym: introducing an open gym data set for activity recognition collected using myo armband
Koskimäki, H., Siirtola, P., & Röning, J. (2017) · Proceedings of the 2017 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2017 ACM International Symposium on Wearable Computers · doi:10.1145/3123024.3124400 · Cited by 46 · cited 1×
“The Activity Recognition Chain as summarized in [19].”
Cross-Domain HAR: Few Shot Transfer Learning for Human Activity Recognition
Thukral, M., Haresamudram, H., & Ploetz, T. (2023) · arXiv · doi:10.48550/arxiv.2310.14390 · Cited by 45
Self-citationHu, R., Chen, L., Miao, S., & Tang, X. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i5.25743 · Cited by 44
MMTSA: Multi-Modal Temporal Segment Attention Network for Efficient Human Activity Recognition
Gao, Z., Wang, Y., Chen, J., Xing, J., Patel, S., Liu, X., & Shi, Y. (2023) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3610872 · Cited by 44
Rey, V. F., Hevesi, P., Kovalenko, O., & Lukowicz, P. (2019) · Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers · doi:10.1145/3341162.3345590 · Cited by 43 · cited 4×
“Efforts to bridge the gap between video and Inertial Measurement Unit (IMU) data have gained prominence, with recent works addressing the challenge of translating video data into IMU representations [103, 104, 153, 154].”
InfluentialSelf-citationAdversarial Multi-view Networks for Activity Recognition
Bai, L., Yao, L., Wang, X., Kanhere, S. S., Guo, B., & Yu, Z. (2020) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3397323 · Cited by 40 · cited 2×
“Furthermore, such methods [9, 109, 168] may suffer when the feature extractor or generator overly focuses on fooling the discriminator.”
Self-supervised Learning for Accelerometer-based Human Activity Recognition: A Survey
Logacjov, A. (2024) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3699767 · Cited by 40 · cited 1×
“While this tutorial remains highly relevant, its primary focus was on classical machine learning techniques, leaving current trends such as the rise of deep learning models, representation learning with unlabeled data, and more recently, learning from multi-modal data under-explored.”
Leng, Z., Kwon, H., & Ploetz, T. (2023) · Proceedings of the 2023 International Symposium on Wearable Computers · doi:10.1145/3594738.3611361 · Cited by 39
Activity Recognition and Healthier Food Preparation
Plötz, T., Moynihan, P., Pham, C., & Olivier, P. (2011) · Atlantis Ambient and Pervasive Intelligence · doi:10.2991/978-94-91216-05-3_14 · Cited by 39
Self-citationSensor-data augmentation for human activity recognition with time-warping and data masking
Jeong, C. Y., Shin, H. C., & Kim, M. (2021) · Multimedia Tools and Applications · doi:10.1007/s11042-021-10600-0 · Cited by 38 · cited 1×
“Efforts to bridge the gap between video and Inertial Measurement Unit (IMU) data have gained prominence, with recent works addressing the challenge of translating video data into IMU representations [103, 104, 153, 154].”
Co-simulation of human digital twins and wearable inertial sensors to analyse gait event estimation
Uhlenberg, L., Derungs, A., & Amft, O. (2023) · Frontiers in Bioengineering and Biotechnology · doi:10.3389/fbioe.2023.1104000 · Cited by 36 · cited 1×
“That may sound as not obviously being useful for anything beyond text processing, especially not for sensor-based HAR.”
Ciliberto, M., Fortes Rey, V., Calatroni, A., Lukowicz, P., & Roggen, D. (2021) · Frontiers in Computer Science · doi:10.3389/fcomp.2021.792065 · Cited by 35 · cited 1×
“…the chosen timesteps, processing is as follows: (i) with a probability of 80%, the data are set to zero (i.e., the masking is performed); (ii) the data are left unchanged with probability of 10%; and (iii) the data are replaced with a random timestep from within the frame with probability of 10%.”
Self-citationKwon, H., Abowd, G. D., & Plötz, T. (2018) · Proceedings of the 2018 ACM International Symposium on Wearable Computers · doi:10.1145/3267242.3267258 · Cited by 35
Self-citationAdaimi, R., & Thomaz, E. (2022) · Sensors · doi:10.3390/s22186881 · Cited by 35 · cited 1×
“…recognition is yet to be verified, these works provide a good vision of how foundational models for human activity recognition can be: that training with a well-selected set of datasets and modest model size could offer much more practical and effective solutions for this mobile sensing task.”
Patient Health Monitoring System using IOT
Reddy, M. V. K., Achyuth, K. S., & Babu, S. (2022) · International Conference on Smart Structures and Systems (ICSSS) · doi:10.1109/icsss54381.2022.9782193 · Cited by 35
PlÖtz, T. (2022) · ACM Computing Surveys · doi:10.1145/3459666 · Cited by 34
Self-citationWorker Activity Recognition in Manufacturing Line Using Near-Body Electric Field
Suh, S., Rey, V. F., Bian, S., Huang, Y.-C., Rožanec, J. M., Ghinani, H. T., Zhou, B., & Lukowicz, P. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3330372 · Cited by 34
Self-citationLabel-preserving data augmentation for mobile sensor data
Kim, M., & Jeong, C. Y. (2021) · Multidimensional Systems and Signal Processing · doi:10.1007/s11045-020-00731-2 · Cited by 32 · cited 2×
“Kim et al. [98] raised concerns about changing labels when augmenting sensor signals, leading to potential performance degradation due to the similarity of augmented signals to other activity classes.”
Tang, J., He, B., Xu, J., Tan, T., Wang, Z., Zhou, Y., & Jiang, S. (2024) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2024.3370396 · Cited by 31 · cited 1×
“However, there are two things to consider: (1) People write text to describe their experiences in, and perception of the real world.”
Kwon, H., Abowd, G. D., & Plötz, T. (2021) · Sensors · doi:10.3390/s21248337 · Cited by 30 · cited 4×
“Efforts to bridge the gap between video and Inertial Measurement Unit (IMU) data have gained prominence, with recent works addressing the challenge of translating video data into IMU representations [103, 104, 153, 154].”
InfluentialSelf-citationConvBoost: Boosting ConvNets for Sensor-based Activity Recognition
Shao, S., Guan, Y., Zhai, B., Missier, P., & Plötz, T. (2023) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3596234 · Cited by 30
Self-citationAbedin, A., Rezatofighi, S. H., Shi, Q., & Ranasinghe, D. C. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/801 · Cited by 29
Improving CNN-based activity recognition by data augmentation and transfer learning
Kalouris, G., Zacharaki, E. I., & Megalooikonomou, V. (2019) · 2019 IEEE 17th International Conference on Industrial Informatics (INDIN) · doi:10.1109/indin41052.2019.8972135 · Cited by 28 · cited 2×
“The obtained orientations enable the transformation of 3D joint positions into the IMU’s frame-of-reference, facilitating the computation of acceleration and angular velocity.”
Weakly Supervised Multi-Task Representation Learning for Human Activity Analysis Using Wearables
Sheng, T., & Huber, M. (2020) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3397330 · Cited by 28 · cited 1×
“Sheng et al. [163] proposed weakly supervised multi-task representation learning using Siamese networks, mitigating environmental differences through similarity-based multi-task learning.”
Zhou, B., Suh, S., Rey, V. F., Altamirano, C. A. V., & Lukowicz, P. (2022) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3534610 · Cited by 28
Self-citationLarge Language Models are Zero-Shot Recognizers for Activities of Daily Living
Civitarese, G., Fiori, M., Choudhary, P., & Bettini, C. (2024) · arXiv · doi:10.48550/arxiv.2407.01238 · Cited by 28
Investigating Enhancements to Contrastive Predictive Coding for Human Activity Recognition
Haresamudram, H., Essa, I., & Plötz, T. (2023) · 2023 IEEE International Conference on Pervasive Computing and Communications (PerCom) · doi:10.1109/percom56429.2023.10099197 · Cited by 27
Self-citationProceedings of the 2022 ACM International Symposium on Wearable Computers
Kingdom, U. '. T. 2. A. I. J. C. O. P. A. U. C. 2. C. U., Association for Computing Machinery-Digital Library, ACM Special Interest Group on Computer-Human Interaction, ACM Special Interest Group on Mobility of Systems, U., ACM Special Interest Group on Spatial Information, & SIGs, A. (2022) · doi:10.1145/3544794 · Cited by 26
Adversarial Deep Feature Extraction Network for User Independent Human Activity Recognition
Suh, S., Rey, V. F., & Lukowicz, P. (2022) · 2022 IEEE International Conference on Pervasive Computing and Communications (PerCom) · doi:10.1109/percom53586.2022.9762387 · Cited by 25
Self-citationAutoAugHAR: Automated Data Augmentation for Sensor-based Human Activity Recognition
Zhou, Y., Zhao, H., Huang, Y., Röddiger, T., Kurnaz, M., Riedel, T., & Beigl, M. (2024) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3659589 · Cited by 25
Xia, C., & Sugiura, Y. (2022) · Proceedings of the 2022 ACM International Symposium on Wearable Computers · doi:10.1145/3544794.3558460 · Cited by 22
LiftRight: Quantifying strength training performance using a wearable sensor
Milanko, S., & Jain, S. (2020) · Smart Health · doi:10.1016/j.smhl.2020.100115 · Cited by 22 · cited 1×
“In fact, many commercially available wearables such as smart watches include–variants of–HAR as a central service element and even selling point.”
Thukral, M., Dhekane, S. G., Hiremath, S. K., Haresamudram, H., & Ploetz, T. (2024) · arXiv · doi:10.48550/arxiv.2405.12368 · Cited by 21
Self-citationA Study on Diffusion Modelling For Sensor-based Human Activity Recognition
Shao, S., & Sánchez, V. (2023) · International Workshop on Biometrics and Forensics (IWBF) · doi:10.1109/iwbf57495.2023.10157482 · Cited by 20
Haresamudram, H., Beedu, A., Rabbi, M., Saha, S., Essa, I., & Ploetz, T. (2024) · AAAI Conference on Artificial Intelligence · doi:10.48550/arxiv.2408.12023 · Cited by 20
Self-citationTS2ACT: Few-Shot Human Activity Sensing with Cross-Modal Co-Learning
Xia, K., Li, W., Gan, S., & Lu, S. (2023) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3631445 · Cited by 19
On specialized window lengths and detector based human activity recognition
Li, H., Abowd, G. D., & Plötz, T. (2018) · Proceedings of the 2018 ACM International Symposium on Wearable Computers · doi:10.1145/3267242.3267246 · Cited by 19
Self-citationPRimuS: Pretraining IMU Encoders with Multimodal Self-Supervision
Das, A. M., Tang, C. I., Kawsar, F., & Malekzadeh, M. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10888874 · Cited by 19 · cited 1×
“Looking beyond works in re-purposing language and vision foundation models for human activity recognition, significant efforts have been dedicated by researchers to developing foundational models for time-series data.”
Self-citationModel-based data augmentation for user-independent fatigue estimation
Jiang, Y., Malliaras, P., Chen, B. K., & Kulić, D. (2021) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2021.104839 · Cited by 19 · cited 1×
“It goes from seeing them as mere “statistical parrots” to highly speculative discussions about the ability of such systems to reason and even emerge consciousness-like behavior [11].”
T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete RepresentationsPreprint
Zhang, J., Zhang, Y., Cun, X., Huang, S., Zhang, Y., Zhao, H., Lu, H., & Shen, X. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2301.06052 · Cited by 19
Improving Cross-Subject Activity Recognition via Adversarial Learning
Leite, C. F. S., & Xiao, Y. (2020) · IEEE Access · doi:10.1109/access.2020.2993818 · Cited by 18 · cited 1×
“These methods include jittering, scaling, rotation, and random sampling, which have demonstrated efficacy in enhancing the diversity of the training dataset.”
Young, A. D., Ling, M. J., & Arvind, D. K. (2010) · ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN) · doi:10.1145/1791212.1791243 · Cited by 18
Fortes Rey, V., Garewal, K. K., & Lukowicz, P. (2021) · Applied Sciences · doi:10.3390/app11073094 · Cited by 17 · cited 3×
“Virtual IMU data generation has emerged as a viable solution to overcome data scarcity, with cross-modality transfer approaches [53, 103, 104, 153, 154] being instrumental in extracting virtual IMU data from 2D RGB videos of human activities.”
Self-citationZhou, Y., King, T., Zhao, H., Huang, Y., Riedel, T., & Beigl, M. (2024) · Proceedings of the 2024 ACM International Symposium on Wearable Computers · doi:10.1145/3675095.3676624 · Cited by 17 · cited 2×
“For example, Contrastive Predictive Coding (CPC) [76, 136] utilizes a contrastive future prediction task, and it was found that predicting one or two timesteps in the future is (too) easy and results in poor performance, whereas predicting multiple timesteps into the future is much more complex,…”
PaliGemma: A versatile 3B VLM for transfer
Beyer, L., Steiner, A., Pinto, A. S., Kolesnikov, A., Wang, X., Salz, D., Neumann, M., Alabdulmohsin, I., Tschannen, M., Bugliarello, E., et al. (2024) · arXiv · doi:10.48550/arxiv.2407.07726 · Cited by 17 · cited 1×
“In abstract terms, the various approaches can be summarized as consisting of two steps: (i) creation of a representation of the activities for which we want to generate data for; and (ii) the translation of that representation into synthetic sensor data.”
Augmented Adversarial Learning for Human Activity Recognition with Partial Sensor Sets
Kang, H., Huang, Q., & Zhang, Q. (2022) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3550285 · Cited by 16 · cited 1×
“However, acquiring high-quality training data for HAR, particularly from wearable sensors, presents unique challenges.”
If only we had more data!: Sensor-Based Human Activity Recognition in Challenging Scenarios
Plötz, T. (2023) · 2023 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops) · doi:10.1109/percomworkshops56833.2023.10150267 · Cited by 16 · cited 1×
“Furthermore, labeling sensor data is much more difficult than labeling images [144].”
Self-citationLLaSA: Large Multimodal Agent for Human Activity Analysis Through Wearable SensorsPreprint
Shouborno, S. A. I., Khan, M. N. H., Biswas, S., & Islam, B. (2024) · arXiv.org · doi:10.48550/arxiv.2406.14498 · Cited by 16
Ray, L. S. S., Zhou, B., Suh, S., & Lukowicz, P. (2023) · IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops) · doi:10.1109/percomworkshops56833.2023.10150221 · Cited by 16
Self-citationRecognizing Hospital Care Activities with a Coat Pocket Worn Smartphone
Bahle, G., Gruenerbl, A., Bignotti, E., Zeni, M., Giunchiglia, F., & Lukowicz, P. (2014) · Proceedings of the 6th International Conference on Mobile Computing, Applications and Services · doi:10.4108/icst.mobicase.2014.257777 · Cited by 15 · cited 1×
“Thus for example in [7] it has been shown that complex activities in a nursing scenario can be recognized using a mobile phone in a pocket of a loose nurse’s coat (which is a very poor source of information) by integrating high-level semantic knowledge about the activities in the recognition…”
Self-citation
Research works printed but not linked: 14
- 10.5555/1622407.1622416 · doi:10.5555/1622407.1622416
- Spyros Gidaris, Andrei Bursuc, Jean-Baptiste Alayrac, Adrià Recasens, Mathilde Caron, Olivier Hénaff, Aäron van den Oord, and Relja Arandjelović. 2021. CVPR 2021 Tutorial on Self-Supervised Representation Learning --- gidariss.github.io. https://gidariss.github.io/self-supervised-learning-cvpr2021/. [Accessed 28-10-2024].
- Ian Goodfellow, Yoshua Bengio, and Aaron Courville. 2016. Deep Learning. MIT Press. http://www.deeplearningbook.org.
- Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. Denoising Diffusion Probabilistic Models. In Advances in Neural Information Processing Systems H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (Eds.), Vol. 33. Curran Associates, Inc., 6840--6851. https://proceedings.neurips.cc/paper_files/paper/2020/file/4c5bcfec8584af0d967f1ab10179ca4b-Paper.pdf
- Yifan Hu. 2023. BSDGAN: Balancing Sensor Data Generative Adversarial Networks for Human Activity Recognition. In 2023 International Joint Conference on Neural Networks (IJCNN). IEEE, 1--8.
- Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems 25 (2012).
- Lin Liao, Dieter Fox, and Henry Kautz. 2005. Location-based activity recognition. Advances in neural information processing systems 18 (2005).
- Hiroki Ohashi, M Al-Nasser, Sheraz Ahmed, Takayuki Akiyama, Takuto Sato, Phong Nguyen, Katsuyuki Nakamura, and Andreas Dengel. 2017. Augmenting wearable sensor data with physical constraint for DNN-based human-action recognition. In ICML 2017 times series workshop. 6--11.
- Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learning. PMLR, 8748--8763.
- Milyun Ni'ma Shoumi and Sozo Inoue. 2024. Leveraging the Large Language Model for Activity Recognition: A Comprehensive Review. International Journal of Activity and Behavior Computing 2024, 2 (2024), 1--27.
- Jiaming Song, Chenlin Meng, and Stefano Ermon. 2021. Denoising Diffusion Implicit Models. In International Conference on Learning Representations. https://openreview.net/forum?id=St1giarCHLP
- Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems 30 (2017).
- Minkai Xu, Lantao Yu, Yang Song, Chence Shi, Stefano Ermon, and Jian Tang. 2022. GeoDiff: A Geometric Diffusion Model for Molecular Conformation Generation. In International Conference on Learning Representations. https://openreview.net/forum?id=PzcvxEMzvQC
- Jinsung Yoon, Daniel Jarrett, and Mihaela Van der Schaar. 2019. Time-series generative adversarial networks. Advances in neural information processing systems 32 (2019).
Cited by
Liu, X., Cheng, D., Fu, Z., Zhang, L., Wu, H., & Song, A. (2026) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2026.3664424 · Cited by 16
Ayub, M., & El-Alfy, E.-S. M. (2025) · IEEE Access · doi:10.1109/access.2025.3631392 · Cited by 6
“The study [47] surveyed the current state-of-the-art techniques in HAR, identified the challenges, and proposed future research options.”
Ubiquitous Lingering Technologies: What's Left Behind by the Past “Proximate Futures”?
Lu, A. J., & Sun, Y. (2026) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3789676 · Cited by 4
Bian, S., Liu, M., Ray, L. S. S., Zhou, B., Guo, B., Yu, Z., Plötz, T., Lukowicz, P., Yuan, S., & Rey, V. F. (2026) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3810230 · Cited by 16
Self-citationCOMODO: Cross-Modal Video-to-IMU Distillation for Efficient Egocentric Human Activity Recognition
Chen, B., Wongso, W., Li, Z., Khaokaew, Y., Xue, H., & Salim, F. D. (2026) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3810218 · Cited by 1
“A key challenge is that annotating IMU data is much more labor-intensive and costly than labeling images or text, as it often requires external reference signals and precise temporal alignment [21].”
Metadata
- Authors
- Haresamudram, H., Tang, C. I., Suh, S., Lukowicz, P., & Plötz, T.
- Affiliations
- Georgia Institute of Technology (US)
- Nokia (United Kingdom) (GB)
- Korea University (KR)
- German Research Centre for Artificial Intelligence (DE)
- Venue
- Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies
- Published
- 2025-06-09
- Volume
- 9
- Issue
- 2
- Pages
- 1-44
- ISSN
- 2474-9567 (ISSN-L)
- OpenAlex
- W4411403377
- Semantic Scholar
- 72ad9f2cfb7d3aa3ffbc250be956cd5e05f405e6
- Cited by
- 33
- References
- 156
- Citations by year
- 2026: 26 · 2025: 7
- FWCI
- 11.47
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Horizon 2020 Framework Programme (52026 HumanE- AI-Net project)
- NSF (IS2112633)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Physical Activity and Health
- Physical Sciences › Engineering › Biomedical Engineering › Gait Recognition and Analysis
- Keywords
- human activity recognition, wearable sensors, sensor-based activity recognition, wearable computing, feature engineering, foundation models, benchmark performance
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-05, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-05 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.