If only we had more data!: Sensor-Based Human Activity Recognition in Challenging Scenarios
The DOI matches its Crossref record; 71 of the 71 references it cites were found in a registry or an index.
Abstract
With the proliferation of miniaturized movement sensing capabilities, human activity recognition using wearables and other forms of pervasive sensing (HAR) has seen an enormous boost in both research and practical deployments. Prominent application domains are health assessments, wellbeing monitoring, and novel interaction paradigms. Arguably, and especially when compared to video based activity recognition, the use of movement sensors such as inertial measurement units has many practical advantages including mobility, more direct movement capturing, and less privacy invasive sensing. However, those advantages come with a cost with regards to the design of machine learning based pipelines for the analysis of the sensor data streams, i.e., multi-variate time series data. Probably the biggest challenge lies in the absence of (very) large, labeled datasets that can readily be used for developing and training complex analysis models. In this paper I explore the challenges for deriving machine learning based systems for sensor-based human activity recognition. My focus is on: i) the role of representation learning in modern HAR systems that overcome the need for explicit feature engineering; ii) self-supervised methods for most effective use of small, labeled datasets; and iii) cross-modality transfer. I conclude with an outlook of next steps in the field that promise to lead to increased robustness and more flexible applicability of contemporary, machine-learnlng based HAR systems.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Furthermore, labeling sensor data is much more difficult than labeling images [152].”
From Past, Present, and Future of Sensor-based Human Activity Recognition Using Wearables: A Surveying Tutorial on a Still Challenging Task (2025) · cites this paper as [152]
In the citing paper
“While collecting and curating large labeled datasets could help alleviate this limitation, it is also costly.”
From Siamese masked reconstruction with temporal rotation consistency for inertial-based human activity recognition (2026) · cites this paper as [19]
In the citing paper
“Despite substantial improvements in recognition accuracy, supervised HAR methods often require large labeled datasets and may generalise poorly under realistic deployment conditions.”
From Siamese masked reconstruction with temporal rotation consistency for inertial-based human activity recognition (2026) · cites this paper as [19]
In the citing paper
“In inertial-based HAR, this shift is produced by the way different individuals implement the same activity (inter-person variability), the sensor position (placement variability), and hardware differences between sensors18,19,37, among others.”
From Siamese masked reconstruction with temporal rotation consistency for inertial-based human activity recognition (2026) · cites this paper as [19]
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173 · cited 1×
“SSL methods in general have shown great promise across a range of domains, including computer vision [31], natural language processing [32]–[34], and speech processing [35]–[37].”
BackgroundBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 1×
“SSL methods in general have shown great promise across a range of domains, including computer vision [31], natural language processing [32]–[34], and speech processing [35]–[37].”
BackgroundMomentum Contrast for Unsupervised Visual Representation Learning
He, K., Fan, H., Wu, Y., Xie, S., & Girshick, R. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00975 · Cited by 13,347
A simple framework for contrastive learning of visual representations
Chen, T., Kornblith, S., Norouzi, M., & Hinton, G. (2020) · arXiv · doi:10.48550/arxiv.2002.05709 · Cited by 10,857 · cited 1×
“Methods such as SimCLR [42], MoCo [43], SimSiam [44], and Barlow Twins [45] have (almost) universally applicable frameworks that allow for easy extension into other domains, including sensor-based HAR.”
MethodLarge-Scale Video Classification with Convolutional Neural Networks
Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., & Fei-Fei, L. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.223 · Cited by 6,354 · cited 1×
“For example, video based activity recognition is a mature field [54] for which very large databases have been collected and annotated , e.g. [55], [56].”
BackgroundBootstrap Your Own Latent: A New Approach to Self-Supervised Learning
Grill, J.-B., Strub, F., Altché, F., Tallec, C., Richemond, P. H., Buchatskaya, E., Doersch, C., Pires, B. A., Guo, Z. D., Azar, M. G., et al. (2020) · arXiv · doi:10.48550/arxiv.2006.07733 · Cited by 3,846 · cited 1×
“In [46] the SimCLR framework was applied to activity recognition for healthcare, ditto for [47] and [48].”
MethodExploring Simple Siamese Representation Learning
Chen, X., & He, K. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01549 · Cited by 3,725 · cited 2×
“Methods such as SimCLR [42], MoCo [43], SimSiam [44], and Barlow Twins [45] have (almost) universally applicable”
MethodDeep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition
Ordóñez, F., & Roggen, D. (2016) · Sensors · doi:10.3390/s16010115 · Cited by 2,761 · cited 1×
“The success of end-to-end models such as DeepConvLSTM [15] then seemed to justify an even more drastic paradigm shift from optimizing individual components of the ARC, even if including learning representations as in [26], to fully automated approaches.”
BackgroundUnsupervised Representation Learning by Predicting Image Rotations
Gidaris, S., Singh, P., & Komodakis, N. (2018) · arXiv · doi:10.48550/arxiv.1803.07728 · Cited by 2,477 · cited 1×
“SSL methods in general have shown great promise across a range of domains, including computer vision [31], natural language processing [32]–[34], and speech processing [35]–[37].”
Background
Research works printed but not linked: 3
- radford, 2019, Language models are unsupervised multitask learners, OpenAIRE blog, 1, 9
- radford, 2018, Improving language understanding by generative pre-training
- 10.1145/3550299 · DOI · Paper page
Cited by
IMUGPT 2.0: Language-Based Cross Modality Transfer for Sensor-Based Human Activity Recognition
Leng, Z., Bhattacharjee, A., Rajasekhar, H., Zhang, L., Bruda, E., Kwon, H., & Plötz, T. (2024) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3678545 · Cited by 61
“HAR finds extensive application in various domains, including fitness tracking [15, 33], health monitoring [7, 46], sign language recognition [49, 54], and in identifying critical events like vehicular accidents and falls [39, 48].”
Self-citationHaresamudram, H., Tang, C. I., Suh, S., Lukowicz, P., & Plötz, T. (2025) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3729467 · Cited by 33
“Furthermore, labeling sensor data is much more difficult than labeling images [144].”
Self-citationCheng, D., Zhang, L., Qin, L., Wang, S., Wu, H., & Song, A. (2024) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2024.3373019 · Cited by 31
Bukit, T. A., Pillado, E. P. B., Yahya, B. N., & Lee, S. (2025) · Applied Soft Computing · doi:10.1016/j.asoc.2025.113793 · Cited by 8
Emotion Recognition on the Go: Utilizing Wearable IMUs for Personalized Emotion Recognition
Leng, Z., Jung, M., Hwang, S., Oh, S., Zhang, L., Plötz, T., & Kim, K. K. (2024) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/3675094.3678452 · Cited by 10
“In order to develop a wearable-based AER system, it is essential to gather IMU data for various emotional states; however, this task is notably labor-intensive, time-consuming, can often times be also impractical due to privacy reasons [29].”
Self-citation
Metadata
- Authors
- Plötz, T.
- Affiliations
- Georgia Institute of Technology (US)
- Corresponding author
- Plötz, Thomas
- Venue
- 2023 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops)
- Publisher
- IEEE
- Published
- 2023-03-13
- Pages
- 565-570
- OpenAlex
- W4381746737
- Cited by
- 16
- References
- 71
- Citations by year
- 2026: 3 · 2025: 7 · 2024: 6
- FWCI
- 1.71
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Physical Sciences › Computer Science › Signal Processing › Time Series Analysis and Forecasting
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Keywords
- human activity recognition, wearable sensors, inertial measurement unit, sensor data streams, representation learning, self-supervised learning, small labeled datasets, cross-modality transfer, health monitoring, well-being monitoring, data scarcity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: If only we had more data!: Sensor-Based Human Activity Recognition in Challenging Scenarios. https://app.cochat.ai/papers/10.1109/percomworkshops56833.2023.10150267
- The paper
- Plötz, T. (2023). If only we had more data!: Sensor-Based Human Activity Recognition in Challenging Scenarios. 2023 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops). https://doi.org/10.1109/percomworkshops56833.2023.10150267
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.