On the role of features in human activity recognition
The DOI matches its Crossref record; 41 of the 43 references it cites were found in a registry or an index.
Abstract
Traditionally, the sliding window based activity recognition chain (ARC) has been dominating practical applications, in which features are carefully optimized towards scenario specifics. Recently, end-to-end, deep learning methods, that do not discriminate between representation learning and classifier optimization, have become very popular also for HAR using wearables, promising "out-of-the-box" modeling with superior recognition capabilities. In this paper, we revisit and analyze specifically the role feature representations play in HAR using wearables. In a systematic exploration we evaluate eight different feature extraction methods, including conventional heuristics and recent representation learning methods, and assess their capabilities for effective activity recognition on five benchmarks. Optimized feature learning integrated into the conventional ARC leads to comparable if not better recognition results as if using end-to-end learning methods, while at the same time offering practitioners more flexibility to optimize their systems towards specifics of wearables and their constraints and limitations.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“As shown by Haresamudram et al. [76], these features are not computationally intensive and can be computed on the fly on many wearable devices, even those with severe resource limitations.”
From Past, Present, and Future of Sensor-based Human Activity Recognition Using Wearables: A Surveying Tutorial on a Still Challenging Task (2025) · cites this paper as Haresamudram et al. [76]
In the citing paper
“In such cases distribution-based features (e.g., ECDF [72]) are a surprisingly strong option and can be computed on-the-fly [76].”
From Past, Present, and Future of Sensor-based Human Activity Recognition Using Wearables: A Surveying Tutorial on a Still Challenging Task (2025) · cites this paper as [76]
In the citing paper
“In contrast, previous studies applied self-supervised training and fine-tuning only on the same data sources24,27,40, making it necessary to pre-train on every new dataset in practical applications.”
From Self-supervised learning for human activity recognition using 700,000 person-days of wearable data (2024) · cites this paper as [24]
In the citing paper
“Recent work towards such unsupervised pre-training has gone beyond the early introduction using Restricted Boltzmann Machines (RBMs) [63], involving (variants of) autoencoders [31, 74], and self-supervision [32, 69].”
From Contrastive Predictive Coding for Human Activity Recognition (2021) · cites this paper as [31]
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 1×
“In a recurrent autoencoder, Vanilla RNNs or more powerful variants such as long short-term memory networks (LSTM) [22], or gated recurrent units (GRU) [8] are used for both encoder and decoder [40].”
MethodA fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336 · cited 2×
“While we study the performance of both unsupervised and supervised methods, special emphasis is placed on unsupervised methods, as they have the advantage of automatically deriving generalizable features from unlabelled data [21, 37].”
Background(2016) · Cited by 11,679
Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation
Powers, D. M. W. (2020) · arXiv · doi:10.48550/arxiv.2010.16061 · Cited by 5,440
WaveNet: a generative model for raw audio
Oord, A. V. D., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1609.03499 · Cited by 4,911 · cited 1×
“They have been used, e.g., for generating raw audio [ 42].”
MethodDeep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition
Ordóñez, F., & Roggen, D. (2016) · Sensors · doi:10.3390/s16010115 · Cited by 2,761 · cited 5×
“We also compare overall classification capabilities of the explored feature representations within the ARC paradigm [6] to a state-of-the-art end-to-end HAR system that does not separate representation learning from classifier training (DeepConvLSTM [34]).”
BackgroundMethodInfluentialSelf-Supervised Visual Feature Learning With Deep Neural Networks: A Survey
Jing, L., & Tian, Y. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.2992393 · Cited by 2,036 · cited 1×
“They are utilized to learn features by predicting geometric transformations, and solving jigsaw puzzles [24].”
MethodA tutorial on human activity recognition using body-worn inertial sensors
Bulling, A., Blanke, U., & Schiele, B. (2014) · ACM Computing Surveys · doi:10.1145/2499621 · Cited by 1,668 · cited 4×
“The standard activity recognition chain (ARC, [6]) defines a series of processing steps, where each step has a clearly defined goal.”
BackgroundMethodInfluentialDeconvolution and Checkerboard Artifacts
Odena, A., Dumoulin, V., & Olah, C. (2016) · Distill · doi:10.23915/distill.00003 · Cited by 1,640 · cited 1×
“The decoder inverts the encoding operations step-by-step including convolution, upsampling and interpolation, reshaping, and padding operations to match the sizes of the corresponding encoder convolution blocks [33].”
MethodExperiencing SAX: A novel symbolic representation of time series
Lin, J., Keogh, E., Wei, L., & Lonardi, S. (2007) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-007-0064-z · Cited by 1,618
Introducing a New Benchmarked Dataset for Activity Monitoring
Reiss, A., & Stricker, D. (2012) · 2012 16th International Symposium on Wearable Computers · doi:10.1109/iswc.2012.13 · Cited by 1,456 · cited 1×
“PAMAP2 contains a total of 12 activities of daily living such as domestic activities, and various sportive exercises (nordic walking, running, etc) [39].”
Background(2015) · Cited by 989
Convolutional Neural Networks for Human Activity Recognition using Mobile Sensors
Zeng, M., Nguyen, L. T., Yu, B., Mengshoel, O. J., Zhu, J., Wu, P., & Zhang, J. (2014) · Proceedings of the 6th International Conference on Mobile Computing, Applications and Services · doi:10.4108/icst.mobicase.2014.257786 · Cited by 876 · cited 2×
“Similarly, the authors in [46] designed a CNN, which outperformed other representations based on distributions, PCA, statistical metrics, and fully connected neural networks.”
BackgroundThe Opportunity challenge: A benchmark database for on-body sensor-based activity recognition
Chavarriaga, R., Sagha, H., Calatroni, A., Digumarti, S. T., Tröster, G., Millán, J. D. R., & Roggen, D. (2013) · Pattern Recognition Letters · doi:10.1016/j.patrec.2012.12.014 · Cited by 742 · cited 1×
“Opportunity Opportunity contains recordings from four participants wearing a range of sensors while pursuing kitchen routines [7].”
BackgroundWearable Assistant for Parkinson’s Disease Patients With the Freezing of Gait Symptom
Bachlin, M., Plotnik, M., Roggen, D., Maidan, I., Hausdorff, J. M., Giladi, N., & Troster, G. (2010) · IEEE Transactions on Information Technology in Biomedicine · doi:10.1109/titb.2009.2036165 · Cited by 693 · cited 1×
“This dataset contains data from ten subjects with Parkinson’s Disease, who experience freezing of gait (FoG) in daily life [4].”
MethodDeep, Convolutional, and Recurrent Models for Human Activity Recognition Using Wearables
Hammerla, N. Y., Halloran, S., & Ploetz, T. (2016) · arXiv · doi:10.48550/arxiv.1604.08880 · Cited by 606 · cited 3×
“For training and evaluation, we employ the same protocol as [18]: The second run from participant 1 is used for validation, while runs 4 and 5 from participants 2 and 3 are used for test.”
MethodResultInfluentialPreprocessing techniques for context recognition from accelerometer data
Figo, D., Diniz, P. C., Ferreira, D. R., & Cardoso, J. M. P. (2010) · Personal and Ubiquitous Computing · doi:10.1007/s00779-010-0293-9 · Cited by 578 · cited 1×
“A wealth of heuristics has been developed that aim at extracting either time-domain features, such as statistical moments, or spectral features that directly encode frequency characteristics [9].”
BackgroundUSC-HAD: a daily activity dataset for ubiquitous activity recognition using wearable sensors
Zhang, M., & Sawchuk, A. A. (2012) · Proceedings of the 2012 ACM Conference on Ubiquitous Computing · doi:10.1145/2370216.2370438 · Cited by 554 · cited 1×
“The USC-HAD dataset [47] was collected on the MotionN-ode sensing platform and consists of data from 14 subjects.”
MethodEnsembles of Deep LSTM Learners for Activity Recognition using Wearables
Guan, Y., & Plötz, T. (2017) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3090076 · Cited by 511 · cited 2×
“The wearables community also has adopted such techniques and remarkable recognition results on benchmark dataset can now be achieved–offline–using variants of end-to-end learning methods, e.g., [15, 34, 46].”
BackgroundMethodSelf-citationWearable Activity Tracking in Car Manufacturing
Stiefmeier, T., Roggen, D., Ogris, G., Lukowicz, P., & Tr, G. (2008) · IEEE Pervasive Computing · doi:10.1109/mprv.2008.40 · Cited by 367 · cited 1×
“Skoda This dataset was recorded in a manufacturing scenario aiming to recognize activities of assembly-line workers in a car production environment [41].”
MethodAnalyzing features for activity recognition
Huynh, T., & Schiele, B. (2005) · Proceedings of the 2005 joint conference on Smart objects and ambient intelligence: innovative context-aware services: usages and technologies · doi:10.1145/1107548.1107591 · Cited by 339
Feature Learning for Activity Recognition in Ubiquitous Computing
Plötz, T., Hammerla, N., & Olivier, P. (2011) · Newcastle University · doi:10.25405/ncl.34175319 · Cited by 337
Self-citationTo recognize shapes, first learn to generate images
Hinton, G. E. (2007) · Progress in Brain Research · doi:10.1016/s0079-6123(06)65034-6 · Cited by 309 · cited 1×
“Generative modelling assumes that the characteristics of the data can be discovered by learning how to generate them, and that sub-sets of the characteristics are then suitable for differentiating between classes [20].”
BackgroundDevice-Free Wireless Localization and Activity Recognition: A Deep Learning Approach
Wang, J., Zhang, X., Gao, Q., Yue, H., & Wang, H. (2017) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2016.2635161 · Cited by 255 · cited 2×
“In [43] and [2], vanilla (or stacked) autoencoders have already been studied.”
BackgroundBhattacharya, S., & Lane, N. D. (2016) · Proceedings of the 14th ACM Conference on Embedded Networked Sensor Systems CD-ROM · doi:10.1145/2994551.2994564 · Cited by 236 · cited 1×
“…the performance boost comes with the price of requiring substantial computational resources (even though considerable progress has been made in the efficient deployment of deep learning models on resource-constrained devices [1, 5, 11, 26]), and considerable training sets of annotated sample data.”
BackgroundSqueezing Deep Learning into Mobile and Embedded Devices
Lane, N. D., Bhattacharya, S., Mathur, A., Georgiev, P., Forlivesi, C., & Kawsar, F. (2017) · IEEE Pervasive Computing · doi:10.1109/mprv.2017.2940968 · Cited by 224 · cited 1×
“…the performance boost comes with the price of requiring substantial computational resources (even though considerable progress has been made in the efficient deployment of deep learning models on resource-constrained devices [1, 5, 11, 26]), and considerable training sets of annotated sample data.”
BackgroundZeng, M., Gao, H., Yu, T., Mengshoel, O. J., Langseth, H., Lane, I., & Liu, X. (2018) · Proceedings of the 2018 ACM International Symposium on Wearable Computers · doi:10.1145/3267242.3267286 · Cited by 185 · cited 1×
“The application of recurrent neural networks for HAR is extended in [45], where continuous attention mechanisms over the sensory channels as well as time are developed to improve the performance.”
BackgroundPD Disease State Assessment in Naturalistic Environments Using Deep Learning
Hammerla, N., Fisher, J., Andras, P., Rochester, L., Walker, R., & Ploetz, T. (2015) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v29i1.9484 · Cited by 179 · cited 1×
“…focus on compact signal representations, thereby minimizing reconstruction loss [19]; and iii) Learned features , that derive representations directly from raw sensor data themselves, typically using unsupervised or supervised learning methods and dimensionality reduction techniques [2, 17, 37].”
BackgroundOn attention models for human activity recognition
Murahari, V. S., & Plötz, T. (2018) · Proceedings of the 2018 ACM International Symposium on Wearable Computers · doi:10.1145/3267242.3267287 · Cited by 157 · cited 1×
“Improvements to the DeepConvLSTM have been proposed in [32], where an attention mechanism is added on the long short-term memory (LSTM) network to determine the ‘important’ time steps.”
BackgroundSelf-citationActivity and Gait Recognition with Time-Delay Embeddings
Frank, J., Mannor, S., & Precup, D. (2010) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v24i1.7724 · Cited by 151 · cited 1×
“Alternatively, direct encodings of temporal aspects of the sensor data have been used, e.g., through time-delay embedding [10], or through discretization of the time-series (e.g., [28, 30]).”
MethodProc. Advances in Neural Information Processing Systems (NIPS '08)
Ting, J.-A., Kalakrishnan, M., Vijayakumar, S., & Stefan Schaal, A. (2008) · Cited by 143
Hammerla, N. Y., Kirkham, R., Andras, P., & Ploetz, T. (2013) · Proceedings of the 2013 International Symposium on Wearable Computers · doi:10.1145/2493988.2494353 · Cited by 138 · cited 5×
“…completely abstract away from the application domain and instead focus on compact signal representations, thereby minimizing reconstruction loss [19]; and iii) Learned features , that derive representations directly from raw sensor data themselves, typically using unsupervised or supervised…”
BackgroundMethodInfluentialAn effective deep autoencoder approach for online smartphone-based human activity recognition
Almaslukh, B., Artoli, A. M., & Al-Muhtadi, J. F. (2017) · Cited by 132
Deep Learning for Human Activity Recognition in Mobile Computing
Plotz, T., & Guan, Y. (2018) · Computer · doi:10.1109/mc.2018.2381112 · Cited by 99 · cited 1×
“Notably, the focus of model developments here is often on recognition performance alone, ignoring challenges inherent to wearable computing in terms of availability and quality of annotated training data, and substantial resource constraints [35].”
BackgroundBootstrapping Personalised Human Activity Recognition Models Using Online Active Learning
Miu, T., Missier, P., & Plotz, T. (2015) · 2015 IEEE International Conference on Computer and Information Technology; Ubiquitous Computing and Communications; Dependable, Autonomic and Secure Computing; Pervasive Intelligence and Computing · doi:10.1109/cit/iucc/dasc/picom.2015.170 · Cited by 54 · cited 1×
“Dependence on amount of training data: The size of the datasets in HAR is generally smaller than in other domains, which is reasoned by practicalities of how data from wearables is recorded and annotated [31].”
BackgroundSelf-citationActivity Recognition and Healthier Food Preparation
Plötz, T., Moynihan, P., Pham, C., & Olivier, P. (2011) · Atlantis Ambient and Pervasive Intelligence · doi:10.2991/978-94-91216-05-3_14 · Cited by 39
Self-citationKwon, H., Abowd, G. D., & Plötz, T. (2018) · Proceedings of the 2018 ACM International Symposium on Wearable Computers · doi:10.1145/3267242.3267258 · Cited by 35 · cited 1×
“Distribution based approaches represent the state-of-the-art for conventional feature extraction in HAR, where quan-tiles of the (inverted) empirical cumulative density function represent sensor readings within an analysis frame [19, 25].”
BackgroundSelf-citationAccelerating Mobile Audio Sensing Algorithms through On-Chip GPU Offloading
Georgiev, P., Lane, N. D., Mascolo, C., & Chu, D. C. Y. (2017) · International Conference on Mobile Systems, Applications, and Services · doi:10.1145/3081333.3081358 · Cited by 27 · cited 1×
“…the performance boost comes with the price of requiring substantial computational resources (even though considerable progress has been made in the efficient deployment of deep learning models on resource-constrained devices [1, 5, 11, 26]), and considerable training sets of annotated sample data.”
BackgroundOn specialized window lengths and detector based human activity recognition
Li, H., Abowd, G. D., & Plötz, T. (2018) · Proceedings of the 2018 ACM International Symposium on Wearable Computers · doi:10.1145/3267242.3267246 · Cited by 19 · cited 2×
“The results are very different for Skoda, a dataset with fewer activities and less variability overall [27].”
ResultSelf-citationActivity recognition in naturalistic environments using body-worn sensors
Hammerla, N. (2015) · Cited by 4
Using Pre-trained Full-Precision Models to Speed Up Training Binary Networks For Mobile Devices
Alizadeh, M., & Lane, N. D. (2018) · International Conference on Mobile Systems, Applications, and Services · doi:10.1145/3210240.3210821 · Cited by 1 · cited 1×
“…the performance boost comes with the price of requiring substantial computational resources (even though considerable progress has been made in the efficient deployment of deep learning models on resource-constrained devices [1, 5, 11, 26]), and considerable training sets of annotated sample data.”
Background
Research works printed but not linked: 11
- Amiriparian S., Proc. DCASE 2017 Workshop.
- Chung J., Proc. Int. Conf. Machine Learning (ICML).
- Frank J., Proc. AAAI Conf. on Art. Intelligence (AAAI).
- A. Graves and M. Ranzato. 2018. Unsupervised Deep Learning Tutorial - Part 1 NeurIPS 2018. https://ranzato.github.io/publications/tutorial_deep_unsup_learning_part1_NeurIPS2018.pdf A. Graves and M. Ranzato. 2018. Unsupervised Deep Learning Tutorial - Part 1 NeurIPS 2018. https://ranzato.github.io/publications/tutorial_deep_unsup_learning_part1_NeurIPS2018.pdf
- A. Graves and M. Ranzato. 2018. Unsupervised Deep Learning Tutorial - Part 2 NeurIPS 2018. https://ranzato.github.io/publications/tutorial_deep_unsup_learning_part2_NeurIPS2018.pdf A. Graves and M. Ranzato. 2018. Unsupervised Deep Learning Tutorial - Part 2 NeurIPS 2018. https://ranzato.github.io/publications/tutorial_deep_unsup_learning_part2_NeurIPS2018.pdf
- Hammerla N., Proc. AAAI Conf. on Art. Intelligence (AAAI).
- Hammerla N., Proc. Int. Joint Conf. on Art. Intelligence (IJCAI).
- Plötz T., Proc. Int. Joint Conf. on Art. Intelligence (IJCAI).
- Srivastava N., Proc. Int. Conf. Machine Learning (ICML).
- Yang J., Proc. Int. Joint Conf. on Art. Intelligence (IJCAI).
- Zeng M., Proc. Int. Symp. Wearable Computing (ISWC).
Cited by
Self-supervised learning for human activity recognition using 700,000 person-days of wearable data
Yuan, H., Chan, S., Creagh, A. P., Tong, C., Acquah, A., Clifton, D. A., & Doherty, A. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01062-3 · Cited by 166
“Current human activity recognition (HAR) models typically rely on manual feature engineering (Twomey et al., 2018; Haresamudram et al., 2019) partly due to the very limited size of existing labelled datasets.”
Contrastive Predictive Coding for Human Activity Recognition
Haresamudram, H., Essa, I., & Plötz, T. (2021) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3463506 · Cited by 150
“There is not yet a consensus on the gold standard feature representation for HAR using wearables [31].”
BackgroundMethodInfluentialSelf-citationMasked reconstruction based self-supervision for human activity recognition
Haresamudram, H., Beedu, A., Agrawal, V., Grady, P. L., Essa, I., Hoffman, J., & Plötz, T. (2020) · Proceedings of the 2020 International Symposium on Wearable Computers · doi:10.1145/3410531.3414306 · Cited by 146
“For USC-HAD, we followed the protocol from [12] with data from participants 1 − 10 for training, from participants 11 and 12 for validation, and from participants 13 and 14 for testing.”
BackgroundMethodResultInfluentialSelf-citationAbedin, A., Ehsanpour, M., Shi, Q., Rezatofighi, H., & Ranasinghe, D. C. (2021) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3448083 · Cited by 107
“bution imbalance for Opportunity and Hospital datasets. 4.2 Unified Evaluation Protocol To ensure a fair comparison, we directly adopt the evaluation protocol and metrics used in the recent literature [13, 37, 15, 47]. Where possible, sensor data are down-sampled to 33 Hz to achieve a consistent temporal resolution wit…”
MethodInfluentialAssessing the State of Self-Supervised Human Activity Recognition Using Wearables
Haresamudram, H., Essa, I., & Plötz, T. (2022) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3550299 · Cited by 107
“Similar to the original papers, the learning rate for classifiers for CPC, Masked reconstruction, and Autoencoder [25] reduces by a factor of 0.8 every 10 epochs.”
BackgroundMethodResultInfluentialSelf-citation
Metadata
- Authors
- Haresamudram, H., Anderson, D. V., & Plötz, T.
- Affiliations
- Georgia Institute of Technology (US)
- Venue
- Proceedings of the 23rd International Symposium on Wearable Computers
- Publisher
- ACM
- Published
- 2019-09-09
- Pages
- 78-88
- OpenAlex
- W2972118594
- Cited by
- 90
- References
- 43
- Citations by year
- 2026: 7 · 2025: 14 · 2024: 19 · 2023: 14 · 2022: 16 · 2021: 12 · 2020: 8
- FWCI
- 2.90
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Symposium on Wearable Computers (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Physical Sciences › Computer Science › Signal Processing › Time Series Analysis and Forecasting
- Keywords
- human activity recognition, wearable sensors, feature extraction, representation learning, end-to-end deep learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.