The DOI matches its Crossref record; 36 of the 37 references it cites were found in a registry or an index.
Let's (not) stick together: pairwise similarity biases cross-validation in activity recognition
Abstract
The ability to generalise towards either new users or unforeseen behaviours is a key requirement for activity recognition systems in ubiquitous computing. Differences in recognition performance for the two application cases can be significant, and user-dependent performance is typically assumed to be an upper bound on performance. We demonstrate that this assumption does not hold for the widely used cross-validation evaluation scheme that is typically employed both during system bootstrapping and for reporting results. We describe how the characteristics of segmented time-series data render random cross-validation a poor fit, as adjacent segments are not statistically independent. We develop an alternative approach -- meta-segmented cross validation -- that explicitly circumvents this issue and evaluate it on two data-sets. Results indicate a significant drop in performance across a variety of feature extraction and classification methods if this bias is removed, and that prolonged, repetitive activities are particularly affected.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“subsequent, often overlapping, windows of sensor readings are not i.i.d. and thus care needs to be taken during model training and evaluation[73]”
From Past, Present, and Future of Sensor-based Human Activity Recognition Using Wearables: A Surveying Tutorial on a Still Challenging Task (2025) · cites this paper as [73]
In the citing paper
“Overall, every participant appears exactly once as part of the test split and pairwise correlations of analysis windows are effectively eliminated through the user-based splits [22].”
From Assessing the State of Self-Supervised Human Activity Recognition Using Wearables (2022) · cites this paper as [22]
Citations
Cites
LIBSVM: A library for support vector machines
Chang, C.-C., & Lin, C.-J. (2011) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/1961189.1961199 · Cited by 40,918
Book Review: Pattern Recognition and Machine Learning
Nasrabadi, N. M. (2007) · Journal of Electronic Imaging · doi:10.1117/1.2819119 · Cited by 22,066
The WEKA data mining software: an update
Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., & Witten, I. H. (2009) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1656274.1656278 · Cited by 17,750 · cited 1×
“We do so as classifiers are usually selected as an off-the-shelf component made available in common machine learning frameworks such as WEKA [13].”
MethodPattern Recognition and Machine Learning
(2006) · doi:10.1007/978-0-387-45528-0 · Cited by 9,824
(2006) · Springer eBooks · Cited by 8,123
A tutorial on human activity recognition using body-worn inertial sensors
Bulling, A., Blanke, U., & Schiele, B. (2014) · ACM Computing Surveys · doi:10.1145/2499621 · Cited by 1,668
Introducing a New Benchmarked Dataset for Activity Monitoring
Reiss, A., & Stricker, D. (2012) · 2012 16th International Symposium on Wearable Computers · doi:10.1109/iswc.2012.13 · Cited by 1,456 · cited 2×
“It is the de-facto standard that subsequent frames overlap with the last segment to a certain extent, such as 50% [24] or more, e.g. 80% in [30].”
BackgroundEstimating classification error rate: Repeated cross-validation, repeated hold-out and bootstrap
Kim, J.-H. (2009) · Computational Statistics & Data Analysis · doi:10.1016/j.csda.2009.04.009 · Cited by 792
Collecting complex activity datasets in highly rich networked sensor environments
Roggen, D., Calatroni, A., Rossi, M., Holleczek, T., Forster, K., Troster, G., Lukowicz, P., Bannach, D., Pirkl, G., Ferscha, A., et al. (2010) · 2010 Seventh International Conference on Networked Sensing Systems (INSS) · doi:10.1109/inss.2010.5573462 · Cited by 788
SEGMENTING TIME SERIES: A SURVEY AND NOVEL APPROACH
Keogh, E., Chu, S., Hart, D., & Pazzani, M. (2004) · Series in Machine Perception and Artificial Intelligence · doi:10.1142/9789812565402_0001 · Cited by 600
Research works printed but not linked: 1
- Kose , M. , Incel , O. D. , and Ersoy , C . Online human activity recognition on smart phones . In Workshop on Mobile Sensing: From Smartphones and Wearables to Big Data ( 2012 ), 11--15. Kose, M., Incel, O. D., and Ersoy, C. Online human activity recognition on smart phones. In Workshop on Mobile Sensing: From Smartphones and Wearables to Big Data (2012), 11--15.
Cited by
Deep, Convolutional, and Recurrent Models for Human Activity Recognition Using Wearables
Hammerla, N. Y., Halloran, S., & Ploetz, T. (2016) · arXiv · doi:10.48550/arxiv.1604.08880 · Cited by 606
Self-citationEnsembles of Deep LSTM Learners for Activity Recognition using Wearables
Guan, Y., & Plötz, T. (2017) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3090076 · Cited by 511
Self-citationREFORMS: Consensus-based Recommendations for Machine-learning-based Science
Kapoor, S., Cantrell, E. M., Peng, K., Pham, T. H., Bail, C. A., Gundersen, O. E., Hofman, J. M., Hullman, J. R., Lones, M. A., Malik, M. M., et al. (2024) · Science Advances · doi:10.1126/sciadv.adk3452 · Cited by 309
A review of state-of-the-art techniques for abnormal human activity recognition
Dhiman, C., & Vishwakarma, D. K. (2019) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2018.08.014 · Cited by 265
Wang, R., Aung, M. S. H., Abdullah, S., Brian, R., Campbell, A. T., Choudhury, T., Hauser, M., Kane, J., Merrill, M., Scherer, E. A., et al. (2016) · Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing · doi:10.1145/2971648.2971740 · Cited by 257
“We use a 10-fold blocked cross validation method [12,16,28, 48] to evaluate the prediction performance of the individual models and mixed models.”
Method
Metadata
- Authors
- Hammerla, N. Y., & Plötz, T.
- Affiliations
- Newcastle University (GB)
- Venue
- Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing
- Publisher
- ACM
- Published
- 2015-09-07
- Pages
- 1041-1051
- OpenAlex
- W2155961734
- Cited by
- 91
- References
- 37
- Citations by year
- 2026: 3 · 2025: 8 · 2024: 9 · 2023: 8 · 2022: 7 · 2021: 13 · 2020: 15 · 2019: 5 · 2018: 8 · 2017: 10 · 2016: 5
- FWCI
- 3.14
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) (publishedVersion)
- Funders
- Engineering and Physical Sciences Research Council (EP/G066019/1)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Physical Sciences › Computer Science › Signal Processing › Time Series Analysis and Forecasting
- Social Sciences › Transportation › Human Mobility and Location-Based Analysis
- Keywords
- activity recognition, time series segmentation, ubiquitous computing, feature extraction, classification
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Let's (not) stick together: pairwise similarity biases cross-validation in activity recognition. https://app.cochat.ai/papers/10.1145/2750858.2807551
- The paper
- Hammerla, N. Y., & Plötz, T. (2015). Let's (not) stick together: pairwise similarity biases cross-validation in activity recognition. Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing. https://doi.org/10.1145/2750858.2807551
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.