On the Generality of Codebook Approach for Sensor-Based Human Activity Recognition
Open accessThe DOI matches its Crossref record; 46 of the 47 references it cites were found in a registry or an index.
Abstract
With the recent spread of mobile devices equipped with different sensors, it is possible to continuously recognise and monitor activities in daily life. This sensor-based human activity recognition is formulated as sequence classification to categorise sequences of sensor values into appropriate activity classes. One crucial problem is how to model features that can precisely represent characteristics of each sequence and lead to accurate recognition. It is laborious and/or difficult to hand-craft such features based on prior knowledge and manual investigation about sensor data. To overcome this, we focus on a feature learning approach that extracts useful features from a large amount of data. In particular, we adopt a simple but effective one, called codebook approach, which groups numerous subsequences collected from sequences into clusters. Each cluster centre is called a codeword and represents a statistically distinctive subsequence. Then, a sequence is encoded as a feature expressing the distribution of codewords. The extensive experiments on different recognition tasks for physical, mental and eye-based activities validate the effectiveness, generality and usability of the codebook approach.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures

![Figure 2 Examples of four types of physiological sequences in the dataset [11]. From the left, the first, second, third and last sequences represent values of Blood Volume Pressure (BVP), Galvanic Skin Response (GSR), RESpiration (RES) and ElectroMyoGram (EMG), respectively. Note that these sequences are normalised to have the mean zero and the variance one. In addition, to emphasise value changes, each type of sequence is plotted using the vertical axis with a different range.](https://figures.cochat.ai/paper-figures/daa7b57679fe4fb3a645e62138bc899f/on-the-generality-of-codebook-approach-for-sensor-based-human-activity-figure-2.png)



![Figure 6 Examples of three-dimensional sequences from six accelerometers for the activity 'jumping in place' in [22].](https://figures.cochat.ai/paper-figures/ae80b09eec7d4598ad006417650da88e/on-the-generality-of-codebook-approach-for-sensor-based-human-activity-figure-6.png)

![Figure 8 Examples of horizontal and vertical ElectroOculoGram (hEOG and vEOG) sequences in [10].](https://figures.cochat.ai/paper-figures/10eb86b750c248db8dd02cbd357f3955/on-the-generality-of-codebook-approach-for-sensor-based-human-activity-figure-8.png)
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“They were used in previous works for time series classification [23] or HAR [24].”
From Comparison of Feature Learning Methods for Human Activity Recognition Using Wearable Sensors (2018) · cites this paper as [24]
In the citing paper
“First, it is widely accepted that denser sampling of subsequences yields a better performance [24,39], so l is set to 1.”
From Comparison of Feature Learning Methods for Human Activity Recognition Using Wearable Sensors (2018) · cites this paper as [24]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LIBSVM: A library for support vector machines
Chang, C.-C., & Lin, C.-J. (2011) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/1961189.1961199 · Cited by 40,918
Representation Learning: A Review and New Perspectives
Bengio, Y., Courville, A., & Vincent, P. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.50 · Cited by 13,001
He, H., & Garcia, E. A. (2009) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2008.239 · Cited by 10,359 · cited 1×
“The following two points deserve attention: The first is the “imbalanced problem” that in general negative sequences significantly outnumber positive ones, since a huge variety of sequences can be negative [42].”
BackgroundLibsvm : A library for support vector machines
Ferrari, V. (2008) · Medical Entomology and Zoology · Cited by 10,122
DEAP: A Database for Emotion Analysis ;Using Physiological Signals
Koelstra, S., Muhl, C., Soleymani, M., Lee, J.-S., Yazdani, A., Ebrahimi, T., Pun, T., Nijholt, A., & Patras, I. (2012) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2011.15 · Cited by 5,093
An analysis of single-layer networks in unsupervised feature learning
Coates, A., Ng, A. Y., & Lee, H. (2011) · International Conference on Artificial Intelligence and Statistics · Cited by 4,675
Visual categorization with bags of keypoints
Csurka, G. (2004) · Cited by 4,379
A Survey on Human Activity Recognition using Wearable Sensors
Lara, O. D., & Labrador, M. A. (2013) · IEEE Communications Surveys & Tutorials · doi:10.1109/surv.2012.110112.00192 · Cited by 2,662
Toward machine emotional intelligence: analysis of affective physiological state
Picard, R. W., Vyzas, E., & Healey, J. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.954607 · Cited by 2,334 · cited 9×
“, emotions like “joy”, “anger” and “hate”) using physiological data, such as blood volume pressures, galvanic skin conductances and respiration rates [11].”
BackgroundMethodResultInfluentialPractical Recommendations for Gradient-Based Training of Deep Architectures
Bengio, Y. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-35289-8_26 · Cited by 2,259 · cited 1×
“, learning rate, momentum, weight decay and sparsity regularisation) [19,35].”
BackgroundZhang, J., Marszałek, M., Lazebnik, S., & Schmid, C. (2007) · International Journal of Computer Vision · doi:10.1007/s11263-006-9794-4 · Cited by 1,923
A tutorial on human activity recognition using body-worn inertial sensors
Bulling, A., Blanke, U., & Schiele, B. (2014) · ACM Computing Surveys · doi:10.1145/2499621 · Cited by 1,668
A Multimodal Database for Affect Recognition and Implicit Tagging
Soleymani, M., Lichtenauer, J., Pun, T., & Pantic, M. (2012) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2011.25 · Cited by 1,664
Emotion recognition based on physiological changes in music listening
Kim, J., & Andre, E. (2008) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.26 · Cited by 1,082
Sensor-Based Activity Recognition
Chen, L., Hoey, J., Nugent, C. D., Cook, D. J., & Yu, Z. (2012) · IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) · doi:10.1109/tsmcc.2012.2198883 · Cited by 1,074
(2015) · Cited by 989
Sampling Strategies for Bag-of-Features Image Classification
Nowak, E., Jurie, F., & Triggs, B. (2006) · Lecture Notes in Computer Science · doi:10.1007/11744085_38 · Cited by 927 · cited 1×
“In image/video classification, it is widely accepted that denser sampling leads to a better performance [45,47].”
BackgroundEarly versus late fusion in semantic video analysis
Snoek, C. G. M., Worring, M., & Smeulders, A. W. M. (2005) · Proceedings of the 13th annual ACM international conference on Multimedia · doi:10.1145/1101149.1101236 · Cited by 879
The devil is in the details: an evaluation of recent feature encoding methods
Chatfield, K., Lempitsky, V., Vedaldi, A., & Zisserman, A. (2011) · Procedings of the British Machine Vision Conference 2011 · doi:10.5244/c.25.76 · Cited by 834 · cited 2×
“Actually, the combination of the codebook approach and an SVM has been justified in many image/video classification tasks using features with thousands of dimensions [21,40].”
Methodvan Gemert, J. C., Veenman, C. J., Smeulders, A. W. M., & Geusebroek, J.-M. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.132 · Cited by 737
Eye Movement Analysis for Activity Recognition Using Electrooculography
Bulling, A., Ward, J. A., Gellersen, H., & Tröster, G. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.86 · Cited by 718
Berkeley MHAD: A comprehensive Multimodal Human Action Database
Ofli, F., Chaudhry, R., Kurillo, G., Vidal, R., & Bajcsy, R. (2013) · 2013 IEEE Workshop on Applications of Computer Vision (WACV) · doi:10.1109/wacv.2013.6474999 · Cited by 484
uWave: Accelerometer-based personalized gesture recognition and its applications
Liu, J., Zhong, L., Wickramasuriya, J., & Vasudevan, V. (2009) · Pervasive and Mobile Computing · doi:10.1016/j.pmcj.2009.07.007 · Cited by 479
A Bag-of-Features Framework to Classify Time Series
Baydogan, M. G., Runger, G., & Tuv, E. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.72 · Cited by 361
Plarre, K., Raij, A. B., Hossain, S. M., Ali, A. A., Nakajima, M., Al’Absi, M., Ertin, E., Kamarck, T. W., Kumar, S., Scott, M. S., et al. (2011) · ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN) · Cited by 337
Rotation-invariant similarity in time series using bag-of-patterns representation
Lin, J., Khade, R., & Li, Y. (2012) · Journal of Intelligent Information Systems · doi:10.1007/s10844-012-0196-5 · Cited by 295
Learning deep physiological models of affect
Martinez, H. P., Bengio, Y., & Yannakakis, G. N. (2013) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2013.2247823 · Cited by 287 · cited 2×
“Deep learning has been recently applied to sensor-based human activity recognition [33,34].”
MethodWearable inertial sensors for human movement analysis
Iosa, M., Picerno, P., Paolucci, S., & Morone, G. (2016) · Expert Review of Medical Devices · doi:10.1080/17434440.2016.1198694 · Cited by 281
Wearable Inertial Sensors for Human Motion Analysis: A Review
Lopez-Nava, I. H., & Munoz-Melendez, A. (2016) · IEEE Sensors Journal · doi:10.1109/jsen.2016.2609392 · Cited by 263 · cited 1×
“This allows continuous recording of sensor data related to the daily life of a person [4,5].”
BackgroundRepresentations of Keypoint-Based Semantic Concept Detection: A Comprehensive Study
Jiang, Y.-G., Yang, J., Ngo, C.-W., & Hauptmann, A. G. (2010) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2009.2036235 · Cited by 260
Clustering Time Series Using Unsupervised-Shapelets
Zakaria, J., Mueen, A., & Keogh, E. (2012) · 2012 IEEE 12th International Conference on Data Mining · doi:10.1109/icdm.2012.26 · Cited by 231
Garbarino, M., Lai, M., Tognetti, S., Picard, R., & Bender, D. (2014) · Proceedings of the 4th International Conference on Wireless Mobile Communication and Healthcare - "Transforming healthcare through innovations in mobile and wireless technologies" · doi:10.4108/icst.mobihealth.2014.257418 · Cited by 215
Bag-of-words representation for biomedical time series classification
Wang, J., Liu, P., She, M. F. H., Nahavandi, S., & Kouzani, A. Z. (2013) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2013.06.004 · Cited by 168
Weakly supervised discriminative localization and classification: a joint learning process
Nguyen, M. H., Torresani, L., de la Torre, F., & Rother, C. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459426 · Cited by 159
Scalable Recognition of Daily Activities with Wearable Sensors
Huỳnh, T., Blanke, U., & Schiele, B. (2007) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-75160-1_4 · Cited by 158
A Pattern Mining Approach to Sensor-Based Human Activity Recognition
Gu, T., Wang, L., Wu, Z., Tao, X., & Lü, J. (2010) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2010.184 · Cited by 153
Motion primitive-based human activity recognition using a bag-of-features approach
Zhang, M., & Sawchuk, A. A. (2012) · Proceedings of the 2nd ACM SIGHIT International Health Informatics Symposium · doi:10.1145/2110363.2110433 · Cited by 125
Finding Structural Similarity in Time Series Data Using Bag-of-Patterns Representation
Lin, J., & Li, Y. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-02279-1_33 · Cited by 113
Understanding physiological responses to stressors during physical activity
Hong, J.-H., Ramos, J., & Dey, A. K. (2012) · Proceedings of the 2012 ACM Conference on Ubiquitous Computing · doi:10.1145/2370216.2370260 · Cited by 64
Smart Eyewear for Interaction and Activity Recognition
Ishimaru, S., Kunze, K. S., Tanaka, K., Uema, Y., Kise, K., & İnami, M. (2015) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/2702613.2725449 · Cited by 45
Using Modified Multivariate Bag-of-Words Models to Classify Physiological Data
Ordóñez, P., Armstrong, T., Oates, T., & Fackler, J. (2011) · IEEE ... International Conference on Data Mining workshops · doi:10.1109/icdmw.2011.174 · Cited by 34
An Ensemble Dynamic Time Warping Classifier with Application to Activity Recognition
McGlynn, D., & Madden, M. G. (2010) · Springer eBooks · doi:10.1007/978-0-85729-130-1_26 · Cited by 26
Shirahama, K., & Grzegorzek, M. (2016) · Multimedia Tools and Applications · doi:10.1007/s11042-014-2292-8 · Cited by 24
Self-citationLatent Maximum Margin Clustering
Zhou, G.-T., Lan, T., Vahdat, A., & Mori, G. (2013) · neural information processing systems · Cited by 22
Kobe University and Muroran Institute of Technology at TRECVID 20112 Semantic Indexing Task.
Shirahama, K., & Uehara, K. (2012) · TRECVID · Cited by 9
Self-citationHybrid negative example selection using visual and conceptual features
Shirahama, K., Matsuoka, Y., & Uehara, K. (2011) · Multimedia Tools and Applications · doi:10.1007/s11042-011-0886-y · Cited by 3
Self-citation
Research works printed but not linked: 3
- Han, J., Kamber, M., and Pei, J. (2011). Data Mining: Concepts and Techniques, Morgan Kaufmann. [3rd ed.].
- Murphy, K.P. (2012). Machine Learning: A Probabilistic Perspective, The MIT Press.
- Vapnik, V.N. (1998). Statistical Learning Theory, Wiley-Interscience.
Cited by
Comparison of Feature Learning Methods for Human Activity Recognition Using Wearable Sensors
Li, F., Shirahama, K., Nisar, M., Köping, L., & Grzegorzek, M. (2018) · Sensors · doi:10.3390/s18020679 · Cited by 302
Self-citationTrends in human activity recognition using smartphones
Ferrari, A., Micucci, D., Mobilio, M., & Napoletano, P. (2021) · Journal of Reliable Intelligent Environments · doi:10.1007/s40860-021-00147-0 · Cited by 110
A survey on unsupervised learning for wearable sensor-based activity recognition
Ige, A. O., & Noor, M. H. M. (2022) · Applied Soft Computing · doi:10.1016/j.asoc.2022.109363 · Cited by 99
A general framework for sensor-based human activity recognition
Köping, L., Shirahama, K., & Grzegorzek, M. (2018) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2017.12.025 · Cited by 93
Self-citationAsymmetric Residual Neural Network for Accurate Human Activity Recognition
Long, J., Sun, W., Yang, Z., & Raymond, O. I. (2019) · Information · doi:10.3390/info10060203 · Cited by 57
Metadata
- Authors
- Shirahama, K., & Grzegorzek, M.
- Affiliations
- University of Siegen (DE)
- University of Economics in Katowice (PL)
- Corresponding author
- Shirahama, Kimiaki
- Venue
- Electronics
- Publisher
- MDPI AG
- Published
- 2017-06-01
- Volume
- 6
- Issue
- 2
- Pages
- 44
- ISSN
- 2079-9292 (ISSN-L)
- OpenAlex
- W2621021458
- Cited by
- 29
- References
- 47
- Citations by year
- 2025: 1 · 2024: 6 · 2023: 4 · 2022: 1 · 2021: 2 · 2020: 5 · 2019: 7 · 2018: 2
- FWCI
- 1.59
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Electronics (publishedVersion, cc-by)
- Funders
- Bundesministerium für Bildung und Forschung (16SV7223K)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Physical Sciences › Computer Science › Signal Processing › Time Series Analysis and Forecasting
- Physical Sciences › Engineering › Biomedical Engineering › Gait Recognition and Analysis
- Keywords
- human activity recognition, sensor-based activity recognition, representation learning, sequence classification, code words, mobile devices, physical activity, mental activity, activity monitoring
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.