Improving CNN-based activity recognition by data augmentation and transfer learning
The DOI matches its Crossref record; 45 of the 46 references it cites were found in a registry or an index.
Abstract
Activity classification is a challenging problem due to large signal dimensionality, high intra-and inter-subject variability in activity patterns, presence of transitional classes showing mixture of patterns, and dominance of the null class. Supervised learning has been the prevalent choice with deep neural networks (DNNs) showing some promising potential. Deep learning however requires a large number of labeled samples which is difficult to acquire, especially from vulnerable older people. In this paper we implement 3 different convolutional neural network architectures trained on data from older people, incorporating Bayesian optimization for efficient hyperparameter tuning. We exploit various augmentation methods for time-series to make invariant predictions and also cross-utilize knowledge about physical activity of younger persons in order to improve generalization in our models designed for older adults.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“In sensor-based HAR leveraging the temporal nature of IMU signals allows for the application of augmentation methods grounded in signal processing [32,34,99,132,141,178,193].”
From Past, Present, and Future of Sensor-based Human Activity Recognition Using Wearables: A Surveying Tutorial on a Still Challenging Task (2025) · cites this paper as [99]
In the citing paper
“Kalouris et al. [99] leverage transfer learning and five augmentation methods, including rotation, 3D rotation, scaling, jittering, and permutation, resulting in general performance enhancements, which emphasizes the efficacy of specific augmentation combinations.”
From Past, Present, and Future of Sensor-based Human Activity Recognition Using Wearables: A Surveying Tutorial on a Still Challenging Task (2025) · cites this paper as [99]
Citations
Cites
TensorFlow: A system for large-scale machine learning
Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.08695 · Cited by 11,822
Practical Bayesian Optimization of Machine Learning Algorithms
Snoek, J., Larochelle, H., & Adams, R. P. (2012) · arXiv · doi:10.48550/arxiv.1206.2944 · Cited by 6,691 · cited 1×
“Bayesian optimization lately became very popular for tuning the hyperparameters of deep networks [37] due to its potential to handle multi-parametric problems with costly objective functions when first- or second-order derivatives are not available.”
BackgroundHow transferable are features in deep neural networks?
Yosinski, J., Clune, J., Bengio, Y., & Lipson, H. (2014) · arXiv · doi:10.48550/arxiv.1411.1792 · Cited by 6,061 · cited 1×
“Since these datasets are similar to the target dataset (FrailSafe), we used the convolutional networks as fixed feature extractors, as in [41].”
MethodDeep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition
Ordóñez, F., & Roggen, D. (2016) · Sensors · doi:10.3390/s16010115 · Cited by 2,761 · cited 3×
“1% and 90% depending on the context and method applied, while the accuracy is increased in a more recent work implementing deep convolutional and LSTM recurrent neural networks [17].”
BackgroundMethod(2013) · RECERCAT (Consorci de Serveis Universitaris de Catalunya) · Cited by 1,959
Introducing a New Benchmarked Dataset for Activity Monitoring
Reiss, A., & Stricker, D. (2012) · 2012 16th International Symposium on Wearable Computers · doi:10.1109/iswc.2012.13 · Cited by 1,456 · cited 4×
“2% for leave-onesubject-out 9-fold cross-validation in [6] and 96% in [7], evaluated however most possibly on data-split across time windows and not across subjects.”
BackgroundMethodResultInfluential(2015) · Cited by 989
Nweke, H. F., Teh, Y. W., Al-garadi, M. A., & Alo, U. R. (2018) · Expert Systems with Applications · doi:10.1016/j.eswa.2018.03.056 · Cited by 930 · cited 1×
“A review on different deep, convolutional, and recurrent network architectures based on physiological activity signals recorded by wearable sensors can be found in [24], while extensive discussion on feature representation and learning approaches for human activity recognition systems can be found in the recent review…”
BackgroundConvolutional Neural Networks for Human Activity Recognition using Mobile Sensors
Zeng, M., Nguyen, L. T., Yu, B., Mengshoel, O. J., Zhu, J., Wu, P., & Zhang, J. (2014) · Proceedings of the 6th International Conference on Mobile Computing, Applications and Services · doi:10.4108/icst.mobicase.2014.257786 · Cited by 876 · cited 2×
“With the domination of deep neural networks and the evidence that feature or metric learning approaches usually perform better than traditional techniques in many computer vision and text or speech recognition applications, lots of interest has been shown also for the use of deep learning architectures, such as convol…”
BackgroundMethodCollecting complex activity datasets in highly rich networked sensor environments
Roggen, D., Calatroni, A., Rossi, M., Holleczek, T., Forster, K., Troster, G., Lukowicz, P., Bannach, D., Pirkl, G., Ferscha, A., et al. (2010) · 2010 Seventh International Conference on Networked Sensing Systems (INSS) · doi:10.1109/inss.2010.5573462 · Cited by 788
Um, T. T., Pfister, F. M. J., Pichler, D., Endo, S., Lang, M., Hirche, S., Fietzek, U., & Kulić, D. (2017) · Proceedings of the 19th ACM International Conference on Multimodal Interaction · doi:10.1145/3136755.3136817 · Cited by 729 · cited 4×
“Augmentation [26] [27] [28] is typically performed in data space or, as shown more recently, in feature space.”
BackgroundMethodInfluentialHuman Activity Recognition Using Wearable Sensors by Deep Convolutional Neural Networks
Jiang, W., & Yin, Z. (2015) · Proceedings of the 23rd ACM international conference on Multimedia · doi:10.1145/2733373.2806333 · Cited by 676 · cited 3×
“Deep 2D CNN were implemented in [18] by stacking the raw signals row-by-row such that every signal sequence becomes adjacent to every other sequence and extracting the magnitude of the 2D Discrete Fourier Transform of the created 2D image.”
BackgroundMethodDeep, Convolutional, and Recurrent Models for Human Activity Recognition Using Wearables
Hammerla, N. Y., Halloran, S., & Ploetz, T. (2016) · arXiv · doi:10.48550/arxiv.1604.08880 · Cited by 606 · cited 1×
“A review on different deep, convolutional, and recurrent network architectures based on physiological activity signals recorded by wearable sensors can be found in [24], while extensive discussion on feature representation and learning approaches for human activity recognition systems can be found in the recent review…”
BackgroundUm, T. T. (n.d.) · mediaTUM – the media and publications repository of the Technical University Munich (Technical University Munich) · Cited by 602
Activity recognition on streaming sensor data
Krishnan, N. C., & Cook, D. J. (2014) · Pervasive and Mobile Computing · doi:10.1016/j.pmcj.2012.07.003 · Cited by 488 · cited 1×
“Therefore, a lot of interest has been shown in the last years for the development of wearable devices or smart environments monitoring activity [1].”
BackgroundData Augmentation for Deep Neural Network Acoustic Modeling
Cui, X., Goel, V., & Kingsbury, B. (2015) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2015.2438544 · Cited by 376
Feature Learning for Activity Recognition in Ubiquitous Computing
Plötz, T., Hammerla, N., & Olivier, P. (2011) · Newcastle University · doi:10.25405/ncl.34175319 · Cited by 337
Elderly activities recognition and classification for applications in assisted living
Chernbumroong, S., Cang, S., Atkins, A., & Yu, H. (2013) · Expert Systems with Applications · doi:10.1016/j.eswa.2012.09.004 · Cited by 317 · cited 1×
“Other studies [2] [44] [45] [46] do not describe in details the validation procedure or assign the individual instances (time windows) to the training or test set without considering the subject exclusiveness in each set.”
BackgroundPersonalizing EEG-based affective models with transfer learning
Zheng, W., & Lu, B. (2016) · International Joint Conference on Artificial Intelligence · Cited by 265
Human action recognition using Dynamic Time Warping
Sempena, S., Maulidevi, N. U., & Aryan, P. R. (2011) · Proceedings of the 2011 International Conference on Electrical Engineering and Informatics · doi:10.1109/iceei.2011.6021605 · Cited by 184 · cited 1×
“The classification of time series has long time been explored with traditional methods performing either feature-based classification based on predefined feature extraction [2] [6] [7] [8] [9], embedding [10] and decomposition techniques [11], or evaluating similarity after some elastic matching that reduces time-scal…”
MethodDesign considerations for the WISDM smart phone-based sensor mining architecture
Lockhart, J. W., Weiss, G. M., Xue, J. C., Gallagher, S. T., Grosner, A. B., & Pulickal, T. T. (2011) · Proceedings of the Fifth International Workshop on Knowledge Discovery from Sensor Data · doi:10.1145/2003653.2003656 · Cited by 169 · cited 1×
“Evaluation was based on recordings collected using a cell phone in a controlled laboratory environment [22].”
MethodBenchmarking classification techniques using the Opportunity human activity dataset
Sagha, H., Digumarti, S. T., del R. Millan, J., Chavarriaga, R., Calatroni, A., Roggen, D., & Troster, G. (2011) · 2011 IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/icsmc.2011.6083628 · Cited by 109 · cited 1×
“CNNs were used for recognition of specific activities (actions in a kitchen) [15] monitored by a variety of body-worn, object-based and ambient sensors [19, 20], as well as for human activity recognition using data from smartphone sensors [21].”
MethodFeature Space Transfer for Data Augmentation
Liu, B., Wang, X., Dixit, M., Kwitt, R., & Vasconcelos, N. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00947 · Cited by 101
Instrumented shoes for activity classification in the elderly
el Achkar, C. M., Lenoble-Hoskovec, C., Paraschiv-Ionescu, A., Major, K., Büla, C., & Aminian, K. (2015) · Gait & Posture · doi:10.1016/j.gaitpost.2015.10.016 · Cited by 94 · cited 1×
“Other studies [2] [44] [45] [46] do not describe in details the validation procedure or assign the individual instances (time windows) to the training or test set without considering the subject exclusiveness in each set.”
BackgroundData augmentation for deep convolutional neural network acoustic modeling
Cui, X., Goel, V., & Kingsbury, B. (2015) · 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2015.7178831 · Cited by 77
Improving classification of epileptic and non-epileptic EEG events by feature selection
Pippa, E., Zacharaki, E. I., Mporas, I., Tsirka, V., Richardson, M. P., Koutroumanidis, M., & Megalooikonomou, V. (2015) · Neurocomputing · doi:10.1016/j.neucom.2015.06.071 · Cited by 68 · cited 1×
“The classification of time series has long time been explored with traditional methods performing either feature-based classification based on predefined feature extraction [2] [6] [7] [8] [9], embedding [10] and decomposition techniques [11], or evaluating similarity after some elastic matching that reduces time-scal…”
MethodSelf-citationHuman activity recognition and monitoring for elderly people
Sebestyen, G., Stoica, I., & Hângan, A. (2016) · IEEE International Conference on Intelligent Computer Communication and Processing (ICCP) · doi:10.1109/iccp.2016.7737171 · Cited by 51 · cited 1×
“in [5] a scaled model of a house was used along with a simulated sequence of activities.”
MethodRokni, S. A., & Ghasemzadeh, H. (2018) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2018.2789890 · Cited by 47 · cited 1×
“A transfer learning methodology for activity recognition without using new labeled data has previously been proposed in [29].”
BackgroundClassification of Daily Activities for the Elderly Using Wearable Sensors
Liu, J., Sohn, J.-H., & Kim, S. (2017) · Journal of Healthcare Engineering · doi:10.1155/2017/8934816 · Cited by 45 · cited 1×
“Other studies [2] [44] [45] [46] do not describe in details the validation procedure or assign the individual instances (time windows) to the training or test set without considering the subject exclusiveness in each set.”
BackgroundRecognizing Physical Activity of Older People from Wearable Sensors and Inconsistent Data
Papagiannaki, A., Zacharaki, E. I., Kalouris, G., Kalogiannis, S., Deltouzos, K., Ellul, J., & Megalooikonomou, V. (2019) · Sensors · doi:10.3390/s19040880 · Cited by 44 · cited 7×
“The obtained hyper-parameter values and their relative contribution in each classification model can be found in [32] for”
BackgroundMethodInfluentialSelf-citationA Stratified Analysis of Bayesian Optimization MethodsPreprint
Dewancker, I., McCourt, M., Clark, S., Hayes, P., Johnson, A., & Ke, G. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.09441 · Cited by 38 · cited 2×
“We optimized the networks hyperparameters by the Bayesian hyperparameter optimization platform SigOpt [38], which is a standardized and scalable platform, accompanied by an API facilitating the generation of well performing models.”
MethodTensor Decomposition for Multiple‐Instance Classification of High‐Order Medical Data
Παπαστεργίου, Θ., Zacharaki, E. I., & Megalooikonomou, V. (2018) · Complexity · doi:10.1155/2018/8651930 · Cited by 36 · cited 1×
“The classification of time series has long time been explored with traditional methods performing either feature-based classification based on predefined feature extraction [2] [6] [7] [8] [9], embedding [10] and decomposition techniques [11], or evaluating similarity after some elastic matching that reduces time-scal…”
MethodSelf-citationSpike pattern recognition by supervised classification in low dimensional embedding space
Zacharaki, E. I., Mporas, I., Garganis, K., & Megalooikonomou, V. (2016) · Brain Informatics · doi:10.1007/s40708-016-0044-4 · Cited by 35 · cited 1×
“The classification of time series has long time been explored with traditional methods performing either feature-based classification based on predefined feature extraction [2] [6] [7] [8] [9], embedding [10] and decomposition techniques [11], or evaluating similarity after some elastic matching that reduces time-scal…”
MethodSelf-citationAn Ensemble Dynamic Time Warping Classifier with Application to Activity Recognition
McGlynn, D., & Madden, M. G. (2010) · Springer eBooks · doi:10.1007/978-0-85729-130-1_26 · Cited by 26 · cited 1×
“The classification of time series has long time been explored with traditional methods performing either feature-based classification based on predefined feature extraction [2] [6] [7] [8] [9], embedding [10] and decomposition techniques [11], or evaluating similarity after some elastic matching that reduces time-scal…”
MethodAutomated activity recognition and monitoring of elderly using wireless sensors: Research challenges
Ranasinghe, D. C., Torres, R. L. S., & Wickramasinghe, A. (2013) · International Workshop on Advances in Sensors and Interfaces (IWASI) · doi:10.1109/iwasi.2013.6576067 · Cited by 21 · cited 1×
“Although many frameworks have been reported in the literature for activity recognition of older people, most of them have been tested on data from young and healthy participants [2, 3, 4], or the experiments were performed on laboratory conditions, e.”
BackgroundIntegrating an openEHR-based personalized virtual model for the ageing population within HBase
Kalogiannis, S., Δελτούζος, Κ., Zacharaki, E. I., Vasilakis, A. A., Μουστάκας, Κ., Ellul, J., & Megalooikonomou, V. (2019) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-019-0745-8 · Cited by 21 · cited 1×
“The data collection was part of the FrailSafe European research project [34] that aims at the development of a technological platform for early detection and prevention of frailty based on non-intrusive sensing (of physical, cognitive, psychological, social domains) for the ageing population [35] [36].”
BackgroundSelf-citationA closed-loop deep learning architecture for robust activity recognition using wearable sensors
Saeedi, R., Norgaard, S., & Gebremedhin, A. H. (2017) · 2017 IEEE International Conference on Big Data (Big Data) · doi:10.1109/bigdata.2017.8257960 · Cited by 20 · cited 1×
“Combination of convolutional and long short-term memory (LSTM) layers were used to model the temporal dependencies for multimodal activity recognition [17] [23].”
MethodFeature Selection Evaluation for Light Human Motion Identification in Frailty Monitoring System
Pippa, E., Mporas, I., & Megalooikonomou, V. (2016) · International Conference on Information and Communication Technologies for Ageing Well and e-Health (ICT4AWE) · doi:10.5220/0005912200880095 · Cited by 19 · cited 4×
“This was performed based on previously reported work that suggests that these classes are not easily separable [3].”
BackgroundMethodInfluentialSelf-citationMporas, I., Tsirka, V., Zacharaki, E. I., Koutroumanidis, M., & Megalooikonomou, V. (2014) · International Conference on PErvasive Technologies Related to Assistive Environments (PETRA) · doi:10.1145/2674396.2674412 · Cited by 13 · cited 1×
“The classification of time series has long time been explored with traditional methods performing either feature-based classification based on predefined feature extraction [2] [6] [7] [8] [9], embedding [10] and decomposition techniques [11], or evaluating similarity after some elastic matching that reduces time-scal…”
MethodSelf-citationAli, H., Messina, E., & Bisiani, R. (2013) · UKSim European Symposium on Computer Modeling and Simulation (EMS) · doi:10.1109/ems.2013.7 · Cited by 11 · cited 2×
“Similarly, another work [31] has applied semisupervised clustering to reduce data variations.”
BackgroundMeeting challenges of activity recognition for ageing population in real life settings
Papagiannaki, A., Zacharaki, E. I., Δελτούζος, Κ., Orselli, R., Freminet, A., Cela, S., Aristodemou, E., Polycarpou, M., Kotsani, M., Bénétos, A., et al. (2018) · IEEE International Conference on e-Health Networking, Applications and Services (Healthcom) · doi:10.1109/healthcom.2018.8531105 · Cited by 10 · cited 2×
“89%) our previous method evaluated on the FrailSafe dataset [42], which achieved 81.”
BackgroundMethodSelf-citationDynamic Weight Alignment for Temporal Convolutional Neural Networks
Iwana, B. K., & Uchida, S. (2019) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2019.8682908 · Cited by 10 · cited 6×
“With the domination of deep neural networks and the evidence that feature or metric learning approaches usually perform better than traditional techniques in many computer vision and text or speech recognition applications, lots of interest has been shown also for the use of deep learning architectures, such as convol…”
BackgroundInfluentialGeriatric group analysis by clustering non-linearly embedded multi-sensor data
Kalogiannis, S., Zacharaki, E. I., Δελτούζος, Κ., Kotsani, M., Ellul, J., Bénétos, A., & Megalooikonomou, V. (2018) · International Conference on Innovations in Intelligent Systems and Applications (INISTA) · doi:10.1109/inista.2018.8466269 · Cited by 9 · cited 1×
“The data collection was part of the FrailSafe European research project [34] that aims at the development of a technological platform for early detection and prevention of frailty based on non-intrusive sensing (of physical, cognitive, psychological, social domains) for the ageing population [35] [36].”
BackgroundSelf-citationADL Detection for the Active Ageing of Elderly People
Andò, B., Baglio, S., Lombardo, C. O., Marletta, V., Pergolizzi, E. A., Pistorio, A., & Valastro, A. (2015) · Biosystems & biorobotics · doi:10.1007/978-3-319-18374-9_27 · Cited by 5 · cited 3×
“The classification of time series has long time been explored with traditional methods performing either feature-based classification based on predefined feature extraction [2] [6] [7] [8] [9], embedding [10] and decomposition techniques [11], or evaluating similarity after some elastic matching that reduces time-scal…”
BackgroundMethodFekri, M., & Shafiq, M. O. (2018) · IEEE International Conference on e-Health Networking, Applications and Services (Healthcom) · doi:10.1109/healthcom.2018.8531150 · Cited by 3 · cited 1×
“CNNs were used for recognition of specific activities (actions in a kitchen) [15] monitored by a variety of body-worn, object-based and ambient sensors [19, 20], as well as for human activity recognition using data from smartphone sensors [21].”
Method
Research works printed but not linked: 2
- chollet, 2015, Keras
- goodfellow, 2016, Deep Learning
Cited by
Bloch, V., Frondelius, L., Arcidiacono, C., Mancino, M., & Pastell, M. (2023) · Sensors · doi:10.3390/s23052611 · Cited by 57
“Different types of data augmentation have been used to enlarge the training dataset [12,20]: rotation, permutation, jittering and scaling performed for the original signal, or local averaging as a down-sampling technique and shuffling in the feature space [21].”
MethodHuman Activity Recognition Data Analysis: History, Evolutions, and New Trends
Ariza-Colpas, P. P., Vicario, E., Oviedo-Carrascal, A. I., Butt Aziz, S., Piñeres-Melo, M. A., Quintero-Linero, A., & Patara, F. (2022) · Sensors · doi:10.3390/s22093401 · Cited by 52
Data Augmentation for Human Activity Recognition With Generative Adversarial Networks
Lupión, M., Cruciani, F., Cleland, I., Nugent, C., & Ortigosa, P. M. (2024) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2024.3364910 · Cited by 44
Transfer learning and its extensive appositeness in human activity recognition: A survey
Ray, A., & Kolekar, M. H. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.122538 · Cited by 39
Sensor-data augmentation for human activity recognition with time-warping and data masking
Jeong, C. Y., Shin, H. C., & Kim, M. (2021) · Multimedia Tools and Applications · doi:10.1007/s11042-021-10600-0 · Cited by 38
“However, because accelerometer signals have characteristics of time-series data, augmentation methods [4, 12, 18, 24, 33] using signal processing (e.g., jittering, scaling, rotation, and random sampling) are widely used in the field of HAR.”
BackgroundMethodResultInfluential
Metadata
- Authors
- Kalouris, G., Zacharaki, E. I., & Megalooikonomou, V.
- Affiliations
- University of Patras (GR)
- Venue
- 2019 IEEE 17th International Conference on Industrial Informatics (INDIN)
- Publisher
- IEEE
- Published
- 2019-07
- Pages
- 1387-1394
- ISSN
- 1935-4576 (ISSN-L)
- OpenAlex
- W3004151317
- Cited by
- 28
- References
- 46
- Citations by year
- 2026: 2 · 2025: 1 · 2024: 7 · 2023: 3 · 2022: 6 · 2021: 6 · 2020: 3
- FWCI
- 1.22
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE International Conference on Industrial Informatics (INDIN) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Physical Sciences › Computer Science › Signal Processing › Time Series Analysis and Forecasting
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Physical Activity and Health
- Keywords
- activity recognition, older adults, convolutional neural networks, data augmentation, transfer learning, Bayesian optimization, hyperparameter optimization, physical activity classification, interindividual variability, generalization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.