Synthetic Sensor Data for Human Activity Recognition
Abstract
Human activity recognition (HAR) based on wearable sensors has emerged as an active topic of research in machine learning and human behavior analysis because of its applications in several fields, including health, security and surveillance, and remote monitoring. Machine learning algorithms are frequently applied in HAR systems to learn from labeled sensor data. The effectiveness of these algorithms generally relies on having access to lots of accurately labeled training data. But labeled data for HAR is hard to come by and is often heavily imbalanced in favor of one or other dominant classes, which in turn leads to poor recognition performance. In this study we introduce a generative adversarial network (GAN)-based approach for HAR that we use to automatically synthesize balanced and realistic sensor data. GANs are robust generative networks, typically used to create synthetic images that cannot be distinguished from real images. Here we explore and construct a model for generating several types of human activity sensor data using a Wasserstein GAN (WGAN). We assess the synthetic data using two commonly-used classifier models, Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM). We evaluate the quality and diversity of the synthetic data by training on synthetic data and testing on real sensor data, and vice versa. We then use synthetic sensor data to oversample the imbalanced training set. We demonstrate the efficacy of the proposed method on two publicly available human activity datasets, the Sussex-Huawei Locomotion (SHL) and Smoking Activity Dataset (SAD). We achieve improvements of using WGAN augmented training data over the imbalanced case, for both SHL (0.85 to 0.95 F1-score), and for SAD (0.70 to 0.77 F1-score) when using a CNN activity classifier.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In SHL and SAD datasets, a CNN-layer discriminator demonstrated synthetic data with high quality and diversity.
“In two public datasets, Sussex-Huawei Locomotion (SHL) and Smoking Activity Dataset (SAD), the discriminator was built with CNN layers, and the results demonstrated synthetic data with high quality and diversity with two public datasets.”
From Deep Learning in Human Activity Recognition with Wearable Sensors: A Review on Advances (2022) · cites this paper as [240]
Alharbi et al. assessed synthetic data with CNN or LSTM models as a generator.
“Similarly, Alharbi et al. [240] assessed synthetic data with CNN or LSTM models as a generator.”
From Deep Learning in Human Activity Recognition with Wearable Sensors: A Review on Advances (2022) · cites this paper as [240]
Cites
Scikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016 · see citing works
SMOTE synthetic minority over-sampling technique
Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002) · Journal of Artificial Intelligence Research · doi:10.1613/jair.953 · Cited by 32,865 · see citing works
He, H., & Garcia, E. A. (2009) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2008.239 · Cited by 10,359 · see citing works
A systematic analysis of performance measures for classification tasks
Sokolova, M., & Lapalme, G. (2009) · Information Processing & Management · doi:10.1016/j.ipm.2009.03.002 · Cited by 7,515 · see citing works
Generative Adversarial Networks
Goodfellow, I. J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1406.2661 · Cited by 4,477 · see citing works
Conditional Image Synthesis With Auxiliary Classifier GANsPreprint
Odena, A., Olah, C., & Shlens, J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.09585 · Cited by 2,968 · see citing works
A Survey on Human Activity Recognition using Wearable Sensors
Lara, O. D., & Labrador, M. A. (2013) · IEEE Communications Surveys & Tutorials · doi:10.1109/surv.2012.110112.00192 · Cited by 2,662 · see citing works
Generative Adversarial Networks
Drori, I. (2022) · Cambridge University Press eBooks · doi:10.1017/9781108891530.013 · Cited by 2,401 · see citing works
SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient
Yu, L., Zhang, W., Wang, Y.-X., & Yu, Y. H. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10804 · Cited by 2,342 · see citing works
Deep learning for sensor-based activity recognition: A survey
Wang, J., Chen, Y., Hao, S., Peng, X., & Hu, L. (2019) · Pattern Recognition Letters · doi:10.1016/j.patrec.2018.02.010 · Cited by 2,014 · cited 2× · see citing works
“In contrast, deep-learning methods are capable of automatically learning feature representation and extracting features directly from the sensor data [9].”
MethodA tutorial on human activity recognition using body-worn inertial sensors
Bulling, A., Blanke, U., & Schiele, B. (2014) · ACM Computing Surveys · doi:10.1145/2499621 · Cited by 1,668 · see citing works
(2015) · Cited by 989 · see citing works
Nweke, H. F., Teh, Y. W., Al-garadi, M. A., & Alo, U. R. (2018) · Expert Systems with Applications · doi:10.1016/j.eswa.2018.03.056 · Cited by 930 · see citing works
Convolutional Neural Networks for Human Activity Recognition using Mobile Sensors
Zeng, M., Nguyen, L. T., Yu, B., Mengshoel, O. J., Zhu, J., Wu, P., & Zhang, J. (2014) · Proceedings of the 6th International Conference on Mobile Computing, Applications and Services · doi:10.4108/icst.mobicase.2014.257786 · Cited by 876 · see citing works
DeepSense: A Unified Deep Learning Framework for Time-Series Mobile Sensing Data Processing
Yao, S., Hu, S., Zhao, Y., Zhang, A., & Abdelzaher, T. (2017) · Proceedings of the 26th International Conference on World Wide Web · doi:10.1145/3038912.3052577 · Cited by 667 · see citing works
Preece, S. J., Goulermas, J. Y., Kenney, L. P. J., & Howard, D. (2009) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2008.2006190 · Cited by 551 · see citing works
Deep Recurrent Neural Networks for Human Activity Recognition
Murad, A., & Pyun, J.-Y. (2017) · Sensors · doi:10.3390/s17112556 · Cited by 524 · see citing works
A Survey of Online Activity Recognition Using Mobile Phones
Shoaib, M., Bosch, S., Incel, O., Scholten, H., & Havinga, P. (2015) · Sensors · doi:10.3390/s150102059 · Cited by 489 · see citing works
Transfer learning for activity recognition: a survey
Cook, D., Feuz, K. D., & Krishnan, N. C. (2013) · Knowledge and Information Systems · doi:10.1007/s10115-013-0665-3 · Cited by 447 · see citing works
Complex Human Activity Recognition Using Smartphone and Wrist-Worn Motion Sensors
Shoaib, M., Bosch, S., Incel, O., Scholten, H., & Havinga, P. (2016) · Sensors · doi:10.3390/s16040426 · Cited by 423 · see citing works
Shmelkov, K., Schmid, C., & Alahari, K. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01216-8_14 · Cited by 394 · see citing works
Comparison of Feature Learning Methods for Human Activity Recognition Using Wearable Sensors
Li, F., Shirahama, K., Nisar, M., Köping, L., & Grzegorzek, M. (2018) · Sensors · doi:10.3390/s18020679 · Cited by 302 · see citing works
Recent trends in machine learning for human activity recognition—A survey
Ramamurthy, S. R., & Roy, N. (2018) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.1254 · Cited by 301 · see citing works
Performance Analysis of Smartphone-Sensor Behavior for Human Activity Recognition
Chen, Y., & Shen, C. Q. (2017) · IEEE Access · doi:10.1109/access.2017.2676168 · Cited by 297 · see citing works
Gjoreski, H., Ciliberto, M., Wang, L., Morales, F. J. O., Mekki, S., Valentin, S., & Roggen, D. (2018) · IEEE Access · doi:10.1109/access.2018.2858933 · Cited by 277 · see citing works
Steven Eyobu, O., & Han, D. S. (2018) · Sensors · doi:10.3390/s18092892 · Cited by 259 · see citing works
LSTM Networks for Mobile Human Activity Recognition
Chen, Y., Zhong, K., Zhang, J., Sun, Q., & Zhao, X. (2016) · International Conference on Artificial Intelligence Technologies and Applications (ICAITA) · doi:10.2991/icaita-16.2016.13 · Cited by 204 · see citing works
Convolutional Neural Networks for Human Activity Recognition Using Body-Worn Sensors
Moya Rueda, F., Grzeszick, R., Fink, G. A., Feldhorst, S., & Ten Hompel, M. (2018) · Informatics · doi:10.3390/informatics5020026 · Cited by 196 · see citing works
SenseGen: A deep learning architecture for synthetic sensor data generation
Alzantot, M., Chakraborty, S., & Srivastava, M. (2017) · 2017 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops) · doi:10.1109/percomw.2017.7917555 · Cited by 153 · see citing works
Jaques, N., Taylor, S., Sano, A., & Picard, R. W. (2017) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii.2017.8273601 · Cited by 135 · see citing works
Better Physical Activity Classification using Smartphone Acceleration Sensor
Arif, M., Bilal, M., Kattan, A. J., & Ahamed, S. I. (2014) · Journal of Medical Systems · doi:10.1007/s10916-014-0095-0 · Cited by 97 · see citing works
Wang, J., Chen, Y., Gu, Y., Xiao, Y., & Pan, H. (2018) · 2018 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2018.8489106 · Cited by 92 · see citing works
Wu, D., Wang, Z., Chen, Y., et al. (2016) · Neurocomputing · doi:10.1016/j.neucom.2015.11.095 · Cited by 91 · see citing works
Generative Adversarial Networks: A Survey and Taxonomy.Preprint
Wang, Z., She, Q., & Ward, T. E. (2019) · arXiv (Cornell University) · Cited by 65 · see citing works
A hierarchical lazy smoking detection algorithm using smartwatch sensors
Shoaib, M. A., Scholten, H., Havinga, P., & İncel, Ö. D. (2016) · IEEE International Conference on e-Health Networking, Applications and Services (Healthcom) · doi:10.1109/healthcom.2016.7749439 · Cited by 62 · see citing works
Antar, A. D., Ahmed, M., Ishrak, M. S., et al. (2018) · doi:10.1145/3267305.3267516 · Cited by 37 · see citing works
A novel feature extraction technique for human activity recognition
Elvira, V., Nazábal, A., & Artés-Rodrı́guez, A. (2014) · IEEE Statistical Signal Processing Workshop (SSP) · doi:10.1109/ssp.2014.6884604 · Cited by 11 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Deep Learning in Human Activity Recognition with Wearable Sensors: A Review on Advances
Zhang, S., Li, Y., Zhang, S., Shahabi, F., Xia, S., Deng, Y., & Alshurafa, N. (2022) · Sensors · doi:10.3390/s22041476 · Cited by 625 · see citing works
“Similarly, [13] assesses synthetic data with CNN or LSTM models as a generator.”
MethodMeng, L., Zhang, A., Chen, C., Wang, X., Jiang, X., Tao, L., Fan, J., Wu, X., Dai, C., Zhang, Y., et al. (2021) · Sensors · doi:10.3390/s21030799 · Cited by 45 · see citing works
- Authors
- Alharbi, F., Ouarbya, L., & Ward, J. A.
- Affiliations
- University of London (GB)
- Venue
- 2020 International Joint Conference on Neural Networks (IJCNN)
- Publisher
- IEEE
- Published
- 2020-07
- Pages
- 1-9
- ISSN
- 2161-4407 (ISSN-L)
- OpenAlex
- W3028846921
- Cited by
- 40
- References
- 37
- Citations by year
- 2026: 2 · 2025: 8 · 2024: 9 · 2023: 7 · 2022: 8 · 2021: 6
- FWCI
- 1.62
- Type
- conference paper
- Language
- en
- Open access
- closed
- Licence
- Locations
- International Joint Conference on Neural Networks (IJCNN) (publishedVersion)
- Goldsmiths (University of London) (acceptedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Physical Sciences › Computer Science › Artificial Intelligence › Imbalanced Data Classification Techniques
- Physical Sciences › Engineering › Biomedical Engineering › Gait Recognition and Analysis
- Keywords
- human activity recognition, synthetic sensor data, generative adversarial networks, Wasserstein GAN, data augmentation, class imbalance, oversampling, wearable sensors, convolutional neural networks, long short-term memory
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
