MARS: Mixed Virtual and Real Wearable Sensors for Human Activity Recognition With Multidomain Deep Learning Model
Open accessThe DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.
Abstract
Together with the rapid development of the Internet of Things, human activity recognition (HAR) using wearable inertial measurement units (IMUs) becomes a promising technology for many research areas. Recently, deep-learning-based methods pave a new way of understanding and performing analysis of the complex data in the HAR system. However, the performance of these methods is mostly based on the quality and quantity of the collected data. In this article, we innovatively propose to build a large data set based on virtual IMUs and then address technical issues by introducing a multiple-domain deep learning framework consisting of three technical parts. In the first part, we propose to learn the single-frame human activity from the noisy IMU data with hybrid convolutional neural networks in the semisupervised form. For the second part, the extracted data features are fused according to the principle of uncertainty-aware consistency, which reduces the uncertainty by weighting the importance of the features. The transfer learning is performed in the last part based on the newly released archive of motion capture as surface shapes data set, containing abundant synthetic human poses, which enhances the variety and diversity of the training data set and is beneficial for the process of training and feature transfer in the proposed method. The efficiency and effectiveness of the proposed method have been demonstrated in the real deep inertial poser data set. The experimental results show that the proposed methods can surprisingly converge within a few iterations and outperform all competing methods.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
A Comprehensive Survey on Transfer Learning
Zhuang, F., Qi, Z., Duan, K., Xi, D., Zhu, Y., Zhu, H., Xiong, H., & He, Q. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2020.3004555 · Cited by 6,678 · cited 2×
“Interested readers are referred to the classical review paper [51] and a very recent tutorial work [53] for details.”
BackgroundMethodEfficient Processing of Deep Neural Networks: A Tutorial and Survey
Sze, V., Chen, Y.-H., Yang, T.-J., & Emer, J. S. (2017) · Proceedings of the IEEE · doi:10.1109/jproc.2017.2761740 · Cited by 4,215 · cited 1×
“In this work, following the prior work [41] and considering the characteristics of IMU data samples, the large stride size is employed, instead of the pooling strategies, to persevere the spatial structure of the considered data.”
MethodLoper, M., Mahmood, N., Romero, J., Pons-Moll, G., & J, B. M. (2015) · ACM Transactions on Graphics · doi:10.1145/2816795.2818013 · Cited by 3,734 · cited 1×
“[39], which contains 23 joint point posture descriptions of the standard bone model and a basic human body orientation descrip-”
BackgroundActivity recognition using cell phone accelerometers
Kwapisz, J. R., Weiss, G. M., & Moore, S. A. (2011) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1964897.1964918 · Cited by 2,912 · cited 1×
“example, [18], [19] manually extracted different features, such as mean, energy, frequency-domain entropy and correlation,”
BackgroundDeep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition
Ordóñez, F., & Roggen, D. (2016) · Sensors · doi:10.3390/s16010115 · Cited by 2,761 · cited 6×
“Deep ConvLSTM [30]: This model uses a deep convolutional neural network to extract feature and an RNN to learn time dependencies.”
MethodInfluentialA Survey on Human Activity Recognition using Wearable Sensors
Lara, O. D., & Labrador, M. A. (2013) · IEEE Communications Surveys & Tutorials · doi:10.1109/surv.2012.110112.00192 · Cited by 2,662
Convolutional Neural Networks for Human Activity Recognition using Mobile Sensors
Zeng, M., Nguyen, L. T., Yu, B., Mengshoel, O. J., Zhu, J., Wu, P., & Zhang, J. (2014) · Proceedings of the 6th International Conference on Mobile Computing, Applications and Services · doi:10.4108/icst.mobicase.2014.257786 · Cited by 876
The Opportunity challenge: A benchmark database for on-body sensor-based activity recognition
Chavarriaga, R., Sagha, H., Calatroni, A., Digumarti, S. T., Tröster, G., Millán, J. D. R., & Roggen, D. (2013) · Pattern Recognition Letters · doi:10.1016/j.patrec.2012.12.014 · Cited by 742 · cited 3×
“3055859 modes, the HAR has been approached in three different ways: 1) vision-based HAR [1], [8]; 2) radio-based HAR [9], [10]; and 3) wearable-device-based HAR [11]–[13].”
BackgroundMethodHuman Activity Recognition Using Wearable Sensors by Deep Convolutional Neural Networks
Jiang, W., & Yin, Z. (2015) · Proceedings of the 23rd ACM international conference on Multimedia · doi:10.1145/2733373.2806333 · Cited by 676
Research works printed but not linked: 40
- parthasarathy, 2015, Gait analysis using virtual body sensor networks for biomedical applications, Int J Pharm Med Biol Sci, 4, 123
- 10.1109/JSEN.2011.2121059 · DOI · Paper page
- 10.1109/TKDE.2019.2891659 · DOI
- 10.1109/ICCV.2019.00554 · DOI · Paper page
- 10.1109/CVPR.2019.00049 · DOI · Paper page
- 10.1109/TPAMI.2018.2858821 · DOI · Paper page
- laine, 2017, Temporal ensembling for semi-supervised learning, Proc Int Conf Learn Represent (ICLR), 1
- 10.1109/JSEN.2018.2885796 · DOI · Paper page
- 10.1109/MPRV.2010.7 · DOI · Paper page
- 10.1023/A:1021709817809 · DOI · Paper page
- 10.1007/s10044-016-0570-y · DOI
- 10.1007/11890348_39 · DOI · Paper page
- yang, 2015, Deep convolutional neural networks on multichannel time series for human activity recognition, Proc Int Joint Conf Artif Intell (IJCAI), 3995
- 10.4018/978-1-60566-766-9.ch011 · DOI · Paper page
- joze, 2020, MMTM: Multimodal transfer module for CNN fusion, Proc IEEE Conf Comput Vis and Pattern Recog, 13289
- liaw, 2002, Classification and regression by randomForest, R News, 2, 18
- zhuang, 2015, Supervised representation learning: Transfer learning with deep autoencoders, Proc 24th Int Joint Conf Artif Intell, 4119
- 10.1109/JIOT.2016.2624800 · DOI · Paper page
- vincent, 2010, Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion, J Mach Learn Res, 11, 3371
- 10.1109/JIOT.2019.2949715 · DOI · Paper page
- xiao, 2020, A deep learning method for complex human activity recognition using virtual wearable sensors, Proc ACM SpatialDI, 1
- 10.1109/JIOT.2019.2901927 · DOI
- bhanujyothi, 2017, A comparative study of random forest & K-nearest Neighbors on HAR dataset using caret, Proc IJIRT, 3, 6
- 10.1109/JSEN.2018.2830743 · DOI
- 10.3390/s120506155 · DOI
- ravi, 2005, Activity recognition from accelerometer data, Proc AAAI, 5, 1541
- 10.3390/s130201402 · DOI · Paper page
- 10.1109/JBHI.2013.2253613 · DOI · Paper page
- 10.1109/ACCESS.2018.2868792 · doi:10.1109/access.2018.2868792
- 10.1109/ACCESS.2018.2880454 · DOI · Paper page
- avci, 2010, Activity recognition using inertial sensing for healthcare, wellbeing and sports applications: A survey, Proc Int Conf Archit Comput Syst (ARCS), 1
- donini, 2018, Empirical risk minimization under fairness constraints, Proc Adv Neural Inf Process Syst, 2791
- 10.1109/JIOT.2016.2558659 · DOI · Paper page
- 10.1016/S0031-3203(02)00262-5 · DOI · Paper page
- tang, 2020, Rethinking 1D-CNN for time series classification A stronger baseline
- sugiyama, 2007, Covariate shift adaptation by importance weighted cross validation, J Mach Learn Res, 8, 985
- 10.4103/0256-4602.64604 · DOI · Paper page
- goodfellow, 2016, Deep Learning, 1
- 10.18653/v1/P19-2040 · DOI
- 10.1109/ASRU46091.2019.9003730 · DOI
Cited by
A Survey on Deep Learning for Human Activity Recognition
Gu, F., Chung, M.-H., Chignell, M., Valaee, S., Zhou, B., & Liu, X. (2022) · ACM Computing Surveys · doi:10.1145/3472290 · Cited by 272
Approaching the Real-World: Supporting Activity Recognition Training with Virtual IMU Data
Kwon, H., Wang, B., Abowd, G. D., & Plötz, T. (2021) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3478096 · Cited by 48
“The actual activity classifier was then fine-tuned using the small amount of real IMU data [79, 106].”
MethodTransfer Learning in Sensor-Based Human Activity Recognition: A Survey
Dhekane, S. G., & Ploetz, T. (2025) · ACM Computing Surveys · doi:10.1145/3717608 · Cited by 42
Zhou, F., Wen, G., Ma, Y., Geng, H., Huang, R., Pei, L., Yu, W., Chu, L., & Qiu, R. (2022) · Applied Sciences · doi:10.3390/app12115336 · Cited by 41
Self-citation
Metadata
- Authors
- Pei, L., Xia, S., Chu, L., Xiao, F., Wu, Q., Yu, W., & Qiu, R.
- Venue
- IEEE Internet of Things Journal
- Published
- 2021-06-01
- Volume
- 8
- Issue
- 11
- Pages
- 9383-9396
- ISSN
- 2327-4662 (ISSN-L)
- 2372-2541
- OpenAlex
- W3127033546
- Cited by
- 50
- References
- 16
- FWCI
- 3.00
- Type
- journal article
- Language
- en
- Open access
- green
- Licence
- Funders
- NSFC (61873163, 61871265)
- International Postdoctoral Exchange Fellowship from the China Postdoctoral Council (20200053)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: MARS: Mixed Virtual and Real Wearable Sensors for Human Activity Recognition With Multidomain Deep Learning Model. https://app.cochat.ai/papers/10.1109/jiot.2021.3055859
- The paper
- Pei, L., Xia, S., Chu, L., Xiao, F., Wu, Q., Yu, W., & Qiu, R. (2021). MARS: Mixed Virtual and Real Wearable Sensors for Human Activity Recognition With Multidomain Deep Learning Model. IEEE Internet of Things Journal. https://doi.org/10.1109/jiot.2021.3055859
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.