IMUTube: Automatic Extraction of Virtual on-body Accelerometry from Video for Human Activity Recognition
Open accessAbstract
The lack of large-scale, labeled data sets impedes progress in developing robust and generalized predictive models for on-body sensor-based human activity recognition (HAR). Labeled data in human activity recognition is scarce and hard to come by, as sensor data collection is expensive, and the annotation is time-consuming and error-prone. To address this problem, we introduce IMUTube, an automated processing pipeline that integrates existing computer vision and signal processing techniques to convert videos of human activity into virtual streams of IMU data. These virtual IMU streams represent accelerometry at a wide variety of locations on the human body. We show how the virtually-generated IMU data improves the performance of a variety of models on known HAR datasets. Our initial results are very promising, but the greater promise of this work lies in a collective approach by the computer vision, signal processing, and activity recognition communities to extend this work in ways that we outline. This should lead to on-body, sensor-based HAR becoming yet another success story in large-dataset breakthroughs in recognition.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Recent approaches addressing the lack of annotated datasets extract virtual movement data from modalities such as mocap and videos, which contain a large number of participants and a diverse set of actions and environments.
“Recent approaches addressing the lack of annotated datasets involve extraction of virtual movement data from modalities such as mocap [37, 72] and videos [44], which contain a large number of participants and diverse set of actions and environments.”
From Contrastive Predictive Coding for Human Activity Recognition (2021) · cites this paper as [44]
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · see citing works
You Only Look Once: Unified, Real-Time Object Detection
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.91 · Cited by 53,803 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
Fast R-CNNPreprint
Girshick, R. B. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1504.08083 · Cited by 29,416 · see citing works
A Method for Registration of 3-D Shapes
Besl, P., & Mckay, N. D. (1992) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.121791 · Cited by 22,163 · see citing works
GAN（Generative Adversarial Nets）
淳司, 柴. (2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,268 · see citing works
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate ShiftPreprint
Ioffe, S. (2024) · arXiv (Cornell University) · doi:10.57702/o9raffed · Cited by 15,273 · see citing works
(2010) · International Conference on Machine Learning · Cited by 13,995 · see citing works
A survey on Image Data Augmentation for Deep Learning
Shorten, C., & Khoshgoftaar, T. M. (2019) · Journal of Big Data · doi:10.1186/s40537-019-0197-0 · Cited by 13,203 · see citing works
Hinton, G., Deng, L., Yu, D., Dahl, G., Mohamed, A.-R., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T., et al. (2012) · IEEE Signal Processing Magazine · doi:10.1109/msp.2012.2205597 · Cited by 10,260 · cited 1× · see citing works
“Regardless of its utility, the HAR field has yet to experience significant improvements in recognition accuracy, in contrast to the breakthroughs in other fields, such as speech recognition [34], natural language processing [19], and computer vision [32].”
BackgroundGANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., & Hochreiter, S. (2017) · arXiv · doi:10.48550/arxiv.1706.08500 · Cited by 8,365 · see citing works
Realtime Multi-person 2D Pose Estimation Using Part Affinity Fields
Cao, Z., Simon, T., Wei, S.-E., & Sheikh, Y. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.143 · Cited by 7,539 · cited 2× · see citing works
“The 2D pose estimator used in IMUTube, OpenPose, has previously known failure scenarios including partial detection of joints, swapping between left and right for rare poses, self-occlusion from the camera viewpoints, and partially visible bodies [10].”
MethodHow transferable are features in deep neural networks?
Yosinski, J., Clune, J., Bengio, Y., & Lipson, H. (2014) · arXiv · doi:10.48550/arxiv.1411.1792 · Cited by 6,061 · see citing works
WaveNet: a generative model for raw audio
Oord, A. V. D., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1609.03499 · Cited by 4,911 · see citing works
Ucf101: A dataset of 101 human actions classes from videos in the wild
Soomro, K., Zamir, A. R., & Shah, M. (2012) · arXiv · doi:10.48550/arxiv.1212.0402 · Cited by 4,575 · see citing works
Simple online and realtime tracking
Bewley, A., Ge, Z., Ott, L., Ramos, F., & Upcroft, B. (2016) · 2016 IEEE International Conference on Image Processing (ICIP) · doi:10.1109/icip.2016.7533003 · Cited by 4,433 · see citing works
Hmdb: a large video database for human motion recognition
Kuehne, H., Jhuang, H., Garrote, E., Poggio, T., & Serre, T. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126543 · Cited by 3,965 · see citing works
Efficient variants of the ICP algorithm
Rusinkiewicz, S., & Levoy, M. (2002) · Proceedings Third International Conference on 3-D Digital Imaging and Modeling · doi:10.1109/im.2001.924423 · Cited by 3,945 · see citing works
Rank transformations as a bridge between parametric and nonparametric statistics
Conover, W. J., & Iman, R. L. (1981) · The American Statistician · doi:10.1080/00031305.1981.10479327 · Cited by 3,922 · see citing works
Loper, M., Mahmood, N., Romero, J., Pons-Moll, G., & J, B. M. (2015) · ACM Transactions on Graphics · doi:10.1145/2816795.2818013 · Cited by 3,734 · see citing works
Unsupervised Learning of Depth and Ego-Motion from Video
Zhou, T., Brown, M., Snavely, N., & Lowe, D. G. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.700 · Cited by 2,910 · see citing works
Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition
Ordóñez, F., & Roggen, D. (2016) · Sensors · doi:10.3390/s16010115 · Cited by 2,761 · cited 15× · see citing works
“Method We employ two machine learning models, Random Forest and DeepConvLSTM [54].”
MethodInfluential2D Human Pose Estimation: New Benchmark and State of the Art Analysis
Andriluka, M., Pishchulin, L., Gehler, P., & Schiele, B. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.471 · Cited by 2,699 · see citing works
ActivityNet: A large-scale video benchmark for human activity understanding
Heilbron, F. C., Escorcia, V., Ghanem, B., & Niebles, J. C. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298698 · Cited by 2,578 · see citing works
SMOTE for Learning from Imbalanced Data: Progress and Challenges, Marking the 15-year Anniversary
Fernández, A., García, S., Herrera, F., & Chawla, N. V. (2018) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.11192 · Cited by 2,234 · see citing works
End-to-End Recovery of Human Shape and Pose
Kanazawa, A., Black, M. J., Jacobs, D. W., & Malik, J. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00744 · Cited by 1,899 · see citing works
NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding
Liu, J., Shahroudy, A., Pérez, M., Wang, G., Duan, L., & Kot, A. C. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2916873 · Cited by 1,841 · see citing works
Deconvolution and Checkerboard Artifacts
Odena, A., Dumoulin, V., & Olah, C. (2016) · Distill · doi:10.23915/distill.00003 · Cited by 1,639 · see citing works
Action recognition based on a bag of 3D points
Li, W., Zhang, Z., & Liu, Z. (2010) · 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2010.5543273 · Cited by 1,478 · see citing works
Introducing a New Benchmarked Dataset for Activity Monitoring
Reiss, A., & Stricker, D. (2012) · 2012 16th International Symposium on Wearable Computers · doi:10.1109/iswc.2012.13 · Cited by 1,456 · cited 4× · see citing works
“pproach and relevant technical details and explore the potential of the approach on practical recognition scenarios. Through a series of experiments on three benchmark datasets—RealWorld [86], PAMAP2 [72], and Opportunity [14]—we demonstrate the effectiveness of our approach. We discuss the overall potential of models…”
BackgroundMethodInfluentialAMASS: Archive of Motion Capture As Surface Shapes
Mahmood, N., Ghorbani, N., Troje, N. F., Pons-Moll, G., & Black, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00554 · Cited by 1,323 · see citing works
3D Human Pose Estimation in Video With Temporal Convolutions and Semi-Supervised Training
Pavllo, D., Feichtenhofer, C., Grangier, D., & Auli, M. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00794 · Cited by 1,321 · see citing works
Are GANs Created Equal? A Large-Scale StudyPreprint
Lučić, M., Kurach, K., Michalski, M., Gelly, S., & Bousquet, O. (2017) · Neural Information Processing Systems · doi:10.48550/arxiv.1711.10337 · Cited by 1,175 · see citing works
AVA: A Video Dataset of Spatio-Temporally Localized Atomic Visual Actions
Gu, C., Sun, C., Ross, D. A., Vondrick, C., Pantofaru, C., Li, Y., Vijayanarasimhan, S., Toderici, G. D., Ricco, S., Sukthankar, R., et al. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00633 · Cited by 1,029 · see citing works
The Opportunity challenge: A benchmark database for on-body sensor-based activity recognition
Chavarriaga, R., Sagha, H., Calatroni, A., Digumarti, S. T., Tröster, G., Millán, J. D. R., & Roggen, D. (2013) · Pattern Recognition Letters · doi:10.1016/j.patrec.2012.12.014 · Cited by 742 · cited 6× · see citing works
“Sensor data collection for human activity recognition is often performed by conducting user studies [14, 72, 97].”
BackgroundMethodInfluentialA Review of Point Cloud Registration Algorithms for Mobile Robotics
Pomerleau, F., Colas, F., & Siegwart, R. (2015) · Foundations and Trends in Robotics · doi:10.1561/2300000035 · Cited by 717 · see citing works
Using vanishing points for camera calibration
Caprile, B., & Torre, V. (1990) · International Journal of Computer Vision · doi:10.1007/bf00127813 · Cited by 715 · see citing works
Extrinsic calibration of a camera and laser range finder (improves camera calibration)
Zhang, Q., & Pless, R. B. (2005) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2004.1389752 · Cited by 697 · see citing works
Wearable Assistant for Parkinson’s Disease Patients With the Freezing of Gait Symptom
Bachlin, M., Plotnik, M., Roggen, D., Maidan, I., Hausdorff, J. M., Giladi, N., & Troster, G. (2010) · IEEE Transactions on Information Technology in Biomedicine · doi:10.1109/titb.2009.2036165 · Cited by 693 · cited 2× · see citing works
“Tong, et al. 1 INTRODUCTION On-body sensor-based human activity recognition (HAR) is widely utilized for behavioral analysis, such as user authentication, healthcare, and tracking everyday activities [5, 14, 50, 78, 97]. Regardless of its utility, the HAR field has yet to experience significant improvements in recogni…”
BackgroundDeep, Convolutional, and Recurrent Models for Human Activity Recognition Using Wearables
Hammerla, N. Y., Halloran, S., & Ploetz, T. (2016) · arXiv · doi:10.48550/arxiv.1604.08880 · Cited by 606 · see citing works
Um, T. T. (n.d.) · mediaTUM – the media and publications repository of the Technical University Munich (Technical University Munich) · Cited by 602 · see citing works
USC-HAD: a daily activity dataset for ubiquitous activity recognition using wearable sensors
Zhang, M., & Sawchuk, A. A. (2012) · Proceedings of the 2012 ACM Conference on Ubiquitous Computing · doi:10.1145/2370216.2370438 · Cited by 554 · see citing works
Ensembles of Deep LSTM Learners for Activity Recognition using Wearables
Guan, Y., & Plötz, T. (2017) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3090076 · Cited by 511 · see citing works
Pons‐Moll, G., Pujades, S., Hu, S., & Black, M. J. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073711 · Cited by 495 · see citing works
Berkeley MHAD: A comprehensive Multimodal Human Action Database
Ofli, F., Chaudhry, R., Kurillo, G., Vidal, R., & Bajcsy, R. K. (2013) · Proceedings · doi:10.1109/wacv.2013.6474999 · Cited by 490 · see citing works
Depth From Videos in the Wild: Unsupervised Monocular Depth Learning From Unknown Cameras
Gordon, A., Li, H., Jonschkowski, R., & Angelova, A. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00907 · Cited by 389 · see citing works
A Direct Least-Squares (DLS) method for PnP
Hesch, J. A., & Roumeliotis, S. I. (2011) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2011.6126266 · Cited by 381 · see citing works
On-body localization of wearable devices: An investigation of position-aware activity recognition
Sztyler, T., & Stuckenschmidt, H. (2016) · 2016 IEEE International Conference on Pervasive Computing and Communications (PerCom) · doi:10.1109/percom.2016.7456521 · Cited by 369 · see citing works
Shen, Z., Wang, W., Lu, X., Shen, J., Ling, H., Xu, T., & Shao, L. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00567 · Cited by 359 · see citing works
Colored Point Cloud Registration Revisited
Park, J., Zhou, Q.-Y., & Koltun, V. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.25 · Cited by 350 · see citing works
Multi-task Self-Supervised Learning for Human Activity Detection
Saeed, A., Ozcelebi, T., & Lukkien, J. (2019) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3328932 · Cited by 346 · see citing works
Huang, Y., Kaufmann, M., Aksan, E., Black, M. J., Hilliges, O., & Pons-Moll, G. (2018) · ACM Transactions on Graphics · doi:10.1145/3272127.3275108 · Cited by 342 · see citing works
Hovsepian, K., Al’Absi, M., Ertin, E., Kamarck, T. W., Nakajima, M., & Kumar, S. (2015) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/2750858.2807526 · Cited by 341 · see citing works
Data Augmentation for Time Series Classification using Convolutional Neural Networks
Guennec, A. L., Malinowski, S., & Tavenard, R. (2016) · HAL · Cited by 341 · see citing works
Times-series data augmentation and deep learning for construction equipment activity recognition
Rashid, K. M., & Louis, J. (2019) · Advanced Engineering Informatics · doi:10.1016/j.aei.2019.100944 · Cited by 341 · see citing works
Learning to Reconstruct People in Clothing From a Single RGB Camera
Alldieck, T., Magnor, M. A., Bhatnagar, B. L., Theobalt, C., & Pons‐Moll, G. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00127 · Cited by 339 · see citing works
A Short Note on the Kinetics-700-2020 Human Action Dataset
Smaira, L., Carreira, J., Noland, E., Clancy, E., Wu, A., & Zisserman, A. (2020) · arXiv · doi:10.48550/arxiv.2010.10864 · Cited by 334 · see citing works
Pons‐Moll, G., Romero, J., Mahmood, N., & Black, M. J. (2015) · ACM Transactions on Graphics · doi:10.1145/2766993 · Cited by 323 · see citing works
Total Capture: 3D Human Pose Estimation Fusing Video and Inertial Sensors
Trumble, M., Gilbert, A., Malleson, C., Hilton, A., & Collomosse, J. P. (2017) · British Machine Vision Conference (BMVC) · doi:10.5244/c.31.14 · Cited by 301 · see citing works
A practical approach for recognizing eating moments with wrist-mounted inertial sensing
Thomaz, E., Essa, I. A., & Abowd, G. D. (2015) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/2750858.2807545 · Cited by 300 · see citing works
Self-citationDiscriminative Blur Detection Features
Shi, J., Xu, L., & Jia, J. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.379 · Cited by 266 · see citing works
Steven Eyobu, O., & Han, D. S. (2018) · Sensors · doi:10.3390/s18092892 · Cited by 259 · see citing works
Hollywood in Homes: Crowdsourcing Data Collection for Activity UnderstandingPreprint
Sigurdsson, G. A., Varol, G., Wang, X., Farhadi, A., Laptev, I., & Gupta, A. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1604.01753 · Cited by 221 · see citing works
Shah, S. K., & Aggarwal, J. K. (1996) · Pattern Recognition · doi:10.1016/0031-3203(96)00038-6 · Cited by 197 · see citing works
Lane, N. D., Xu, Y., Lu, H., Hu, S., Choudhury, T., Campbell, A. T., & Zhao, F. (2011) · Proceedings of the 13th international conference on Ubiquitous computing · doi:10.1145/2030112.2030160 · Cited by 171 · see citing works
Self-citationSensing Fine-Grained Hand Activity with Smartwatches
Laput, G., & Harrison, C. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300568 · Cited by 147 · see citing works
Cross-people mobile-phone based activity recognition
Zhao, Z., Chen, Y., Liu, J., Shen, Z., & Liu, M. (2011) · doi:10.5591/978-1-57735-516-8/ijcai11-423 · Cited by 144 · see citing works
A Systematic Study of Unsupervised Domain Adaptation for Robust Human-Activity Recognition
Chang, Y., Mathur, A., Isopoussu, A., Song, J., & Kawsar, F. (2020) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3380985 · Cited by 140 · see citing works
Hammerla, N. Y., Kirkham, R., Andras, P., & Ploetz, T. (2013) · Proceedings of the 2013 International Symposium on Wearable Computers · doi:10.1145/2493988.2494353 · Cited by 138 · cited 1× · see citing works
“ix2R, we follow the same scheme but use both real and virtual IMU data. Method We employ two machine learning models, Random Forest and DeepConvLSTM [54]. Random forest is trained using ECDF features [25] with 15 components, and DeepConvLSTM is trained on raw data. For DeepConvLSTM, we train the model for a maximum of…”
MethodReview on Smart Electro-Clothing Systems (SeCSs)
Sayem, A. S. M., Teay, S. H., Shahariar, H., Fink, P. L., & Albarbar, A. (2020) · Sensors · doi:10.3390/s20030587 · Cited by 133 · see citing works
Data augmentation using synthetic data for time series classification with deep residual networksPreprint
Fawaz, H. I., Forestier, G., Weber, J., Idoumghar, L., & Müller, P.-A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.02455 · Cited by 129 · see citing works
Bogdan, O., Eckstein, V., Rameau, F., & Bazin, J. (2018) · Conference on Visual Media Production · doi:10.1145/3278471.3278479 · Cited by 124 · see citing works
Golestaneh, S. A., & Karam, L. J. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.71 · Cited by 119 · see citing works
Distribution Matching in Variational InferencePreprint
Rosca, M., Lakshminarayanan, B., & Mohamed, S. (2018) · arXiv.org · doi:10.48550/arxiv.1802.06847 · Cited by 107 · see citing works
Biosignal Data Augmentation Based on Generative Adversarial Networks
Haradal, S., Hayashi, H., & Uchida, S. (2018) · doi:10.1109/embc.2018.8512396 · Cited by 107 · see citing works
On the role of features in human activity recognition
Haresamudram, H., Anderson, D. V., & Plötz, T. (2019) · Proceedings of the 23rd International Symposium on Wearable Computers · doi:10.1145/3341163.3347727 · Cited by 90 · see citing works
Self-citationFew-shot learning-based human activity recognition
Feng, S., & Duarte, M. F. (2019) · Expert Systems with Applications · doi:10.1016/j.eswa.2019.06.070 · Cited by 83 · see citing works
Ramponi, G., Protopapas, P., Brambilla, M., & Janssen, R. R. P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.08295 · Cited by 81 · see citing works
Mathur, A., Zhang, T., Bhattacharya, S., Velickovic, P., Joffe, L., Lane, N. D., Kawsar, F., & Lio, P. (2018) · 2018 17th ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN) · doi:10.1109/ipsn.2018.00048 · Cited by 70 · cited 1× · see citing works
“increase variability in the datasets. They involve techniques that systematically transform the data during the training process in order to make classifiers more robust to noise and other variations [55]. They artificially inflate the training data by utilizing methods, which perform data warping, or oversampling [82…”
MethodSelf-citationRobust Video Stabilization by Optimization in CNN Weight Space
Yu, J.-Y., & Ramamoorthi, R. (2019) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2019.00392 · Cited by 66 · see citing works
Liaqat, D., Abdalla, M., Abed-Esfahani, P., Gabel, M., Son, T., Wu, R. C., Gershon, A. S., Rudzicz, F., & de Lara, E. (2019) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3328927 · Cited by 59 · see citing works
IMUSim: A simulation environment for inertial sensing algorithm design and evaluation
Young, A. D., Ling, M. J., & Arvind, D. K. (2011) · Information Processing in Sensor Networks · Cited by 56 · see citing works
SenseGAN: Enabling Deep Learning for Internet of Things with a Semi-Supervised Framework
Yao, S., Zhao, Y., Shao, H., Zhang, C., Zhang, A., Hu, S., Liu, D., Liu, S., Su, L., & Abdelzaher, T. (2018) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3264954 · Cited by 54 · see citing works
Personalized mobile physical activity recognition
Reiss, A., & Stricker, D. (2013) · International Symposium on Wearable Computers · doi:10.1145/2493988.2494349 · Cited by 53 · see citing works
Plötz, T., Chen, C., Hammerla, N., & Abowd, G. D. (2012) · International Symposium on Wearable Computers · doi:10.1109/iswc.2012.15 · Cited by 51 · see citing works
Self-citationA Deep Learning Method for Complex Human Activity Recognition Using Virtual Wearable Sensors
Xiao, F., Pei, L., Chu, L., Zou, D., Yu, W., Zhu, Y., & Li, T. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-69873-7_19 · Cited by 49 · see citing works
Wearables in the wet lab: a laboratory system for capturing and guiding experiments
Scholl, P. M., Wille, M., & Van Laerhoven, K. (2015) · Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing · doi:10.1145/2750858.2807547 · Cited by 45 · see citing works
Rey, V. F., Hevesi, P., Kovalenko, O., & Lukowicz, P. (2019) · Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers · doi:10.1145/3341162.3345590 · Cited by 43 · cited 2× · see citing works
“he majority of such mocap datasets include dancing, quick locomotion transitions, and martial arts, which are less relevant to recognizing daily human activities. Most related to our work, Rey et al. [74] also proposed to collect virtual sensor data from online videos and demonstrated the effectiveness of the virtual…”
BackgroundKang, S.-W., Choi, H., Park, H.-I., Choi, B.-G., Im, H., Shin, D., Jung, Y.-G., Lee, J., Park, H.-W., Park, S., et al. (2017) · Sensors · doi:10.3390/s17112560 · Cited by 36 · see citing works
Are Accelerometers for Activity Recognition a Dead-end?
Tong, C., Tailor, S. A., & Lane, N. D. (2020) · IEEE Workshop on Mobile Computing Systems and Applications (WMCSA) · doi:10.1145/3376897.3377867 · Cited by 36 · see citing works
Self-citationA self-calibration approach to extrinsic parameter estimation of stereo cameras
Zhuang, H. (1995) · Robotics and Autonomous Systems · doi:10.1016/0921-8890(95)00020-g · Cited by 24 · see citing works
A Multi-Sensor Setting Activity Recognition Simulation Tool
Takeda, S., Okita, T., Lago, P., & Inoue, S. (2018) · Proceedings of the 2018 ACM International Joint Conference and 2018 International Symposium on Pervasive and Ubiquitous Computing and Wearable Computers · doi:10.1145/3267305.3267509 · Cited by 18 · see citing works
Towards Machine Learning with Zero Real-World Data
Kang, C., Jung, H., & Lee, Y. (2019) · WearSys@MobiSys · doi:10.1145/3325424.3329662 · Cited by 14 · see citing works
Spline Function Simulation Data Generation for Walking Motion Using Foot-Mounted Inertial Sensors
Pham, T. T., & Suh, Y. S. (2018) · Electronics · doi:10.3390/electronics8010018 · Cited by 10 · see citing works
Asare, P., Dickerson, R. F., Wu, X., Lach, J., & Stankovic, J. A. (2013) · ACM Conference on Embedded Networked Sensor Systems (SenSys) · doi:10.1145/2517351.2517392 · Cited by 8 · see citing works
Flagged first, then by citations
2 of the 99 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Contrastive Predictive Coding for Human Activity Recognition
Haresamudram, H., Essa, I., & Plötz, T. (2021) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3463506 · Cited by 150 · see citing works
Self-citationAhuja, K., Jiang, Y., Goel, M., & Harrison, C. (2021) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445138 · Cited by 119 · see citing works
Assessing the State of Self-Supervised Human Activity Recognition Using Wearables
Haresamudram, H., Essa, I., & Plötz, T. (2022) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3550299 · Cited by 107 · see citing works
Self-citationZhou, H., Lu, T., Liu, Y., Zhang, S., Liu, R., & Gowda, M. (2023) · IEEE/ACM International Conference on Internet-of-Things Design and Implementation (IoTDI) · doi:10.1145/3576842.3582382 · Cited by 53 · see citing works
Transfer Learning in Sensor-Based Human Activity Recognition: A Survey
Dhekane, S. G., & Ploetz, T. (2025) · ACM Computing Surveys · doi:10.1145/3717608 · Cited by 42 · see citing works
Liu, Y., Zhang, S., & Gowda, M. (2021) · Proceedings of the International Conference on Internet-of-Things Design and Implementation · doi:10.1145/3450268.3453537 · Cited by 39 · see citing works
Deng, K., Zhao, D. Y., Han, Q., Zhang, Z., Wang, S., Zhou, A., & Ma, H. (2023) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3580872 · Cited by 37 · see citing works
Liu, S., Guo, B., Ma, K., Yu, Z., & Du, J. (2021) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3448125 · Cited by 33 · see citing works
Investigating Enhancements to Contrastive Predictive Coding for Human Activity Recognition
Haresamudram, H., Essa, I., & Plötz, T. (2023) · 2023 IEEE International Conference on Pervasive Computing and Communications (PerCom) · doi:10.1109/percom56429.2023.10099197 · Cited by 27 · see citing works
Self-citationDeep generative cross-modal on-body accelerometer data synthesis from videos
Zhang, S., & Alshurafa, N. I. (2020) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/3410530.3414329 · Cited by 21 · see citing works
Wang, H. D., & Ma, M. (2022) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3570349 · Cited by 17 · see citing works
Teikari, P., & Pietrusz, A. (2021) · doi:10.31236/osf.io/w734a · Cited by 11 · see citing works
From Virtual to Real World: Applying Animation to Design the Activity Recognition System
Xia, C., & Sugiura, Y. (2021) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3411763.3451677 · Cited by 3 · see citing works
The Impact of Synthetic Data on Fall Detection Application
Debnath, M., Kabir, M. S., Ni, J., & Ngu, A. H. H. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-66538-7_21 · Cited by 1 · see citing works
Building generalizable deep learning solutions for mobile sensing
Xu, H. (2024) · doi:10.32657/10356/174809 · Cited by 1 · see citing works
Image Recognition Applied to Security Systems: The Case of Burkina Faso
Tiendrebeogo, A. (2022) · New Media and Mass Communication · doi:10.7176/nmmc/102-04
- Authors
- Kwon, H., Tong, C., Haresamudram, H., Gao, Y., Abowd, G. D., Lane, N. D., & Ploetz, T.
- Affiliations
- Georgia Institute of Technology (US)
- University of Oxford (GB)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2020
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2006.05675
- OpenAlex
- W3034646170
- Cited by
- 14
- References
- 99
- Citations by year
- 2025: 1 · 2024: 2 · 2023: 4 · 2022: 2 · 2021: 4 · 2020: 1
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Physical Sciences › Engineering › Biomedical Engineering › Gait Recognition and Analysis
- Keywords
- human activity recognition, on-body sensors, inertial measurement unit, accelerometry, labeled data scarcity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
