IMUTube: Automatic Extraction of Virtual on-body Accelerometry from Video for Human Activity Recognition
The DOI matches its Crossref record; 64 of the 64 references it cites were found in a registry or an index.
Abstract
The lack of large-scale, labeled data sets impedes progress in developing robust and generalized predictive models for on-body sensor-based human activity recognition (HAR). Labeled data in human activity recognition is scarce and hard to come by, as sensor data collection is expensive, and the annotation is time-consuming and error-prone. To address this problem, we introduce IMUTube, an automated processing pipeline that integrates existing computer vision and signal processing techniques to convert videos of human activity into virtual streams of IMU data. These virtual IMU streams represent accelerometry at a wide variety of locations on the human body. We show how the virtually-generated IMU data improves the performance of a variety of models on known HAR datasets. Our initial results are very promising, but the greater promise of this work lies in a collective approach by the computer vision, signal processing, and activity recognition communities to extend this work in ways that we outline. This should lead to on-body, sensor-based HAR becoming yet another success story in large-dataset breakthroughs in recognition.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Due to these factors, publicly available wearable sensor datasets are typically limited in size and variability, often containing only 10-20 participants and a handful of activities, recorded over a few hours [109].”
From Past, Present, and Future of Sensor-based Human Activity Recognition Using Wearables: A Surveying Tutorial on a Still Challenging Task (2025) · cites this paper as [109]
In the citing paper
“The system leads to improving model performance when integrating real and virtual IMU data [108,109].”
From Past, Present, and Future of Sensor-based Human Activity Recognition Using Wearables: A Surveying Tutorial on a Still Challenging Task (2025) · cites this paper as [109]
In the citing paper
“Cross-modality transfer approaches [54,108,109,161,162] are instrumental for extracting virtual IMU data from 2D RGB videos of activities.”
From Past, Present, and Future of Sensor-based Human Activity Recognition Using Wearables: A Surveying Tutorial on a Still Challenging Task (2025) · cites this paper as [109]
In the citing paper
“IMUTube [109] extracts virtual IMU data from 2D RGB videos using pose tracking.”
From Past, Present, and Future of Sensor-based Human Activity Recognition Using Wearables: A Surveying Tutorial on a Still Challenging Task (2025) · cites this paper as [109]
In the citing paper
“In recent years, there has been a growing interest in the combination of data obtained from videobased systems together with the time-series data originating from on-body sensors Rey et al. (2019); Kwon et al. (2020); Fortes Rey et al. (2021).”
From Opportunity++: A Multimodal Dataset for Video- and Wearable, Object and Ambient Sensors-Based Human Activity Recognition (2021) · cites this paper as Kwon et al. (2020)
8 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Girshick, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.169 · Cited by 28,105
A method for registration of 3-D shapes
Besl, P. J., & McKay, N. D. (1992) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.121791 · Cited by 18,103
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S. (2024) · arXiv (Cornell University) · doi:10.57702/o9raffed · Cited by 14,214
A survey on Image Data Augmentation for Deep Learning
Shorten, C., & Khoshgoftaar, T. M. (2019) · Journal of Big Data · doi:10.1186/s40537-019-0197-0 · Cited by 13,203 · cited 2×
“Recently, data generation using either oversampling or generative adversarial networks (GANs [82]) have also been successfully introduced to sensor-based human activity recognition [93].”
BackgroundMethodHinton, G., Deng, L., Yu, D., Dahl, G., Mohamed, A.-R., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T., et al. (2012) · IEEE Signal Processing Magazine · doi:10.1109/msp.2012.2205597 · Cited by 10,260 · cited 1×
“Regardless of its utility, the HAR field has yet to experience significant improvements in recognition accuracy, in contrast to the breakthroughs in other fields, such as speech recognition [34], natural language processing [19], and computer vision [32].”
BackgroundGANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., & Hochreiter, S. (2017) · arXiv · doi:10.48550/arxiv.1706.08500 · Cited by 8,365
How transferable are features in deep neural networks?
Yosinski, J., Clune, J., Bengio, Y., & Lipson, H. (2014) · arXiv · doi:10.48550/arxiv.1411.1792 · Cited by 6,061 · cited 1×
“In particular, if the target dataset is significantly smaller compared to the base dataset, transfer learning enables training a large target network without overfitting [94], and typically results in improved performance.”
BackgroundUcf101: A dataset of 101 human actions classes from videos in the wild
Soomro, K., Zamir, A. R., & Shah, M. (2012) · arXiv · doi:10.48550/arxiv.1212.0402 · Cited by 4,575
Simple online and realtime tracking
Bewley, A., Ge, Z., Ott, L., Ramos, F., & Upcroft, B. (2016) · 2016 IEEE International Conference on Image Processing (ICIP) · doi:10.1109/icip.2016.7533003 · Cited by 4,433
Hmdb: a large video database for human motion recognition
Kuehne, H., Jhuang, H., Garrote, E., Poggio, T., & Serre, T. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126543 · Cited by 3,965 · cited 1×
“ing and cycling. To efficiently locate such videos, we extract annotated video segments from activity video datasets in the computer vision domain, including ActivityNet [9], Kinetics700 [12], HMDB51 [44], MPIIHPD [3], UCF101 [84], Charades [83], AVA [26], MSRdailyactivity3D [49], and NTU RGBD [51]. The resulting vide…”
MethodEfficient variants of the ICP algorithm
Rusinkiewicz, S., & Levoy, M. (2002) · Proceedings Third International Conference on 3-D Digital Imaging and Modeling · doi:10.1109/im.2001.924423 · Cited by 3,945 · cited 2×
“ra ego-motion (rotation and translation) parameters between two consecutive frame point clouds. A popular method for registering groups of point clouds is the Iterative Closest Points (ICP) algorithm [6, 66, 76]. Fixing a point cloud as a reference, ICP iteratively finds closest point pairs between two point clouds an…”
MethodRank transformations as a bridge between parametric and nonparametric statistics
Conover, W. J., & Iman, R. L. (1981) · The American Statistician · doi:10.1080/00031305.1981.10479327 · Cited by 3,922
2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
(2015) · doi:10.1109/cvpr31182.2015 · Cited by 3,737
Loper, M., Mahmood, N., Romero, J., Pons-Moll, G., & J, B. M. (2015) · ACM Transactions on Graphics · doi:10.1145/2816795.2818013 · Cited by 3,734
2017 IEEE International Conference on Computer Vision (ICCV)
(2017) · doi:10.1109/iccv37128.2017 · Cited by 3,056
Unsupervised Learning of Depth and Ego-Motion from Video
Zhou, T., Brown, M., Snavely, N., & Lowe, D. G. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.700 · Cited by 2,910 · cited 1×
“Specialized video stabilization strategies or camera ego-motion techniques can address these issues [80, 96, 100].”
BackgroundDeep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition
Ordóñez, F., & Roggen, D. (2016) · Sensors · doi:10.3390/s16010115 · Cited by 2,761 · cited 15×
“Method We employ two machine learning models, Random Forest and DeepConvLSTM [54].”
MethodInfluentialActivityNet: A large-scale video benchmark for human activity understanding
Heilbron, F. C., Escorcia, V., Ghanem, B., & Niebles, J. C. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298698 · Cited by 2,578
NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding
Liu, J., Shahroudy, A., Perez, M., Wang, G., Duan, L.-Y., & Kot, A. C. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2916873 · Cited by 1,818
Deconvolution and Checkerboard Artifacts
Odena, A., Dumoulin, V., & Olah, C. (2016) · Distill · doi:10.23915/distill.00003 · Cited by 1,640
British Machine Vision Conference (BMVC)
Damen, D., & Hogg, D. (2007) · Cited by 1,213
Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI 2017)
Amendola, G., Ricca, F., & Truszczyński, M. (2017) · International Joint Conference on Artificial Intelligence · Cited by 1,179
One Billion Word Benchmark for Measuring Progress in Statistical Language Modeling
Chelba, C., Mikolov, T., Schuster, M., Ge, Q., Brants, T., Koehn, P., & Robinson, T. (2013) · arXiv · doi:10.48550/arxiv.1312.3005 · Cited by 1,051
Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems
(2024) · doi:10.1145/3613904 · Cited by 896
The Opportunity challenge: A benchmark database for on-body sensor-based activity recognition
Chavarriaga, R., Sagha, H., Calatroni, A., Digumarti, S. T., Tröster, G., Millán, J. D. R., & Roggen, D. (2013) · Pattern Recognition Letters · doi:10.1016/j.patrec.2012.12.014 · Cited by 742 · cited 6×
“Sensor data collection for human activity recognition is often performed by conducting user studies [14, 72, 97].”
BackgroundMethodInfluentialA Review of Point Cloud Registration Algorithms for Mobile Robotics
Pomerleau, F., Colas, F., & Siegwart, R. (2015) · Foundations and Trends® in Robotics · doi:10.1561/2300000035 · Cited by 717
Using vanishing points for camera calibration
Caprile, B., & Torre, V. (1990) · International Journal of Computer Vision · doi:10.1007/bf00127813 · Cited by 711
Wearable Assistant for Parkinson’s Disease Patients With the Freezing of Gait Symptom
Bachlin, M., Plotnik, M., Roggen, D., Maidan, I., Hausdorff, J. M., Giladi, N., & Troster, G. (2010) · IEEE Transactions on Information Technology in Biomedicine · doi:10.1109/titb.2009.2036165 · Cited by 693 · cited 2×
“Tong, et al. 1 INTRODUCTION On-body sensor-based human activity recognition (HAR) is widely utilized for behavioral analysis, such as user authentication, healthcare, and tracking everyday activities [5, 14, 50, 78, 97]. Regardless of its utility, the HAR field has yet to experience significant improvements in recogni…”
BackgroundOne billion word benchmark for measuring progress in statistical language modeling
Chelba, C., Mikolov, T., Schuster, M., Ge, Q., Brants, T., Koehn, P., & Robinson, T. (2014) · Interspeech 2014 · doi:10.21437/interspeech.2014-564 · Cited by 627
Deep, Convolutional, and Recurrent Models for Human Activity Recognition Using Wearables
Hammerla, N. Y., Halloran, S., & Ploetz, T. (2016) · arXiv · doi:10.48550/arxiv.1604.08880 · Cited by 606
Ensembles of Deep LSTM Learners for Activity Recognition using Wearables
Guan, Y., & Plötz, T. (2017) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3090076 · Cited by 511 · cited 1×
“available datasets for human activity recognition contain imbalanced classes. For example, approximately 75% of the Opportunity dataset (which has 18 classes in total) [14] consists of the null class [27], making it challenging to design classifiers. The activities being studied also impact the class imbalance to some…”
BackgroundInfluentialSelf-citationClothCap: seamless 4D clothing capture and retargeting
Pons-Moll, G., Pujades, S., Hu, S., & Black, M. J. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073711 · Cited by 495
Depth From Videos in the Wild: Unsupervised Monocular Depth Learning From Unknown Cameras
Gordon, A., Li, H., Jonschkowski, R., & Angelova, A. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00907 · Cited by 389
On-body localization of wearable devices: An investigation of position-aware activity recognition
Sztyler, T., & Stuckenschmidt, H. (2016) · 2016 IEEE International Conference on Pervasive Computing and Communications (PerCom) · doi:10.1109/percom.2016.7456521 · Cited by 369
2023 IEEE International Conference on Image Processing (ICIP)
(2023) · doi:10.1109/icip49359.2023 · Cited by 353
Huang, Y., Kaufmann, M., Aksan, E., Black, M. J., Hilliges, O., & Pons-Moll, G. (2018) · ACM Transactions on Graphics · doi:10.1145/3272127.3275108 · Cited by 342
Times-series data augmentation and deep learning for construction equipment activity recognition
Rashid, K. M., & Louis, J. (2019) · Advanced Engineering Informatics · doi:10.1016/j.aei.2019.100944 · Cited by 340
A Short Note on the Kinetics-700-2020 Human Action Dataset
Smaira, L., Carreira, J., Noland, E., Clancy, E., Wu, A., & Zisserman, A. (2020) · arXiv · doi:10.48550/arxiv.2010.10864 · Cited by 334
Dyna: a model of dynamic human shape in motion
Pons-Moll, G., Romero, J., Mahmood, N., & Black, M. J. (2015) · ACM Transactions on Graphics · doi:10.1145/2766993 · Cited by 323
Steven Eyobu, O., & Han, D. S. (2018) · Sensors · doi:10.3390/s18092892 · Cited by 259 · cited 1×
“ng, rotation and warping are detailed and their effects relative to no augmentation is studied. Further, the authors observed that combining multiple transformations results in higher performance. In [85], augmentation is performed on IMU spectrogram features to improve the activity recognition performance. Another ap…”
BackgroundShah, S., & Aggarwal, J. K. (1996) · Pattern Recognition · doi:10.1016/0031-3203(96)00038-6 · Cited by 194
Sensing Fine-Grained Hand Activity with Smartwatches
Laput, G., & Harrison, C. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300568 · Cited by 147
A Systematic Study of Unsupervised Domain Adaptation for Robust Human-Activity Recognition
Chang, Y., Mathur, A., Isopoussu, A., Song, J., & Kawsar, F. (2020) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3380985 · Cited by 138
Hammerla, N. Y., Kirkham, R., Andras, P., & Ploetz, T. (2013) · Proceedings of the 2013 International Symposium on Wearable Computers · doi:10.1145/2493988.2494353 · Cited by 138 · cited 1×
“ix2R, we follow the same scheme but use both real and virtual IMU data. Method We employ two machine learning models, Random Forest and DeepConvLSTM [54]. Random forest is trained using ECDF features [25] with 15 components, and DeepConvLSTM is trained on raw data. For DeepConvLSTM, we train the model for a maximum of…”
MethodSelf-citationReview on Smart Electro-Clothing Systems (SeCSs)
Muhammad Sayem, A. S., Hon Teay, S., Shahariar, H., Luise Fink, P., & Albarbar, A. (2020) · Sensors · doi:10.3390/s20030587 · Cited by 132
USA, U. '. T. 2. A. I. J. C. O. P. A. U. C. 2. V., Association for Computing Machinery-Digital Library, ACM Special Interest Group on Computer-Human Interaction, ACM Special Interest Group on Mobility of Systems, U., ACM Special Interest Group on Spatial Information, & SIGs, A. (2021) · doi:10.1145/3460418 · Cited by 110
Few-shot learning-based human activity recognition
Feng, S., & Duarte, M. F. (2019) · Expert Systems with Applications · doi:10.1016/j.eswa.2019.06.070 · Cited by 91
Are GANs Created Equal? A Large-Scale Study
Lučić, M., Kurach, K., Michalski, M., Gelly, S., & Bousquet, O. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.10337 · Cited by 73
Data augmentation using synthetic data for time series classification with deep residual networks
Fawaz, H. I., Forestier, G., Weber, J., Idoumghar, L., & Müller, P.-A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.02455 · Cited by 72
WearBreathing: Real World Respiratory Rate Monitoring Using Smartwatches
Liaqat, D., Abdalla, M., Abed-Esfahani, P., Gabel, M., Son, T., Wu, R., Gershon, A., Rudzicz, F., & Lara, E. D. (2019) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3328927 · Cited by 58
Wearables in the wet lab: a laboratory system for capturing and guiding experiments
Scholl, P. M., Wille, M., & Van Laerhoven, K. (2015) · Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing · doi:10.1145/2750858.2807547 · Cited by 45
Kang, S.-W., Choi, H., Park, H.-I., Choi, B.-G., Im, H., Shin, D., Jung, Y.-G., Lee, J., Park, H.-W., Park, S., et al. (2017) · Sensors · doi:10.3390/s17112560 · Cited by 41
Proceedings of the 2022 ACM International Symposium on Wearable Computers
Kingdom, U. '. T. 2. A. I. J. C. O. P. A. U. C. 2. C. U., Association for Computing Machinery-Digital Library, ACM Special Interest Group on Computer-Human Interaction, ACM Special Interest Group on Mobility of Systems, U., ACM Special Interest Group on Spatial Information, & SIGs, A. (2022) · doi:10.1145/3544794 · Cited by 26
A self-calibration approach to extrinsic parameter estimation of stereo cameras
Zhuang, H. (1995) · Robotics and Autonomous Systems · doi:10.1016/0921-8890(95)00020-g · Cited by 24
SMOTE for learning from imbalanced data
FernándezAlberto, GarcíaSalvador, HerreraFrancisco, & ChawlaNitesh, V. (2018) · doi:10.5555/3241691.3241712 · Cited by 15
Spline Function Simulation Data Generation for Walking Motion Using Foot-Mounted Inertial Sensors
Pham, T. T., & Suh, Y. S. (2018) · Electronics · doi:10.3390/electronics8010018 · Cited by 9
Asare, P., Dickerson, R. F., Wu, X., Lach, J., & Stankovic, J. A. (2013) · Proceedings of the 11th ACM Conference on Embedded Networked Sensor Systems · doi:10.1145/2517351.2517392 · Cited by 8
Research works printed but not linked: 40
- S. Alireza Golestaneh and L. Karam . 2017. Spatially-varying blur detection based on multiscale fused and sorted transform coefficients of gradient magnitudes . In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 5800--5809 . S. Alireza Golestaneh and L. Karam. 2017. Spatially-varying blur detection based on multiscale fused and sorted transform coefficients of gradient magnitudes. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 5800--5809.
- T. Alldieck , M. Magnor , B. Bhatnagar , C. Theobalt , and G. Pons-Moll . 2019. Learning to reconstruct people in clothing from a single RGB camera . In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 1175--1186 . T. Alldieck, M. Magnor, B. Bhatnagar, C. Theobalt, and G. Pons-Moll. 2019. Learning to reconstruct people in clothing from a single RGB camera. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 1175--1186.
- M. Andriluka , L. Pishchulin , P. Gehler , and B. Schiele . 2014. 2D Human Pose Estimation: New Benchmark and State of the Art Analysis . In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). M. Andriluka, L. Pishchulin, P. Gehler, and B. Schiele. 2014. 2D Human Pose Estimation: New Benchmark and State of the Art Analysis. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
- O. Bogdan , V. Eckstein , F. Rameau , and J. Bazin . 2018 . DeepCalib: a deep learning approach for automatic intrinsic calibration of wide field-of-view cameras . In Proceedings of the ACM SIGGRAPH European Conference on Visual Media Production. ACM , 6 : 1 - 6 :10. O. Bogdan, V. Eckstein, F. Rameau, and J. Bazin. 2018. DeepCalib: a deep learning approach for automatic intrinsic calibration of wide field-of-view cameras. In Proceedings of the ACM SIGGRAPH European Conference on Visual Media Production. ACM, 6:1-6:10.
- Z. Cao , T. Simon , S. Wei , and Y. Sheikh . 2017. Realtime multi-person 2d pose estimation using part affinity fields . In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 7291--7299 . Z. Cao, T. Simon, S. Wei, and Y. Sheikh. 2017. Realtime multi-person 2d pose estimation using part affinity fields. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 7291--7299.
- Blender Online Community . 2018. Blender - a 3D modelling and rendering package . Blender Foundation, Stichting Blender Foundation, Amsterdam . http://www.blender.org Blender Online Community. 2018. Blender - a 3D modelling and rendering package. Blender Foundation, Stichting Blender Foundation, Amsterdam. http://www.blender.org
- C. Gu , C. Sun , D. Ross , C. Vondrick , C. Pantofaru , Y. Li , S. Vijayanarasimhan , G. Toderici , S. Ricco , R. Sukthankar , C. Schmid , and J. Malik . 2018. Ava: A video dataset of spatio-temporally localized atomic visual actions . In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 6047--6056 . C. Gu, C. Sun, D. Ross, C. Vondrick, C. Pantofaru, Y. Li, S. Vijayanarasimhan, G. Toderici, S. Ricco, R. Sukthankar, C. Schmid, and J. Malik. 2018. Ava: A video dataset of spatio-temporally localized atomic visual actions. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 6047--6056.
- S. Haradal , H. Hayashi , and S. Uchida . 2018. Biosignal data augmentation based on generative adversarial networks . In Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE, 368--371 . S. Haradal, H. Hayashi, and S. Uchida. 2018. Biosignal data augmentation based on generative adversarial networks. In Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE, 368--371.
- K. He , X. Zhang , S. Ren , and J. Sun . 2016. Deep residual learning for image recognition . In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 770--778 . K. He, X. Zhang, S. Ren, and J. Sun. 2016. Deep residual learning for image recognition. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 770--778.
- I. Joel , A.and Stergios. 2011 . A Direct Least-Squares (DLS) method for PnP . In IEEE International Conference on Computer Vision (ICCV). IEEE. I. Joel, A.and Stergios. 2011. A Direct Least-Squares (DLS) method for PnP. In IEEE International Conference on Computer Vision (ICCV). IEEE.
- A. Kanazawa , M. Black , D. Jacobs , and J. Malik . 2018. End-to-end recovery of human shape and pose . In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 7122--7131 . A. Kanazawa, M. Black, D. Jacobs, and J. Malik. 2018. End-to-end recovery of human shape and pose. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 7122--7131.
- C. Kang , H. Jung , and Y. Lee . 2019 . Towards Machine Learning with Zero Real-World Data. In The ACM Workshop on Wearable Systems and Applications. 41--46 . C. Kang, H. Jung, and Y. Lee. 2019. Towards Machine Learning with Zero Real-World Data. In The ACM Workshop on Wearable Systems and Applications. 41--46.
- P. Karlsson , B. Lo , and G. Z. Yang . 2014. Inertial sensing simulations using modified motion capture data . In Proceedings of the International Conference on Wearable and Implantable Body Sensor Networks (BSN). 16--19 . P. Karlsson, B. Lo, and G. Z. Yang. 2014. Inertial sensing simulations using modified motion capture data. In Proceedings of the International Conference on Wearable and Implantable Body Sensor Networks (BSN). 16--19.
- D. Kingma and J. Ba . 2014 . Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014). D. Kingma and J. Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014). · arXiv
- N. Lane , Y. Xu , H. Lu , S. Hu , T. Choudhury , A. Campbell , and F. Zhao . 2011. Enabling Large-Scale Human Activity Inference on Smartphones Using Community Similarity Networks . In Proceedings of the International Conference on Ubiquitous Computing. ACM, 355--364 . N. Lane, Y. Xu, H. Lu, S. Hu, T. Choudhury, A. Campbell, and F. Zhao. 2011. Enabling Large-Scale Human Activity Inference on Smartphones Using Community Similarity Networks. In Proceedings of the International Conference on Ubiquitous Computing. ACM, 355--364.
- A. Le Guennec , S. Malinowski , and R. Tavenard . 2016. Data Augmentation for Time Series Classification using Convolutional Neural Networks . In ECML/PKDD Workshop on Advanced Analytics and Learning on Temporal Data. A. Le Guennec, S. Malinowski, and R. Tavenard. 2016. Data Augmentation for Time Series Classification using Convolutional Neural Networks. In ECML/PKDD Workshop on Advanced Analytics and Learning on Temporal Data.
- N. Mahmood , N. Ghorbani , N. Troje , G. Pons-Moll , and M. Black . 2019. AMASS: Archive of motion capture as surface shapes . In IEEE International Conference on Computer Vision (ICCV). 5442--5451 . N. Mahmood, N. Ghorbani, N. Troje, G. Pons-Moll, and M. Black. 2019. AMASS: Archive of motion capture as surface shapes. In IEEE International Conference on Computer Vision (ICCV). 5442--5451.
- A. Mathur , T. Zhang , S. Bhattacharya , P. Velickovic , L. Joffe , N. Lane , F. Kawsar , and P. Lió . 2018. Using deep data augmentation training to address software and hardware heterogeneities in wearable and smartphone sensing devices . In IEEE International Conference on Information Processing in Sensor Networks (IPSN). IEEE, 200--211 . A. Mathur, T. Zhang, S. Bhattacharya, P. Velickovic, L. Joffe, N. Lane, F. Kawsar, and P. Lió. 2018. Using deep data augmentation training to address software and hardware heterogeneities in wearable and smartphone sensing devices. In IEEE International Conference on Information Processing in Sensor Networks (IPSN). IEEE, 200--211.
- V. Nair and G. Hinton . 2010. Rectified linear units improve restricted boltzmann machines . In Proceedings of the international conference on machine learning (ICML). 807--814 . V. Nair and G. Hinton. 2010. Rectified linear units improve restricted boltzmann machines. In Proceedings of the international conference on machine learning (ICML). 807--814.
- F. Ofli , R. Chaudhry , G. Kurillo , R. Vidal , and R. Bajcsy . 2013. Berkeley mhad: A comprehensive multimodal human action database . In IEEE Workshop on Applications of Computer Vision (WACV). IEEE, 53--60 . F. Ofli, R. Chaudhry, G. Kurillo, R. Vidal, and R. Bajcsy. 2013. Berkeley mhad: A comprehensive multimodal human action database. In IEEE Workshop on Applications of Computer Vision (WACV). IEEE, 53--60.
- A. Oord , S. Dieleman , H. Zen , K. Simonyan , O. Vinyals , A. Graves , N. Kalchbrenner , A. Senior , and K. Kavukcuoglu . 2016 . Wavenet: A generative model for raw audio. arXiv preprint arXiv:1609.03499 (2016). A. Oord, S. Dieleman, H. Zen, K. Simonyan, O. Vinyals, A. Graves, N. Kalchbrenner, A. Senior, and K. Kavukcuoglu. 2016. Wavenet: A generative model for raw audio. arXiv preprint arXiv:1609.03499 (2016). · arXiv
- D. Pavllo , C. Feichtenhofer , D. Grangier , and M. Auli . 2019. 3D human pose estimation in video with temporal convolutions and semi-supervised training . In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 7753--7762 . D. Pavllo, C. Feichtenhofer, D. Grangier, and M. Auli. 2019. 3D human pose estimation in video with temporal convolutions and semi-supervised training. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 7753--7762.
- T. Plötz , C. Chen , N. Hammerla , and G. Abowd . 2012. Automatic synchronization of wearable sensors and video-cameras for ground truth annotation-a practical approach . In Proceedings of the ACM International Symposium on Wearable Computers. IEEE, 100--103 . T. Plötz, C. Chen, N. Hammerla, and G. Abowd. 2012. Automatic synchronization of wearable sensors and video-cameras for ground truth annotation-a practical approach. In Proceedings of the ACM International Symposium on Wearable Computers. IEEE, 100--103.
- G. Ramponi , P. Protopapas , M. Brambilla , and R. Janssen . 2018 . T-cgan: Conditional generative adversarial network for data augmentation in noisy time series with irregular sampling. arXiv preprint arXiv:1811.08295 (2018). G. Ramponi, P. Protopapas, M. Brambilla, and R.Janssen. 2018. T-cgan: Conditional generative adversarial network for data augmentation in noisy time series with irregular sampling. arXiv preprint arXiv:1811.08295 (2018). · arXiv
- J. Redmon , S. Divvala , R. Girshick , and A. Farhadi . 2016. You only look once: Unified, real-time object detection . In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 779--788 . J. Redmon, S. Divvala, R. Girshick, and A. Farhadi. 2016. You only look once: Unified, real-time object detection. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 779--788.
- A. Reiss and D. Stricker . 2012. Introducing a new benchmarked dataset for activity monitoring . In Proceedings of the ACM International Symposium on Wearable Computers. IEEE, 108--109 . A. Reiss and D. Stricker. 2012. Introducing a new benchmarked dataset for activity monitoring. In Proceedings of the ACM International Symposium on Wearable Computers. IEEE, 108--109.
- A. Reiss and D. Stricker . 2013. Personalized mobile physical activity recognition . In Proceedings of the ACM International Symposium on Wearable Computers. 25--28 . A. Reiss and D. Stricker. 2013. Personalized mobile physical activity recognition. In Proceedings of the ACM International Symposium on Wearable Computers. 25--28.
- M. Rosca B. Lakshminarayanan and S. Mohamed. 2018. Distribution matching in variational inference. arXiv preprint arXiv:1802.06847 (2018). M. Rosca B. Lakshminarayanan and S. Mohamed. 2018. Distribution matching in variational inference. arXiv preprint arXiv:1802.06847 (2018). · arXiv
- A. Saeed , T. Ozcelebi , and J. Lukkien . 2019. Multi-task Self-Supervised Learning for Human Activity Detection . Proceedings of the ACM on interactive, mobile, wearable and ubiquitous technologies (IMWUT) 3 , 2 ( 2019 ), 1--30. A. Saeed, T. Ozcelebi, and J. Lukkien. 2019. Multi-task Self-Supervised Learning for Human Activity Detection. Proceedings of the ACM on interactive, mobile, wearable and ubiquitous technologies (IMWUT) 3, 2 (2019), 1--30.
- Z. Shen , W. Wang , X. Lu , J. Shen , H. Ling , T. Xu , and L. Shao . 2019 . Human-Aware Motion Deblurring. In IEEE International Conference on Computer Vision (ICCV). 5572--5581 . Z. Shen, W. Wang, X. Lu, J. Shen, H. Ling, T. Xu, and L. Shao. 2019. Human-Aware Motion Deblurring. In IEEE International Conference on Computer Vision (ICCV). 5572--5581.
- J. Shi , L. Xu , and J. Jia . 2014. Discriminative blur detection features . In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2965--2972 . J. Shi, L. Xu, and J. Jia. 2014. Discriminative blur detection features. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2965--2972.
- G. Sigurdsson , G. Varol , X. Wang , I. Laptev , A. Farhadi , and A. Gupta . 2016 . Hollywood in Homes: Crowdsourcing Data Collection for Activity Understanding. arXiv preprint arXiv:1604.01753 (2016). G. Sigurdsson, G. Varol, X. Wang, I. Laptev, A. Farhadi, and A. Gupta. 2016. Hollywood in Homes: Crowdsourcing Data Collection for Activity Understanding. arXiv preprint arXiv:1604.01753 (2016). · arXiv
- E. Thomaz , I. Essa , and G. Abowd . 2015. A practical approach for recognizing eating moments with wrist-mounted inertial sensing . In Proceedings of the ACM International Joint Conference on Pervasive and Ubiquitous Computing. 1029--1040 . E. Thomaz, I. Essa, and G. Abowd. 2015. A practical approach for recognizing eating moments with wrist-mounted inertial sensing. In Proceedings of the ACM International Joint Conference on Pervasive and Ubiquitous Computing. 1029--1040.
- C. Tong , S. Tailor , and N. Lane . 2020. Are Accelerometers for Activity Recognition a Dead-End? . In Proceedings of the International Workshop on Mobile Computing Systems and Applications. ACM, 39--44 . C. Tong, S. Tailor, and N. Lane. 2020. Are Accelerometers for Activity Recognition a Dead-End?. In Proceedings of the International Workshop on Mobile Computing Systems and Applications. ACM, 39--44.
- F. Xiao L. Pei L. Chu D. Zou W. Yu Y. Zhu and T. Li. 2020. A Deep Learning Method for Complex Human Activity Recognition Using Virtual Wearable Sensors. arXiv preprint arXiv:2003.01874 (2020). F. Xiao L. Pei L. Chu D. Zou W. Yu Y. Zhu and T. Li. 2020. A Deep Learning Method for Complex Human Activity Recognition Using Virtual Wearable Sensors. arXiv preprint arXiv:2003.01874 (2020). · arXiv
- S. Yao , Y. Zhao , H. Shao , C. Zhang , A. Zhang , S. Hu , D. Liu , S. Liu , Lu Su , and T. Abdelzaher . 2018. Sensegan: Enabling deep learning for internet of things with a semi-supervised framework . Proceedings of the ACM on interactive, mobile, wearable and ubiquitous technologies (IMWUT) 2 , 3 ( 2018 ), 1--21. S. Yao, Y. Zhao, H. Shao, C. Zhang, A. Zhang, S. Hu, D. Liu, S. Liu, Lu Su, and T. Abdelzaher. 2018. Sensegan: Enabling deep learning for internet of things with a semi-supervised framework. Proceedings of the ACM on interactive, mobile, wearable and ubiquitous technologies (IMWUT) 2, 3 (2018), 1--21.
- A. Young , M. Ling , and D. Arvind . 2011. IMUSim: A simulation environment for inertial sensing algorithm design and evaluation . In Proceedings of the International Conference on Information Processing in Sensor Networks (IPSN). IEEE, 199--210 . A. Young, M. Ling, and D. Arvind. 2011. IMUSim: A simulation environment for inertial sensing algorithm design and evaluation. In Proceedings of the International Conference on Information Processing in Sensor Networks (IPSN). IEEE, 199--210.
- J. Yu and R. Ramamoorthi . 2019. Robust Video Stabilization by Optimization in CNN Weight Space . In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 3800--3808 . J. Yu and R. Ramamoorthi. 2019. Robust Video Stabilization by Optimization in CNN Weight Space. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 3800--3808.
- M. Zhang and A. A. Sawchuk . 2012. USC-HAD: a daily activity dataset for ubiquitous activity recognition using wearable sensors . In Proceedings of the International Conference on Ubiquitous Computing. M. Zhang and A. A. Sawchuk. 2012. USC-HAD: a daily activity dataset for ubiquitous activity recognition using wearable sensors. In Proceedings of the International Conference on Ubiquitous Computing.
- Q. Zhang and R. Pless . 2004. Extrinsic calibration of a camera and laser range finder (improves camera calibration) . In IEEE International Conference on Intelligent Robots and Systems (IROS). IEEE. Q. Zhang and R. Pless. 2004. Extrinsic calibration of a camera and laser range finder (improves camera calibration). In IEEE International Conference on Intelligent Robots and Systems (IROS). IEEE.
Cited by
Deep Learning in Human Activity Recognition with Wearable Sensors: A Review on Advances
Zhang, S., Li, Y., Zhang, S., Shahabi, F., Xia, S., Deng, Y., & Alshurafa, N. (2022) · Sensors · doi:10.3390/s22041476 · Cited by 625
Contrastive Predictive Coding for Human Activity Recognition
Haresamudram, H., Essa, I., & Plötz, T. (2021) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3463506 · Cited by 150
Self-citationAssessing the State of Self-Supervised Human Activity Recognition Using Wearables
Haresamudram, H., Essa, I., & Plötz, T. (2022) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3550299 · Cited by 107
Self-citationImproving Deep Learning for HAR with Shallow LSTMs
Bock, M., Hölzemann, A., Moeller, M., & Van Laerhoven, K. (2021) · 2021 International Symposium on Wearable Computers · doi:10.1145/3460421.3480419 · Cited by 74
Towards Generalized mmWave-based Human Pose Estimation through Signal Augmentation
Xue, H., Cao, Q., Miao, C., Ju, Y., Hu, H., Zhang, A., & Su, L. (2023) · ACM Annual International Conference on Mobile Computing and Networking (MobiCom) · doi:10.1145/3570361.3613302 · Cited by 95
Metadata
- Authors
- Kwon, H., Tong, C., Haresamudram, H., Gao, Y., Abowd, G. D., Lane, N. D., & Plötz, T.
- Affiliations
- Georgia Institute of Technology (US)
- University of Oxford (GB)
- Venue
- Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
- Published
- 2020-09-04
- Volume
- 4
- Issue
- 3
- Pages
- 1-29
- ISSN
- 2474-9567 (ISSN-L)
- OpenAlex
- W3083323811
- Cited by
- 152
- References
- 49
- Citations by year
- 2026: 20 · 2025: 29 · 2024: 24 · 2023: 32 · 2022: 23 · 2021: 24
- FWCI
- 7.68
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies (publishedVersion)
- Funders
- Engineering and Physical Sciences Research Council (1892895)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Physical Sciences › Engineering › Biomedical Engineering › Gait Recognition and Analysis
- Keywords
- human activity recognition, on-body sensors, inertial measurement unit, automatic annotation, accelerometry, wearable sensors, labeled data scarcity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.