IMUTube: Automatic Extraction of Virtual on-body Accelerometry from Video for Human Activity Recognition
The DOI matches its Crossref record; 70 of the 70 references it cites were found in a registry or an index.
Abstract
The lack of large-scale, labeled data sets impedes progress in developing robust and generalized predictive models for on-body sensor-based human activity recognition (HAR). Labeled data in human activity recognition is scarce and hard to come by, as sensor data collection is expensive, and the annotation is time-consuming and error-prone. To address this problem, we introduce IMUTube, an automated processing pipeline that integrates existing computer vision and signal processing techniques to convert videos of human activity into virtual streams of IMU data. These virtual IMU streams represent accelerometry at a wide variety of locations on the human body. We show how the virtually-generated IMU data improves the performance of a variety of models on known HAR datasets. Our initial results are very promising, but the greater promise of this work lies in a collective approach by the computer vision, signal processing, and activity recognition communities to extend this work in ways that we outline. This should lead to on-body, sensor-based HAR becoming yet another success story in large-dataset breakthroughs in recognition.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Due to these factors, publicly available wearable sensor datasets are typically limited in size and variability, often containing only 10-20 participants and a handful of activities, recorded over a few hours [109].”
From Past, Present, and Future of Sensor-based Human Activity Recognition Using Wearables: A Surveying Tutorial on a Still Challenging Task (2025) · cites this paper as [109]
In the citing paper
“The system leads to improving model performance when integrating real and virtual IMU data [108,109].”
From Past, Present, and Future of Sensor-based Human Activity Recognition Using Wearables: A Surveying Tutorial on a Still Challenging Task (2025) · cites this paper as [109]
In the citing paper
“Cross-modality transfer approaches [54,108,109,161,162] are instrumental for extracting virtual IMU data from 2D RGB videos of activities.”
From Past, Present, and Future of Sensor-based Human Activity Recognition Using Wearables: A Surveying Tutorial on a Still Challenging Task (2025) · cites this paper as [109]
In the citing paper
“IMUTube [109] extracts virtual IMU data from 2D RGB videos using pose tracking.”
From Past, Present, and Future of Sensor-based Human Activity Recognition Using Wearables: A Surveying Tutorial on a Still Challenging Task (2025) · cites this paper as [109]
In the citing paper
“In recent years, there has been a growing interest in the combination of data obtained from videobased systems together with the time-series data originating from on-body sensors Rey et al. (2019); Kwon et al. (2020); Fortes Rey et al. (2021).”
From Opportunity++: A Multimodal Dataset for Video- and Wearable, Object and Ambient Sensors-Based Human Activity Recognition (2021) · cites this paper as Kwon et al. (2020)
8 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Girshick, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.169 · Cited by 28,105
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818
A method for registration of 3-D shapes
Besl, P. J., & McKay, N. D. (1992) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.121791 · Cited by 18,103
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S. (2024) · arXiv (Cornell University) · doi:10.57702/o9raffed · Cited by 14,214
A survey on Image Data Augmentation for Deep Learning
Shorten, C., & Khoshgoftaar, T. M. (2019) · Journal of Big Data · doi:10.1186/s40537-019-0197-0 · Cited by 13,203 · cited 2×
“Recently, data generation using either oversampling or generative adversarial networks (GANs [82]) have also been successfully introduced to sensor-based human activity recognition [93].”
BackgroundMethodHinton, G., Deng, L., Yu, D., Dahl, G., Mohamed, A.-R., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T., et al. (2012) · IEEE Signal Processing Magazine · doi:10.1109/msp.2012.2205597 · Cited by 10,260 · cited 1×
“Regardless of its utility, the HAR field has yet to experience significant improvements in recognition accuracy, in contrast to the breakthroughs in other fields, such as speech recognition [34], natural language processing [19], and computer vision [32].”
BackgroundGANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., & Hochreiter, S. (2017) · arXiv · doi:10.48550/arxiv.1706.08500 · Cited by 8,365
How transferable are features in deep neural networks?
Yosinski, J., Clune, J., Bengio, Y., & Lipson, H. (2014) · arXiv · doi:10.48550/arxiv.1411.1792 · Cited by 6,061 · cited 1×
“In particular, if the target dataset is significantly smaller compared to the base dataset, transfer learning enables training a large target network without overfitting [94], and typically results in improved performance.”
Background
Research works printed but not linked: 37
- S. Alireza Golestaneh and L. Karam . 2017. Spatially-varying blur detection based on multiscale fused and sorted transform coefficients of gradient magnitudes . In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 5800--5809 . S. Alireza Golestaneh and L. Karam. 2017. Spatially-varying blur detection based on multiscale fused and sorted transform coefficients of gradient magnitudes. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 5800--5809.
- T. Alldieck , M. Magnor , B. Bhatnagar , C. Theobalt , and G. Pons-Moll . 2019. Learning to reconstruct people in clothing from a single RGB camera . In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 1175--1186 . T. Alldieck, M. Magnor, B. Bhatnagar, C. Theobalt, and G. Pons-Moll. 2019. Learning to reconstruct people in clothing from a single RGB camera. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 1175--1186.
- M. Andriluka , L. Pishchulin , P. Gehler , and B. Schiele . 2014. 2D Human Pose Estimation: New Benchmark and State of the Art Analysis . In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). M. Andriluka, L. Pishchulin, P. Gehler, and B. Schiele. 2014. 2D Human Pose Estimation: New Benchmark and State of the Art Analysis. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
- O. Bogdan , V. Eckstein , F. Rameau , and J. Bazin . 2018 . DeepCalib: a deep learning approach for automatic intrinsic calibration of wide field-of-view cameras . In Proceedings of the ACM SIGGRAPH European Conference on Visual Media Production. ACM , 6 : 1 - 6 :10. O. Bogdan, V. Eckstein, F. Rameau, and J. Bazin. 2018. DeepCalib: a deep learning approach for automatic intrinsic calibration of wide field-of-view cameras. In Proceedings of the ACM SIGGRAPH European Conference on Visual Media Production. ACM, 6:1-6:10.
- Z. Cao , T. Simon , S. Wei , and Y. Sheikh . 2017. Realtime multi-person 2d pose estimation using part affinity fields . In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 7291--7299 . Z. Cao, T. Simon, S. Wei, and Y. Sheikh. 2017. Realtime multi-person 2d pose estimation using part affinity fields. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 7291--7299.
- Blender Online Community . 2018. Blender - a 3D modelling and rendering package . Blender Foundation, Stichting Blender Foundation, Amsterdam . http://www.blender.org Blender Online Community. 2018. Blender - a 3D modelling and rendering package. Blender Foundation, Stichting Blender Foundation, Amsterdam. http://www.blender.org
- C. Gu , C. Sun , D. Ross , C. Vondrick , C. Pantofaru , Y. Li , S. Vijayanarasimhan , G. Toderici , S. Ricco , R. Sukthankar , C. Schmid , and J. Malik . 2018. Ava: A video dataset of spatio-temporally localized atomic visual actions . In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 6047--6056 . C. Gu, C. Sun, D. Ross, C. Vondrick, C. Pantofaru, Y. Li, S. Vijayanarasimhan, G. Toderici, S. Ricco, R. Sukthankar, C. Schmid, and J. Malik. 2018. Ava: A video dataset of spatio-temporally localized atomic visual actions. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 6047--6056.
- S. Haradal , H. Hayashi , and S. Uchida . 2018. Biosignal data augmentation based on generative adversarial networks . In Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE, 368--371 . S. Haradal, H. Hayashi, and S. Uchida. 2018. Biosignal data augmentation based on generative adversarial networks. In Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE, 368--371.
- K. He , X. Zhang , S. Ren , and J. Sun . 2016. Deep residual learning for image recognition . In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 770--778 . K. He, X. Zhang, S. Ren, and J. Sun. 2016. Deep residual learning for image recognition. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 770--778.
- I. Joel , A.and Stergios. 2011 . A Direct Least-Squares (DLS) method for PnP . In IEEE International Conference on Computer Vision (ICCV). IEEE. I. Joel, A.and Stergios. 2011. A Direct Least-Squares (DLS) method for PnP. In IEEE International Conference on Computer Vision (ICCV). IEEE.
- C. Kang , H. Jung , and Y. Lee . 2019 . Towards Machine Learning with Zero Real-World Data. In The ACM Workshop on Wearable Systems and Applications. 41--46 . C. Kang, H. Jung, and Y. Lee. 2019. Towards Machine Learning with Zero Real-World Data. In The ACM Workshop on Wearable Systems and Applications. 41--46.
- P. Karlsson , B. Lo , and G. Z. Yang . 2014. Inertial sensing simulations using modified motion capture data . In Proceedings of the International Conference on Wearable and Implantable Body Sensor Networks (BSN). 16--19 . P. Karlsson, B. Lo, and G. Z. Yang. 2014. Inertial sensing simulations using modified motion capture data. In Proceedings of the International Conference on Wearable and Implantable Body Sensor Networks (BSN). 16--19.
- D. Kingma and J. Ba . 2014 . Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014). D. Kingma and J. Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014). · arXiv
- N. Lane , Y. Xu , H. Lu , S. Hu , T. Choudhury , A. Campbell , and F. Zhao . 2011. Enabling Large-Scale Human Activity Inference on Smartphones Using Community Similarity Networks . In Proceedings of the International Conference on Ubiquitous Computing. ACM, 355--364 . N. Lane, Y. Xu, H. Lu, S. Hu, T. Choudhury, A. Campbell, and F. Zhao. 2011. Enabling Large-Scale Human Activity Inference on Smartphones Using Community Similarity Networks. In Proceedings of the International Conference on Ubiquitous Computing. ACM, 355--364.
- A. Le Guennec , S. Malinowski , and R. Tavenard . 2016. Data Augmentation for Time Series Classification using Convolutional Neural Networks . In ECML/PKDD Workshop on Advanced Analytics and Learning on Temporal Data. A. Le Guennec, S. Malinowski, and R. Tavenard. 2016. Data Augmentation for Time Series Classification using Convolutional Neural Networks. In ECML/PKDD Workshop on Advanced Analytics and Learning on Temporal Data.
- N. Mahmood , N. Ghorbani , N. Troje , G. Pons-Moll , and M. Black . 2019. AMASS: Archive of motion capture as surface shapes . In IEEE International Conference on Computer Vision (ICCV). 5442--5451 . N. Mahmood, N. Ghorbani, N. Troje, G. Pons-Moll, and M. Black. 2019. AMASS: Archive of motion capture as surface shapes. In IEEE International Conference on Computer Vision (ICCV). 5442--5451.
- A. Mathur , T. Zhang , S. Bhattacharya , P. Velickovic , L. Joffe , N. Lane , F. Kawsar , and P. Lió . 2018. Using deep data augmentation training to address software and hardware heterogeneities in wearable and smartphone sensing devices . In IEEE International Conference on Information Processing in Sensor Networks (IPSN). IEEE, 200--211 . A. Mathur, T. Zhang, S. Bhattacharya, P. Velickovic, L. Joffe, N. Lane, F. Kawsar, and P. Lió. 2018. Using deep data augmentation training to address software and hardware heterogeneities in wearable and smartphone sensing devices. In IEEE International Conference on Information Processing in Sensor Networks (IPSN). IEEE, 200--211.
- V. Nair and G. Hinton . 2010. Rectified linear units improve restricted boltzmann machines . In Proceedings of the international conference on machine learning (ICML). 807--814 . V. Nair and G. Hinton. 2010. Rectified linear units improve restricted boltzmann machines. In Proceedings of the international conference on machine learning (ICML). 807--814.
- F. Ofli , R. Chaudhry , G. Kurillo , R. Vidal , and R. Bajcsy . 2013. Berkeley mhad: A comprehensive multimodal human action database . In IEEE Workshop on Applications of Computer Vision (WACV). IEEE, 53--60 . F. Ofli, R. Chaudhry, G. Kurillo, R. Vidal, and R. Bajcsy. 2013. Berkeley mhad: A comprehensive multimodal human action database. In IEEE Workshop on Applications of Computer Vision (WACV). IEEE, 53--60.
- A. Oord , S. Dieleman , H. Zen , K. Simonyan , O. Vinyals , A. Graves , N. Kalchbrenner , A. Senior , and K. Kavukcuoglu . 2016 . Wavenet: A generative model for raw audio. arXiv preprint arXiv:1609.03499 (2016). A. Oord, S. Dieleman, H. Zen, K. Simonyan, O. Vinyals, A. Graves, N. Kalchbrenner, A. Senior, and K. Kavukcuoglu. 2016. Wavenet: A generative model for raw audio. arXiv preprint arXiv:1609.03499 (2016). · arXiv
- D. Pavllo , C. Feichtenhofer , D. Grangier , and M. Auli . 2019. 3D human pose estimation in video with temporal convolutions and semi-supervised training . In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 7753--7762 . D. Pavllo, C. Feichtenhofer, D. Grangier, and M. Auli. 2019. 3D human pose estimation in video with temporal convolutions and semi-supervised training. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 7753--7762.
- T. Plötz , C. Chen , N. Hammerla , and G. Abowd . 2012. Automatic synchronization of wearable sensors and video-cameras for ground truth annotation-a practical approach . In Proceedings of the ACM International Symposium on Wearable Computers. IEEE, 100--103 . T. Plötz, C. Chen, N. Hammerla, and G. Abowd. 2012. Automatic synchronization of wearable sensors and video-cameras for ground truth annotation-a practical approach. In Proceedings of the ACM International Symposium on Wearable Computers. IEEE, 100--103.
- J. Redmon , S. Divvala , R. Girshick , and A. Farhadi . 2016. You only look once: Unified, real-time object detection . In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 779--788 . J. Redmon, S. Divvala, R. Girshick, and A. Farhadi. 2016. You only look once: Unified, real-time object detection. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 779--788.
- A. Reiss and D. Stricker . 2012. Introducing a new benchmarked dataset for activity monitoring . In Proceedings of the ACM International Symposium on Wearable Computers. IEEE, 108--109 . A. Reiss and D. Stricker. 2012. Introducing a new benchmarked dataset for activity monitoring. In Proceedings of the ACM International Symposium on Wearable Computers. IEEE, 108--109.
- A. Reiss and D. Stricker . 2013. Personalized mobile physical activity recognition . In Proceedings of the ACM International Symposium on Wearable Computers. 25--28 . A. Reiss and D. Stricker. 2013. Personalized mobile physical activity recognition. In Proceedings of the ACM International Symposium on Wearable Computers. 25--28.
- A. Saeed , T. Ozcelebi , and J. Lukkien . 2019. Multi-task Self-Supervised Learning for Human Activity Detection . Proceedings of the ACM on interactive, mobile, wearable and ubiquitous technologies (IMWUT) 3 , 2 ( 2019 ), 1--30. A. Saeed, T. Ozcelebi, and J. Lukkien. 2019. Multi-task Self-Supervised Learning for Human Activity Detection. Proceedings of the ACM on interactive, mobile, wearable and ubiquitous technologies (IMWUT) 3, 2 (2019), 1--30.
- Z. Shen , W. Wang , X. Lu , J. Shen , H. Ling , T. Xu , and L. Shao . 2019 . Human-Aware Motion Deblurring. In IEEE International Conference on Computer Vision (ICCV). 5572--5581 . Z. Shen, W. Wang, X. Lu, J. Shen, H. Ling, T. Xu, and L. Shao. 2019. Human-Aware Motion Deblurring. In IEEE International Conference on Computer Vision (ICCV). 5572--5581.
- J. Shi , L. Xu , and J. Jia . 2014. Discriminative blur detection features . In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2965--2972 . J. Shi, L. Xu, and J. Jia. 2014. Discriminative blur detection features. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2965--2972.
- G. Sigurdsson , G. Varol , X. Wang , I. Laptev , A. Farhadi , and A. Gupta . 2016 . Hollywood in Homes: Crowdsourcing Data Collection for Activity Understanding. arXiv preprint arXiv:1604.01753 (2016). G. Sigurdsson, G. Varol, X. Wang, I. Laptev, A. Farhadi, and A. Gupta. 2016. Hollywood in Homes: Crowdsourcing Data Collection for Activity Understanding. arXiv preprint arXiv:1604.01753 (2016). · arXiv
- E. Thomaz , I. Essa , and G. Abowd . 2015. A practical approach for recognizing eating moments with wrist-mounted inertial sensing . In Proceedings of the ACM International Joint Conference on Pervasive and Ubiquitous Computing. 1029--1040 . E. Thomaz, I. Essa, and G. Abowd. 2015. A practical approach for recognizing eating moments with wrist-mounted inertial sensing. In Proceedings of the ACM International Joint Conference on Pervasive and Ubiquitous Computing. 1029--1040.
- C. Tong , S. Tailor , and N. Lane . 2020. Are Accelerometers for Activity Recognition a Dead-End? . In Proceedings of the International Workshop on Mobile Computing Systems and Applications. ACM, 39--44 . C. Tong, S. Tailor, and N. Lane. 2020. Are Accelerometers for Activity Recognition a Dead-End?. In Proceedings of the International Workshop on Mobile Computing Systems and Applications. ACM, 39--44.
- F. Xiao L. Pei L. Chu D. Zou W. Yu Y. Zhu and T. Li. 2020. A Deep Learning Method for Complex Human Activity Recognition Using Virtual Wearable Sensors. arXiv preprint arXiv:2003.01874 (2020). F. Xiao L. Pei L. Chu D. Zou W. Yu Y. Zhu and T. Li. 2020. A Deep Learning Method for Complex Human Activity Recognition Using Virtual Wearable Sensors. arXiv preprint arXiv:2003.01874 (2020). · arXiv
- S. Yao , Y. Zhao , H. Shao , C. Zhang , A. Zhang , S. Hu , D. Liu , S. Liu , Lu Su , and T. Abdelzaher . 2018. Sensegan: Enabling deep learning for internet of things with a semi-supervised framework . Proceedings of the ACM on interactive, mobile, wearable and ubiquitous technologies (IMWUT) 2 , 3 ( 2018 ), 1--21. S. Yao, Y. Zhao, H. Shao, C. Zhang, A. Zhang, S. Hu, D. Liu, S. Liu, Lu Su, and T. Abdelzaher. 2018. Sensegan: Enabling deep learning for internet of things with a semi-supervised framework. Proceedings of the ACM on interactive, mobile, wearable and ubiquitous technologies (IMWUT) 2, 3 (2018), 1--21.
- A. Young , M. Ling , and D. Arvind . 2011. IMUSim: A simulation environment for inertial sensing algorithm design and evaluation . In Proceedings of the International Conference on Information Processing in Sensor Networks (IPSN). IEEE, 199--210 . A. Young, M. Ling, and D. Arvind. 2011. IMUSim: A simulation environment for inertial sensing algorithm design and evaluation. In Proceedings of the International Conference on Information Processing in Sensor Networks (IPSN). IEEE, 199--210.
- J. Yu and R. Ramamoorthi . 2019. Robust Video Stabilization by Optimization in CNN Weight Space . In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 3800--3808 . J. Yu and R. Ramamoorthi. 2019. Robust Video Stabilization by Optimization in CNN Weight Space. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 3800--3808.
- M. Zhang and A. A. Sawchuk . 2012. USC-HAD: a daily activity dataset for ubiquitous activity recognition using wearable sensors . In Proceedings of the International Conference on Ubiquitous Computing. M. Zhang and A. A. Sawchuk. 2012. USC-HAD: a daily activity dataset for ubiquitous activity recognition using wearable sensors. In Proceedings of the International Conference on Ubiquitous Computing.
- Q. Zhang and R. Pless . 2004. Extrinsic calibration of a camera and laser range finder (improves camera calibration) . In IEEE International Conference on Intelligent Robots and Systems (IROS). IEEE. Q. Zhang and R. Pless. 2004. Extrinsic calibration of a camera and laser range finder (improves camera calibration). In IEEE International Conference on Intelligent Robots and Systems (IROS). IEEE.
Cited by
Items 101–156 of 156
Behavioral Biometrics in VR: Changing Sensor Signal Modalities
Sawicki, A., Saeed, K., & Walendziuk, W. (2025) · Sensors · doi:10.3390/s25185899 · Cited by 2
“Additionally, in [21], the authors, based on [19], developed a solution for generating IMU data from recordings available on the YouTube platform.”
VirtualIMU: Generating Virtual Wearable Inertial Data from Video for Deep Learning Applications
Gavier, I., Liu, Y., & Lee, S. I. (2023) · Wearable and Implantable Body Sensor Networks · doi:10.1109/bsn58485.2023.10331242 · Cited by 3
mTanaaw: A System for Assessment and Analysis of Mental Health with Wearables
Pinge, A., Jaisinghani, D., Ghosh, S., Challa, A., & Sen, S. (2024) · International Conference on Communication Systems & Networks (COMSNETS) · doi:10.1109/comsnets59351.2024.10427432 · Cited by 2
Shah, H., & Holia, M. S. (2024) · Indian Journal of Science and Technology · doi:10.17485/ijst/v17i21.3203 · Cited by 3
Techniques for Generating Sign Language a Comprehensive Review
Waghmare, P. P. (2024) · Journal of The Institution of Engineers (India) Series B · doi:10.1007/s40031-024-01118-8 · Cited by 2
RAPNet: Resolution-Adaptive and Predictive Early Exit Network for Efficient Image Recognition
Hu, Y., Cheng, Y., Zhou, Z., Cao, Z., Lu, A., Liu, J., Zhang, M., & Li, Z. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3428554 · Cited by 3
SensorGPT: A New Paradigm for Synthetic Sensory Data Generation
Sharma, R., Khamis, A., Khalifa, S., Bretherton, D., & Kusý, B. (2025) · IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops) · doi:10.1109/percomworkshops65533.2025.00061 · Cited by 2
Augmentation of Accelerometer and Gyroscope Signals in Biometric Gait Systems
Sawicki, A. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-10539-5_3 · Cited by 3
Virtual Sensors With 3D Digital Human Motion for Interactive Simulation
Xia, C., & Sugiura, Y. (2023) · Computer · doi:10.1109/mc.2023.3239344 · Cited by 2
Spatiotemporal Deformable Models for Long-Term Complex Activity Detection.Preprint
Khan, S., & Fabio Cuzzolin, A. (2021) · arXiv (Cornell University) · Cited by 2
COMODO: Cross-Modal Video-to-IMU Distillation for Efficient Egocentric Human Activity Recognition
Chen, B., Wongso, W., Li, Z., Khaokaew, Y., Xue, H., & Salim, F. D. (2026) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3810218 · Cited by 1
Oishi, N., Birch, P., Lago, P., & Roggen, D. (2026) · Knowledge-Based Systems · doi:10.1016/j.knosys.2026.116367 · Cited by 1
Tsuji, H., Yamamoto, T., Kobayashi, M., Sasaki, K., Aso, N., & Sugiura, Y. (2025) · IEEE/SICE International Symposium on System Integration (SII) · doi:10.1109/sii59315.2025.10870587 · Cited by 1
Vsens Toolkit: AR-based Open-ended System for Virtual Sensors
Liang, F., Min, T., Xia, C., & Sugiura, Y. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3721177 · Cited by 1
Pose based human activity recognition using deep learning convolutional neural network
Chopde, N. R., Gulhane, S., Yawale, P., Chokhat, A. D., & Solanke, S. B. (2025) · AIP conference proceedings · doi:10.1063/5.0295158 · Cited by 1
W2W: A Simulated Exploration of IMU Placement Across the Human Body for Designing Smarter Wearable
Ray, L. S. S., Zhou, B., & Lukowicz, P. (2025) · ACM International Symposium on Wearable Computers (ISWC) · doi:10.1145/3715071.3750417 · Cited by 7
Vsens: Incorporating XR into the Process of Collecting Virtual IMU Data
Liang, F., Min, T., Xia, C., & Sugiura, Y. (2026) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3790110 · Cited by 1
Wang, Q., & Xia, C. (2025) · International Conference on Cyberworlds (CW) · doi:10.1109/cw68232.2025.00058 · Cited by 1
Xu, C., Li, H., Li, Z., Chen, X., Rathore, A. S., Zhang, H., Wang, K., & Xu, W. (2022) · IEEE International Conference on Healthcare Informatics (ICHI) · doi:10.1109/ichi54592.2022.00053 · Cited by 1
On the Effectiveness of Virtual IMU Data for Eating Detection with Wrist Sensors
Jain, Y., Kwon, H., & Plötz, T. (2022) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/3544793.3560337 · Cited by 4
Self-citationAn Extensive Survey on Machine Learning-Enabled Automated Human Action Recognition Models
Jandhyam, L. A., Ragupathy, R., & Satyala, N. (2023) · Lecture notes on data engineering and communications technologies · doi:10.1007/978-981-99-0835-6_31 · Cited by 1
Haresamudram, H., Tang, C. I., Suh, S., Lukowicz, P., & Plötz, T. (2023) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/3594739.3605102 · Cited by 3
“However, existing works have demonstrated that the collection of large-scale labeled data for mobile sensing tasks is particularly difficult [1, 7, 9], due to the difficulty in interpreting sensor streams (such as accelerometer traces).”
BackgroundSelf-citationNeural network algorithm for predicting human speed based on computer vision and machine learning
Obukhov, A. D., Teselkin, D. V., Олеговна, С. Е., Komissarov, A., & Shilcin, M. (2024) · ITM Web of Conferences · doi:10.1051/itmconf/20245903003 · Cited by 1
Hwang, S., Leng, Z., Oh, S., Kim, K. K., & Plötz, T. (2024) · ACM International Symposium on Wearable Computers (ISWC) · doi:10.1145/3675095.3676617 · Cited by 1
Self-citationDeep Learning Based Approach for Human Intention Estimation in Lower-Back Exoskeleton
Zanina, V., Dlamini, G., & Palyonov, V. (2023) · Lecture notes in networks and systems · doi:10.1007/978-3-031-28073-3_12 · Cited by 1
Haresamudram, H., Tang, C. I., Suh, S., Lukowicz, P., & Ploetz, T. (2024) · arXiv · Cited by 31
Self-citationInvestigating Enhancements to Contrastive Predictive Coding for Human Activity Recognition
Haresamudram, H., Essa, I., & Ploetz, T. (2022) · arXiv · Cited by 27
Self-citationThe Three A’s of Wearable and Ubiquitous Computing: Activity, Affect, and Attention
Laerhoven, K. V. (2021) · Frontiers in Computer Science · doi:10.3389/fcomp.2021.691622
“…include taking existing computer vision and signal processing techniques to convert videos of human activity into virtual streams of IMU data (Kwon et al., 2020), for instance, so that much larger repositories can be used to retrieve realistic human pose sequences for given activities.”
BackgroundCan You See It? Good, So We Can Sense It!
Kwon, H., Tong, C., Haresamudram, H., Gao, Y., Abowd, G. D., Lane, N. D., & Plötz, T. (2021) · GetMobile Mobile Computing and Communications · doi:10.1145/3486880.3486891
Self-citationImage Recognition Applied to Security Systems: The Case of Burkina Faso
Tiendrebeogo, A. (2022) · New Media and Mass Communication · doi:10.7176/nmmc/102-04
Generating Virtual Head-Mounted Gyroscope Signals From Video Data
Lu, M., Chen, C.-H., Dawton, B., Nakamura, Y., & Arakawa, Y. (2023) · IEEE International Conference on Consumer Electronics - Taiwan (ICCE-Taiwan) · doi:10.1109/icce-taiwan58799.2023.10227010
Locally Controllable Attitude B-Spline Function
Suh, Y. S. (2024) · IEEE Access · doi:10.1109/access.2024.3488172
Synthetic Sensory Data Generators: How Much Progress Are We Making?
Sharma, R., Khamis, A., Khalifa, S., Bretherton, D., & Kusý, B. (2025) · IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops) · doi:10.1109/percomworkshops65533.2025.00084 · Cited by 1
Ray, L. S. S., Liu, M., Gurung, D., Zhou, B., Suh, S., & Lukowicz, P. (2025) · ACM International Symposium on Wearable Computers (ISWC) · doi:10.1145/3715071.3750424 · Cited by 2
Patidar, P., Arakawa, R., Graça, R., Moutinho, R., Soares, A., Vasconcelos, A., Talami, F., da Silva, J. C., Silva, I., Mendes-Santos, C., et al. (2025) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3770674 · Cited by 4
Augmenting Intra-Class Difference Limb Diversity for Variety 3D Human Motions Generation
Huang, L., & Xia, C. (2025) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3746058.3758353
Uhlenberg, L., Gebelein, M. J., Haeusler, L. O., & Amft, O. (2025) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/3714394.3754444
Bian, Q., Wang, H. B., Alsayed, K. A., & Ding, Z. (2026) · Frontiers in Bioengineering and Biotechnology · doi:10.3389/fbioe.2025.1737916 · Cited by 1
Text2IMU: Advancing Human Activity Recognition by Text-Driven IMU Data Synthesis
Haeusler, L. O., Uhlenberg, L., & Amft, O. (2025) · Wearable and Implantable Body Sensor Networks · doi:10.1109/bsn66969.2025.11337798 · Cited by 5
Zhang, Z., Zhong, C., Fu, Y., Chen, Y., Jia, J., Li, H., & Xu, C. (2025) · Wearable and Implantable Body Sensor Networks · doi:10.1109/bsn66969.2025.11337466
Chen, H., Wang, C., Zhi, D., & Yang, Q. (2026) · Displays · doi:10.1016/j.displa.2026.103415
Jana, A., Adhikari, P., De, S., Bhattacharjee, S., & Singh, P. K. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-17181-8_14 · Cited by 1
Img2Spec : A Generalized Cross-Modal Gesture Recognition Framework Based on Local Descriptors
Zou, Y., Chen, W., He, Y., Dong, F., Leung, V. C. M., & Wu, K. (2026) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2026.3689384
Ji, Q., Li, H., Shi, C., Wang, Y., Cheng, J. Q., & Chen, Y. J. (2026) · International Conference on Mobile Systems, Applications, and Services · doi:10.1145/3745756.3809233
Physical Self-Supervised Learning: IMU Sensing without Manual Labels
Leng, Y., Liu, R., Hu, S., Zhao, P., Chen, C.-F., Chen, S., & Yao, S. (2026) · International Conference on Mobile Systems, Applications, and Services · doi:10.1145/3745756.3809252 · Cited by 2
Recent Advances in Human Activity Recognition for Healthcare (2019–2025)
Yu, H.-N., Wang, Q., Cang, S., He, Z., Wang, Y., & Nguyễn, L. V. (2026) · 2026 6th International Symposium on Intelligent Robotics and Systems (ISoIRS) · doi:10.1109/isoirs70157.2026.11545278
Huang, L., & Xia, C. (2026) · International Conference on Multimedia Retrieval · doi:10.1145/3805622.3810642
RadSyn: Synthesizing Large-Scale Realistic mmWave Radar Training Data via Video and Text Fusion
Deng, K., Ling, Y., Xing, L., Cao, Z., Wu, H., & Ma, H. (2026) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2026.3704562
A Quantitative Comparison of Human Body Modeling Methods to Synthesize IMU Data
Uhlenberg, L., Kempfle, J., Haeusler, L. O., & Amft, O. (2026) · IEEE Access · doi:10.1109/access.2026.3726408
Fast Personalization of Real-Time Human Activity Recognition via Transfer Learning
You, Z., Yuan, Y., Tohfafarosh, M., & Zhou, X. (2026) · IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob) · doi:10.1109/biorob66782.2026.11681440
Spilz, A., Oppel, H., & Münz, M. (2026) · AI · doi:10.3390/ai7090381
Adapting Pretrained Large Vision Models for Sensor-based Activity Recognition
Cai, Y.-Z., Feng, R.-X., Cai, K.-L., Liu, Y.-H., Guo, B.-S., & Hong, Z.-Q. (2026) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3832003
SolarSim: Physics-Based Simulation of Wearable Solar Harvester Signals for Motion Sensing
Ghalwash, Y., Khamis, A., Khalifa, S., Sandhu, M. M., & Jurdak, R. (2026) · ACM International Symposium on Wearable Computers (ISWC) · doi:10.1145/3830727.3834813
AutoResearcHAR: A Multi-Agent Framework for Dataset Adaptive IMU Based Human Activity Recognition
Ray, L. S. S., Rey, V. F., Bian, S., Suh, S., Lukowicz, P., & Zhou, B. (2026) · ACM International Symposium on Wearable Computers (ISWC) · doi:10.1145/3830727.3834823
Ahmed, M. S., Petz, K. D., French, N. J., Lakhtakia, T., Sangani, A. N., Rucker, M., Chen, X., Teachman, B. A., & Barnes, L. E. (2026) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3832043
Metadata
- Authors
- Kwon, H., Tong, C., Haresamudram, H., Gao, Y., Abowd, G. D., Lane, N. D., & Plötz, T.
- Affiliations
- Georgia Institute of Technology (US)
- University of Oxford (GB)
- Venue
- Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
- Published
- 2020-09-04
- Volume
- 4
- Issue
- 3
- Pages
- 1-29
- ISSN
- 2474-9567 (ISSN-L)
- OpenAlex
- W3083323811
- Cited by
- 152
- References
- 49
- Citations by year
- 2026: 20 · 2025: 29 · 2024: 24 · 2023: 32 · 2022: 23 · 2021: 24
- FWCI
- 7.68
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies (publishedVersion)
- Funders
- Engineering and Physical Sciences Research Council (1892895)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Physical Sciences › Engineering › Biomedical Engineering › Gait Recognition and Analysis
- Keywords
- human activity recognition, on-body sensors, inertial measurement unit, automatic annotation, accelerometry, wearable sensors, labeled data scarcity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.