The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Let there be IMU data: generating training data for wearable, motion sensor based activity recognition from monocular RGB videos
Abstract
Recent advances in Machine Learning, in particular Deep Learning have been driving rapid progress in fields such as computer vision and natural language processing. Human activity recognition (HAR) using wearable sensors, which has been a thriving research field for the last 20 years, has benefited much less from such advances. This is largely due to the lack of adequate amounts of labeled training data. In this paper we propose a method to mitigate the labeled data problem in wearable HAR by generating wearable motion data from monocular RGB videos, which can be collected from popular video platforms such as YouTube. Our method works by extracting 2D poses from video frames and then using a regression model to map them to sensor signals. We have validated it on fitness exercises as the domain for which activity recognition is trained and shown that we can improve a HAR recognition model using data that was produced from a YouTube video.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Cross-modality transfer approaches [54,108,109,161,162] are instrumental for extracting virtual IMU data from 2D RGB videos of activities.”
From Past, Present, and Future of Sensor-based Human Activity Recognition Using Wearables: A Surveying Tutorial on a Still Challenging Task (2025) · cites this paper as [162]
In the citing paper
“In recent years, there has been a growing interest in the combination of data obtained from videobased systems together with the time-series data originating from on-body sensors Rey et al. (2019); Kwon et al. (2020); Fortes Rey et al. (2021).”
From Opportunity++: A Multimodal Dataset for Video- and Wearable, Object and Ambient Sensors-Based Human Activity Recognition (2021) · cites this paper as Rey et al. (2019)
In the citing paper
“Recently, a paradigm change to data collection for supervised recognizer training has been introduced where existing repositories of videos of human activities are utilized to generate virtual IMU training data [14,15,16,17].”
From Complex Deep Neural Networks from Large Scale Virtual IMU Data for Effective Human Activity Recognition Using Wearables (2021) · cites this paper as [17]
In the citing paper
“Video repositories have recently attracted attention as potential training resources for sensor-based HAR models [14,15,17].”
From Complex Deep Neural Networks from Large Scale Virtual IMU Data for Effective Human Activity Recognition Using Wearables (2021) · cites this paper as [17]
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
Realtime Multi-person 2D Pose Estimation Using Part Affinity Fields
Cao, Z., Simon, T., Wei, S.-E., & Sheikh, Y. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.143 · Cited by 7,539 · cited 1×
“We used the OpenPose library [6, 7] to extract frame by frame the 2D poses of subjects.”
MethodStacked Hourglass Networks for Human Pose Estimation
Newell, A., Yang, K., & Deng, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46484-8_29 · Cited by 5,634 · cited 1×
“Other approaches focus on generating only 2D poses[6, 17], with Openpose [6] also being able to generate 3D ones when two cameras are available.”
BackgroundAn Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling
Bai, S., Kolter, J. Z., & Koltun, V. (2018) · arXiv · doi:10.48550/arxiv.1803.01271 · Cited by 4,655
A Survey on Human Activity Recognition using Wearable Sensors
Lara, O. D., & Labrador, M. A. (2013) · IEEE Communications Surveys & Tutorials · doi:10.1109/surv.2012.110112.00192 · Cited by 2,662
Revisiting Unreasonable Effectiveness of Data in Deep Learning Era
Sun, C., Shrivastava, A., Singh, S., & Gupta, A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.97 · Cited by 2,484
Deep learning for sensor-based activity recognition: A survey
Wang, J., Chen, Y., Hao, S., Peng, X., & Hu, L. (2019) · Pattern Recognition Letters · doi:10.1016/j.patrec.2018.02.010 · Cited by 2,014
End-to-End Recovery of Human Shape and Pose
Kanazawa, A., Black, M. J., Jacobs, D. W., & Malik, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00744 · Cited by 1,870
Keep It SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image
Bogo, F., Kanazawa, A., Lassner, C., Gehler, P., Romero, J., & Black, M. J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46454-1_34 · Cited by 1,498
Research works printed but not linked: 3
- {n. d.}. Apple Special Event . June 3, 2019 . https://www.apple.com/apple-events/june-2019/. {n. d.}. Apple Special Event. June 3, 2019. https://www.apple.com/apple-events/june-2019/.
- Zhe Cao Tomas Simon Shih-En Wei and Yaser Sheikh. 2017. Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields. In CVPR. Zhe Cao Tomas Simon Shih-En Wei and Yaser Sheikh. 2017. Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields. In CVPR . · DOI · Paper page
- Pramod Murthy , Onorina Kovalenko , Ahmed Elhayek , Christiano Couto Gava, and Didier Stricker . 2017 . 3D Human Pose Tracking inside Car using Single RGB Spherical Camera. In CSCS. Pramod Murthy, Onorina Kovalenko, Ahmed Elhayek, Christiano Couto Gava, and Didier Stricker. 2017. 3D Human Pose Tracking inside Car using Single RGB Spherical Camera. In CSCS.
Cited by
Towards Generalized mmWave-based Human Pose Estimation through Signal Augmentation
Xue, H., Cao, Q., Miao, C., Ju, Y., Hu, H., Zhang, A., & Su, L. (2023) · ACM Annual International Conference on Mobile Computing and Networking (MobiCom) · doi:10.1145/3570361.3613302 · Cited by 95
IMUGPT 2.0: Language-Based Cross Modality Transfer for Sensor-Based Human Activity Recognition
Leng, Z., Bhattacharjee, A., Rajasekhar, H., Zhang, L., Bruda, E., Kwon, H., & Plötz, T. (2024) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3678545 · Cited by 61
“The most notable ones are: (i) paucity of labeled data – the time-consuming and expensive nature of wearable data collection along with its inherent privacy concerns have led to datasets being relatively smaller in size; (ii) difficulty in data annotation – ambiguity in the target activities and…”
3D Human Action Recognition: Through the eyes of researchers
Sarkar, A., Banerjee, A., Singh, P. K., & Sarkar, R. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.116424 · Cited by 43
Zero-Shot Learning for IMU-Based Activity Recognition Using Video Embeddings
Tong, C., Ge, J., & Lane, N. D. (2021) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3494995 · Cited by 61
“This motivation is shared by a growing body of recent works which use videos as a source domain to generate virtual embeddings for activity recognition on IMU data [31, 32, 35, 46] as well as other sensing data [2, 9].”
BackgroundLiu, Y., Zhang, S., & Gowda, M. (2021) · Proceedings of the International Conference on Internet-of-Things Design and Implementation · doi:10.1145/3450268.3453537 · Cited by 39
“Although in a similar spirit to recent works [38, 55, 66, 74] showing the feasibility of harvesting training data from videos for identifying upto ten classes of human activities, ZeroNet differs from the above works in following ways: (i) Shows the feasibility of harvesting training data from videos for a gesture rec…”
Background
Metadata
- Authors
- Rey, V. F., Hevesi, P., Kovalenko, O., & Lukowicz, P.
- Affiliations
- DFKI, Kaiserslautern, Germany
- German Research Centre for Artificial Intelligence (DE)
- Venue
- Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers
- Publisher
- ACM
- Published
- 2019-09-09
- Pages
- 699-708
- OpenAlex
- W2972527328
- Cited by
- 43
- References
- 20
- Citations by year
- 2026: 3 · 2025: 4 · 2024: 8 · 2023: 8 · 2022: 6 · 2021: 10 · 2020: 4
- FWCI
- 1.63
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) (publishedVersion)
- Funders
- Deutsche Forschungsgemeinschaft (LU 1574/2-1)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Physical Sciences › Computer Science › Human-Computer Interaction › Hand Gesture Recognition Systems
- Keywords
- human activity recognition, wearable sensors, inertial measurement unit, monocular RGB video, 2D pose estimation, physical conditioning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Let there be IMU data: generating training data for wearable, motion sensor based activity recognition from monocular RGB videos. https://app.cochat.ai/papers/10.1145/3341162.3345590
- The paper
- Rey, V. F., Hevesi, P., Kovalenko, O., & Lukowicz, P. (2019). Let there be IMU data: generating training data for wearable, motion sensor based activity recognition from monocular RGB videos. Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers. https://doi.org/10.1145/3341162.3345590
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.