Complex Deep Neural Networks from Large Scale Virtual IMU Data for Effective Human Activity Recognition Using Wearables
Open accessThe DOI matches its Crossref record; 85 of the 87 references it cites were found in a registry or an index, and 2 are not checked yet; 5 of the 50 sources its claims cite were checked for support.
Abstract
Supervised training of human activity recognition (HAR) systems based on body-worn inertial measurement units (IMUs) is often constrained by the typically rather small amounts of labeled sample data. Systems like IMUTube have been introduced that employ cross-modality transfer approaches to convert videos of activities of interest into virtual IMU data. We demonstrate for the first time how such large-scale virtual IMU datasets can be used to train HAR systems that are substantially more complex than the state-of-the-art. Complexity is thereby represented by the number of model parameters that can be trained robustly. Our models contain components that are dedicated to capture the essentials of IMU data as they are of relevance for activity recognition, which increased the number of trainable parameters by a factor of 1100 compared to state-of-the-art model architectures. We evaluate the new model architecture on the challenging task of analyzing free-weight gym exercises, specifically on classifying 13 dumbbell execises. We have collected around 41 h of virtual IMU data using IMUTube from exercise videos available from YouTube. The proposed model is trained with the large amount of virtual IMU data and calibrated with a mere 36 min of real IMU data. The trained model was evaluated on a real IMU dataset and we demonstrate the substantial performance improvements of 20% absolute F1 score compared to the state-of-the-art convolutional models in HAR.
Verified 2026-11-19 · 2 references not checked yet
Overview
CoChat Summary
TL;DR
The paper shows that large-scale virtual IMU data generated by IMUTube can train much more complex HAR models than previously feasible, using a model with about 1100× more trainable parameters than a standard ConvNet. On 13 dumbbell exercises, the best model reaches 80.2% mean F1, improving by 17.6 absolute points over a CNN baseline and 6.2 over DeepConvLSTM.
Key findings
- The proposed model trained with virtual IMU data achieved 80.2% mean F1, a +17.6% absolute improvement over the CNN baseline and +6.2% absolute over DeepConvLSTM.
- The full architecture contains approximately 1100× more trainable parameters than the convolutional baseline, with reported gains of 12.64% absolute F1 using real IMU only and 17.63% absolute F1 using real + virtual IMU.
- Adaptive trimming alone increased parameters by about 10× and improved F1 by 4.33% on average; multi-window kernels added another ~10× complexity and 3.44% absolute F1.
- Multi-view kernels gave 3.58% absolute F1 on average, skip connections 2.65% with virtual IMU, while recurrent layers and uncertainty modeling gave only marginal, non-significant improvements (0.19% and 0.27% absolute F1, respectively).
Methods
- The model combines adaptive sensor-window trimming, multi-scale hourglass convolutions, multi-window kernels, multi-view time-channel kernels, skip connections, multi-scale recurrent aggregation, and heteroscedastic uncertainty modeling for noisy samples.
- Virtual IMU data were generated with IMUTube from YouTube videos for 13 dumbbell exercises, yielding >41 h of virtual data; real IMU data came from MyoGym (10 users, 30 free-weight activities, 2.81 min per activity on average, about 36 min total for calibration).
- Evaluation used 30 Hz data, 4-second windows with 1-second overlap, subjects 1–5 for training, 6–7 for validation, 8–10 for testing, mean F1 score, and Wilson score intervals from 15 runs; models were pretrained on calibrated virtual IMU data and fine-tuned on real IMU data.
Limitations
- The paper states the virtual dataset is still relatively small compared with datasets like ImageNet, and that increasing virtual data may introduce noise that could counteract performance gains.
- The paper notes the studied gym exercises are relatively constrained, the architecture components were manually designed rather than learned via NAS, and uncertainty modeling had only marginal effect.
- Evident from the text: evaluation uses a single wrist sensor, 13 exercises, only 10 real users, weak labels from YouTube search terms, and a domain gap between virtual and real IMU data that requires calibration.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures
![Figure 1 Deriving human activity recognition systems from large scale virtual IMU data generated through IMUTube [14]. Videos for specific activities are retrieved from public repositories such as YouTube through keyword-based queries. These search terms serve as weak labels for all generated sensor data. The IMUTube system then processes the retrieved videos to (i) suppress irrelevant videos; (ii) eliminate portions of the videos that are of insufficient quality (motion blur, occlusions, etc.); (iii) generates virtual IMU data for on-body sensor positions as specified by the query; and (iv) calibrates the virtual IMU data towards more realistic sensor data. The resulting, large-scale virtual IMU dataset is then used for HAR model training.](https://figures.cochat.ai/paper-figures/3288fe2b6c4c4ebf947652421be54793/complex-deep-neural-networks-from-large-scale-virtual-imu-data-for-effective-figure-1.jpg)

![Figure 3 Non-linear multi-scale feature extraction (MS-Conv) with rescaling factors of [1/2,1/4,1/8]. To capture low dimension temporal representation at each scaling factor, we adopt hourglass networks, which are often used to capture low dimensional spatial representations of 2D images in computer vision scenarios [72]. The hourglass network first downsamples input with the convolutional model and then upsamples back to the original scale with the transposed convolutional model with ReLU activation. We apply separate hourglass network models to capture each temporal scale information independently, which is then aggregated and remixed with the bottleneck convolutional layer for final output.](https://figures.cochat.ai/paper-figures/9381c37a402c4b69926d7207b733b295/complex-deep-neural-networks-from-large-scale-virtual-imu-data-for-effective-figure-3.jpg)
![Figure 4 Multi-kernel window convolution (MW-Conv) with kernel sizes of [0.15,0.5,1,1.5,2] seconds, respectively, to capture short and long motions in human activities. We used different colors for each kernels to show that they are independent and not sharing the weights. Features from each kernel are concatenated along the feature map axis and recombined through the bottleneck layer. We have explored other feature aggregation approaches, such as summation or bilinear pooling [82], but concatenation was most effective over others.](https://figures.cochat.ai/paper-figures/85eefc2d8e77409b8c6e627039bdb146/complex-deep-neural-networks-from-large-scale-virtual-imu-data-for-effective-figure-4.jpg)
![Figure 5 Multi-view convolution (MV-Conv) with kernel shapes t×1, 1×c, t×c, and 1×1 are used to capture the time and channel correlations. We used different colors for each kernels to show that they are independent and not sharing the weights. Features from each kernel are concatenated along the the feature map axis and recombined through the bottleneck layer. We have explored other feature aggregation approaches, such as summation or bilinear pooling [82], but concatenation was most effective over others.](https://figures.cochat.ai/paper-figures/bffbd5f4a38a4147975104a9401b613c/complex-deep-neural-networks-from-large-scale-virtual-imu-data-for-effective-figure-5.jpg)


Verification
Claims the paper makes
5 Valid · 7 Not valid · 28 Not checked · 75 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“The popular DeepConvLSTM model [7] can only afford four, moderately-sized convolution layers to learn effective data representations, and two LSTM layers for sequence modeling.”
§ Complex Deep Neural Networks from Large Scale Virtual IMU Data for Effective Human Activity Recognition Using Wearables > 1. Introduction
What the sources say
- Partly supportedcites [7] Ordóñez, F.J., and Roggen, D. (2016). Deep convolutional and lstm recurrent neural networks for multimodal wearable activity recognition. Sensors, 16. sourceExperimental — accuracy not yet measured
Section 3.1 (Architecture) confirms the structural part of the claim: DeepConvLSTM transforms sensor data 'through four convolutional operations' and adds two recurrent dense layers, shorthand 'C(64)-C(64)-C(64)-C(64)-R(128)-R(128)-Sm', with the two-layer recurrent depth chosen following prior work showing at least two recurrent layers beneficial. However, the 'can only afford' limitation is not what the source says: Section 5.4 (Discussion) states it is an eight-layer network while noting 'increasing further the number of layers may be beneficial', and Section 5.3 (Hyperparameters Evaluation, Figure 8) reports performance improving by ~1% per added convolutional layer on OPPORTUNITY. The paper frames the layer count as a deliberate trade-off with training time and computational budget, not a capacity the model cannot exceed. (LLM, conf=0.80)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“Recently, a paradigm change to data collection for supervised recognizer training has been introduced where existing repositories of videos of human activities are utilized to generate virtual IMU training data [14,15,16,17].”
§ Complex Deep Neural Networks from Large Scale Virtual IMU Data for Effective Human Activity Recognition Using Wearables > 1. Introduction
What the sources say
- Not checked yetcites [14] Kwon, 2020, IMUTube: Automatic extraction of virtual on-body accelerometry from video for human activity recognition, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. source
- Not checked yetcites [15] Kwon, 2020, Approaching the Real-World: Supporting Activity Recognition Training with Virtual IMU Data, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. source
- Supportedcites [16] Liu, Y., Zhang, S., and Gowda, M. (2021, January 18–21). When Video meets Inertial Sensors: Zero-shot Domain Adaptation for Finger Motion Analytics with Inertial Sensors. Proceedings of the International Conference on Internet-of-Things Design and Implementation, Charlottesville, VA, USA. source
Section 2.1 (p. 3) says publicly available videos can be used to harvest training data and identifies a public sign-language tutorial as this paper’s source. Section 4 (p. 4) states that locations captured from video are transformed into synthetic accelerometer and orientation data for training ML models. (LLM, conf=0.94)
- Not checked yetcites [17] Rey, V., Hevesi, P., Kovalenko, O., and Lukowicz, P. (2019). Let there be IMU data: Generating training data for wearable, motion sensor based activity recognition from monocular RGB videos. Adjunct Proceedings of the ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the ACM International Symposium on Wearable Computers, ACM. source
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Adding so-called skip-connections from intermediate layers directly to layers later in the model seemingly helps to counteract the vanishing gradient by preserving the flow of backpropagated gradient signals [86,87].”
§ Complex Deep Neural Networks from Large Scale Virtual IMU Data for Effective Human Activity Recognition Using Wearables > 3. Complex Deep Neural Networks for Human Activity Recognition > 3.4. Full Feature Extraction Model with Skip-Connection and Temporal Aggregation
What the sources say
- Not checked yetcites [86] He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA. source
- Partly supportedcites [87] Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7–12). Going deeper with convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA. sourceExperimental — accuracy not yet measured
Our copy of 'Going deeper with convolutions' supports only the gradient-flow half of the claim: Sec. 5 GoogLeNet (p. 6) states that, because effective gradient propagation through the 22-layer depth 'was a concern', adding auxiliary classifiers to intermediate layers (Inception 4a and 4d) was expected to 'increase the gradient signal that gets propagated back'. However, the paper's mechanism is side-branch auxiliary classifiers discarded at inference, not skip-connections routed from intermediate layers to later layers; no passage in our copy (searched: skip/shortcut connections, vanishing gradient, identity mapping) describes or uses skip connections, and the paper never uses the term 'vanishing gradient'. The skip-connection half of the attributed claim therefore rests on the other citation [86 or 87], not on this source. (LLM, conf=0.72)
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“Previous work showed that training a large model is challenging because it becomes more likely that the gradient during optimization becomes infinitesimally small at the bottom layers [84,85,86].”
§ Complex Deep Neural Networks from Large Scale Virtual IMU Data for Effective Human Activity Recognition Using Wearables > 3. Complex Deep Neural Networks for Human Activity Recognition > 3.4. Full Feature Extraction Model with Skip-Connection and Temporal Aggregation
What the sources say
- Not checked yetcites [84] Kremer, S.C., and Kolen, J.F. (2001). Gradient Flow in Recurrent Nets: The Difficulty of Learning Long-Term Dependencies. A Field Guide to Dynamical Recurrent Networks, IEEE Press.
- Supportedcites [85] Goh, 2017, Deep learning for computational chemistry, J. Comput. Chem. source
Section 2, “Deep Learning 101,” explains that error signals decay as they pass backward through multiple layers, calling this the vanishing gradient problem. It also says this problem made training deep networks difficult and practically capped their depth. (LLM, conf=0.90)
- Not checked yetcites [86] He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA. source
hedgedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“which increased the number of trainable parameters by a factor of 1100 compared to state-of-the-art model architectures.”
- Also stated in: § 1. Introduction · plain
“Compared to state-of-the-art model architectures, we increase the number of trainable parameters by a factor of 1100.”
§ Complex Deep Neural Networks from Large Scale Virtual IMU Data for Effective Human Activity Recognition Using Wearables > Abstract
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 1. Introduction · plain
60 more claims
Create a free account to see all 115 claims and to verify the 45 references still open.
- Probably not a substantial claim. 35
- It may state two separate claims as one. 19
- The quote only partly states this claim. 1
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“The system leads to improving model performance when integrating real and virtual IMU data [108,109].”
From Past, Present, and Future of Sensor-based Human Activity Recognition Using Wearables: A Surveying Tutorial on a Still Challenging Task (2025) · cites this paper as [108]
The cited source says what the claim attributes to it.
In the citing paper
“Cross-modality transfer approaches [54,108,109,161,162] are instrumental for extracting virtual IMU data from 2D RGB videos of activities.”
From Past, Present, and Future of Sensor-based Human Activity Recognition Using Wearables: A Surveying Tutorial on a Still Challenging Task (2025) · cites this paper as [108]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Complex Deep Neural Networks from Large Scale Virtual IMU Data for Effective Human Activity Recognition Using Wearables > 5. Discussion > 5.1. Collect Even Larger Datasets of Virtual IMU Data
“a total of approximately 41 hours, i.e., about 150,000 samples of 4-second data frames”
§ Complex Deep Neural Networks from Large Scale Virtual IMU Data for Effective Human Activity Recognition Using Wearables > 4. Case Study: Analyzing Free Weight Gym Exercises > 4.3. Evaluation Protocol
“we use a 4-second window with 1-second overlap across all experiments”
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 3×
“Previous work showed that training a large model is challenging because it becomes more likely that the gradient during optimization becomes infinitesimally small at the bottom layers [84–86].”
BackgroundMethodImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 1×
“With such a massive dataset it is now possible to train very large models, such as the popular AlexNet [99], or ResNet-152 [86] that consists of 60 million trainable parameters.”
MethodHochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 1×
“Combined with recurrent neural network elements such as long short-term memory (LSTM) cells [27], effective models can be derived that capture both short-and long-term temporal dependencies by aggregating historical features along the sequence [7,25,26,28].”
BackgroundGoing deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · cited 1×
“For example, the popular ImageNet dataset consists of about 14 million annotated images [98], which is two orders of magnitude larger than our dataset.”
MethodLabaca-Castro, R. (2023) · Machine Learning under Malware Attack · doi:10.1007/978-3-658-40442-0_9 · Cited by 19,272
OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields
Cao, Z., Hidalgo, G., Simon, T., Wei, S.-E., & Sheikh, Y. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2929257 · Cited by 5,536 · cited 2×
“Standard multi-person 2D pose detectors often confuse people in a scene with non-human objects that may look like humans, which results in erroneous tracking [52].”
BackgroundProceedings European Conference on Computer Vision 2002
Heyden, A., Sparr, G., Nielsen, M., & Johansen, P. (2002) · Lund University Publications (Lund University) · Cited by 5,153
Jaderberg, M., Simonyan, K., Zisserman, A., & Kavukcuoglu, K. (2015) · arXiv · doi:10.48550/arxiv.1506.02025 · Cited by 5,137 · cited 1×
“The cropping operation, F crop ( X , s , e ) = F sampler ( F grid _ gen ( X ) , s , e ) , of AdapTrimm is fully differentiable, which is adapted from the grid generator, ´ X = F grid _ gen ( X ) , and sampler, F sampler ( ´ X , s , e ) , used in spatial transformer networks (STN) [71].”
BackgroundProbable Inference, the Law of Succession, and Statistical Inference
Wilson, E. B. (1927) · Journal of the American Statistical Association · doi:10.1080/01621459.1927.10502953 · Cited by 4,476 · cited 1×
“For the statistical significance test, Wilson score intervals are derived from 15 runs for each experiment [96].”
MethodSimple online and realtime tracking
Bewley, A., Ge, Z., Ott, L., Ramos, F., & Upcroft, B. (2016) · 2016 IEEE International Conference on Image Processing (ICIP) · doi:10.1109/icip.2016.7533003 · Cited by 4,433 · cited 1×
“In order to estimate 2D pose changes across video sequences, a multiperson tracker is utilized [58,59].”
MethodRank transformations as a bridge between parametric and nonparametric statistics
Conover, W. J., & Iman, R. L. (1981) · The American Statistician · doi:10.1080/00031305.1981.10479327 · Cited by 3,922 · cited 1×
“In order to minimize the discrepancies between the extracted virtual IMU data and the target real IMU sensor data, IMUTube applies distribution mapping [67].”
MethodMulti-task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics
Cipolla, R., Gal, Y., & Kendall, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00781 · Cited by 3,151
Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition
Ordóñez, F., & Roggen, D. (2016) · Sensors · doi:10.3390/s16010115 · Cited by 2,761 · cited 10×
“Recurrent neural networks (RNNs) are often used in combination with convolutional operators in an effort to capture global temporal dynamics of the feature representations learned by the convolutional part of the model [7].”
BackgroundMethodInfluentialUnsupervised Representation Learning by Predicting Image Rotations
Gidaris, S., Singh, P., & Komodakis, N. (2018) · arXiv · doi:10.48550/arxiv.1803.07728 · Cited by 2,477 · cited 1×
“Recently, semi-supervised and especially self-supervised learning methods [101–103] have become popular in many machine learning application domains including human activity recognition using wearables [13,43,44].”
BackgroundSeqGAN: Sequence Generative Adversarial Nets with Policy Gradient
Yu, L., Zhang, W., Wang, J., & Yu, Y. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10804 · Cited by 2,285
SMOTE for Learning from Imbalanced Data: Progress and Challenges, Marking the 15-year Anniversary
Fernandez, A., Garcia, S., Herrera, F., & Chawla, N. V. (2018) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.11192 · Cited by 2,219 · cited 1×
“Fernández et al. [33] also proposed techniques to oversample minority samples to tackle the label imbalance problem.”
BackgroundBilinear CNN Models for Fine-Grained Visual Recognition
Lin, T.-Y., RoyChowdhury, A., & Maji, S. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.170 · Cited by 2,135 · cited 2×
“We have explored other feature aggregation approaches, such as summation or bilinear pooling [82],”
BackgroundMethodRMPE: Regional Multi-person Pose Estimation
Fang, H.-S., Xie, S., Tai, Y.-W., & Lu, C. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.256 · Cited by 1,884 · cited 1×
“To do so, the system estimates 2D poses [52,57] from detected humans using a human detector [53].”
BackgroundNeural Architecture Search: A Survey
Elsken, T., Metzen, J. H., & Hutter, F. (2018) · arXiv · doi:10.48550/arxiv.1808.05377 · Cited by 1,763
A tutorial on human activity recognition using body-worn inertial sensors
Bulling, A., Blanke, U., & Schiele, B. (2014) · ACM Computing Surveys · doi:10.1145/2499621 · Cited by 1,668
Introducing a New Benchmarked Dataset for Activity Monitoring
Reiss, A., & Stricker, D. (2012) · 2012 16th International Symposium on Wearable Computers · doi:10.1109/iswc.2012.13 · Cited by 1,456 · cited 1×
“The advancements in IMUTube have made it easier to collect weakly-labeled virtual IMU datasets, which have been successfully used for exemplary HAR tasks, including classification of locomotions, home activities, and gym exercises [31,68,69].”
AMASS: Archive of Motion Capture As Surface Shapes
Mahmood, N., Ghorbani, N., Troje, N. F., Pons-Moll, G., & Black, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00554 · Cited by 1,323 · cited 1×
“However, the usefulness of virtual IMU data from virtual 3D characters has inspired other studies to explore large, public motion capture datasets available, for example, in the computer graphics community [47–49].”
Background3D Human Pose Estimation in Video With Temporal Convolutions and Semi-Supervised Training
Pavllo, D., Feichtenhofer, C., Grangier, D., & Auli, M. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00794 · Cited by 1,297 · cited 1×
“Then each 2D pose sequence is lifted to 3D poses using 3D pose estimators [60].”
MethodHow many steps/day are enough? for adults
Tudor-Locke, C., Craig, C. L., Brown, W. J., Clemes, S. A., De Cocker, K., Giles-Corti, B., Hatano, Y., Inoue, S., Matsudo, S. M., Mutrie, N., et al. (2011) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/1479-5868-8-79 · Cited by 1,087
Omni-Scale Feature Learning for Person Re-Identification
Zhou, K., Yang, Y., Cavallaro, A., & Xiang, T. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00380 · Cited by 1,005 · cited 1×
“In order to estimate 2D pose changes across video sequences, a multiperson tracker is utilized [58,59].”
MethodGradient Flow in Recurrent Nets: the Difficulty of Learning Long-Term Dependencies
Hochreiter, S., Bengio, Y., Informatik, F. F., Bengio, Y., Frasconi, P., & Schmidhuber, J. (2001) · Cited by 863
Deep learning for computational chemistry
Goh, G. B., Hodas, N. O., & Vishnu, A. (2017) · Journal of Computational Chemistry · doi:10.1002/jcc.24764 · Cited by 809
The Opportunity challenge: A benchmark database for on-body sensor-based activity recognition
Chavarriaga, R., Sagha, H., Calatroni, A., Digumarti, S. T., Tröster, G., Millán, J. D. R., & Roggen, D. (2013) · Pattern Recognition Letters · doi:10.1016/j.patrec.2012.12.014 · Cited by 742 · cited 3×
“The advancements in IMUTube have made it easier to collect weakly-labeled virtual IMU datasets, which have been successfully used for exemplary HAR tasks, including classification of locomotions, home activities, and gym exercises [31,68,69].”
BackgroundMethodUm, T. T., Pfister, F. M. J., Pichler, D., Endo, S., Lang, M., Hirche, S., Fietzek, U., & Kulić, D. (2017) · Proceedings of the 19th ACM International Conference on Multimodal Interaction · doi:10.1145/3136755.3136817 · Cited by 729 · cited 4×
“To tackle this “small data” problem, the research community has explored many different avenues, ranging from, for example, data augmentation [10,11], transfer learning [12], to self-supervised learning techniques [13].”
BackgroundMethodInfluentialWearable Assistant for Parkinson’s Disease Patients With the Freezing of Gait Symptom
Bachlin, M., Plotnik, M., Roggen, D., Maidan, I., Hausdorff, J. M., Giladi, N., & Troster, G. (2010) · IEEE Transactions on Information Technology in Biomedicine · doi:10.1109/titb.2009.2036165 · Cited by 693 · cited 1×
“For example, the Daphnet freeze of gait dataset consists of a mere five hours of sensor data from 10 users [29], and the Wetlab activity dataset covers 13 h of sensor data from 22 users [30].”
MethodDeep Learning for Sensor-based Human Activity Recognition
Chen, K., Zhang, D., Yao, L., Guo, B., Yu, Z., & Liu, Y. (2022) · ACM Computing Surveys · doi:10.1145/3447744 · Cited by 690 · cited 2×
“Recently, end-to-end learning approaches have been introduced for supervised feature learning in HAR [22,25].”
BackgroundMethodData Augmentation for Time Series Classification using Convolutional Neural Networks
Guennec, A. L., Malinowski, S., & Tavenard, R. (2016) · HAL · Cited by 604
Um, T. T. (n.d.) · mediaTUM – the media and publications repository of the Technical University Munich (Technical University Munich) · Cited by 602
How many steps/day are enough? for children and adolescents
Tudor-Locke, C., Craig, C. L., Beets, M. W., Belton, S., Cardon, G. M., Duncan, S., Hatano, Y., Lubans, D. R., Olds, T. S., Raustorp, A., et al. (2011) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/1479-5868-8-78 · Cited by 552 · cited 1×
“For example, the average duration of a (walking) step is between 0.3 and 0.6 s depending on age [75–77].”
BackgroundInfluentialBerkeley MHAD: A comprehensive Multimodal Human Action Database
Ofli, F., Chaudhry, R., Kurillo, G., Vidal, R., & Bajcsy, R. (2013) · 2013 IEEE Workshop on Applications of Computer Vision (WACV) · doi:10.1109/wacv.2013.6474999 · Cited by 484
Step Counting: A Review of Measurement Considerations and Health-Related Applications
Bassett, D. R., Toth, L. P., LaMunion, S. R., & Crouter, S. E. (2017) · Sports Medicine · doi:10.1007/s40279-016-0663-1 · Cited by 460 · cited 1×
“For example, the average duration of a (walking) step is between 0.3 and 0.6 s depending on age [75–77].”
BackgroundInfluentialDepth From Videos in the Wild: Unsupervised Monocular Depth Learning From Unknown Cameras
Gordon, A., Li, H., Jonschkowski, R., & Angelova, A. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00907 · Cited by 389 · cited 1×
“IMUTube does so by estimating the background depth maps from each scene and lifting them to 3D point cloud models [62,63].”
BackgroundMulti-task Self-Supervised Learning for Human Activity Detection
Saeed, A., Ozcelebi, T., & Lukkien, J. (2019) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3328932 · Cited by 346 · cited 1×
“Saeed et al. [41] proposed a multi-task pretext task to predict eight different types of data transformations applied to the input data.”
MethodFeature Learning for Activity Recognition in Ubiquitous Computing
Plötz, T., Hammerla, N., & Olivier, P. (2011) · Newcastle University · doi:10.25405/ncl.34175319 · Cited by 337
Self-citationColored Point Cloud Registration Revisited
Park, J., Zhou, Q.-Y., & Koltun, V. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.25 · Cited by 331 · cited 1×
“Based on subsequent 3D point clouds, IMUTube calculates the camera’s ego-motion using the Iterative Closest Points (ICP) method [64].”
MethodHuman activity recognition from accelerometer data using Convolutional Neural Network
Lee, S.-M., Yoon, S. M., & Cho, H. (2017) · 2017 IEEE International Conference on Big Data and Smart Computing (BigComp) · doi:10.1109/bigcomp.2017.7881728 · Cited by 331 · cited 1×
“Modeling such variations of motion lengths is important for deriving effective feature representations, be it explicitly through feature engineering or implicitly through end-to-end learning [80,81].”
BackgroundDeep Learning for Sensor-based Human Activity Recognition: Overview, Challenges and Opportunities
Chen, K., Zhang, D., Yao, L., Guo, B., Yu, Z., & Liu, Y. (2020) · arXiv · doi:10.48550/arxiv.2001.07416 · Cited by 314 · cited 2×
“Recently, end-to-end learning approaches have been introduced for supervised feature learning in HAR [22,25].”
BackgroundMethodPose2Seg: Detection Free Human Instance Segmentation
Zhang, S.-H., Li, R., Dong, X., Rosin, P., Cai, Z., Han, X., Yang, D., Huang, H., & Hu, S.-M. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00098 · Cited by 231 · cited 1×
“To do so, the system applies human segmentation [54] and body part parsing [55] for a detected human bounding box.”
BackgroundSchoenfeld, B. J., Ogborn, D. I., & Krieger, J. W. (2015) · Sports Medicine · doi:10.1007/s40279-015-0304-0 · Cited by 183 · cited 1×
“Depending on the specifics of eccentric motions in, e.g., gym exercises (as we study them in this paper) these durations can get extended to 1–3 s [78,79].”
BackgroundLiu, L., Zhang, J., He, R., Liu, Y., Wang, Y., Tai, Y., Luo, D., Wang, C., Li, J., & Huang, F. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00652 · Cited by 183
Biosignal Data Augmentation Based on Generative Adversarial Networks
Haradal, S., Hayashi, H., & Uchida, S. (2018) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc.2018.8512396 · Cited by 155 · cited 1×
“For example, Harada et al. [8] and Yao et al. [36] augmented biosignals and IoT (Internet of Things) sensor signals with GANs, respectively.”
BackgroundKwon, H., Tong, C., Haresamudram, H., Gao, Y., Abowd, G. D., Lane, N. D., & Plötz, T. (2020) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3411841 · Cited by 152
Self-citationContrastive Predictive Coding for Human Activity Recognition
Haresamudram, H., Essa, I., & Plötz, T. (2021) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3463506 · Cited by 150 · cited 2×
“Recently, semi-supervised and especially self-supervised learning methods [101–103] have become popular in many machine learning application domains including human activity recognition using wearables [13,43,44].”
BackgroundSelf-citationMasked reconstruction based self-supervision for human activity recognition
Haresamudram, H., Beedu, A., Agrawal, V., Grady, P. L., Essa, I., Hoffman, J., & Plötz, T. (2020) · Proceedings of the 2020 International Symposium on Wearable Computers · doi:10.1145/3410531.3414306 · Cited by 146 · cited 3×
“To tackle this “small data” problem, the research community has explored many different avenues, ranging from, for example, data augmentation [10,11], transfer learning [12], to self-supervised learning techniques [13].”
BackgroundMethodSelf-citationMorales, F. J. O., & Roggen, D. (2016) · Proceedings of the 2016 ACM International Symposium on Wearable Computers · doi:10.1145/2971763.2971764 · Cited by 136 · cited 3×
“Combined with recurrent neural network elements such as long short-term memory (LSTM) cells [27], effective models can be derived that capture both short- and long-term temporal dependencies by aggregating historical features along the sequence [7,25,26,28].”
BackgroundMethodClassification of gait patterns in the time–frequency domain
Nyan, M. N., Tay, F. E. H., Seah, K. H. W., & Sitoh, Y. Y. (2005) · Journal of Biomechanics · doi:10.1016/j.jbiomech.2005.08.014 · Cited by 136
Gao, Y., Wang, W., Phoha, V. V., Sun, W., & Jin, Z. (2019) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3351239 · Cited by 126
Accelerometry Based Classification of Walking Patterns Using Time-frequency Analysis
Wang, N., Ambikairajah, E., Lovell, N. H., & Celler, B. G. (2007) · 2007 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society · doi:10.1109/iembs.2007.4353438 · Cited by 117
Federated Self-Supervised Learning of Multisensor Representations for Embedded Intelligence
Saeed, A., Salim, F. D., Ozcelebi, T., & Lukkien, J. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3009358 · Cited by 106 · cited 2×
“Recently, semi-supervised and especially self-supervised learning methods [101–103] have become popular in many machine learning application domains including human activity recognition using wearables [13,43,44].”
BackgroundPMC: A Privacy-preserving Deep Learning Model Customization Framework for Edge Computing
Liu, B., Li, Y., Liu, Y., Guo, Y., & Chen, X. (2020) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3432208 · Cited by 95
Cross-domain activity recognition via transfer learning
Hu, D. H., Zheng, V. W., & Yang, Q. (2010) · Pervasive and Mobile Computing · doi:10.1016/j.pmcj.2010.11.005 · Cited by 93
Federated Self-Supervised Learning of Multisensor Representations for Embedded Intelligence
Saeed, A., Salim, F. D., Özçelebi, T., & Lukkien, J. J. (2024) · Figshare · Cited by 92
On the role of features in human activity recognition
Haresamudram, H., Anderson, D. V., & Plötz, T. (2019) · Proceedings of the 23rd International Symposium on Wearable Computers · doi:10.1145/3341163.3347727 · Cited by 90 · cited 2×
“Other variants of feature learning utilized more contemporary forms of autoencoder networks showing similar improvements in feature generalization capabilities [23,24].”
BackgroundSelf-citationHuman Activity Recognition with Convolutional Neural Networks
Bevilacqua, A., MacDonald, K., Rangarej, A., Widjaya, V., Caulfield, B., & Kechadi, T. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-10997-4_33 · Cited by 86 · cited 1×
“Combined with recurrent neural network elements such as long short-term memory (LSTM) cells [27], effective models can be derived that capture both short-and long-term temporal dependencies by aggregating historical features along the sequence [7,25,26,28].”
BackgroundBaz-Valle, E., Fontes-Villalba, M., & Santos-Concejero, J. (2021) · Journal of Strength & Conditioning Research · doi:10.1519/jsc.0000000000002776 · Cited by 84 · cited 2×
“, gym exercises (as we study them in this paper) these durations can get extended to 1–3 s [78,79].”
BackgroundMethodIoT Structured Long-Term Wearable Social Sensing for Mental Wellbeing
Yang, S., Gao, B., Jiang, L., Jin, J., Gao, Z., Ma, X., & Woo, W. L. (2018) · IEEE Internet of Things Journal · doi:10.1109/jiot.2018.2889966 · Cited by 83 · cited 3×
“Specifically, MVMW-Conv feature representations X MV = F mwmv ( X MS ) are extracted as follows: where multi-view features are captured for different temporal lengths [ 3, 4, · · · , 61 ] .”
BackgroundMethodHuman activity recognition based on smartphone and wearable sensors using multiscale DCNN ensemble
Sena, J., Barreto, J., Caetano, C., Cramer, G., & Schwartz, W. R. (2021) · Neurocomputing · doi:10.1016/j.neucom.2020.04.151 · Cited by 81 · cited 1×
“Modeling such variations of motion lengths is important for deriving effective feature representations, be it explicitly through feature engineering or implicitly through end-to-end learning [80,81].”
BackgroundGrapy-ML: Graph Pyramid Mutual Learning for Cross-Dataset Human Parsing
He, H., Zhang, J., Zhang, Q., & Tao, D. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6728 · Cited by 62
IMUSim: A simulation environment for inertial sensing algorithm design and evaluation
Young, A. D., Ling, M. J., & Arvind, D. K. (2011) · Information Processing in Sensor Networks · Cited by 56
SenseGAN: Enabling Deep Learning for Internet of Things with a Semi-Supervised Framework
Yao, S., Zhao, Y., Shao, H., Zhang, C., Zhang, A., Hu, S., Liu, D., Liu, S., Su, L., & Abdelzaher, T. (2018) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3264954 · Cited by 54 · cited 1×
“[36] augmented biosignals and IoT (Internet of Things) sensor signals with GANs, respectively.”
BackgroundA Deep Learning Method for Complex Human Activity Recognition Using Virtual Wearable Sensors
Xiao, F., Pei, L., Chu, L., Zou, D., Yu, W., Zhu, Y., & Li, T. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-69873-7_19 · Cited by 53
Handling annotation uncertainty in human activity recognition
Kwon, H., Abowd, G. D., & Plötz, T. (2019) · Proceedings of the 23rd International Symposium on Wearable Computers · doi:10.1145/3341163.3347744 · Cited by 52
Self-citationPlotz, T., Chen, C., Hammerla, N. Y., & Abowd, G. D. (2012) · 2012 16th International Symposium on Wearable Computers · doi:10.1109/iswc.2012.15 · Cited by 51 · cited 3×
“Assume [ 12 , 14 , 18 ] are defined as the range for the temporal scales with three branches in our hourglass network model from X C ∈ R h × T × d , where h is the number of feature map, T is the temporal length, and d is the number of sensor modalities (Figure 3).”
BackgroundMethodInfluentialSelf-citationApproaching the Real-World: Supporting Activity Recognition Training with Virtual IMU Data
Kwon, H., Wang, B., Abowd, G. D., & Plötz, T. (2021) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3478096 · Cited by 48
Self-citationMyoGym: introducing an open gym data set for activity recognition collected using myo armband
Koskimäki, H., Siirtola, P., & Röning, J. (2017) · Proceedings of the 2017 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2017 ACM International Symposium on Wearable Computers · doi:10.1145/3123024.3124400 · Cited by 46 · cited 5×
“For the MyoGym real IMU dataset, we use subjects 1 through 5 as training set, subjects 6 and 7 as validation set, and data from subjects 8 through 10 as the test set for all experiments.”
BackgroundMethodInfluentialWearables in the wet lab: a laboratory system for capturing and guiding experiments
Scholl, P. M., Wille, M., & Van Laerhoven, K. (2015) · Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing · doi:10.1145/2750858.2807547 · Cited by 45 · cited 1×
“For example, the Daphnet freeze of gait dataset consists of a mere five hours of sensor data from 10 users [29], and the Wetlab activity dataset covers 13 h of sensor data from 22 users [30].”
MethodDeep Auto-Set: A Deep Auto-Encoder-Set Network for Activity Recognition Using Wearables
Varamin, A. A., Abbasnejad, E., Shi, Q., Ranasinghe, D. C., & Rezatofighi, H. (2018) · Proceedings of the 15th EAI International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services · doi:10.1145/3286978.3287024 · Cited by 44 · cited 2×
“Other variants of feature learning utilized more contemporary forms of autoencoder networks showing similar improvements in feature generalization capabilities [23,24].”
BackgroundMethodRey, V. F., Hevesi, P., Kovalenko, O., & Lukowicz, P. (2019) · Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers · doi:10.1145/3341162.3345590 · Cited by 43 · cited 3×
“Video repositories have recently attracted attention as potential training resources for sensor-based HAR models [14,15,17].”
BackgroundMethodAdversarial Multi-view Networks for Activity Recognition
Bai, L., Yao, L., Wang, X., Kanhere, S. S., Guo, B., & Yu, Z. (2020) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3397323 · Cited by 40 · cited 1×
“For effective activity recognition, feature representations for multi-channel sensor data in HAR models need to capture the relationships between time and channel information [83].”
BackgroundLiu, Y., Zhang, S., & Gowda, M. (2021) · Proceedings of the International Conference on Internet-of-Things Design and Implementation · doi:10.1145/3450268.3453537 · Cited by 39 · cited 1×
“[16] demonstrated that virtual IMU data from videos could be used for American Sign Language (ASL) recognition tasks.”
Kwon, H., Abowd, G. D., & Plötz, T. (2018) · Proceedings of the 2018 ACM International Symposium on Wearable Computers · doi:10.1145/3267242.3267258 · Cited by 35 · cited 1×
“For each analysis window, features are calculated, often based on hand-crafted representations that incorporate statistical, frequency, or distribution information [19–21].”
MethodSelf-citationFinger Gesture Tracking for Interactive Applications: A Pilot Study with Sign Languages
Liu, Y., Jiang, F., & Gowda, M. (2020) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3414117 · Cited by 35 · cited 5×
“As discussed in Section 3.4.3, we applied non-linear multi-scale operation on the input at each layer of GRU with scaling factors of [ 1, 12 , 14 ] .”
BackgroundMethodInfluentialLearning to Recognize Human Activities Using Soft Labels
Hu, N., Englebienne, G., Lou, Z., & Krose, B. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2621761 · Cited by 29 · cited 1×
“Previous work showed that handling outlier samples in HAR datasets is important and, if successful, can significantly improve the performance of the resulting HAR model [91–93].”
BackgroundCross-dataset deep transfer learning for activity recognition
Gjoreski, M., Kalabakov, S., Luštrek, M., Gams, M., & Gjoreski, H. (2019) · Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers · doi:10.1145/3341162.3344865 · Cited by 28 · cited 2×
“Over the years, the HAR research community has developed numerous approaches to derive effective recognition systems [6–9].”
BackgroundMulti-Scale Recurrent Neural Network for Sound Event Detection
Lü, R., Duan, Z., & Zhang, C. (2018) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2018.8462006 · Cited by 26 · cited 1×
“Furthermore, studies have shown that multi-scale recurrent neural networks are effective through hierarchically aggregating the inherent temporal dynamics of the data sequences analyzed [88–90].”
BackgroundDigging deeper: towards a better understanding of transfer learning for human activity recognition
Hoelzemann, A., & Van Laerhoven, K. (2020) · Proceedings of the 2020 International Symposium on Wearable Computers · doi:10.1145/3410531.3414311 · Cited by 20 · cited 7×
“As discussed in Section 3.4.3, we applied non-linear multi-scale operation on the input at each layer of GRU with scaling factors of [ 1, 12 , 14 ] .”
BackgroundMethodInfluentialA Multi-Sensor Setting Activity Recognition Simulation Tool
Takeda, S., Okita, T., Lago, P., & Inoue, S. (2018) · Proceedings of the 2018 ACM International Joint Conference and 2018 International Symposium on Pervasive and Ubiquitous Computing and Wearable Computers · doi:10.1145/3267305.3267509 · Cited by 18 · cited 1×
“[51] trained HAR models with virtual IMU data extracted from hundreds of subjects performing thousands of motions in the motion capture dataset.”
MethodTowards Machine Learning with Zero Real-World Data
Kang, C., Jung, H., & Lee, Y. (2019) · WearSys@MobiSys · doi:10.1145/3325424.3329662 · Cited by 14 · cited 2×
“To capture varying durations for activities under study, we employed 10 different 1D kernel sizes, namely [3, 5, 7, 15, 23, 31, 37, 45, 53, 61].”
MethodClassification of Aggressive Movements Using Smartwatches
Franck, T., Baddour, N., & Lemaire, E. D. (2020) · Sensors · doi:10.3390/s20216377 · Cited by 13 · cited 1×
“These movement signals are then automatically analyzed to recognize and assess activities that are of relevance for many practical applications, including, for example, gesture-based interaction [1], health assessments [2,3], or behavioral authentication [4].”
BackgroundChen, Y., Gu, Y., Jiang, X., & Wang, J. (2016) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/2968219.2971453 · Cited by 12 · cited 1×
“[40] developed an online domain adaptation model under the challenge of dynamically changing feature dimensions, activity classes, and data distributions, simultaneously.”
BackgroundNasir, M., Baucom, B., Georgiou, P., & Narayanan, S. (2015) · 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2015.7178298 · Cited by 9 · cited 1×
“Previous work showed that handling outlier samples in HAR datasets is important and, if successful, can significantly improve the performance of the resulting HAR model [91–93].”
Background
Research works printed but not linked: 18
- Gao, 2019, EarEcho: Using ear canal echo for wearable authentication, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., 3, 1
- Hammerla, N.Y., Halloran, S., and Plötz, T. (2016). Deep, Convolutional, and Recurrent Models for Human Activity Recognition Using Wearables, AAAI Press.
- Fawaz, H., Forestier, G., Weber, J., Idoumghar, L., and Muller, P. (2018). Data augmentation using synthetic data for time series classification with deep residual networks. arXiv.
- Ramponi, G., Protopapas, P., Brambilla, M., and Janssen, R. (2018). T-cgan: Conditional generative adversarial network for data augmentation in noisy time series with irregular sampling. arXiv.
- Yosinski, J., Clune, J., Bengio, Y., and Lipson, H. (2014). How transferable are features in deep neural networks?. arXiv.
- Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., and Polosukhin, I. (2017). Attention is all you need. arXiv.
- Redmon, J., and Farhadi, A. (2018). YOLOv3: An Incremental Improvement. arXiv.
- Joel, A., and Stergios, I. (2011, January 6–13). A Direct Least-Squares (DLS) method for PnP. Proceedings of the 2011 International Conference on Computer Vision, Barcelona, Spain.
- Kingma, D., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv.
- Newell, A., Yang, K., and Deng, J. Stacked hourglass networks for human pose estimation. Proceedings of the European Conference on Computer Vision.
- Li, P., Lin, Y., and Schultz-Fellenz, E. (2018). Contextual hourglass network for semantic segmentation of high resolution aerial imagery. arXiv.
- Oñoro-Rubio, D., and Niepert, M. (2018). Contextual Hourglass Networks for Segmentation and Density Estimation. arXiv.
- Chung, J., Ahn, S., and Bengio, Y. (2016). Hierarchical multiscale recurrent neural networks. arXiv.
- Kádár, A., Côté, M., Chrupała, G., and Alishahi, A. (2018). Revisiting the hierarchical multiscale lstm. arXiv.
- Kendall, A., and Gal, Y. (2017). What uncertainties do we need in bayesian deep learning for computer vision?. arXiv.
- Rolnick, D., Veit, A., Belongie, S., and Shavit, N. (2017). Deep learning is robust to massive label noise. arXiv.
- Misra, I., Zitnick, C.L., and Hebert, M. Shuffle and learn: Unsupervised learning using temporal order verification. Proceedings of the European Conference on Computer Vision.
- Devlin, J., Chang, M., Lee, K., and Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv.
Cited by
IMUGPT 2.0: Language-Based Cross Modality Transfer for Sensor-Based Human Activity Recognition
Leng, Z., Bhattacharjee, A., Rajasekhar, H., Zhang, L., Bruda, E., Kwon, H., & Plötz, T. (2024) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3678545 · Cited by 61
“Sensor-based human activity recognition using wearable devices (HAR), i.e., the automated assessment of what a person is doing (and when), has come a long way and it is considered one of the central pillars of mobile, wearable, and ubiquitous computing [36].”
InfluentialSelf-citationMekruksavanich, S., & Jitpattanakul, A. (2023) · Intelligent Automation & Soft Computing · doi:10.32604/iasc.2023.033542 · Cited by 45
Leng, Z., Kwon, H., & Ploetz, T. (2023) · Proceedings of the 2023 International Symposium on Wearable Computers · doi:10.1145/3594738.3611361 · Cited by 39
“Previous studies [10, 12] have shown that the extracted virtual IMU data lead to improved model performance when mixed with the real IMU data and allowed for effective training of more complex models.”
Self-citationHaresamudram, H., Tang, C. I., Suh, S., Lukowicz, P., & Plötz, T. (2025) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3729467 · Cited by 33
“Efforts to bridge the gap between video and Inertial Measurement Unit (IMU) data have gained prominence, with recent works addressing the challenge of translating video data into IMU representations [103, 104, 153, 154].”
InfluentialSelf-citationSignRing: Continuous American Sign Language Recognition Using IMU Rings and Virtual IMU Data
Li, J., Huang, L., Shah, S., Jones, S. J., Jin, Y., Wang, D., Russell, A., Choi, S., Gao, Y., Yuan, J., et al. (2023) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3610881 · Cited by 44
Metadata
- Authors
- Kwon, H., Abowd, G. D., & Plötz, T.
- Affiliations
- Georgia Institute of Technology (US)
- Northeastern University (US)
- Corresponding author
- Plötz, Thomas
- Venue
- Sensors
- Publisher
- MDPI AG
- Published
- 2021-12-13
- Volume
- 21
- Issue
- 24
- Pages
- 8337
- ISSN
- 1424-8220 (ISSN-L)
- PMC
- PMC8707382
- OpenAlex
- W4200504110
- Cited by
- 30
- References
- 86
- Citations by year
- 2026: 4 · 2025: 10 · 2024: 4 · 2023: 10 · 2022: 2
- FWCI
- 1.44
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Sensors (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Samsung Scholarship Program (N/A)
- Oracle for Research (N/A)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Physical Sciences › Engineering › Biomedical Engineering › Gait Recognition and Analysis
- Keywords
- human activity recognition, inertial measurement unit, wearable sensors, cross-modality transfer, deep neural network, model complexity, dumbbell exercise, transfer learning
- MeSH terms
- Wearable Electronic Devices, Humans, Human Activities, Neural Networks, Computer, Recognition, Psychology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-05, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-05 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash and gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.