AugToAct: scaling complex human activity recognition with few labels
Open accessThe DOI matches its Crossref record; 17 of the 17 references it cites were found in a registry or an index.
Abstract
Human activity recognition (HAR) from wearable sensor data has recently gained widespread adoption in a number of fields. However, recognizing complex human activities, postural and rhythmic body movements (e.g. dance, sports) is challenging due to the lack of domain-specific labeling information, the perpetual variability in human movement kinematics profiles due to age, sex, dexterity and the level of professional training. In this paper, we propose a deep activity recognition model to work with limited labeled data, both for simple and complex human activities. To mitigate the intra and inter-user spatio-temporal variability of movements, we posit novel data augmentation and domain normalization techniques. We depict a semi-supervised technique that learns noise and transformation invariant feature representation from sparsely labeled data to accommodate intra-personal and inter-user variations of human movement kinematics. We also postulate a transfer learning approach to learn domain invariant feature representations by minimizing the feature distribution distance between the source and target domains. We showcase the improved performance of our proposed framework, AugToAct, using a public HAR dataset. We also design our own data collection, annotation and experimental setup on complex dance activity recognition steps and kinematics movements where we achieved higher performance metrics with limited label data compared to simple activity recognition tasks.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition
Ordóñez, F., & Roggen, D. (2016) · Sensors · doi:10.3390/s16010115 · Cited by 2,761
Self-Normalizing Neural Networks
Klambauer, G., Unterthiner, T., Mayr, A., & Hochreiter, S. (2017) · arXiv · doi:10.48550/arxiv.1706.02515 · Cited by 2,267
Transfer Feature Learning with Joint Distribution Adaptation
Long, M., Wang, J., Ding, G., Sun, J., & Yu, P. S. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.274 · Cited by 2,071
Domain adaptation with structural correspondence learning
Blitzer, J., McDonald, R., & Pereira, F. (2006) · Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing - EMNLP '06 · doi:10.3115/1610075.1610094 · Cited by 1,543 · cited 2×
“This flexibility comes at a price though, as the raw features in NLP or computer vision domain often carry more contextual information, unlike inertial data streams.”
MethodRealistic Evaluation of Deep Semi-Supervised Learning Algorithms
Oliver, A., Odena, A., Raffel, C., Cubuk, E. D., & Goodfellow, I. J. (2018) · arXiv · doi:10.48550/arxiv.1804.09170 · Cited by 772
Um, T. T., Pfister, F. M. J., Pichler, D., Endo, S., Lang, M., Hirche, S., Fietzek, U., & Kulić, D. (2017) · Proceedings of the 19th ACM International Conference on Multimodal Interaction · doi:10.1145/3136755.3136817 · Cited by 729 · cited 2×
“This approach works as properly parameterized augmentation transformations can approximate the minute variations noticed when humans perform their activities [49], thus helping to cover unexplored input space and increase the generalizability [13] of the trained model.”
BackgroundStisen, A., Blunck, H., Bhattacharya, S., Prentow, T. S., Kjærgaard, M. B., Dey, A., Sonne, T., & Jensen, M. M. (2015) · Proceedings of the 13th ACM Conference on Embedded Networked Sensor Systems · doi:10.1145/2809695.2809718 · Cited by 714
Adaptive batch normalization for practical domain adaptation
Li, Y., Wang, N., Shi, J., Hou, X., & Liu, J. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2018.03.005 · Cited by 601
Research works printed but not linked: 43
- 10.1145/3302506.3310391 · DOI · Paper page
- 10.1109/ICCV.2015.324 · DOI · Paper page
- 10.1016/j.pmcj.2014.05.006 · DOI
- Rita Chattopadhyay , Qian Sun , Wei Fan , Ian Davidson , Sethuraman Panchanathan , and Jieping Ye. 2012. Multisource domain adaptation and its application to early detection of fatigue. TKDD 6, 4 ( 2012 ), 18:1--18:26. Rita Chattopadhyay, Qian Sun, Wei Fan, Ian Davidson, Sethuraman Panchanathan, and Jieping Ye. 2012. Multisource domain adaptation and its application to early detection of fatigue. TKDD 6, 4 (2012), 18:1--18:26.
- 10.1109/TPAMI.2011.114 · DOI · Paper page
- 10.5555/1756006.1756025 · doi:10.5555/1756006.1756025 · Paper page
- 10.1145/3177102.3177116 · DOI
- 10.1109/ICDM.2017.106 · DOI · Paper page
- Yaroslav Ganin , Evgeniya Ustinova , Hana Ajakan , Pascal Germain , Hugo Larochelle , François Laviolette , Mario Marchand , and Victor S. Lempitsky . 2016 . Domain-Adversarial Training of Neural Networks . Journal of Machine Learning Research 17 (2016), 59:1--59:35. Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor S. Lempitsky. 2016. Domain-Adversarial Training of Neural Networks. Journal of Machine Learning Research 17 (2016), 59:1--59:35.
- 10.5555/2354409.2355024 · doi:10.5555/2354409.2355024
- Ian J. Goodfellow , Yoshua Bengio , and Aaron C . Courville . 2016 . Deep Learning. MIT Press . Ian J. Goodfellow, Yoshua Bengio, and Aaron C. Courville. 2016. Deep Learning. MIT Press.
- Daniel Ho , Eric Liang , Xi Chen , Ion Stoica , and Pieter Abbeel . 2019 . Population Based Augmentation: Efficient Learning of Augmentation Policy Schedules . In Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9--15 June 2019, Long Beach, California, USA. 2731--2741. Daniel Ho, Eric Liang, Xi Chen, Ion Stoica, and Pieter Abbeel. 2019. Population Based Augmentation: Efficient Learning of Augmentation Policy Schedules. In Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9--15 June 2019, Long Beach, California, USA. 2731--2741.
- HM Sajjad Hossain , MD Abdullah AL Hafiz Khan , Nirmalya Roy , and Shimei Pan . 2018 . DeActive: Scaling Activity Recognition with Active Deep Learning . Proceedings of the ACM on Interactive, Multimedia, Wearable and Ubiquitous Technologies (IMWUT 2018) (2018). HM Sajjad Hossain, MD Abdullah AL Hafiz Khan, Nirmalya Roy, and Shimei Pan. 2018. DeActive: Scaling Activity Recognition with Active Deep Learning. Proceedings of the ACM on Interactive, Multimedia, Wearable and Ubiquitous Technologies (IMWUT 2018) (2018).
- Sergey Ioffe and Christian Szegedy . 2015 . Batch normalization: Accelerating deep network training by reducing internal covariate shift . In International Conference on Machine Learning. 448--456 . Sergey Ioffe and Christian Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International Conference on Machine Learning. 448--456.
- 10.1109/PERCOM.2018.8444585 · DOI · Paper page
- Diederik P Kingma and Jimmy Ba . 2014 . Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014). Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014). · arXiv
- Hedda Lausberg and Han Sloetjes . 2009. Coding gestural behavior with the NEUROGES-ELAN system. Behavior research methods 41, 3 ( 2009 ), 841--849. Hedda Lausberg and Han Sloetjes. 2009. Coding gestural behavior with the NEUROGES-ELAN system. Behavior research methods 41, 3 (2009), 841--849.
- Arthur Le Guennec , Simon Malinowski , and Romain Tavenard . 2016 . Data augmentation for time series classification using convolutional neural networks . In ECML/PKDD workshop on advanced analytics and learning on temporal data. Arthur Le Guennec, Simon Malinowski, and Romain Tavenard. 2016. Data augmentation for time series classification using convolutional neural networks. In ECML/PKDD workshop on advanced analytics and learning on temporal data.
- Yann LeCun , Yoshua Bengio , and Geoffrey Hinton . 2015. Deep learning. nature 521, 7553 ( 2015 ), 436. Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. 2015. Deep learning. nature 521, 7553 (2015), 436.
- Mingsheng Long , Han Zhu , Jianmin Wang , and Michael I. Jordan . 2017. Deep Transfer Learning with Joint Adaptation Networks . In Proceedings of the 34th International Conference on Machine Learning, ICML 2017 , Sydney, NSW, Australia, 6- -11 August 2017 . 2208--2217. Mingsheng Long, Han Zhu, Jianmin Wang, and Michael I. Jordan. 2017. Deep Transfer Learning with Joint Adaptation Networks. In Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6--11 August 2017. 2208--2217.
- Kanti V Mardia and Peter E Jupp . 2009. Directional statistics . Vol. 494 . John Wiley & Sons . Kanti V Mardia and Peter E Jupp. 2009. Directional statistics. Vol. 494. John Wiley & Sons.
- Richard M. Murray , Zexiang Li , and Shankar Sastry . 1994. A mathematical introduction to robotics manipulation . CRC Press . Richard M. Murray, Zexiang Li, and Shankar Sastry. 1994. A mathematical introduction to robotics manipulation. CRC Press.
- Sinno Jialin Pan , Ivor W Tsang , James T Kwok , and Qiang Yang . [n. d.]. Domain adaptation via transfer component analysis. Neural Networks , IEEE Transactions , 2011 ([n. d.]). Sinno Jialin Pan, Ivor W Tsang, James T Kwok, and Qiang Yang. [n. d.]. Domain adaptation via transfer component analysis. Neural Networks, IEEE Transactions, 2011 ([n. d.]).
- 10.1109/TNN.2010.2091281 · DOI · Paper page
- 10.1109/TKDE.2009.191 · DOI · Paper page
- Charles Ruizhongtai Qi , Hao Su , Kaichun Mo , and Leonidas J. Guibas . 2017. PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation . In 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017 , Honolulu, HI, USA, July 21--26 , 2017 . 77--85. Charles Ruizhongtai Qi, Hao Su, Kaichun Mo, and Leonidas J. Guibas. 2017. PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation. In 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21--26, 2017. 77--85.
- Rajat Raina , Alexis Battle , Honglak Lee , Benjamin Packer , and Andrew Y. Ng . 2007. Self-taught learning: transfer learning from unlabeled data. In Machine Learning , Proceedings of the Twenty-Fourth International Conference (ICML 2007 ), Corvallis, Oregon, USA, June 20--24 , 2007 . 759--766. Rajat Raina, Alexis Battle, Honglak Lee, Benjamin Packer, and Andrew Y. Ng. 2007. Self-taught learning: transfer learning from unlabeled data. In Machine Learning, Proceedings of the Twenty-Fourth International Conference (ICML 2007), Corvallis, Oregon, USA, June 20--24, 2007. 759--766.
- 10.1109/ISWC.2008.4911590 · DOI · Paper page
- Maja Stikic , Diane Larlus , and Bernt Schiele . 2009 . Multi-graph Based Semi-supervised Learning for Activity Recognition. In 13th IEEE International Symposium on Wearable Computers (ISWC 2009 ), 4--7 September 2009, Linz, Austria. 85--92. Maja Stikic, Diane Larlus, and Bernt Schiele. 2009. Multi-graph Based Semi-supervised Learning for Activity Recognition. In 13th IEEE International Symposium on Wearable Computers (ISWC 2009), 4--7 September 2009, Linz, Austria. 85--92.
- 10.5555/3298483.3298614 · doi:10.5555/3298483.3298614
- 10.1109/TNN.2009.2015974 · DOI · Paper page
- Eric Tzeng , Judy Hoffman , Kate Saenko , and Trevor Darrell . 2017 . Adversarial Discriminative Domain Adaptation. In 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017 , Honolulu, HI, USA, July 21--26 , 2017. 2962--2971. Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell. 2017. Adversarial Discriminative Domain Adaptation. In 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21--26, 2017. 2962--2971.
- Eric Tzeng , Judy Hoffman , Ning Zhang , Kate Saenko , and Trevor Darrell . 2014. Deep Domain Confusion: Maximizing for Domain Invariance. CoRR abs/1412.3474 ( 2014 ). Eric Tzeng, Judy Hoffman, Ning Zhang, Kate Saenko, and Trevor Darrell. 2014. Deep Domain Confusion: Maximizing for Domain Invariance. CoRR abs/1412.3474 (2014). · arXiv
- Eric Tzeng , Judy Hoffman , Ning Zhang , Kate Saenko , and Trevor Darrell . 2014. Deep domain confusion: Maximizing for domain invariance. arXiv preprint arXiv:1412.3474 ( 2014 ). Eric Tzeng, Judy Hoffman, Ning Zhang, Kate Saenko, and Trevor Darrell. 2014. Deep domain confusion: Maximizing for domain invariance. arXiv preprint arXiv:1412.3474 (2014). · arXiv
- Pascal Vincent , Hugo Larochelle , Isabelle Lajoie , Yoshua Bengio , and Pierre-Antoine Manzagol . 2010 . Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion . Journal of machine learning research 11 , Dec (2010), 3371 -- 3408 . Pascal Vincent, Hugo Larochelle, Isabelle Lajoie, Yoshua Bengio, and Pierre-Antoine Manzagol. 2010. Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. Journal of machine learning research 11, Dec (2010), 3371--3408.
- Jindong Wang , Yiqiang Chen , Shuji Hao , Wenjie Feng , and Zhiqi Shen . 2017 . Balanced Distribution Adaptation for Transfer Learning. In 2017 IEEE International Conference on Data Mining, ICDM 2017 , New Orleans, LA, USA, November 18--21 , 2017. 1129--1134. Jindong Wang, Yiqiang Chen, Shuji Hao, Wenjie Feng, and Zhiqi Shen. 2017. Balanced Distribution Adaptation for Transfer Learning. In 2017 IEEE International Conference on Data Mining, ICDM 2017, New Orleans, LA, USA, November 18--21, 2017. 1129--1134.
- Jindong Wang , Yiqiang Chen , Lisha Hu , Xiaohui Peng , and Philip S Yu. 2017. Stratified Transfer Learning for Cross-domain Activity Recognition. arXiv preprint arXiv:1801.00820 ( 2017 ). Jindong Wang, Yiqiang Chen, Lisha Hu, Xiaohui Peng, and Philip S Yu. 2017. Stratified Transfer Learning for Cross-domain Activity Recognition. arXiv preprint arXiv:1801.00820 (2017). · arXiv
- Jindong Wang , Yiqiang Chen , Lisha Hu , Xiaohui Peng , and Philip S. Yu . 2018 . Stratified Transfer Learning for Cross-domain Activity Recognition. In 2018 IEEE International Conference on Pervasive Computing and Communications, PerCom 2018 , Athens, Greece, March 19--23 , 2018 . 1--10. Jindong Wang, Yiqiang Chen, Lisha Hu, Xiaohui Peng, and Philip S. Yu. 2018. Stratified Transfer Learning for Cross-domain Activity Recognition. In 2018 IEEE International Conference on Pervasive Computing and Communications, PerCom 2018, Athens, Greece, March 19--23, 2018. 1--10.
- Wikipedia. 2017. Manipuri Dance --- Wikipedia , The Free Encyclopedia. Wikipedia. 2017. Manipuri Dance --- Wikipedia, The Free Encyclopedia.
- 10.1109/CVPR.2010.5539857 · DOI · Paper page
- Chiyuan Zhang , Samy Bengio , Moritz Hardt , Benjamin Recht , and Oriol Vinyals . 2017 . Understanding deep learning requires rethinking generalization . In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24--26, 2017, Conference Track Proceedings. Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. 2017. Understanding deep learning requires rethinking generalization. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24--26, 2017, Conference Track Proceedings.
- Zhongtang Zhao , Yiqiang Chen , Junfa Liu , Zhiqi Shen , and Mingjie Liu . 2011 . Cross-People Mobile-Phone Based Activity Recognition. In IJCAI 2011, Proceedings of the 22nd International Joint Conference on Artificial Intelligence , Barcelona, Catalonia, Spain, July 16--22 , 2011. 2545--2550. Zhongtang Zhao, Yiqiang Chen, Junfa Liu, Zhiqi Shen, and Mingjie Liu. 2011. Cross-People Mobile-Phone Based Activity Recognition. In IJCAI 2011, Proceedings of the 22nd International Joint Conference on Artificial Intelligence, Barcelona, Catalonia, Spain, July 16--22, 2011. 2545--2550.
- Hong-Yu Zhou , Avital Oliver , Jianxin Wu , and Yefeng Zheng . 2018. When Semi-Supervised Learning Meets Transfer Learning: Training Strategies, Models and Datasets. CoRR abs/1812.05313 ( 2018 ). Hong-Yu Zhou, Avital Oliver, Jianxin Wu, and Yefeng Zheng. 2018. When Semi-Supervised Learning Meets Transfer Learning: Training Strategies, Models and Datasets. CoRR abs/1812.05313 (2018). · arXiv
Cited by
Abedin, A., Ehsanpour, M., Shi, Q., Rezatofighi, H., & Ranasinghe, D. C. (2021) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3448083 · Cited by 107
Suh, S., Rey, V. F., & Lukowicz, P. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2022.110143 · Cited by 107
Human Activity Recognition Data Analysis: History, Evolutions, and New Trends
Ariza-Colpas, P. P., Vicario, E., Oviedo-Carrascal, A. I., Butt Aziz, S., Piñeres-Melo, M. A., Quintero-Linero, A., & Patara, F. (2022) · Sensors · doi:10.3390/s22093401 · Cited by 52
Transfer Learning in Sensor-Based Human Activity Recognition: A Survey
Dhekane, S. G., & Ploetz, T. (2025) · ACM Computing Surveys · doi:10.1145/3717608 · Cited by 42
Adversarial Deep Feature Extraction Network for User Independent Human Activity Recognition
Suh, S., Rey, V. F., & Lukowicz, P. (2022) · 2022 IEEE International Conference on Pervasive Computing and Communications (PerCom) · doi:10.1109/percom53586.2022.9762387 · Cited by 25
Metadata
- Authors
- Faridee, A. Z. M., Khan, M. A. A. H., Pathak, N., & Roy, N.
- Venue
- Proceedings of the 16th EAI International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services
- Publisher
- ACM
- Published
- 2019-11-12
- Pages
- 162-171
- OpenAlex
- W3004729646
- Cited by
- 17
- References
- 17
- FWCI
- 0.84
- Type
- conference paper
- Language
- en
- Open access
- bronze
- Funders
- NSF (1750936)
- Alzheimer's Association (AARG-17-533039)
- ONR (N00014-18-1-2462)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: AugToAct: scaling complex human activity recognition with few labels. https://app.cochat.ai/papers/10.1145/3360774.3360831
- The paper
- Faridee, A. Z. M., Khan, M. A. A. H., Pathak, N., & Roy, N. (2019). AugToAct: scaling complex human activity recognition with few labels. Proceedings of the 16th EAI International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services. https://doi.org/10.1145/3360774.3360831
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.