CoChat

IMU2Doppler: Cross-Modal Domain Adaptation for Doppler-based Activity Recognition Using IMU Data

Bhalla, Sejal, Goel, Mayank, Khurana, Rushil · 2021 · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
Cited by 41

The DOI matches its Crossref record; 7 of the 7 references it cites were found in a registry or an index.

Abstract

The proliferation of sensors powered by state-of-the-art machine learning techniques can now infer context, recognize activities and enable interactions. A key component required to build these automated sensing systems is labeled training data. However, the cost of collecting and labeling new data impedes our ability to deploy new sensors to recognize human activities. We tackle this challenge using domain adaptation i.e., using existing labeled data in a different domain to aid the training of a machine learning model for a new sensor. In this paper, we use off-the-shelf smartwatch IMU datasets to train an activity recognition system for mmWave radar sensor with minimally labeled data. We demonstrate that despite the lack of extensive datasets for mmWave radar, we are able to use our domain adaptation approach to build an activity recognition system that classifies between 10 activities with an accuracy of 70% with only 15 seconds of labeled doppler data. We also present results for a range of available labeled data (10 - 30 seconds) and show that our approach outperforms the baseline in every single scenario. We take our approach a step further and show that multiple IMU datasets can be combined together to act as a single source for our domain adaptation approach. Lastly, we discuss the limitations of our work and how it can impact future research directions.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3494994

Cites

7 of 7 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 56

  1. 10.1145/3411764.3445138 · DOI
  2. 10.1109/TPAMI.2017.2753232 · DOI · Paper page
  3. Yusuf Aytar , Carl Vondrick , and Antonio Torralba . 2016 . Soundnet: Learning sound representations from unlabeled video. Advances in neural information processing systems 29 (2016), 892--900. Yusuf Aytar, Carl Vondrick, and Antonio Torralba. 2016. Soundnet: Learning sound representations from unlabeled video. Advances in neural information processing systems 29 (2016), 892--900.
  4. 10.1145/3313831.3376869 · DOI · Paper page
  5. 10.5555/1610075.1610094 · doi:10.5555/1610075.1610094
  6. 10.1109/LGRS.2015.2452946 · doi:10.1109/lgrs.2015.2452946
  7. 10.1145/3432224 · DOI
  8. 10.1109/LGRS.2019.2942097 · DOI
  9. 10.1109/RADAR.2018.8557324 · DOI
  10. 10.1109/TAES.2006.1603402 · DOI · Paper page
  11. Wenyuan Dai , Yuqiang Chen , Gui-Rong Xue , Qiang Yang , and Yong Yu . 2008 . Translated Learning: Transfer Learning across Different Feature Spaces . In Proceedings of the 21st International Conference on Neural Information Processing Systems (NIPS'08) . Curran Associates Inc., Red Hook, NY, USA, 353--360. event-place: Vancouver, British Columbia, Canada. Wenyuan Dai, Yuqiang Chen, Gui-Rong Xue, Qiang Yang, and Yong Yu. 2008. Translated Learning: Transfer Learning across Different Feature Spaces. In Proceedings of the 21st International Conference on Neural Information Processing Systems (NIPS'08). Curran Associates Inc., Red Hook, NY, USA, 353--360. event-place: Vancouver, British Columbia, Canada.
  12. Marcos Lopez De Prado . 2018. Advances in financial machine learning . John Wiley & Sons . Marcos Lopez De Prado. 2018. Advances in financial machine learning. John Wiley & Sons.
  13. Gulustan Dogan , Iremnaz Cay , Sinem Sena Ertas , Şeref Recep Keskin , Nouran Alotaibi , and Elif Sahin . 2020 . Where are you? Human activity recognition with smartphone sensor data . In Adjunct Proceedings of the 2020 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2020 ACM International Symposium on Wearable Computers. 301--304 . Gulustan Dogan, Iremnaz Cay, Sinem Sena Ertas, Şeref Recep Keskin, Nouran Alotaibi, and Elif Sahin. 2020. Where are you? Human activity recognition with smartphone sensor data. In Adjunct Proceedings of the 2020 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2020 ACM International Symposium on Wearable Computers. 301--304.
  14. Jeff Donahue , Yangqing Jia , Oriol Vinyals , Judy Hoffman , Ning Zhang , Eric Tzeng , and Trevor Darrell . 2014 . DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition . In Proceedings of the 31st International Conference on International Conference on Machine Learning - Volume 32 (ICML '14). JMLR.org, I-647-I-655. event-place: Beijing, China. Jeff Donahue, Yangqing Jia, Oriol Vinyals, Judy Hoffman, Ning Zhang, Eric Tzeng, and Trevor Darrell. 2014. DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition. In Proceedings of the 31st International Conference on International Conference on Machine Learning - Volume 32 (ICML '14). JMLR.org, I-647-I-655. event-place: Beijing, China.
  15. Lixin Duan , Dong Xu , and Ivor W. Tsang . 2012. Learning with Augmented Features for Heterogeneous Domain Adaptation . In Proceedings of the 29th International Coference on International Conference on Machine Learning (ICML '12) . Omnipress, Madison, WI, USA, 667--674. event-place: Edinburgh, Scotland. Lixin Duan, Dong Xu, and Ivor W. Tsang. 2012. Learning with Augmented Features for Heterogeneous Domain Adaptation. In Proceedings of the 29th International Coference on International Conference on Machine Learning (ICML '12). Omnipress, Madison, WI, USA, 667--674. event-place: Edinburgh, Scotland.
  16. 10.1109/TAES.2015.130595 · DOI
  17. 10.1016/j.eswa.2019.06.070 · DOI
  18. Maayan Harel and Shie Mannor . 2011 . Learning from Multiple Outlooks . In Proceedings of the 28th International Conference on International Conference on Machine Learning (ICML '11) . Omnipress, Madison, WI, USA, 401--408. event-place: Bellevue, Washington, USA. Maayan Harel and Shie Mannor. 2011. Learning from Multiple Outlooks. In Proceedings of the 28th International Conference on International Conference on Machine Learning (ICML '11). Omnipress, Madison, WI, USA, 401--408. event-place: Bellevue, Washington, USA.
  19. 10.1109/ACCESS.2019.2938725 · DOI
  20. 10.23919/EuRAD.2018.8546615 · DOI
  21. 10.3390/computers2020088 · DOI · Paper page
  22. 10.1145/3287063 · DOI
  23. 10.1145/3313831.3376822 · doi:10.1145/3313831.3376822
  24. 10.1145/3313831.3376761 · DOI
  25. 10.1109/LGRS.2015.2491329 · DOI · Paper page
  26. 10.1109/JTEHM.2018.2791609 · DOI
  27. 10.1145/3314404 · DOI
  28. 10.1109/JSEN.2018.2872849 · DOI · Paper page
  29. 10.1016/j.neucom.2021.04.124 · DOI
  30. 10.1109/TKDE.2009.191 · DOI · Paper page
  31. 10.1109/TGRS.2020.3019915 · DOI
  32. 10.1145/3369818 · DOI · Paper page
  33. 10.1145/3351242 · DOI · Paper page
  34. 10.1145/3161162 · doi:10.1145/3161162
  35. 10.3390/electronics9040588 · DOI
  36. 10.1145/3090082 · DOI · Paper page
  37. 10.1145/3349624.3356768 · DOI · Paper page
  38. 10.1145/3412060.3418430 · DOI
  39. 10.5555/3016100.3016186 · doi:10.5555/3016100.3016186
  40. 10.5555/3298483.3298614 · doi:10.5555/3298483.3298614
  41. 10.1109/RADAR42522.2020.9114643 · DOI
  42. 10.1109/CVPR.2016.549 · DOI · Paper page
  43. 10.1109/LSENS.2020.2991367 · doi:10.1109/lsens.2020.2991367
  44. 10.1145/3267242.3267276 · DOI
  45. Chang Wang and Sridhar Mahadevan . 2011 . Heterogeneous Domain Adaptation Using Manifold Alignment . In Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence - Volume Volume Two (IJCAI'11). AAAI Press, 1541-- 1546 . event-place: Barcelona, Catalonia, Spain. Chang Wang and Sridhar Mahadevan. 2011. Heterogeneous Domain Adaptation Using Manifold Alignment. In Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence - Volume Volume Two (IJCAI'11). AAAI Press, 1541--1546. event-place: Barcelona, Catalonia, Spain.
  46. 10.1109/TBME.2014.2319333 · doi:10.1109/tbme.2014.2319333
  47. 10.1109/ACCESS.2019.2940729 · DOI · Paper page
  48. 10.5555/1735835.1735838 · doi:10.5555/1735835.1735838
  49. 10.1109/TGRS.2009.2012849 · DOI · Paper page
  50. 10.1117/12.2228397 · doi:10.1117/12.2228397
  51. Cheng Zhang , Qiuyue Xue , Anandghan Waghmare , Sumeet Jain , Yiming Pu , Sinan Hersek , Kent Lyons , Kenneth A Cunefare , Omer T Inan , and Gregory D Abowd . 2017 . Soundtrak: Continuous 3d tracking of a finger using active acoustics . Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1 , 2 (2017), 1 -- 25 . Cheng Zhang, Qiuyue Xue, Anandghan Waghmare, Sumeet Jain, Yiming Pu, Sinan Hersek, Kent Lyons, Kenneth A Cunefare, Omer T Inan, and Gregory D Abowd. 2017. Soundtrak: Continuous 3d tracking of a finger using active acoustics. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1, 2 (2017), 1--25. · DOI
  52. 10.1145/2971763.2971775 · DOI
  53. 10.1109/LSENS.2018.2889060 · DOI
  54. Sicheng Zhao , Bo Li , Pengfei Xu , and Kurt Keutzer . 2020. Multi-source domain adaptation in the deep learning era: A systematic survey. arXiv preprint arXiv:2002.12169 ( 2020 ). Sicheng Zhao, Bo Li, Pengfei Xu, and Kurt Keutzer. 2020. Multi-source domain adaptation in the deep learning era: A systematic survey. arXiv preprint arXiv:2002.12169 (2020). · arXiv
  55. 10.1109/SECON48991.2020.9158431 · DOI
  56. 10.5555/2900423.2900630 · doi:10.5555/2900423.2900630

Cited by

See all 41

Metadata

Authors
Bhalla, S., Goel, M., & Khurana, R.
Venue
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
Published
2021-12-27
Volume
5
Issue
4
Pages
1-20
ISSN
2474-9567 (ISSN-L)
OpenAlex
W4200329580
Cited by
41
References
7
FWCI
1.94
Type
journal article
Language
en
Open access
closed

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.