The DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index.
Interacting with Soli: Exploring Fine-Grained Dynamic Gesture Recognition in the Radio-Frequency Spectrum
Open accessAbstract
This paper proposes a novel machine learning architecture, specifically designed for radio-frequency based gesture recognition. We focus on high-frequency (60]GHz), short-range radar based sensing, in particular Google's Soli sensor. The signal has unique properties such as resolving motion at a very fine level and allowing for segmentation in range and velocity spaces rather than image space. This enables recognition of new types of inputs but poses significant difficulties for the design of input recognition algorithms. The proposed algorithm is capable of detecting a rich set of dynamic gestures and can resolve small motions of fingers in fine detail. Our technique is based on an end-to-end trained combination of deep convolutional and recurrent neural networks. The algorithm achieves high recognition rates (avg 87%) on a challenging set of 11 dynamic gestures and generalizes well across 10 users. The proposed model runs on commodity hardware at 140 Hz (CPU only).
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Hinton, G., Deng, L., Yu, D., Dahl, G., Mohamed, A.-R., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T., et al. (2012) · IEEE Signal Processing Magazine · doi:10.1109/msp.2012.2205597 · Cited by 10,260
Speech recognition with deep recurrent neural networks
Graves, A., Mohamed, A.-R., & Hinton, G. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6638947 · Cited by 8,893 · cited 13×
“(LSTM) architecture [16], which uses purpose-built memory cells to store information, is better at finding and exploiting long range dependencies in the data.”
BackgroundInfluentialTwo-Stream Convolutional Networks for Action Recognition in Videos
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1406.2199 · Cited by 6,756
Backpropagation through time: what it does and how to do it
Werbos, P. J. (1990) · Proceedings of the IEEE · doi:10.1109/5.58337 · Cited by 4,879 · cited 1×
“Gradients and loss values are then summed over time and finally the network’s parameters are optimized, minimizing the combined loss similar to [36].”
BackgroundDeep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition
Ordóñez, F., & Roggen, D. (2016) · Sensors · doi:10.3390/s16010115 · Cited by 2,761 · cited 3×
“Finally, significant improvements in human activity recognition based on accelerometers or gyroscopes have been demonstrated [25] by leveraging similar architectures.”
BackgroundHierarchical recurrent neural network for skeleton based action recognition
Du, Y., Wang, W., & Wang, L. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298714 · Cited by 1,895
Whole-home gesture recognition using wireless signals
Pu, Q., Gupta, S., Gollakota, S., & Patel, S. (2013) · Proceedings of the 19th annual international conference on Mobile computing & networking - MobiCom '13 · doi:10.1145/2500423.2500436 · Cited by 1,109
Soli: ubiquitous gesture sensing with millimeter wave radar
Lien, J., Gillian, N., Karagozler, M. E., Amihood, P., Schwesig, C., Olson, E., Raja, H., & Poupyrev, I. (2016) · ACM Transactions on Graphics · doi:10.1145/2897824.2925953 · Cited by 957 · cited 18×
“While our implementation is specific to the Soli sensor, generalizing the approach to other high-frequency RF signals is straightforward.”
BackgroundMethodInfluentialSelf-citationVision-based hand pose estimation: A review
Erol, A., Bebis, G., Nicolescu, M., Boyle, R. D., & Twombly, X. (2007) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2006.10.012 · Cited by 866
Skinput: appropriating the body as an input surface
Harrison, C., Tan, D., & Morris, D. (2010) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1753326.1753394 · Cited by 694
An HMM-based threshold model approach for gesture recognition
Lee, H.-K., & Kim, J. H. (1999) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.799904 · Cited by 671
Graves, A. (2012) · Studies in Computational Intelligence · doi:10.1007/978-3-642-24797-2_2 · Cited by 545
Capturing the human figure through a wall
Adib, F., Hsu, C.-Y., Mao, H., Katabi, D., & Durand, F. (2015) · ACM Transactions on Graphics · doi:10.1145/2816795.2818072 · Cited by 443
Accurate, Robust, and Flexible Real-time Hand Tracking
Sharp, T., Keskin, C., Robertson, D., Taylor, J., Shotton, J., Kim, D., Rhemann, C., Leichter, I., Vinnikov, A., Wei, Y., et al. (2015) · Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702179 · Cited by 431
Digits: freehand 3D interactions anywhere using a wrist-worn gloveless sensor
Kim, D., Hilliges, O., Izadi, S., Butler, A. D., Chen, J., Oikonomidis, I., & Olivier, P. (2012) · Proceedings of the 25th annual ACM symposium on User interface software and technology · doi:10.1145/2380116.2380139 · Cited by 428
Self-citationGestureWrist and GesturePad: unobtrusive wearable interaction devices
Rekimoto, J. (2002) · Proceedings Fifth International Symposium on Wearable Computers · doi:10.1109/iswc.2001.962092 · Cited by 410
Darrell, T., & Pentland, A. (2002) · Proceedings of IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.1993.341109 · Cited by 353
SoundWave: using the doppler effect to sense gestures
Gupta, S., Morris, D., Patel, S., & Tan, D. (2012) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2207676.2208331 · Cited by 353
Enabling always-available input with muscle-computer interfaces
Saponas, T. S., Tan, D. S., Morris, D., Balakrishnan, R., Turner, J., & Landay, J. A. (2009) · Proceedings of the 22nd annual ACM symposium on User interface software and technology · doi:10.1145/1622176.1622208 · Cited by 328
Touché: enhancing touch interaction on humans, screens, liquids, and everyday objects
Sato, M., Poupyrev, I., & Harrison, C. (2012) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2207676.2207743 · Cited by 322
Self-citationHand Pose Estimation and Hand Shape Classification Using Multi-layered Randomized Decision Forests
Keskin, C., Kıraç, F., Kara, Y. E., & Akarun, L. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33783-3_61 · Cited by 284
SideSight: multi-"touch" interaction around small devices
Butler, A., Izadi, S., & Hodges, S. (2008) · Proceedings of the 21st annual ACM symposium on User interface software and technology · doi:10.1145/1449715.1449746 · Cited by 269
XWand: UI for intelligent spaces
Wilson, A., & Shafer, S. (2003) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/642611.642706 · Cited by 230
Multi-scale Deep Learning for Gesture Detection and Localization
Neverova, N., Wolf, C., Taylor, G. W., & Nebout, F. (2015) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-16178-5_33 · Cited by 229
WiGiG: Multi-gigabit wireless communications in the 60 GHz band
Hansen, C. (2011) · IEEE Wireless Communications · doi:10.1109/mwc.2011.6108325 · Cited by 217
Gesture modeling and recognition using finite state machines
Hong, P., Turk, M., & Huang, T. S. (2002) · Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580) · doi:10.1109/afgr.2000.840667 · Cited by 214
The influence of muscle groups on performance of multiple degree-of-freedom input
Zhai, S., Milgram, P., & Buxton, W. (1996) · Proceedings of the SIGCHI conference on Human factors in computing systems common ground - CHI '96 · doi:10.1145/238386.238534 · Cited by 192
In-air gestures around unmodified mobile devices
Song, J., Sörös, G., Pece, F., Fanello, S., Izadi, S., Keskin, C., & Hilliges, O. (2014) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/2642918.2647373 · Cited by 185
Self-citationuTrack: 3D input using two magnetic sensors
Chen, K.-Y., Lyons, K., White, S., & Patel, S. (2013) · Proceedings of the 26th annual ACM symposium on User interface software and technology · doi:10.1145/2501988.2502035 · Cited by 170
RadarCat: Radar Categorization for Input & Interaction
Yeo, H.-S., Flamich, G., Schrempf, P., Harris‐Birtill, D., & Quigley, A. J. (2016) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/2984511.2984515 · Cited by 161
Humantenna: using the body as an antenna for real-time whole-body interaction
Cohn, G. A., Morris, D., Patel, S., & Tan, D. S. (2012) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2207676.2208330 · Cited by 152
MagGetz: customizable passive tangible controllers on and around conventional mobile devices
Hwang, S., Ahn, M., & Wohn, K. (2013) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/2501988.2501991 · Cited by 83
Dynamic hand gesture recognition: An exemplar-based approach from motion divergence fields
Shen, X., Hua, G., Williams, L., & Wu, Y. N. (2011) · Image and Vision Computing · doi:10.1016/j.imavis.2011.11.003 · Cited by 82
Zhao, C., Chen, K.-Y., Aumi, M. T. I., Patel, S., & Reynolds, M. S. (2014) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/2642918.2647380 · Cited by 62
Type-hover-swipe in 96 bytes: a motion sensing mechanical keyboard
Taylor, S., Keskin, C., Hilliges, O., Izadi, S., & Helmes, J. (2014) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2556288.2557030 · Cited by 51
Self-citation
Cited by
Digital twin: values, challenges and enablers from a modeling perspective
Rasheed, A., San, O., & Kvamsdal, T. (2020) · IEEE Access · doi:10.1109/access.2020.2970143 · Cited by 1,952
Deep Learning in Mobile and Wireless Networking: A Survey
Zhang, C., Patras, P., & Haddadi, H. (2019) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2904897 · Cited by 1,768
Symbiotic human-robot collaborative assembly
Wang, L., Gao, R., Váncza, J., Krüger, J., Wang, X. V., Makris, S., & Chryssolouris, G. (2019) · CIRP Annals · doi:10.1016/j.cirp.2019.05.002 · Cited by 565
Latern: Dynamic Continuous Hand Gesture Recognition Using FMCW Radar Sensor
Zhang, Z., Tian, Z., & Zhou, M. (2018) · IEEE Sensors Journal · doi:10.1109/jsen.2018.2808688 · Cited by 333
A Survey of Deep Learning-Based Human Activity Recognition in Radar
Li, X., He, Y., & Jing, X. (2019) · Remote Sensing · doi:10.3390/rs11091068 · Cited by 290
Metadata
- Authors
- Wang, S., Song, J., Lien, J., Poupyrev, I., & Hilliges, O.
- Venue
- Proceedings of the 29th Annual Symposium on User Interface Software and Technology
- Publisher
- ACM
- Published
- 2016-10-16
- Pages
- 851-860
- OpenAlex
- W2536305597
- Cited by
- 433
- References
- 36
- FWCI
- 20.06
- Type
- conference paper
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.