Device-free gesture tracking using acoustic signals
Open accessThe DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
Abstract
Device-free gesture tracking is an enabling HCI mechanism for small wearable devices because fingers are too big to control the GUI elements on such small screens, and it is also an important HCI mechanism for medium-to-large size mobile devices because it allows users to provide input without blocking screen view. In this paper, we propose LLAP, a device-free gesture tracking scheme that can be deployed on existing mobile devices as software, without any hardware modification. We use speakers and microphones that already exist on most mobile devices to perform device-free tracking of a hand/finger. The key idea is to use acoustic phase to get fine-grained movement direction and movement distance measurements. LLAP first extracts the sound signal reflected by the moving hand/finger after removing the background sound signals that are relatively consistent over time. LLAP then measures the phase changes of the sound signals caused by hand/finger movements and then converts the phase changes into the distance of the movement. We implemented and evaluated LLAP using commercial-off-the-shelf mobile phones. For 1-D hand movement and 2-D drawing in the air, LLAP has a tracking accuracy of 3.5 mm and 4.6 mm, respectively. Using gesture traces tracked by LLAP, we can recognize the characters and short words drawn in the air with an accuracy of 92.3% and 91.2%, respectively.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Huang, N. E., Shen, Z., Long, S. R., Wu, M. C., Shih, H. H., Zheng, Q., Yen, N.-C., Tung, C. C., & Liu, H. H. (1998) · Proceedings of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rspa.1998.0193 · Cited by 23,564
Fundamentals of Wireless Communication
Tse, D., & Viswanath, P. (2005) · doi:10.1017/cbo9780511807213 · Cited by 13,162
The Cricket location-support system
Priyantha, N. B., Chakraborty, A., & Balakrishnan, H. (2000) · Proceedings of the 6th annual international conference on Mobile computing and networking · doi:10.1145/345910.345917 · Cited by 3,716
(2009) · Encyclopedia of Neuroscience · doi:10.1007/978-3-540-29678-2_6014 · Cited by 2,850
Understanding and Modeling of WiFi Signal Based Human Activity Recognition
Wang, W., Liu, A. X., Shahzad, M., Ling, K., & Lu, S. (2015) · Proceedings of the 21st Annual International Conference on Mobile Computing and Networking · doi:10.1145/2789168.2790093 · Cited by 1,110
Self-citationWhole-home gesture recognition using wireless signals
Pu, Q., Gupta, S., Gollakota, S., & Patel, S. (2013) · Proceedings of the 19th annual international conference on Mobile computing & networking - MobiCom '13 · doi:10.1145/2500423.2500436 · Cited by 1,109
E-eyes: device-free location-oriented activity identification using fine-grained WiFi signatures
Wang, Y., Liu, J., Chen, Y., Gruteser, M., Yang, J., & Liu, H. (2014) · Proceedings of the 20th annual international conference on Mobile computing and networking · doi:10.1145/2639108.2639143 · Cited by 817
WiGest: A ubiquitous WiFi-based gesture recognition system
Abdelnasser, H., Youssef, M., & Harras, K. A. (2015) · 2015 IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom.2015.7218525 · Cited by 552
Keystroke Recognition Using WiFi Signals
Ali, K., Liu, A. X., Wang, W., & Shahzad, M. (2015) · Proceedings of the 21st Annual International Conference on Mobile Computing and Networking · doi:10.1145/2789168.2790109 · Cited by 540
Self-citationBeepBeep: a high accuracy acoustic ranging system using COTS mobile devices
Peng, C., Shen, G., Zhang, Y., Li, Y., & Tan, K. (2007) · Proceedings of the 5th international conference on Embedded networked sensor systems · doi:10.1145/1322263.1322265 · Cited by 473
FingerIO: Using Active Sonar for Fine-Grained Finger Tracking
Nandakumar, R., Iyer, V., Tan, D., & Gollakota, S. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858580 · Cited by 408
RF-IDraw: virtual touch screen in the air using RF signals
Wang, J., Vasisht, D., & Katabi, D. (2014) · Proceedings of the 2014 ACM conference on SIGCOMM · doi:10.1145/2619239.2626330 · Cited by 359
SoundWave: using the doppler effect to sense gestures
Gupta, S., Morris, D., Patel, S., & Tan, D. (2012) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2207676.2208331 · Cited by 353
Contactless Sleep Apnea Detection on Smartphones
Nandakumar, R., Gollakota, S., & Watson, N. (2015) · Proceedings of the 13th Annual International Conference on Mobile Systems, Applications, and Services · doi:10.1145/2742647.2742674 · Cited by 343
Multi-person localization via RF body reflections
Adib, F., Kabelac, Z., & Katabi, D. (2015) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 314
WiDraw: Enabling Hands-free Drawing in the Air on Commodity WiFi Devices
Sun, L., Sen, S., Koutsonikolas, D., & Kim, K.-H. (2015) · Proceedings of the 21st Annual International Conference on Mobile Computing and Networking · doi:10.1145/2789168.2790129 · Cited by 299
mTrack: High-Precision Passive Tracking Using Millimeter Wave Radios
Wei, T., & Zhang, X. (2015) · Proceedings of the 21st Annual International Conference on Mobile Computing and Networking · doi:10.1145/2789168.2790113 · Cited by 265
Bringing gesture recognition to all devices
Kellogg, B., Talla, V., & Gollakota, S. (2014) · doi:10.5555/2616448.2616477 · Cited by 240
WiGest demo: A ubiquitous WiFi-based gesture recognition system
Abdelnasser, H., Harras, K. A., & Youssef, M. (2015) · 2015 IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS) · doi:10.1109/infcomw.2015.7179321 · Cited by 212
Detecting driver phone use leveraging car speakers
Yang, J., Sidhom, S., Chandrasekaran, G., Vu, T., Liu, H., Cecan, N., Chen, Y., Gruteser, M., & Martin, R. P. (2011) · Proceedings of the 17th annual international conference on Mobile computing and networking · doi:10.1145/2030613.2030625 · Cited by 208
In-air gestures around unmodified mobile devices
Song, J., Sörös, G., Pece, F., Fanello, S., Izadi, S., Keskin, C., & Hilliges, O. (2014) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/2642918.2647373 · Cited by 185
Snooping Keystrokes with mm-level Audio Ranging on a Single Phone
Liu, J., Wang, Y., Kar, G., Chen, Y., Yang, J., & Gruteser, M. (2015) · Proceedings of the 21st Annual International Conference on Mobile Computing and Networking · doi:10.1145/2789168.2790122 · Cited by 173
Context-free Attacks Using Keyboard Acoustic Emanations
Zhu, T., Ma, Q., Zhang, S., & Liu, Y. (2014) · Proceedings of the 2014 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/2660267.2660296 · Cited by 166
Extended high-frequency (9–20 kHz) audiometry reference thresholds in 645 healthy subjects
Valiente, A. R., Trinidad, A., García‐Berrocal, J. R., Górriz, C., & Ramírez‐Camacho, R. (2014) · International Journal of Audiology · doi:10.3109/14992027.2014.893375 · Cited by 162
Leveraging directional antenna capabilities for fine-grained gesture recognition
Melgarejo, P., Zhang, X., Ramanathan, P., & Chu, D. (2014) · Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing · doi:10.1145/2632048.2632095 · Cited by 133
Wang, J., Zhao, K., Zhang, X., & Peng, C. (2014) · Proceedings of the 12th annual international conference on Mobile systems, applications, and services · doi:10.1145/2594368.2594384 · Cited by 125
SwordFight: enabling a new class of phone-to-phone action games on commodity phones
Zhang, Z., Chu, D., Chen, X. C., & Moscibroda, T. (2012) · International Conference on Mobile Systems, Applications, and Services · doi:10.1145/2307636.2307638 · Cited by 123
EchoTag: Accurate Infrastructure-Free Indoor Location Tagging with Smartphones
Tung, Y.-C., & Shin, K. G. (2015) · ACM Annual International Conference on Mobile Computing and Networking (MobiCom) · doi:10.1145/2789168.2790102 · Cited by 119
Extending Mobile Interaction Through Near-Field Visible Light Sensing
Zhang, C., Tabor, J., Zhang, J., & Zhang, X. (2015) · ACM Annual International Conference on Mobile Computing and Networking (MobiCom) · doi:10.1145/2789168.2790115 · Cited by 114
Shake and walk: Acoustic direction finding and fine-grained indoor localization using smartphones
Huang, W., Xiong, Y., Li, X., Lin, H., Mao, X., Yang, P., & Liu, Y. (2014) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom.2014.6847959 · Cited by 98
AirLink: sharing files between multiple devices using in-air gestures
Chen, K.-Y., Ashbrook, D., Goel, M., Lee, S.-H., & Patel, S. (2014) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/2632048.2632090 · Cited by 97
Aumi, M. T. I., Gupta, S., Goel, M., Larson, E. C., & Patel, S. (2013) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/2493432.2493515 · Cited by 95
Sun, Z., Purohit, A., Bose, R., & Zhang, P. (2013) · International Conference on Mobile Systems, Applications, and Services · doi:10.1145/2462456.2464437 · Cited by 80
Zhang, M., Yang, P., Tian, C., Shi, L., Tang, S., & Xiao, F. (2015) · doi:10.1145/2797044.2797045 · Cited by 32
Xiao, R., Harrison, C., Willis, K. D. D., Poupyrev, I., & Hudson, S. E. (2013) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/2501988.2502022 · Cited by 30
Cited by
Digital twin: values, challenges and enablers from a modeling perspective
Rasheed, A., San, O., & Kvamsdal, T. (2020) · IEEE Access · doi:10.1109/access.2020.2970143 · Cited by 1,952
Towards Environment Independent Device Free Human Activity Recognition
Jiang, W., Miao, C., Ma, F., Yao, S., Wang, Y., Yuan, Y., Xue, H., Song, C., Ma, X., Koutsonikolas, D., et al. (2018) · Proceedings of the 24th Annual International Conference on Mobile Computing and Networking · doi:10.1145/3241539.3241548 · Cited by 573
IndoTrack: Device-Free Indoor Human Tracking with Commodity Wi-Fi
Li, X., Zhang, D., Lv, Q., Xiong, J., Li, S., Zhang, Y., & Mei, H. (2017) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3130940 · Cited by 402
SignFi: Sign Language Recognition Using WiFi
Ma, Y., Zhou, G., Wang, S., Zhao, H., & Jung, W. (2018) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3191755 · Cited by 380
mD-Track: Leveraging Multi-Dimensionality for Passive Indoor Wi-Fi Tracking
Xie, Y., Xiong, J., Li, M., & Jamieson, K. (2019) · The 25th Annual International Conference on Mobile Computing and Networking · doi:10.1145/3300061.3300133 · Cited by 320
Metadata
- Authors
- Wang, W., Liu, A. X., & Sun, K.
- Venue
- Proceedings of the 22nd Annual International Conference on Mobile Computing and Networking
- Publisher
- ACM
- Published
- 2016-10-03
- Pages
- 82-94
- ISSN
- 1543-5679 (ISSN-L)
- OpenAlex
- W2525860450
- Cited by
- 378
- References
- 35
- FWCI
- 30.78
- Type
- conference paper
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.