CoChat

A Deformable Interface for Human Touch Recognition Using Stretchable Carbon Nanotube Dielectric Elastomer Sensors and Deep Neural Networks

Open access
Larson, Chris, Spjut, Josef, Knepper, Ross, Shepherd, Robert · 2019 · Soft Robotics
Cited by 51

The DOI matches its Crossref record; 7 of the 7 references it cites were found in a registry or an index.

Abstract

This article presents a machine learning approach to map outputs from an embedded array of sensors distributed throughout a deformable body to continuous and discrete virtual states, and its application to interpret human touch in soft interfaces. We integrate stretchable capacitors into a rubber membrane, and use a passive addressing scheme to probe sensor arrays in real time. To process the signals from this array, we feed capacitor measurements into convolutional neural networks that classify and localize touch events on the interface. We implement this concept with a device called OrbTouch. To modularize the system, we use a supervised learning approach wherein a user defines a set of touch inputs and trains the interface by giving it examples; we demonstrate this by using OrbTouch to play the popular game Tetris. Our regression model localizes touches with mean test error of 0.09 mm, whereas our classifier recognizes five gestures with a mean test error of 1.2%. In a separate demonstration, we show that OrbTouch can discriminate between 10 different users with a mean test error of 2.4%. At test time, we feed the outputs of these models into a debouncing algorithm to provide a nearly error-free experience.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1089/soro.2018.0086

Cites

7 of 7 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 34

  1. 10.1038/nature14543 · DOI · Paper page
  2. Follmer S, , Leithinger D, , Olwal A, et al. Jamming user interfaces: Programmable particle stiffness and sensing for malleable and shape-changing devices. In Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology. Cambridge, MA: ACM, 2012, pp. 519–528.
  3. 10.1073/pnas.1003250107 · DOI · Paper page
  4. 10.1109/TVCG.2016.2525788 · DOI
  5. 10.1109/TOH.2015.2423492 · DOI · Paper page
  6. Lee S-S, , Kim S, , Jin B, et al. How users manipulate deformable displays as input devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Cambridge, MA: ACM, 2010, pp. 1647–1656.
  7. Rasmussen MK, , Pedersen EW, , Petersen MG, et al. Shape-changing interfaces: A review of the design space and open research questions. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Atlanta, GA: ACM, 2012, pp. 735–744.
  8. Ogata M, , Sugiura Y, , Makino Y, et al. Senskin: Adapting skin as a soft interface. In Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology. St. Andrews, Scotland: ACM, 2013, pp. 539–544.
  9. 10.1145/2556288.2557239 · DOI
  10. Pai. DK, VanDerLoo EW, Sadhukhan S, et al. The tango: A tangible tangoreceptive whole-hand human interface. In Eurohaptics Conference, 2005 and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 2005. World Haptics 2005. First Joint, IEEE, Pisa, Italy, 2005, pp. 141–147.
  11. Han S, , Park J. Grip-ball: A spherical multi-touch interface for interacting with virtual worlds. In Consumer Electronics (ICCE), 2013 IEEE International Conference. Las Vegas, NV: IEEE, 2013, pp. 600–601.
  12. 10.1145/1753846.1753864 · doi:10.1145/1753846.1753864
  13. 10.1109/VR.2013.6549433 · DOI
  14. 10.1145/1518701.1518749 · DOI · Paper page
  15. Steimle J, , Jordt A, , Maes P. Flexpad: Highly flexible bending interactions for projected handheld displays. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Paris, France: ACM, 2013, pp. 237–246.
  16. 10.1126/science.1182383 · DOI · Paper page
  17. 10.1038/nn.2973 · DOI · Paper page
  18. 10.1088/0960-1317/20/12/125029 · DOI · Paper page
  19. 10.1126/science.1121401 · DOI · Paper page
  20. 10.1038/nnano.2011.36 · DOI · Paper page
  21. 10.1038/nnano.2011.184 · DOI · Paper page
  22. Shepherd R Peele B Murray BM et al. Stretchable transducers for kinesthetic interactions in virtual reality. In ACM SIGGRAPH 2017 Emerging Technologies. ACM 2017 p. 21. · DOI
  23. 10.1109/LRA.2018.2799743 · DOI · Paper page
  24. 10.1021/acsnano.5b01613 · DOI · Paper page
  25. He. K, Zhang X, Ren S, et al. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, 2016, pp. 770–778.
  26. Mnih V Kavukcuoglu K Silver D et al. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602 2013. · arXiv
  27. Mikolov T Chen K Corrado G et al. Efficient estimation of word representations in vector space. arxiv: 1301.3781 2013. · arXiv
  28. Bengio. S, Vinyals O, Jaitly N, et al. Scheduled sampling for sequence prediction with recurrent neural networks. In Proceedings of the 28th International Conference on Neural Information Processing Systems. Montreal, Canada, 2015, pp. 1171–1179.
  29. 10.1098/rsta.1952.0013 · doi:10.1098/rsta.1952.0013
  30. Kingma D Ba J. Adam: A method for stochastic optimization. arXiv preprint arXiv: 1412.6980 2014. · arXiv
  31. Abadi M Agarwal A Barham P et al. Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv preprint arXiv: 1603.04467 2016. · arXiv
  32. Nesterov Y. A method of solving a convex programming problem with convergence rate o (1/k2). Soviet Math Doklady, 1983; 27:372–376.
  33. 10.1109/TRO.2011.2171093 · DOI · Paper page
  34. 10.1002/smll.200600061 · DOI · Paper page

Cited by

See all 51

Metadata

Authors
Larson, C., Spjut, J., Knepper, R., & Shepherd, R.
Venue
Soft Robotics
Published
2019-10-01
Volume
6
Issue
5
Pages
611-620
ISSN
  • 2169-5172 (ISSN-L)
  • 2169-5180
OpenAlex
W2795512733
Cited by
51
References
7
FWCI
1.99
Type
journal article
Language
en
Open access
green
Licence
journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.