Cited by 264Robot Manipulation and LearningTeleoperation and Haptic SystemsTactile and Sensory Interactions

Deep learning for tactile understanding from visual and haptic data

Open access
Gao, Yang, Hendricks, Lisa Anne, Kuchenbecker, Katherine J., Darrell, Trevor·2016·2016 IEEE International Conference on Robotics and Automation (ICRA)

Abstract

Robots which interact with the physical world will benefit from a fine-grained tactile understanding of objects and surfaces. Additionally, for certain tasks, robots may need to know the haptic properties of an object before touching it. To enable better tactile understanding for robots, we propose a method of classifying surfaces with haptic adjectives (e.g., compressible or smooth) from both visual and physical interaction data. Humans typically combine visual predictions and feedback from physical interactions to accurately predict haptic properties and interact with the world. Inspired by this cognitive pattern, we propose and explore a purely visual haptic prediction model. Purely visual models enable a robot to “feel” without physical interaction. Furthermore, we demonstrate that using both visual and physical interaction signals together yields more accurate haptic classification. Our models take advantage of recent advances in deep neural networks by employing a unified approach to learning features for physical interaction and visual observations. Even though we employ little domain specific knowledge, our model still achieves better results than methods based on hand-designed features.

Integrity signals

Registry

Crossref · 10.1109/icra.2016.7487176

Cites

36 of 36 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Authors
Gao, Y., Hendricks, L. A., Kuchenbecker, K. J., & Darrell, T.
Affiliations
  • University of California, Berkeley (US)
  • University of Pennsylvania (US)
Venue
2016 IEEE International Conference on Robotics and Automation (ICRA)
Publisher
IEEE
Published
2016-05
Pages
536-543
ISSN
1050-4729 (ISSN-L)
OpenAlex
W2963654998
Cited by
264
References
36
Citations by year
2026: 6 · 2025: 12 · 2024: 18 · 2023: 27 · 2022: 34 · 2021: 32 · 2020: 39 · 2019: 38 · 2018: 28 · 2017: 24 · 2016: 6
FWCI
50.78
Type
conference paper
Language
en
Open access
Locations
  • Proceedings - IEEE International Conference on Robotics and Automation/Proceedings (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • National Science Foundation (1426787, 1427425)
  • Public Health Agency of Canada
  • Defense Advanced Research Projects Agency
  • National Defense Science and Engineering Graduate
Topics
  • Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
  • Physical Sciences › Engineering › Mechanical Engineering › Teleoperation and Haptic Systems
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Tactile and Sensory Interactions
Keywords
classification of surfaces, multimodal fusion

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review