Cited by 1,643Reinforcement Learning in RoboticsRobot Manipulation and LearningMotor Control and Adaptation

Learning dexterous in-hand manipulation

Open access
Andrychowicz, OpenAI: Marcin, Baker, Bowen, Chociej, Maciek, Józefowicz, Rafal, McGrew, Bob, Pachocki, Jakub, Petron, Arthur, Plappert, Matthias +8 more·2020·The International Journal of Robotics Research

Abstract

We use reinforcement learning (RL) to learn dexterous in-hand manipulation policies that can perform vision-based object reorientation on a physical Shadow Dexterous Hand. The training is performed in a simulated environment in which we randomize many of the physical properties of the system such as friction coefficients and an object’s appearance. Our policies transfer to the physical robot despite being trained entirely in simulation. Our method does not rely on any human demonstrations, but many behaviors found in human manipulation emerge naturally, including finger gaiting, multi-finger coordination, and the controlled use of gravity. Our results were obtained using the same distributed RL system that was used to train OpenAI Five. We also include a video of our results: https://youtu.be/jwSbzNHGflM .

Integrity signals

Registry

Crossref · 10.1177/0278364919887447

Cites

48 of 48 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Dynamics randomization can help acquire a robust policy, particularly where a controlling policy is needed.

    “In addition to adding randomization to the visual input, dynamics randomization could also help acquire a robust policy particularly where the controlling policy is needed.”
  • Learning from demonstrations avoids per-task reward engineering but makes data the bottleneck, since contact-rich demonstrations are difficult to collect at scale.

    “Learning from demonstrations is the prevailing route to such skills, avoiding the per-task reward engineering of reinforcement learning (Andrychowicz et al., 2020;Chen et al., 2022), but it shifts the bottleneck to data: contact-rich demonstrations are precisely what is hard to collect at scale.”
  • Domain randomization has a powerful effect, as shown by successful sim-to-real transfer experiments for dexterous in-hand manipulation with a physical five-fingered hand.

    “Their successful sim-to-real transfer experiments show the powerful effect of domain randomization.”

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Authors
Andrychowicz, O. M., Baker, B., Chociej, M., Józefowicz, R., McGrew, B., Pachocki, J., Petron, A., Plappert, M., Powell, G., Ray, A., Schneider, J., Sidor, S., Tobin, J., Welinder, P., Weng, L., & Zaremba, W.
Affiliations
  • OpenAI (United States) (US)
  • OpenAI, San Francisco, CA, USA
Corresponding author
Plappert, Matthias
Venue
The International Journal of Robotics Research
Published
2019-11-18
Volume
39
Issue
1
Pages
3-20
ISSN
  • 0278-3649 (ISSN-L)
  • 1741-3176
OpenAlex
W2990747716
Cited by
1,643
References
48
Citations by year
2026: 120 · 2025: 224 · 2024: 221 · 2023: 243 · 2022: 183 · 2021: 263 · 2020: 244 · 2019: 137 · 2018: 8
FWCI
93.29
Type
journal article
Language
en
Open access
Locations
The International Journal of Robotics Research (publishedVersion, cc-by-nc)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
  • Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Motor Control and Adaptation
Keywords
in-hand manipulation, dexterous manipulation, object reorientation, simulation-to-robot transfer, visual servoing, distributed reinforcement learning, human-robot interaction

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review