Student or educator? Sign up with your school email and get 6 months of Pro free.

The DOI matches its Crossref record; 50 of the 50 references it cites were found in a registry or an index; 2 of the 5 sources its claims cite were checked for support.

Sim-to-Real Transfer of Robotic Control with Dynamics Randomization

Open access
Peng, Xue Bin, Andrychowicz, Marcin, Zaremba, Wojciech, Abbeel, Pieter·2018·2018 IEEE International Conference on Robotics and Automation (ICRA)

Abstract

Simulations are attractive environments for training agents as they provide an abundant source of data and alleviate certain safety concerns during the training process. But the behaviours developed by agents in simulation are often specific to the characteristics of the simulator. Due to modeling error, strategies that are successful in simulation may not transfer to their real world counterparts. In this paper, we demonstrate a simple method to bridge this “reality gap”. By randomizing the dynamics of the simulator during training, we are able to develop policies that are capable of adapting to very different dynamics, including ones that differ significantly from the dynamics on which the policies were trained. This adaptivity enables the policies to generalize to the dynamics of the real world without any training on the physical system. Our approach is demonstrated on an object pushing task using a robotic arm. Despite being trained exclusively in simulation, our policies are able to maintain a similar level of performance when deployed on a real robot, reliably moving an object to a desired location from random initial configurations. We explore the impact of various design decisions and show that the resulting policies are robust to significant calibration error.

Overview

CoChat Summary

TL;DR

The paper demonstrates that randomizing simulator dynamics during training lets recurrent policies transfer directly to a real 7-DOF Fetch arm for a puck-pushing task without any real-world training. The LSTM policy retained comparable performance in simulation and reality, while feedforward and non-randomized baselines transferred poorly or failed.

Key findings

  • The LSTM policy trained only in simulation achieved real-robot success of 0.89 ± 0.06 over 28 trials, comparable to its simulated success of 0.91 ± 0.03 over 100 randomized-dynamics trials.
  • Memoryless feedforward policies transferred worse: FF no Rand had sim 0.51 ± 0.05 and real 0.0 ± 0.0 (10 trials); FF had sim 0.83 ± 0.04 and real 0.67 ± 0.14 (12 trials); FF + Hist had sim 0.87 ± 0.03 and real 0.70 ± 0.10 (20 trials).
  • Ablating randomization hurt real-robot success: fixed action timestep 0.29 ± 0.11 (17 trials), no observation noise 0.25 ± 0.12 (12 trials), fixed link mass 0.64 ± 0.10 (22 trials), fixed puck friction 0.48 ± 0.10 (27 trials), versus 0.89 ± 0.06 (28 trials) for all randomized parameters.
  • When contact dynamics were altered by attaching a packet of chips to the puck, the LSTM policy achieved 0.91 ± 0.04 success, comparable to 0.89 ± 0.06 without the attachment.

Methods

  • Task: a 7-DOF Fetch Robotics arm pushed a puck to a randomly sampled target on a table; reward was binary (0 if puck within 0.07 m of target, otherwise -1). State was 52D (joint positions/velocities, gripper position, puck position/orientation/linear and angular velocities); action was 7D target joint-angle offsets. Simulation used MuJoCo with 0.002 s timestep and 20 simulation steps per control step; episodes were 100 control steps in simulation and 200 timesteps on the real robot.
  • Dynamics randomization: at each episode start, 95 parameters were sampled and held fixed, including link masses, joint damping, puck mass/friction/damping, table height, controller gains, action timestep, and observation noise. Action timestep varied per step via Δt ∼ Δt0 + Exp(λ), with Δt0 = 0.04 s and λ fixed per episode; observation noise was Gaussian with standard deviation 5% of each feature’s running standard deviation; action exploration noise had standard deviation 0.01 rad.
  • Training and analysis: policies were trained with Recurrent Deterministic Policy Gradient (RDPG) and Hindsight Experience Replay (HER, k = 0.8), using an LSTM policy and an omniscient critic that received simulator dynamics parameters. ADAM used stepsize 5 × 10⁻⁴, batches of 128 episodes with 100 steps each, about 8000 update iterations, roughly 100 million samples, and about 8 hours on a 100-core cluster. Performance was compared across architectures and randomization ablations in simulation and on the real robot.

Limitations

  • Stated: little calibration was performed to make simulation match the real robot, and the authors note that more extensive calibration would likely improve performance; they also state future work should extend to richer tasks and modalities such as vision.
  • Evident from the text: evaluation is limited to a single puck-pushing task with a 7-DOF arm, and real-robot trials are small (10–28 trials per condition), so broader task and statistical generalization are not established.
  • Evident from the text: the method relies on simulation-only training and does not test direct real-world fine-tuning or vision-based transfer.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1109/icra.2018.8460528

Cites

50 of 50 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Fig. 1 A recurrent neural network policy trained for a pushing task in simulation is deployed directly on a Fetch Robotics arm. The red marker indicates the target location for the puck.
Fig. 1 A recurrent neural network policy trained for a pushing task in simulation is deployed directly on a Fetch Robotics arm. The red marker indicates the target location for the puck.From the PDF
Fig. 2 Our experiments are conducted on a 7-DOF Fetch Robotics arm. Left: Real robot. Right: Simulated MuJoCo model.
Fig. 2 Our experiments are conducted on a 7-DOF Fetch Robotics arm. Left: Real robot. Right: Simulated MuJoCo model.From the PDF
Fig. 3 LSTM policy deployed on the Fetch arm. Bottom: The contact dynamics of the puck was modified by attaching a packet of chips to the bottom.
Fig. 3 LSTM policy deployed on the Fetch arm. Bottom: The contact dynamics of the puck was modified by attaching a packet of chips to the bottom.From the PDF
Fig. 4 Schematic illustrations of the policy network (top) , and value network (bottom) . Features that are relevant for inferring the dynamics of the environment are processed by the recurrent branch, while the other inputs are processed by the feedforward branch.
Fig. 4 Schematic illustrations of the policy network (top) , and value network (bottom) . Features that are relevant for inferring the dynamics of the environment are processed by the recurrent branch, while the other inputs are processed by the feedforward branch.From the PDF
Fig. 5 Joint trajectories recorded from the simulated and real robot when executing the same target trajectories. The joints correspond to the shoulder, elbow, and wrist of the Fetch arm.
Fig. 5 Joint trajectories recorded from the simulated and real robot when executing the same target trajectories. The joints correspond to the shoulder, elbow, and wrist of the Fetch arm.From the PDF
Fig. 6 Learning curves of different network architectures. Four policies are trained for each architecture with different random seeds. Performance is evaluated over 100 episodes in simulation with random dynamics.
Fig. 6 Learning curves of different network architectures. Four policies are trained for each architecture with different random seeds. Performance is evaluated over 100 episodes in simulation with random dynamics.From the PDF
Fig. 7 Performance of different models when deployed on the simulated and real robot for the pushing task. Policies are trained using only data from simulation.
Fig. 7 Performance of different models when deployed on the simulated and real robot for the pushing task. Policies are trained using only data from simulation.From the PDF

Verification

Claims the paper makes

1 Valid · 0 Not valid · 2 Not checked · 29 Own results and interpretations

13 more claims

Create a free account to see all 32 claims and to verify the 3 references still open.

See all claims — free account
  • It may state two separate claims as one. 9
  • Probably not a substantial claim. 6

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

1 more claim about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Citations

Cites

Show all 39 references

Research works printed but not linked: 1

  1. ignasi clavera, 2017, Policy transfer via modularity, IROS IEEE

Cited by

See all 1,309

Metadata

Authors
Peng, X. B., Andrychowicz, M., Zaremba, W., & Abbeel, P.
Affiliations
  • University of California, Berkeley (US)
  • OpenAI (United States) (US)
Venue
2018 IEEE International Conference on Robotics and Automation (ICRA)
Publisher
IEEE
Published
2018-05
Pages
3803-3810
ISSN
1050-4729 (ISSN-L)
OpenAlex
W2767050701
Cited by
1,309
References
39
Citations by year
2026: 149 · 2025: 167 · 2024: 155 · 2023: 158 · 2022: 159 · 2021: 197 · 2020: 178 · 2019: 109 · 2018: 34
FWCI
52.88
Type
conference paper
Language
en
Open access
Locations
  • Proceedings - IEEE International Conference on Robotics and Automation/Proceedings (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
  • Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
Keywords
sim-to-real transfer, robot control, reality gap, object pushing, robot manipulators, policy generalization

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Sim-to-Real Transfer of Robotic Control with Dynamics Randomization. https://app.cochat.ai/papers/10.1109/icra.2018.8460528
The paper
Peng, X. B., Andrychowicz, M., Zaremba, W., & Abbeel, P. (2018). Sim-to-Real Transfer of Robotic Control with Dynamics Randomization. 2018 IEEE International Conference on Robotics and Automation (ICRA). https://doi.org/10.1109/icra.2018.8460528

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.