The DOI matches its Crossref record; 50 of the 50 references it cites were found in a registry or an index; 2 of the 5 sources its claims cite were checked for support.
Sim-to-Real Transfer of Robotic Control with Dynamics Randomization
Open accessAbstract
Simulations are attractive environments for training agents as they provide an abundant source of data and alleviate certain safety concerns during the training process. But the behaviours developed by agents in simulation are often specific to the characteristics of the simulator. Due to modeling error, strategies that are successful in simulation may not transfer to their real world counterparts. In this paper, we demonstrate a simple method to bridge this “reality gap”. By randomizing the dynamics of the simulator during training, we are able to develop policies that are capable of adapting to very different dynamics, including ones that differ significantly from the dynamics on which the policies were trained. This adaptivity enables the policies to generalize to the dynamics of the real world without any training on the physical system. Our approach is demonstrated on an object pushing task using a robotic arm. Despite being trained exclusively in simulation, our policies are able to maintain a similar level of performance when deployed on a real robot, reliably moving an object to a desired location from random initial configurations. We explore the impact of various design decisions and show that the resulting policies are robust to significant calibration error.
Overview
CoChat Summary
TL;DR
The paper demonstrates that randomizing simulator dynamics during training lets recurrent policies transfer directly to a real 7-DOF Fetch arm for a puck-pushing task without any real-world training. The LSTM policy retained comparable performance in simulation and reality, while feedforward and non-randomized baselines transferred poorly or failed.
Key findings
- The LSTM policy trained only in simulation achieved real-robot success of 0.89 ± 0.06 over 28 trials, comparable to its simulated success of 0.91 ± 0.03 over 100 randomized-dynamics trials.
- Memoryless feedforward policies transferred worse: FF no Rand had sim 0.51 ± 0.05 and real 0.0 ± 0.0 (10 trials); FF had sim 0.83 ± 0.04 and real 0.67 ± 0.14 (12 trials); FF + Hist had sim 0.87 ± 0.03 and real 0.70 ± 0.10 (20 trials).
- Ablating randomization hurt real-robot success: fixed action timestep 0.29 ± 0.11 (17 trials), no observation noise 0.25 ± 0.12 (12 trials), fixed link mass 0.64 ± 0.10 (22 trials), fixed puck friction 0.48 ± 0.10 (27 trials), versus 0.89 ± 0.06 (28 trials) for all randomized parameters.
- When contact dynamics were altered by attaching a packet of chips to the puck, the LSTM policy achieved 0.91 ± 0.04 success, comparable to 0.89 ± 0.06 without the attachment.
Methods
- Task: a 7-DOF Fetch Robotics arm pushed a puck to a randomly sampled target on a table; reward was binary (0 if puck within 0.07 m of target, otherwise -1). State was 52D (joint positions/velocities, gripper position, puck position/orientation/linear and angular velocities); action was 7D target joint-angle offsets. Simulation used MuJoCo with 0.002 s timestep and 20 simulation steps per control step; episodes were 100 control steps in simulation and 200 timesteps on the real robot.
- Dynamics randomization: at each episode start, 95 parameters were sampled and held fixed, including link masses, joint damping, puck mass/friction/damping, table height, controller gains, action timestep, and observation noise. Action timestep varied per step via Δt ∼ Δt0 + Exp(λ), with Δt0 = 0.04 s and λ fixed per episode; observation noise was Gaussian with standard deviation 5% of each feature’s running standard deviation; action exploration noise had standard deviation 0.01 rad.
- Training and analysis: policies were trained with Recurrent Deterministic Policy Gradient (RDPG) and Hindsight Experience Replay (HER, k = 0.8), using an LSTM policy and an omniscient critic that received simulator dynamics parameters. ADAM used stepsize 5 × 10⁻⁴, batches of 128 episodes with 100 steps each, about 8000 update iterations, roughly 100 million samples, and about 8 hours on a 100-core cluster. Performance was compared across architectures and randomization ablations in simulation and on the real robot.
Limitations
- Stated: little calibration was performed to make simulation match the real robot, and the authors note that more extensive calibration would likely improve performance; they also state future work should extend to richer tasks and modalities such as vision.
- Evident from the text: evaluation is limited to a single puck-pushing task with a 7-DOF arm, and real-robot trials are small (10–28 trials per condition), so broader task and statistical generalization are not established.
- Evident from the text: the method relies on simulation-only training and does not test direct real-world fine-tuning or vision-based transfer.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures







Verification
Claims the paper makes
1 Valid · 0 Not valid · 2 Not checked · 29 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Furthermore, it has been observed that DeepRL policies are prone to exploiting idiosyncrasies of the simulator to realize behaviours that are infeasible in the real world [2], [7].”
What the sources say
- Could not verifycites [2] [2] T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, and D. Wierstra, 'Continuous control with deep reinforcement learning,' CoRR , vol. abs/1509.02971, 2015. [Online]. Available: http://arxiv.org/abs/1509.02971 - sourceExperimental — accuracy not yet measured
The Results (p. 4) describe testing DDPG in simulated physical environments, but do not report policies exploiting simulator quirks. Searches for simulator artifacts, unrealistic behavior, and real-world feasibility found no passage addressing the claimed tendency. (LLM, conf=0.72)
- Partly supportedcites [7] [7] N. Heess, D. TB, S. Sriram, J. Lemmon, J. Merel, G. Wayne, Y. Tassa, T. Erez, Z. Wang, S. M. A. Eslami, M. A. Riedmiller, and D. Silver, 'Emergence of locomotion behaviours in rich environments,' CoRR , vol. abs/1707.02286, 2017. [Online]. Available: http://arxiv.org/abs/1707.02286 - sourceExperimental — accuracy not yet measured
The Introduction (p. 1) says agents may find idiosyncratic solutions that work in one setting but are not robust across settings. The Results (p. 8) report that a simulated humanoid can exploit redundancies in the task specification, producing visually unsatisfactory gaits. Neither passage says the policies exploit quirks of the simulator or that their behaviours are infeasible in the real world. (LLM, conf=0.82)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We explore the impact of various design decisions and show that the resulting policies are robust to significant calibration error.”
- Also stated in: § B. Domain Randomization · p. 2 · plain
“Unlike previous methods which often require meticulous calibration of the simulation to closely conform to the physical system, our policies are able to adapt to significant calibration error.”
- Also stated in: § C. Non-prehensile Manipulation · p. 2 · plain
“The resulting policies are able accommodate large calibration errors when deployed on a real robot and also generalize to variability in the dynamics of the physical system.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § B. Domain Randomization · p. 2 · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The feedforward network trained without randomization is unable to perform the task under the real world dynamics.”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Given the same target trajectory, the pose trajectories of the simulated and real robot differ significantly, with varying degrees of mismatch across joints.”
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Policies trained without randomizing the action timestep and observation noise show particularly noticeable drops in performance.”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
13 more claims
Create a free account to see all 32 claims and to verify the 3 references still open.
- It may state two separate claims as one. 9
- Probably not a substantial claim. 6
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“We instantiate SIMPLER on top of the SAPIEN physics simulator [68], but show in Section VI-D that our contributions are independent of the used simulator and can be reproduced in Isaac Sim [49].”
From Evaluating Real-World Robot Manipulation Policies in Simulation (2024) · cites this paper as [49]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Residual policy learning restores dynamic feasibility on the robot by learning small corrections to the retargeted reference, reusing the same residual formulation as physics refinement, so that the resulting policy transfers directly to hardware (Peng et al., 2018).”
From Dexterous Robot Manipulation from Human Demonstrations via Contact-Anchored Retargeting and Residual Policy Learning (2026) · cites this paper as Peng et al., 2018
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Synthesis rollouts under domain randomization (Peng et al., 2018) produce additional dynamically feasible robot trajectories from each retargeted reference.”
From Dexterous Robot Manipulation from Human Demonstrations via Contact-Anchored Retargeting and Residual Policy Learning (2026) · cites this paper as Peng et al., 2018
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Another is randomization of physical parameters that vary between simulation and reality, such that the controller becomes robust to a range of conditions that cover those that arise in physical deployment, without the necessity to precisely model these conditions a priori [21].”
From Learning quadrupedal locomotion over challenging terrain (2020) · cites this paper as [21]
The cited source says what the claim attributes to it.
In the citing paper
“Peng et al. [44] shows the effectiveness of memory-based policies (i.e., policies learnt by using past memory for future learning [95] [97]) to deal with the RG, introducing DR to generalise environment dynamics.”
From Crossing the Reality Gap: A Survey on Sim-to-Real Transferability of Robot Controllers in Reinforcement Learning (2021) · cites this paper as [44]
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357 · cited 1×
“In simulated domains, agents have been developed to perform a diverse array of challenging tasks [1], [2], [3].”
BackgroundContinuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782
Domain-Adversarial Training of Neural Networks
Ganin, Y., Ustinova, E., Ajakan, H., Germain, P., Larochelle, H., Laviolette, F., Marchand, M., & Lempitsky, V. (2017) · Advances in Computer Vision and Pattern Recognition · doi:10.1007/978-3-319-58347-1_10 · Cited by 8,318
(1999) · Cited by 5,147
MuJoCo: A physics engine for model-based control
Todorov, E., Erez, T., & Tassa, Y. (2012) · 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2012.6386109 · Cited by 4,767 · cited 1×
“All simulations are performed using the MuJoCo physics engine [39] with a simulation timestep of 0.”
MethodMulti-Agent Actor-Critic for Mixed Cooperative-Competitive Environments
Lowe, R., Wu, Y., Tamar, A., Harb, J., Abbeel, P., & Mordatch, I. (2017) · arXiv · doi:10.48550/arxiv.1706.02275 · Cited by 3,377
Self-citationDomain randomization for transferring deep neural networks from simulation to the real world
Tobin, J., Fong, R., Ray, A., Schneider, J., Zaremba, W., & Abbeel, P. (2017) · 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8202133 · Cited by 3,124
Self-citationLevine, S., Pastor, P., Krizhevsky, A., Ibarz, J., & Quillen, D. (2018) · The International Journal of Robotics Research · doi:10.1177/0278364917710318 · Cited by 1,971
Andrychowicz, M., Wolski, F., Ray, A., Schneider, J., Fong, R., Welinder, P., McGrew, B., Tobin, J., Abbeel, P., & Zaremba, W. (2017) · arXiv · doi:10.48550/arxiv.1707.01495 · Cited by 1,355
Self-citation
Research works printed but not linked: 1
- ignasi clavera, 2017, Policy transfer via modularity, IROS IEEE
Cited by
Deep reinforcement learning for autonomous driving: A survey
Kiran, B. R., Sobh, I., Talpaert, V., Mannion, P., Sallab, A. A. A., Yogamani, S., & Perez, P. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2021.3054625 · Cited by 2,342
“Authors in [118], randomized the dynamics of the simulator during training.”
BackgroundLearning agile and dynamic motor skills for legged robots
Hwangbo, J., Lee, J., Dosovitskiy, A., Bellicoso, D., Tsounis, V., Koltun, V., & Hutter, M. (2019) · Science Robotics · doi:10.1126/scirobotics.aau5872 · Cited by 1,482
Learning quadrupedal locomotion over challenging terrain
Lee, J., Hwangbo, J., Wellhausen, L., Koltun, V., & Hutter, M. (2020) · Science Robotics · doi:10.1126/scirobotics.abc5986 · Cited by 1,156
“Another is randomization of physical parameters that vary between simulation and reality, such that the controller becomes robust to a range of conditions that cover those that arise in physical deployment, without the necessity to precisely model these conditions a priori [21].”
Generalizing to unseen domains: A survey on domain generalization
Wang, J., Lan, C., Liu, C., Ouyang, Y., Qin, T., Lu, W., Chen, Y., Zeng, W., & Yu, P. (2022) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2022.3178128 · Cited by 1,046
“Similar techniques were also used in [26, 27, 28, 24] to strengthen the generalization capability of the models.”
Methodvon Rueden, L., Mayer, S., Beckh, K., Georgiev, B., Giesselbach, S., Heese, R., Kirsch, B., Walczak, M., Pfrommer, J., Pick, A., et al. (2021) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3079836 · Cited by 793
“A promising direction to close the gap is domain adaptation, especially adversarial training [67], [137], or domain randomization [138].”
Background
Metadata
- Authors
- Peng, X. B., Andrychowicz, M., Zaremba, W., & Abbeel, P.
- Affiliations
- University of California, Berkeley (US)
- OpenAI (United States) (US)
- ORCID
- Abbeel, Pieter
- Venue
- 2018 IEEE International Conference on Robotics and Automation (ICRA)
- Publisher
- IEEE
- Published
- 2018-05
- Pages
- 3803-3810
- ISSN
- 1050-4729 (ISSN-L)
- OpenAlex
- W2767050701
- Cited by
- 1,309
- References
- 39
- Citations by year
- 2026: 149 · 2025: 167 · 2024: 155 · 2023: 158 · 2022: 159 · 2021: 197 · 2020: 178 · 2019: 109 · 2018: 34
- FWCI
- 52.88
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Proceedings - IEEE International Conference on Robotics and Automation/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- sim-to-real transfer, robot control, reality gap, object pushing, robot manipulators, policy generalization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Sim-to-Real Transfer of Robotic Control with Dynamics Randomization. https://app.cochat.ai/papers/10.1109/icra.2018.8460528
- The paper
- Peng, X. B., Andrychowicz, M., Zaremba, W., & Abbeel, P. (2018). Sim-to-Real Transfer of Robotic Control with Dynamics Randomization. 2018 IEEE International Conference on Robotics and Automation (ICRA). https://doi.org/10.1109/icra.2018.8460528
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.