Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 98 of the 98 references it cites were found in a registry or an index; 48 of the 90 sources its claims cite were checked for support.

Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey

Open access
Zhao, Wenshuai, Queralta, Jorge Pena, Westerlund, Tomi · 2020 · 2020 IEEE Symposium Series on Computational Intelligence (SSCI)

Abstract

Deep reinforcement learning has recently seen huge success across multiple areas in the robotics domain. Owing to the limitations of gathering real-world data, i.e., sample inefficiency and the cost of collecting it, simulation environments are utilized for training the different agents. This not only aids in providing a potentially infinite data source, but also alleviates safety concerns with real robots. Nonetheless, the gap between the simulated and real worlds degrades the performance of the policies once the models are transferred into real robots. Multiple research efforts are therefore now being directed towards closing this sim-toreal gap and accomplish more efficient policy transfer. Recent years have seen the emergence of multiple methods applicable to different domains, but there is a lack, to the best of our knowledge, of a comprehensive review summarizing and putting into context the different methods. In this survey paper, we cover the fundamental background behind sim-to-real transfer in deep reinforcement learning and overview the main methods being utilized at the moment: domain randomization, domain adaptation, imitation learning, meta-learning and knowledge distillation. We categorize some of the most relevant recent works, and outline the main application scenarios. Finally, we discuss the main opportunities and challenges of the different approaches and point to the most promising directions.

Overview

CoChat Summary

TL;DR

This survey reviews sim-to-real transfer in deep reinforcement learning for robotics, covering domain randomization, domain adaptation, imitation learning, meta-learning, and knowledge distillation. It identifies domain randomization as the most widely adopted method and outlines application scenarios, challenges, and promising directions.

Key findings

  • The main sim-to-real methods reviewed are domain randomization, domain adaptation, imitation learning, meta-learning, and knowledge distillation; domain randomization is identified as the most widely adopted method.
  • Domain randomization is divided into visual randomization and dynamics randomization: visual randomization targets robotic vision tasks such as object localization, detection, pose estimation, and semantic segmentation, while dynamics randomization helps acquire robust control policies. For dexterous in-hand manipulation, one work randomizes object dimensions, object and robot link masses, surface friction coefficients, robot joint damping coefficients, and actuator force gains.
  • Application scenarios include dexterous robotic manipulation (e.g., peg-in-hole, deformable object manipulation, multi-fingered hands, force control) and robotic navigation (e.g., six-legged robots, depth-based mapless navigation, soccer competitions), plus other applications such as plasma jet control, tactile sensing, and multi-agent manipulation.
  • Stated challenges include the lack of formal explanation for why domain randomization works, the homogeneous feature-space assumption in most domain adaptation algorithms, and the need for wider theoretical and empirical studies; promising directions include integrating different methods and using incremental complexity learning, continual learning, and reward shaping.

Methods

  • This is a survey/review paper, not an experimental study; it reviews fundamental background and recent works on sim-to-real transfer in DRL for robotics.
  • It classifies relevant publications in Table I by real robot/platform, learning algorithm, knowledge transfer, simulator/engine, multi-agent learning, and sim-to-real transfer/learning details.
  • The analysis is a qualitative synthesis of literature covering zero-shot transfer, system identification, domain randomization, domain adaptation, learning with disturbances, simulation environments, and application areas; no dataset or sample size is reported.

Limitations

  • The paper states that for domain randomization it is hard to formally explain how and why it works, making efficient simulation and randomization distribution design difficult; for domain adaptation, most existing algorithms focus on homogeneous deep domain adaptation and assume the same feature spaces between source and target domains, which may not hold in many applications.
  • The paper states that wider theoretical and empirical studies are required to better understand these techniques, and that generalization of existing results with more comprehensive analysis is lacking.
  • The paper states its scope excludes specific simulators or techniques for direct learning in real-world settings, focuses on works specifically tackling sim-to-real transfer, and notes that sim-to-real literature for robotic navigation is sparse. Evident from the text: as a survey, it presents no primary experimental validation or quantitative benchmark results; Table I is a classification rather than a comparative performance evaluation.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1109/ssci47803.2020.9308468

Cites

98 of 98 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Fig. 1 Conceptual view of a simulation-to-reality transfer process. One of the most common methods is domain randomization, through which different parameters of the simulator (e..g, colors, textures, dynamics) are randomized to produce more robust policies.
Fig. 1 Conceptual view of a simulation-to-reality transfer process. One of the most common methods is domain randomization, through which different parameters of the simulator (e..g, colors, textures, dynamics) are randomized to produce more robust policies.From the PDF
Fig. 2 Illustration of the different methods related to sim-to-real transfer in deep reinforcement learning and their relationships.
Fig. 2 Illustration of the different methods related to sim-to-real transfer in deep reinforcement learning and their relationships.From the PDF
Image 3
From the PDF

Verification

Claims the paper makes

24 Valid · 8 Not valid · 17 Not checked · 26 Own results and interpretations

8 more claims

Create a free account to see all 75 claims and to verify the 16 references still open.

See all claims — free account
  • Probably not a substantial claim. 48
  • It may state two separate claims as one. 27

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 52 references

Research works printed but not linked: 9

  1. brockman, 2016, OpenAI Gym
  2. coumans, 2016, Pybullet a python module for physics simulation for games robotics and machine learning
  3. van baar, 2018, Simulation to real transfer learning with robustified policies for robot tasks
  4. pomerleau, 1989, Alvinn: An autonomous land vehicle in a neural network, Advances in neural information processing systems
  5. tobin, 2019, Real-World Robotic Perception and Control Using Synthetic Data
  6. kaspar, 0, Reinforcement learning with cartesian commands and sim to real transfer for peg in hole tasks
  7. muratore, 2020, Bayesian domain randomization for sim-to-real transfer
  8. pedersen, 2019, Sim-to-real transfer of robotic gripper pose estimation-using deep reinforcement learning generative adversarial networks and visual servoing
  9. hundt, 2019, “good robot!” Efficient reinforcement learning for multi-step visual tasks via reward shaping

Cited by

See all 941

Metadata

Authors
Zhao, W., Queralta, J. P., & Westerlund, T.
Affiliations
University of Turku (FI)
Venue
2020 IEEE Symposium Series on Computational Intelligence (SSCI)
Publisher
IEEE
Published
2020-12-01
Pages
737-744
OpenAlex
W3088310808
Cited by
941
References
52
Citations by year
2026: 164 · 2025: 217 · 2024: 216 · 2023: 162 · 2022: 133 · 2021: 48 · 2020: 1
FWCI
32.84
Type
conference paper
Language
en
Open access
Locations
  • IEEE Symposium Series on Computational Intelligence (SSCI) (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
Academy of Finland
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
Keywords
sim-to-real transfer, deep reinforcement learning, robotics, domain randomization, domain adaptation, imitation learning, meta-learning, knowledge distillation, sample inefficiency

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey. https://app.cochat.ai/papers/10.1109/ssci47803.2020.9308468
The paper
Zhao, W., Queralta, J. P., & Westerlund, T. (2020). Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey. 2020 IEEE Symposium Series on Computational Intelligence (SSCI). https://doi.org/10.1109/ssci47803.2020.9308468

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.