The DOI matches its Crossref record; 98 of the 98 references it cites were found in a registry or an index; 48 of the 90 sources its claims cite were checked for support.
Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey
Open accessAbstract
Deep reinforcement learning has recently seen huge success across multiple areas in the robotics domain. Owing to the limitations of gathering real-world data, i.e., sample inefficiency and the cost of collecting it, simulation environments are utilized for training the different agents. This not only aids in providing a potentially infinite data source, but also alleviates safety concerns with real robots. Nonetheless, the gap between the simulated and real worlds degrades the performance of the policies once the models are transferred into real robots. Multiple research efforts are therefore now being directed towards closing this sim-toreal gap and accomplish more efficient policy transfer. Recent years have seen the emergence of multiple methods applicable to different domains, but there is a lack, to the best of our knowledge, of a comprehensive review summarizing and putting into context the different methods. In this survey paper, we cover the fundamental background behind sim-to-real transfer in deep reinforcement learning and overview the main methods being utilized at the moment: domain randomization, domain adaptation, imitation learning, meta-learning and knowledge distillation. We categorize some of the most relevant recent works, and outline the main application scenarios. Finally, we discuss the main opportunities and challenges of the different approaches and point to the most promising directions.
Overview
CoChat Summary
TL;DR
This survey reviews sim-to-real transfer in deep reinforcement learning for robotics, covering domain randomization, domain adaptation, imitation learning, meta-learning, and knowledge distillation. It identifies domain randomization as the most widely adopted method and outlines application scenarios, challenges, and promising directions.
Key findings
- The main sim-to-real methods reviewed are domain randomization, domain adaptation, imitation learning, meta-learning, and knowledge distillation; domain randomization is identified as the most widely adopted method.
- Domain randomization is divided into visual randomization and dynamics randomization: visual randomization targets robotic vision tasks such as object localization, detection, pose estimation, and semantic segmentation, while dynamics randomization helps acquire robust control policies. For dexterous in-hand manipulation, one work randomizes object dimensions, object and robot link masses, surface friction coefficients, robot joint damping coefficients, and actuator force gains.
- Application scenarios include dexterous robotic manipulation (e.g., peg-in-hole, deformable object manipulation, multi-fingered hands, force control) and robotic navigation (e.g., six-legged robots, depth-based mapless navigation, soccer competitions), plus other applications such as plasma jet control, tactile sensing, and multi-agent manipulation.
- Stated challenges include the lack of formal explanation for why domain randomization works, the homogeneous feature-space assumption in most domain adaptation algorithms, and the need for wider theoretical and empirical studies; promising directions include integrating different methods and using incremental complexity learning, continual learning, and reward shaping.
Methods
- This is a survey/review paper, not an experimental study; it reviews fundamental background and recent works on sim-to-real transfer in DRL for robotics.
- It classifies relevant publications in Table I by real robot/platform, learning algorithm, knowledge transfer, simulator/engine, multi-agent learning, and sim-to-real transfer/learning details.
- The analysis is a qualitative synthesis of literature covering zero-shot transfer, system identification, domain randomization, domain adaptation, learning with disturbances, simulation environments, and application areas; no dataset or sample size is reported.
Limitations
- The paper states that for domain randomization it is hard to formally explain how and why it works, making efficient simulation and randomization distribution design difficult; for domain adaptation, most existing algorithms focus on homogeneous deep domain adaptation and assume the same feature spaces between source and target domains, which may not hold in many applications.
- The paper states that wider theoretical and empirical studies are required to better understand these techniques, and that generalization of existing results with more comprehensive analysis is lacking.
- The paper states its scope excludes specific simulators or techniques for direct learning in real-world settings, focuses on works specifically tackling sim-to-real transfer, and notes that sim-to-real literature for robotic navigation is sparse. Evident from the text: as a survey, it presents no primary experimental validation or quantitative benchmark results; Table I is a classification rather than a comparative performance evaluation.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures



Verification
Claims the paper makes
24 Valid · 8 Not valid · 17 Not checked · 26 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“From the literature, we summarize three common methods for domain adaptation regardless of their tasks. They are discrepancy-based , adversarial-based , and reconstructionbased methods, which can be also used crossly.”
What the sources say
- Partly supportedcites [65] [65] Eric Tzeng, Judy Hoffman, Ning Zhang, Kate Saenko, and Trevor Darrell. Deep domain confusion: Maximizing for domain invariance. arXiv:1412.3474 , 2014. 67. sourceExperimental — accuracy not yet measured
The Introduction (p. 1) and Section 3 (p. 3) describe a domain-adaptation method that minimizes maximum mean discrepancy, supporting the discrepancy-based part of the claim. The paper does not present adversarial-based and reconstruction-based methods alongside it as three common categories of domain adaptation. (LLM, conf=0.82)
- Not checked — no full text of the sourcecites [68] [68] Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, Franc ¸ois Laviolette, Mario Marchand, and Victor Lempitsky. Domain-adversarial training of neural networks. The Journal of Machine Learning Research , 17(1), 2016. - source
- Partly supportedcites [71] [71] Konstantinos Bousmalis, George Trigeorgis, Nathan Silberman, Dilip Krishnan, and Dumitru Erhan. Domain separation networks. In Advances in neural information processing systems , 2016. - sourceExperimental — accuracy not yet measured
Related Work (p. 2) discusses adversarial training and MMD-based alignment, while the Discussion (p. 8) describes a reconstruction loss used with either DANN or MMD. The paper therefore illustrates the three approaches and their combination, but it does not present them as a three-part classification of common domain-adaptation methods regardless of task; its review focuses on CNN-based unsupervised adaptation. (LLM, conf=0.84)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“In those cases where system identification for one-shot transfer is the objective, researchers have often built or customized specific simulations that meet problem-specific requirements and constraints [32], [36], [41].”
What the sources say
- Not checked — no full text of the sourcecites [32] [32] Matthew Witman, Dogan Gidon, David B Graves, Berend Smit, and Ali Mesbah. Sim-to-real transfer reinforcement learning for control of thermal effects of an atmospheric pressure plasma jet. Plasma Sources Science and Technology , 28(9), 2019. - source
- Partly supportedcites [36] [36] Hansenclever F Bassani, Renie A Delgado, Jose Nilton de O Lima Junior, Heitor R Medeiros, Pedro HM Braga, and Alain Tapp. Learning to play soccer by reinforcement and applying sim-to-real to compete in the real world. arXiv:2003.11102 , 2020. - sourceExperimental — accuracy not yet measured
The passages labeled 25 and 44 describe a customized soccer simulation environment designed for the paper’s RL and sim-to-real problem. They do not describe system identification or one-shot transfer as its objective, or establish that researchers have often taken this approach. (LLM, conf=0.86)
- Partly supportedcites [41] [41] Andrew Hundt, Benjamin Killeen, Heeyeon Kwon, Chris Paxton, and Gregory D Hager. ' good robot!': Efficient reinforcement learning for multi-step visual tasks via reward shaping. arXiv:1909.11730 , 2019. - sourceExperimental — accuracy not yet measured
The Simulation Experiments section (p. 5) describes two simulated tasks of the authors’ design, and the Real World Experiments section (p. 7) reports direct transfer of models trained in simulation. The paper does not describe system identification as the objective or establish that researchers often build or customize simulations to meet problem-specific constraints. (LLM, conf=0.76)
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“An exception to this is, however, robotic tasks involving object manipulation [5], [6].”
What the sources say
- Not checked — no full text of the sourcecites [5] [5] Aravind Rajeswaran, Vikash Kumar, Abhishek Gupta, Giulia Vezzani, John Schulman, Emanuel Todorov, and Sergey Levine. Learning complex dexterous manipulation with deep reinforcement learning and demonstrations. arXiv:1709.10087 , 2017. - source
- Supportedcites [6] [6] Jan Matas, Stephen James, and Andrew J Davison. Sim-to-real reinforcement learning for deformable object manipulation. arXiv:1806.07851 , 2018. - source
Section 5.3 (p. 8) reports testing simulation-trained policies on a real robot for three cloth-manipulation tasks. Table 2 (p. 8) reports real-world successes without additional training, supporting this paper as an example of deep reinforcement learning succeeding beyond simulation in object manipulation. (LLM, conf=0.90)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“In robotic vision tasks including object localization [53], object detection [54], pose estimation [55], and semantic segmentation [56], the training data from simulator always have different textures, lighting, and camera positions from the realistic environments.”
What the sources say
- Partly supportedcites [53] [53] Josh Tobin, Rachel Fong, Alex Ray, Jonas Schneider, Wojciech Zaremba, and Pieter Abbeel. Domain randomization for transferring deep neural networks from simulation to the real world. In IROS , 2017. - sourceExperimental — accuracy not yet measured
The Introduction (p. 1) describes a reality gap between simulated camera images and real-world images, and its Fig. 1 caption says this study randomizes camera positions, lighting, and non-realistic textures. But the claim's “always” extends that description to all simulator training data; Bridging the reality gap (p. 2) also describes approaches that make simulation closely match physical reality. (LLM, conf=0.88)
- Not checked yetcites [54] [54] Jonathan Tremblay, Aayush Prakash, David Acuna, Mark Brophy, Varun Jampani, Cem Anil, Thang To, Eric Cameracci, Shaad Boochoon, and Stan Birchfield. Training deep networks with synthetic data: Bridging the reality gap by domain randomization. In CVPR Workshops , 2018. - source
Checked before — its earlier verdict is held back until a new check runs.
- Not checked — no full text of the sourcecites [55] [55] Martin Sundermeyer, Zoltan-Csaba Marton, Maximilian Durner, Manuel Brucker, and Rudolph Triebel. Implicit 3d orientation learning for 6d object detection from rgb images. In ECCV , 2018. - source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Moreover, a relevant conclusion from this work is that excessive domain randomization can be detrimental. Specifically, when the number of different colors that were being used for each texture was too large, the performance of the real robot was significantly worse.”
What the sources say
- Supportedcites [6] [6] Jan Matas, Stephen James, and Andrew J Davison. Sim-to-real reinforcement learning for deformable object manipulation. arXiv:1806.07851 , 2018. - source
Section 5.3 (p. 8) reports that heavy domain randomisation was detrimental: sampling texture colours uniformly across all colours made the agent’s performance after transfer significantly worse. This supports the citing sentence’s description of excessive colour variation harming real-robot performance. (LLM, conf=0.93)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
8 more claims
Create a free account to see all 75 claims and to verify the 16 references still open.
- Probably not a substantial claim. 48
- It may state two separate claims as one. 27
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Simulation-based approaches provide a cheap and safe development environment for RMSes, but there are challenges to deploying simulation-trained systems into real-world settings [62].”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [62]
The cited source says what the claim attributes to it.
In the citing paper
“Consequently, simulation-to-reality (Sim2Real) transfer [62] (Section 3.4) becomes a key technology in modern robotics for bridging the gap between the simulation-trained policy and the real-world deployment.”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [62]
Citations
Cites
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782
Proximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625
Domain-Adversarial Training of Neural Networks
Ganin, Y., Ustinova, E., Ajakan, H., Germain, P., Larochelle, H., Laviolette, F., Marchand, M., & Lempitsky, V. (2017) · Advances in Computer Vision and Pattern Recognition · doi:10.1007/978-3-319-58347-1_10 · Cited by 8,318
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132
A Comprehensive Survey on Transfer Learning
Zhuang, F., Qi, Z., Duan, K., Xi, D., Zhu, Y., Zhu, H., Xiong, H., & He, Q. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2020.3004555 · Cited by 6,678 · cited 2×
“Transfer learning aims at improving the performance of target learners on target domains by transferring the knowledge contained in different but related source domains [18].”
BackgroundSoft actor-critic Off-policy maximum entropy deep reinforcement learning with a stochastic actor
Haarnoja, T., Zhou, A., Abbeel, P., & Levine, S. (2018) · arXiv · doi:10.48550/arxiv.1801.01290 · Cited by 5,594
MuJoCo: A physics engine for model-based control
Todorov, E., Erez, T., & Tassa, Y. (2012) · 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2012.6386109 · Cited by 4,767 · cited 3×
“The most widely used simulators in the literature are Gazebo [74], Unity3D, and PyBullet [75] or MuJoCo [17].”
BackgroundMethodDeep Reinforcement Learning: A Brief Survey
Arulkumaran, K., Deisenroth, M. P., Brundage, M., & Bharath, A. A. (2017) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2743240 · Cited by 4,733
A Brief Survey of Deep Reinforcement Learning
Arulkumaran, K., Deisenroth, M. P., Brundage, M., & Bharath, A. A. (2017) · arXiv · doi:10.48550/arxiv.1708.05866 · Cited by 4,733
Trust region policy optimization
Schulman, J., Levine, S., Moritz, P., Jordan, M. I., & Abbeel, P. (2015) · arXiv · doi:10.48550/arxiv.1502.05477 · Cited by 4,578
Research works printed but not linked: 9
- brockman, 2016, OpenAI Gym
- coumans, 2016, Pybullet a python module for physics simulation for games robotics and machine learning
- van baar, 2018, Simulation to real transfer learning with robustified policies for robot tasks
- pomerleau, 1989, Alvinn: An autonomous land vehicle in a neural network, Advances in neural information processing systems
- tobin, 2019, Real-World Robotic Perception and Control Using Synthetic Data
- kaspar, 0, Reinforcement learning with cartesian commands and sim to real transfer for peg in hole tasks
- muratore, 2020, Bayesian domain randomization for sim-to-real transfer
- pedersen, 2019, Sim-to-real transfer of robotic gripper pose estimation-using deep reinforcement learning generative adversarial networks and visual servoing
- hundt, 2019, “good robot!” Efficient reinforcement learning for multi-step visual tasks via reward shaping
Cited by
Transfer Learning in Deep Reinforcement Learning: A Survey
Zhu, Z., Lin, K., Jain, A. K., & Zhou, J. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3292075 · Cited by 861
“For instance, [70] consolidated sim-to-real TL methods.”
BackgroundSelf-Driving Laboratories for Chemistry and Materials Science
Tom, G., Schmid, S. P., Baird, S. G., Cao, Y., Darvish, K., Hao, H., Lo, S., Pablo-García, S., Rajaonson, E. M., Skreta, M., et al. (2024) · Chemical Reviews · doi:10.1021/acs.chemrev.4c00055 · Cited by 707
Deep reinforcement learning in smart manufacturing: A review and prospects
Li, C., Zheng, P., Yin, Y., Wang, B., & Wang, L. (2023) · CIRP Journal of Manufacturing Science and Technology · doi:10.1016/j.cirpj.2022.11.003 · Cited by 327
Deep Multi-Agent Reinforcement Learning for Highway On-Ramp Merging in Mixed Traffic
Chen, D., Hajidavalloo, M. R., Li, Z., Chen, K., Wang, Y., Jiang, L., & Wang, Y. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3285442 · Cited by 257
“Interested readers are referred to an extensive survey on sim-to-real paradigms for reinforcement learning [62].”
BackgroundA Comprehensive Survey on Artificial Intelligence Empowered Edge Computing on Consumer Electronics
Syu, J.-H., Lin, J. C.-W., Srivastava, G., & Yu, K. (2023) · IEEE Transactions on Consumer Electronics · doi:10.1109/tce.2023.3318150 · Cited by 212
“The flexibility and generality of reinforcement learning make it broadly used in various fields, especially in automated systems [39], robots [40], and finance [41], [42].”
Background
Metadata
- Authors
- Zhao, W., Queralta, J. P., & Westerlund, T.
- Affiliations
- University of Turku (FI)
- ORCID
- Westerlund, Tomi
- Venue
- 2020 IEEE Symposium Series on Computational Intelligence (SSCI)
- Publisher
- IEEE
- Published
- 2020-12-01
- Pages
- 737-744
- OpenAlex
- W3088310808
- Cited by
- 941
- References
- 52
- Citations by year
- 2026: 164 · 2025: 217 · 2024: 216 · 2023: 162 · 2022: 133 · 2021: 48 · 2020: 1
- FWCI
- 32.84
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Symposium Series on Computational Intelligence (SSCI) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Academy of Finland
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
- Keywords
- sim-to-real transfer, deep reinforcement learning, robotics, domain randomization, domain adaptation, imitation learning, meta-learning, knowledge distillation, sample inefficiency
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey. https://app.cochat.ai/papers/10.1109/ssci47803.2020.9308468
- The paper
- Zhao, W., Queralta, J. P., & Westerlund, T. (2020). Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey. 2020 IEEE Symposium Series on Computational Intelligence (SSCI). https://doi.org/10.1109/ssci47803.2020.9308468
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.