The DOI matches its Crossref record; 132 of the 173 references it cites were found in a registry or an index; 30 of the 71 sources its claims cite were checked for support.
Crossing the Reality Gap: A Survey on Sim-to-Real Transferability of Robot Controllers in Reinforcement Learning
Open accessAbstract
The growing demand for robots able to act autonomously in complex scenarios has widely accelerated the introduction of Reinforcement Learning (RL) in robots control applications. However, the trial and error intrinsic nature of RL may result in long training time on real robots and, moreover, it may lead to dangerous outcomes. While simulators are useful tools to accelerate RL training and to ensure safety, they often are provided only with an approximated model of robot dynamics and of its interaction with the surrounding environment, thus resulting in what is called the reality gap (RG): a mismatch of simulated and real control-law performances caused by the inaccurate representation of the real environment in simulation. The most undesirable result occurs when the controller learnt in simulation fails the task on the real robot, thus resulting in an unsuccessful sim-to-real transfer. The goal of the present survey is threefold: (1) to identify the main approaches to face the RG problem in the context of robot control with RL, (2) to point out their shortcomings, and (3) to outline new potential research areas.
Overview
CoChat Summary
TL;DR
This survey reviews sim-to-real transferability of RL robot controllers and categorizes reality-gap approaches into domain randomisation, adversarial RL, and transfer learning. It finds no task-independent solution or standard reality-gap metric, and notes many methods lack real-robot validation.
Key findings
- The survey identifies three main categories for addressing the reality gap: domain randomisation (DR), adversarial RL (ARL), and transfer learning (TL). Across the surveyed articles, 11 use DR, 2 use ARL, and 7 use TL (Tables 2–5).
- DR and ARL treat sim-to-real transferability as robustness to model perturbations, but the paper argues that robustness to chosen disturbances does not guarantee transferability if those disturbances do not correctly represent simulator inaccuracies.
- TL approaches appear promising because they involve interaction with the real system during training, but they often lack robustness assessment and may be inefficient due to two successive or simultaneous training phases.
- Open challenges include the absence of a general task-independent approach, the need for an index to reveal and quantify the reality gap, and the sample efficiency of RL-based methods.
Methods
- Literature survey: included articles had to explicitly address the reality gap, deal with robot control applications, and employ RL techniques; the survey then categorizes them into DR, ARL, and TL.
- The paper provides a formal RL/control framework and summarizes each surveyed article by algorithm, task, simulator, and whether evaluation was sim-to-sim and/or sim-to-real.
- Qualitative synthesis; no primary experiments or quantitative meta-analysis are conducted.
Limitations
- Stated: task diversity and the lack of a common theoretical framework prevent systematic comparison; many proposed approaches were not experimentally evaluated in an actual sim-to-real scenario; each approach appears tailored to a specific task; a general task-independent approach is still missing; sample efficiency remains a significant open problem.
- Evident: the survey does not report a systematic search protocol, databases, or quantitative meta-analysis, so its categorization and counts depend on the selected articles.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
18 Valid · 11 Not valid · 38 Not checked · 44 Own results and interpretations
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“The analysed task is a Marble maze game driven by a robotic arm and the results show that there is a trade-off between controller robustness and ne-tuning steps.”
What the sources say
- Disputedcites [56] [56] J. Van Baar, A. Sullivan, R. Cordorel, D. Jha, D. Romeres, and D. Nikovski, ''Sim-to-real transfer learning using robusti ed controllers in robotic tasks involving complex dynamics,'' in Proc. Int. Conf. Robot. Automat. (ICRA) , May 2019, pp. 6001 6007. <!-- image --> IEEEAccess" - sourceExperimental — accuracy not yet measured
Section 5.4 (p. 8) describes a 3D maze-navigation game controlled through movement and looking actions, not a marble maze driven by a robotic arm. The cited paper discusses transfer learning and finetuning, but the passages do not report the claimed trade-off between controller robustness and fine-tuning steps. (LLM, conf=0.84)
Checked by a reasoning model — may be wrong.
plainstates an associationSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“However, the authors assume that, when the consequences of a control input applied in simulation differ from those applied on the real robot, real observations should match simulated ones, and that is not always true.”
What the sources say
- Disputedcites [49] [49] P. Christiano, Z. Shah, I. Mordatch, J. Schneider, T. Blackwell, J. Tobin, P. Abbeel, and W. Zaremba, ''Transfer from simulation to real world through learning deep inverse dynamics model,'' 2016, arXiv:1610.03518 . - sourceExperimental — accuracy not yet measured
The Experiments section (p. 4) says that visual discrepancies between simulated and real observations are obvious, rather than assuming they should match. The Introduction (p. 2) describes training a new network on the real robot after training in simulation. The paper does not make the claimed assumption about observations when control inputs have different consequences. (LLM, conf=0.83)
Checked by a reasoning model — may be wrong.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“On the contrary, their evaluations have shown that controllers learnt in simulation with large observation space lead to bad results when transferred to real robot.”
What the sources say
- Partly supportedcites [54] [54] J. Tan, T. Zhang, E. Coumans, A. Iscen, Y. Bai, D. Hafner, S. Bohez, and V. Vanhoucke, ''Sim-to-real: Learning agile locomotion for quadruped robots,'' in Proceedings of Robotics: Science and Systems . Pittsburgh, PA, USA: RSS Foundation, Jun. 2018, pp. 10 20. - sourceExperimental — accuracy not yet measured
The ablation study (4 Experiments, pp. 7–8) found that a baseline controller trained in simulation failed the full task on the real robot. But it attributes that failure to the absence of domain randomisation, not to a large observation space; the fully randomised controller completed all standard real-world trials. (LLM, conf=0.85)
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“In particular, when the cGAN during training is fed with images sampled only from non-successful trajectories, the nal controller results in an unsatisfactory transfer.”
What the sources say
- Could not verifycites [46] [46] S. James, P. Wohlhart, M. Kalakrishnan, D. Kalashnikov, A. Irpan, J. Ibarz, S. Levine, R. Hadsell, and K. Bousmalis, ''Sim-to-real via sim-tosim: Data-ef cient robotic grasping via randomized-to-canonical adaptation networks,'' in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. , Jun. 2019, pp. 12627 12637. - sourceExperimental — accuracy not yet measured
Section 4.1 (p. 4) says the cGAN’s training images were sampled from QT-Opt trajectories, but does not describe training it only on unsuccessful trajectories. Section 5.2 (p. 7) reports transfer performance for other training approaches, not the claimed unsuccessful-trajectory condition. Searches for failed or unsuccessful trajectory sampling found no result addressing its effect on transfer. (LLM, conf=0.75)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
Every source the paper cites for this claim was checked and supports it.
In the paper
“Besides, they show that heavy randomisation can lead to unsuccessful transfers.”
What the sources say
- Supportedcites [45] [45] J. Matas, S. James, and A. J. Davison, ''Sim-to-real reinforcement learning for deformable object manipulation,'' in Proc. 2nd Annu. Conf. Robot Learn. (CoRL) , Z rich, Switzerland, Oct. 2018, pp. 734 743. [Online]. Available: https://proceedings.mlr.press/v87/matas18a.html - source
Section 5.3 Sim-to-real transfer (p. 8) states that across levels of domain randomisation, 'heavy randomisation can be detrimental to learning', and that sampling texture colours uniformly over all colours made the agent's post-transfer performance significantly worse. This is exactly what the citing sentence attributes, including the transfer framing (worse performance after the sim-to-real transfer). (LLM, conf=0.90)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
33 more claims · 5 with their cited sources checked
Create a free account to see all 111 claims and to verify the 12 references still open.
- It may state two separate claims as one. 37
- Probably not a substantial claim. 57
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ V. CONCLUSION AND OPEN CHALLENGES · p. 14
“When performances of the controller in the simulator and in real robot are different, a RG exists and the controller is not sim-to-real transferable.”
§ III. REINFORCEMENT LEARNING IN ROBOTICS · p. 5
“Note that, in general, it is not required that the controller behaves identically when applied to E and E 0 , but, rather, that it is E 0 -toE transferable, i.e., sim-to-real transferable. In practice, this requirement translates to a (subjectively) properly bounded RG.”
Citations
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357 · cited 6×
“We categorise the different TL approaches based on: • the kind of information passed through u , weights of an image processing net, state-control input pairs, policy parameters or even the policy itself); • the transfer timing of u ( k ) exch , either continuous (C-TL) or one shot (1-TL), the latter generally occurri…”
MethodInfluentialReinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · cited 5×
“RL approaches can be categorised in three main categories [1]:”
BackgroundMethodInfluentialReinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Image-to-Image Translation with Conditional Adversarial Networks
Isola, P., Zhu, J.-Y., Zhou, T., & Efros, A. A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.632 · Cited by 22,930 · cited 1×
“The map is learnt by using an image-conditioned Generative Adversarial Network (cGAN) [98], able to map an image of a domain D into an adapted image of the canonical domain Dc; i.”
MethodLabaca-Castro, R. (2023) · Machine Learning under Malware Attack · doi:10.1007/978-3-658-40442-0_9 · Cited by 19,272
GAN(Generative Adversarial Nets)
(2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,270
Proximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625
Explaining and Harnessing Adversarial Examples
Goodfellow, I. J., Shlens, J., & Szegedy, C. (2014) · arXiv · doi:10.48550/arxiv.1412.6572 · Cited by 10,381 · cited 2×
“The agent uses a curriculum learning approach [91], in which ξ (cid:48) , υ (cid:48) ( k ) , and ζ (cid:48) ( k ) alternately assume the form of isometrically scaled versions of Fast Gradient Sign Method (FGSM) [92].”
MethodIntriguing properties of neural networks
Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., & Fergus, R. (2013) · arXiv · doi:10.48550/arxiv.1312.6199 · Cited by 8,143 · cited 1×
“The choice of adversarial perturbations is motivated by the fact that by employing them, the resulting models are likely to generalise well [93].”
BackgroundAsynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132
Research works printed but not linked: 10
- tedrake, 2005, Learning to walk in 20 minutes, Proc 14th Yale Workshop Adapt Learn Syst, 95585, 1412
- bertsekas, 2019, REINFORCEMENT LEARNING AND OPTIMAL CONTROL
- coumans, 2016, Pybullet a python module for physics simulation for games robotics and machine learning
- coumans, 2016, Bullet Physics SDK
- 2020, Blender Open Source 3D Modeling Suit
- 2020, Simspark
- 2020, Gazebo-robot simulation made easy
- 2001, Vortex Studio Real-time simulation and visualization software for system-level modeling of mechatronics and mechanical equipment
- brockman, 2016, OpenAI gym, arXiv 1606 01540 [cs]
- 2020, Poppy Project Open Source Robotics Platform
Cited by
Experiment-free exoskeleton assistance via learning in simulation
Luo, S., Jiang, M., Zhang, S., Zhu, J., Yu, S., Dominguez Silva, I., Wang, T., Rouse, E., Zhou, B., Yuk, H., et al. (2024) · Nature · doi:10.1038/s41586-024-07382-4 · Cited by 166
Hu, X., Li, S., Huang, T., Tang, B., Huai, R., & Chen, L. (2023) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2023.3312777 · Cited by 240
“At the same time, it solves two problems of traditional simulators [88], [89]: 1) simulation testing is not equivalent to real end-to-end testing, and 2) factors such as weather, climate, and lighting are not fully covered.”
Backgrounddel Real, A. J., Andreiana, D. S., Roldán, Á. O., Bustos, A. H., & Acevedo, L. (2022) · Applied Sciences · doi:10.3390/app122312377 · Cited by 128
“…are aggregating demonstration data to the experience of the agent in a model-based RL algorithm to learn better models; including scripted policies to initialise the training, such as in QT-Opt [113] and reward shaping provides additional guidance to exploration, boosting the learning process.”
MethodLee, H., Jeon, J., Lee, D., Park, C., Kim, J., & Lee, D. (2023) · Automation in Construction · doi:10.1016/j.autcon.2023.105060 · Cited by 57
Sim-to-real transfer and reality gap modeling in model predictive control for autonomous driving
Daza, I. G., Izquierdo, R., Martínez, L. M., Benderius, O., & Llorca, D. F. (2022) · Applied Intelligence · doi:10.1007/s10489-022-04148-1 · Cited by 58
“[13] conducted a recent survey about closing the reality gap and reduce the sim-to-real transferability when developing a robot controller based on reinforcement learning.”
Background
Metadata
- Authors
- Salvato, E., Fenu, G., Medvet, E., & Pellegrino, F. A.
- Affiliations
- University of Trieste (IT)
- Venue
- IEEE Access
- Published
- 2021
- Volume
- 9
- Pages
- 153171-153187
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W3212580602
- Cited by
- 200
- References
- 173
- Citations by year
- 2026: 45 · 2025: 51 · 2024: 53 · 2023: 39 · 2022: 12
- FWCI
- 9.15
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- IEEE Access (publishedVersion, cc-by)
- ArTS Archivio della ricerca di Trieste (University of Trieste https://www.units.it/) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Italian Ministry for Research in the framework of the 2017 Program for Research Projects of National Interest (2017YKXYXJ)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- sim-to-real transfer, reality gap, robot control, reinforcement learning, robot simulator, robot dynamics modeling
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Crossing the Reality Gap: A Survey on Sim-to-Real Transferability of Robot Controllers in Reinforcement Learning. https://app.cochat.ai/papers/10.1109/access.2021.3126658
- The paper
- Salvato, E., Fenu, G., Medvet, E., & Pellegrino, F. A. (2021). Crossing the Reality Gap: A Survey on Sim-to-Real Transferability of Robot Controllers in Reinforcement Learning. IEEE Access. https://doi.org/10.1109/access.2021.3126658
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.