Crossing the Reality Gap: A Survey on Sim-to-Real Transferability of Robot Controllers in Reinforcement Learning
Open accessThe DOI matches its Crossref record; 133 of the 173 references it cites were found in a registry or an index; 30 of the 71 sources its claims cite were checked for support.
Abstract
The growing demand for robots able to act autonomously in complex scenarios has widely accelerated the introduction of Reinforcement Learning (RL) in robots control applications. However, the trial and error intrinsic nature of RL may result in long training time on real robots and, moreover, it may lead to dangerous outcomes. While simulators are useful tools to accelerate RL training and to ensure safety, they often are provided only with an approximated model of robot dynamics and of its interaction with the surrounding environment, thus resulting in what is called the reality gap (RG): a mismatch of simulated and real control-law performances caused by the inaccurate representation of the real environment in simulation. The most undesirable result occurs when the controller learnt in simulation fails the task on the real robot, thus resulting in an unsuccessful sim-to-real transfer. The goal of the present survey is threefold: (1) to identify the main approaches to face the RG problem in the context of robot control with RL, (2) to point out their shortcomings, and (3) to outline new potential research areas.
Overview
CoChat Summary
TL;DR
This survey reviews sim-to-real transferability of RL robot controllers and categorizes reality-gap approaches into domain randomisation, adversarial RL, and transfer learning. It finds no task-independent solution or standard reality-gap metric, and notes many methods lack real-robot validation.
Key findings
- The survey identifies three main categories for addressing the reality gap: domain randomisation (DR), adversarial RL (ARL), and transfer learning (TL). Across the surveyed articles, 11 use DR, 2 use ARL, and 7 use TL (Tables 2–5).
- DR and ARL treat sim-to-real transferability as robustness to model perturbations, but the paper argues that robustness to chosen disturbances does not guarantee transferability if those disturbances do not correctly represent simulator inaccuracies.
- TL approaches appear promising because they involve interaction with the real system during training, but they often lack robustness assessment and may be inefficient due to two successive or simultaneous training phases.
- Open challenges include the absence of a general task-independent approach, the need for an index to reveal and quantify the reality gap, and the sample efficiency of RL-based methods.
Methods
- Literature survey: included articles had to explicitly address the reality gap, deal with robot control applications, and employ RL techniques; the survey then categorizes them into DR, ARL, and TL.
- The paper provides a formal RL/control framework and summarizes each surveyed article by algorithm, task, simulator, and whether evaluation was sim-to-sim and/or sim-to-real.
- Qualitative synthesis; no primary experiments or quantitative meta-analysis are conducted.
Limitations
- Stated: task diversity and the lack of a common theoretical framework prevent systematic comparison; many proposed approaches were not experimentally evaluated in an actual sim-to-real scenario; each approach appears tailored to a specific task; a general task-independent approach is still missing; sample efficiency remains a significant open problem.
- Evident: the survey does not report a systematic search protocol, databases, or quantitative meta-analysis, so its categorization and counts depend on the selected articles.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
18 Valid · 11 Not valid · 38 Not checked · 44 Own results and interpretations
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“The analysed task is a Marble maze game driven by a robotic arm and the results show that there is a trade-off between controller robustness and ne-tuning steps.”
What the sources say
- Disputedcites [56] [56] J. Van Baar, A. Sullivan, R. Cordorel, D. Jha, D. Romeres, and D. Nikovski, ''Sim-to-real transfer learning using robusti ed controllers in robotic tasks involving complex dynamics,'' in Proc. Int. Conf. Robot. Automat. (ICRA) , May 2019, pp. 6001 6007. <!-- image --> IEEEAccess" - sourceExperimental — accuracy not yet measured
Section 5.4 (p. 8) describes a 3D maze-navigation game controlled through movement and looking actions, not a marble maze driven by a robotic arm. The cited paper discusses transfer learning and finetuning, but the passages do not report the claimed trade-off between controller robustness and fine-tuning steps. (LLM, conf=0.84)
Checked by a reasoning model — may be wrong.
plainstates an associationSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“However, the authors assume that, when the consequences of a control input applied in simulation differ from those applied on the real robot, real observations should match simulated ones, and that is not always true.”
What the sources say
- Disputedcites [49] [49] P. Christiano, Z. Shah, I. Mordatch, J. Schneider, T. Blackwell, J. Tobin, P. Abbeel, and W. Zaremba, ''Transfer from simulation to real world through learning deep inverse dynamics model,'' 2016, arXiv:1610.03518 . - sourceExperimental — accuracy not yet measured
The Experiments section (p. 4) says that visual discrepancies between simulated and real observations are obvious, rather than assuming they should match. The Introduction (p. 2) describes training a new network on the real robot after training in simulation. The paper does not make the claimed assumption about observations when control inputs have different consequences. (LLM, conf=0.83)
Checked by a reasoning model — may be wrong.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“On the contrary, their evaluations have shown that controllers learnt in simulation with large observation space lead to bad results when transferred to real robot.”
What the sources say
- Partly supportedcites [54] [54] J. Tan, T. Zhang, E. Coumans, A. Iscen, Y. Bai, D. Hafner, S. Bohez, and V. Vanhoucke, ''Sim-to-real: Learning agile locomotion for quadruped robots,'' in Proceedings of Robotics: Science and Systems . Pittsburgh, PA, USA: RSS Foundation, Jun. 2018, pp. 10 20. - sourceExperimental — accuracy not yet measured
The ablation study (4 Experiments, pp. 7–8) found that a baseline controller trained in simulation failed the full task on the real robot. But it attributes that failure to the absence of domain randomisation, not to a large observation space; the fully randomised controller completed all standard real-world trials. (LLM, conf=0.85)
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“In particular, when the cGAN during training is fed with images sampled only from non-successful trajectories, the nal controller results in an unsatisfactory transfer.”
What the sources say
- Could not verifycites [46] [46] S. James, P. Wohlhart, M. Kalakrishnan, D. Kalashnikov, A. Irpan, J. Ibarz, S. Levine, R. Hadsell, and K. Bousmalis, ''Sim-to-real via sim-tosim: Data-ef cient robotic grasping via randomized-to-canonical adaptation networks,'' in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. , Jun. 2019, pp. 12627 12637. - sourceExperimental — accuracy not yet measured
Section 4.1 (p. 4) says the cGAN’s training images were sampled from QT-Opt trajectories, but does not describe training it only on unsuccessful trajectories. Section 5.2 (p. 7) reports transfer performance for other training approaches, not the claimed unsuccessful-trajectory condition. Searches for failed or unsuccessful trajectory sampling found no result addressing its effect on transfer. (LLM, conf=0.75)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
Every source the paper cites for this claim was checked and supports it.
In the paper
“Besides, they show that heavy randomisation can lead to unsuccessful transfers.”
What the sources say
- Supportedcites [45] [45] J. Matas, S. James, and A. J. Davison, ''Sim-to-real reinforcement learning for deformable object manipulation,'' in Proc. 2nd Annu. Conf. Robot Learn. (CoRL) , Z rich, Switzerland, Oct. 2018, pp. 734 743. [Online]. Available: https://proceedings.mlr.press/v87/matas18a.html - source
Section 5.3 Sim-to-real transfer (p. 8) states that across levels of domain randomisation, 'heavy randomisation can be detrimental to learning', and that sampling texture colours uniformly over all colours made the agent's post-transfer performance significantly worse. This is exactly what the citing sentence attributes, including the transfer framing (worse performance after the sim-to-real transfer). (LLM, conf=0.90)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
33 more claims · 5 with their cited sources checked
Create a free account to see all 111 claims and to verify the 12 references still open.
- It may state two separate claims as one. 37
- Probably not a substantial claim. 57
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ V. CONCLUSION AND OPEN CHALLENGES · p. 14
“When performances of the controller in the simulator and in real robot are different, a RG exists and the controller is not sim-to-real transferable.”
§ III. REINFORCEMENT LEARNING IN ROBOTICS · p. 5
“Note that, in general, it is not required that the controller behaves identically when applied to E and E 0 , but, rather, that it is E 0 -toE transferable, i.e., sim-to-real transferable. In practice, this requirement translates to a (subjectively) properly bounded RG.”
Citations
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357 · cited 6×
“We categorise the different TL approaches based on: • the kind of information passed through u , weights of an image processing net, state-control input pairs, policy parameters or even the policy itself); • the transfer timing of u ( k ) exch , either continuous (C-TL) or one shot (1-TL), the latter generally occurri…”
MethodInfluentialReinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · cited 5×
“RL approaches can be categorised in three main categories [1]:”
BackgroundMethodInfluentialReinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Image-to-Image Translation with Conditional Adversarial Networks
Isola, P., Zhu, J.-Y., Zhou, T., & Efros, A. A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.632 · Cited by 22,930 · cited 1×
“The map is learnt by using an image-conditioned Generative Adversarial Network (cGAN) [98], able to map an image of a domain D into an adapted image of the canonical domain Dc; i.”
MethodLabaca-Castro, R. (2023) · Machine Learning under Malware Attack · doi:10.1007/978-3-658-40442-0_9 · Cited by 19,272
GAN(Generative Adversarial Nets)
(2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,270
Proximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625
Explaining and Harnessing Adversarial Examples
Goodfellow, I. J., Shlens, J., & Szegedy, C. (2014) · arXiv · doi:10.48550/arxiv.1412.6572 · Cited by 10,381 · cited 2×
“The agent uses a curriculum learning approach [91], in which ξ (cid:48) , υ (cid:48) ( k ) , and ζ (cid:48) ( k ) alternately assume the form of isometrically scaled versions of Fast Gradient Sign Method (FGSM) [92].”
MethodIntriguing properties of neural networks
Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., & Fergus, R. (2013) · arXiv · doi:10.48550/arxiv.1312.6199 · Cited by 8,143 · cited 1×
“The choice of adversarial perturbations is motivated by the fact that by employing them, the resulting models are likely to generalise well [93].”
BackgroundAsynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132 · cited 1×
“[56] shows the benefits of using DR and Asynchronous Actor Critic Agent (A3C) algorithm [86] (on-policy) for learning the controller, with respect to not using DR.”
Method
Research works printed but not linked: 11
- [32] H. Durrant-Whyte, N. Roy, and P. Abbeel, Learning to Control a LowCost Manipulator Using Data-Ef cient Reinforcement Learning . Cambridge, MA, USA: MIT Press, 2012, pp. 57 64. [Online]. Available: https://ieeexplore.ieee.org/document/6301026 -
- [35] R. Tedrake, T. W. Zhang, and H. S. Seung, ''Learning to walk in 20 minutes,'' in Proc. 14th Yale Workshop Adapt. Learn. Syst. , Beijing, China, vol. 95585, 2005, pp. 1412 1939. <!-- image --> IEEEAccess" -
- [25] D. P. Bertsekas, Reinforcement Learning and Optimal Control . Belmont, MA, USA: Athena Scienti c, 2019. -
- [71] E. Coumans and Y. Bai. (2016). Pybullet, a Python Module for Physics Simulation for Games, Robotics and Machine Learning . [Online]. Available: https://pybullet.org -
- [70] E. Coumans and Y. Bai. (2016 2019). Bullet Physics SDK . [Online]. Available: https://bulletphysics.org -
- [65] Blender Community. (2020). Blender: Open Source 3D Modeling Suit . [Online]. Available: https://www.blender.org -
- [67] Open Source Robotics Foundation. (2020). GAZEBO: Robot Simulation Made Easy . [Online]. Available: https://www.gazebosim.org/ -
- [69] CM Labs. (2001). Vortex Studio Real-Time Simulation and Visualization Software for System-Level Modeling of Mechatronics and Mechanical Equipment . [Online]. Available: https://www.cm-labs.com/vortex-studio/ -
- [91] Y. Bengio, J. Louradour, R. Collobert, and J. Weston, ''Curriculum learning,'' in Proc. 26th Annu. Int. Conf. Mach. Learn. , 2009, pp. 41 48. -
- [113] Poppy Station. (2020). Poppy Project: Open Source Robotics Platform . [Online]. Available: https://www.poppy-project.org/en/ -
- [116] A. Gupta, R. Mendonca, Y. Liu, P. Abbeel, and S. Levine, ''Metareinforcement learning of structured exploration strategies,'' in Proc. Adv. Neural Inf. Process. Syst. , 2018, pp. 5302 5311. -
Cited by
Items 101–198 of 198
Learning to Navigate Through Reinforcement Across the Sim2Real GapPreprint
Azzam, R., Chehadeh, M. S., Hay, O. A., Boiko, I. M., & Zweiri, Y. (2022) · doi:10.36227/techrxiv.20138960 · Cited by 2
Tubeuf, C., Birkelbach, F., Maly, A., Krause, M., & Hofmann, R. (2023) · International Conference on Efficiency, Cost, Optimization, Simulation and Environmental Impact of Energy Systems (ECOS) · doi:10.52202/069564-0289 · Cited by 4
Platform-portable reinforcement learning methods to localize underwater targets
Rusiñol, I. M., Martín, M., O’Reilly, T., et al. (2023) · doi:10.1109/oceanslimerick52467.2023.10244488 · Cited by 1
“, sim-to-real transferability), which is of special importance in robot controllers [1].”
BackgroundEvolutionary Machine Learning in Robotics
Medvet, E., Nadizar, G., Pigozzi, F., & Salvato, E. (2023) · Genetic and evolutionary computation · doi:10.1007/978-981-99-3814-8_23 · Cited by 3
Self-citationBasumatary, H., Adhar, D., & Hazarika, S. M. (2024) · Robotica · doi:10.1017/s0263574724001838 · Cited by 2
“DR acts as a bridge between simulation and reality, akin to robust control in control theory, by designing controllers resilient to parameter variations and noise [25].”
Lee, X. Y., Wang, H., KATSUMATA, D., et al. (2024) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata62323.2024.10825047 · Cited by 1
“Nevertheless, developing a simulator is also often costly, and RL policies trained in simulator environments often suffer from the sim-to-real gap Zhao et al. [2020], Salvato et al. [2021].”
Contact-Based Digital Twins Modeling for Reinforcement Learning of Robotic Disassembly Operations
Qu, M., Pham, D. T., Lan, F., Wu, Z., Zang, Y., Zhang, Y., & Wang, Y. (2025) · IEEE Transactions on Industrial Cyber-Physical Systems · doi:10.1109/ticps.2025.3589351 · Cited by 1
“However, the sim-to-real transfer approach has been known to have a critical issue known as the “reality gap”, which refers to a decrease in performance when implementing the learned policies in real-world settings [23].”
Yao, Y., Wentao, H., Gu, C., Du, J., Tan, F., Zhu, Z., & Lu, J. (2025) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros60139.2025.11246492 · Cited by 8
Sarhan, A. Y., Abdullah, O., Musri, K. T., & Al-Hraishawi, H. (2026) · IEEE Access · doi:10.1109/access.2026.3652482 · Cited by 5
“A policy that performs well in a controlled simulation may fail when confronted with complex real-world channel dynamics, hardware imperfections, or unexpected adversarial behaviors [26].”
Software-defined self-learning control system for industrial robots by using reinforcement learning
Moon, J., Kim, M., Lee, T., & Um, J. (2026) · Scientific Reports · doi:10.1038/s41598-025-28904-8 · Cited by 2
“This ensures that the control policies learned in the virtual environment can be correctly applied to real robotic systems 41 .”
A Real-Time Digital Twin Framework for the TIAGo Service Robot
Herrmann, M., Hempel, T., & Al-Hamadi, A. K. (2025) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc58881.2025.11343553 · Cited by 1
“By integrating the remaining sensors, such as LiDAR and sonar, the sim-to-real gap can be reduced and later overcome by transfer learning or domain randomization methods [4].”
Panichella, S., Khatiri, S., & Birchler, C. (2026) · Springer eBooks · doi:10.1007/978-981-95-1786-2_6 · Cited by 1
Towards Torque-Driven Reinforcement Learning for Quadruped Locomotion
Dowdy, J., & Vaz, J. C. (2026) · IEEE/SICE International Symposium on System Integration (SII) · doi:10.1109/sii64115.2026.11404550 · Cited by 1
“One of the significant issues in simulated environment training is the so-called sim-to-real gap [8].”
Tripura, T., & Chakraborty, S. (2026) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2026.114226 · Cited by 1
Lai, K., Wang, X., Li, Y., Guan, H., Zhang, Y., & Xia, Y. (2026) · Engineering Structures · doi:10.1016/j.engstruct.2026.122740 · Cited by 1
Tubeuf, C., Bernhardt, T., Birkelbach, F., Hofmann, R., & Maly, A. (2026) · Energy and AI · doi:10.1016/j.egyai.2026.100755 · Cited by 1
Synthetic Data-Driven Exoskeleton Control via Contralateral Gait Fusion for Variable-Speed Walking
Shi, J., Zhu, H., Yang, Y., Liu, B., & Wang, X. (2026) · Biomimetics · doi:10.3390/biomimetics11050319 · Cited by 1
“Relying on reward shaping and domain randomization, the trained policy could overcome the sim2real gaps [28–31].”
PhenoMARL-Fert: Safe Multi-Agent DRL for Real-Time Irrigation and Nutrient Control
Kumar, U. N., Narayanan, T., & Kumar, B. (2026) · IEEE International Conference on Computing, Power and Communication Technologies (IC2PCT) · doi:10.1109/ic2pct68894.2026.11584422 · Cited by 1
“…addressed individually from the fertigation system, the PhenoMARL-Fert approach is innovative in the sense that it proposes the use of the cooperative multi-agent approach, the conditioning with phenology, and the feasibility projection during inference for reducing the unsafe actions [1] – [16].”
Operational evidence standards for machine learning in wastewater treatment
Jiang, S., Yang, Y., Zhang, K., Mao, Y., & Cheng, X. (2026) · npj Clean Water · doi:10.1038/s41545-026-00610-6 · Cited by 2
NeuralFloors: Conditional Street-Level Scene Generation From BEV Semantic Maps via Neural Fields
Muşat, V., Martini, D. D., Gadd, M., & Newman, P. M. (2024) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3356793 · Cited by 2
“However,assetsandscenesrequiretediousand time-consumingmanualcreationandthesimulatedworldsuf-fersfromthesim-to-realgap[12].”
Modeling and Simulation Technologies for Effective Multi-agent Research
Schaefer, K. E., Brewer, R. W., Wickwire, J., Scalise, R., & Kessens, C. C. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-61044-8_7 · Cited by 1
Kunbum, P., & Tsuchiya, T. (2024) · IEEE Journal of Indoor and Seamless Positioning and Navigation · doi:10.1109/jispin.2024.3433671 · Cited by 1
“Using the robot model, we present a simulation of exploring, map-building, and navigating a typical office, along with the corresponding experimental results [4].”
From United Steel to Waymo: industrializing simulation
Hind, S. (2024) · AI & Society · doi:10.1007/s00146-024-02051-6 · Cited by 3
“The statistical rules that are meant to govern ‘ sim-to-real transfer’ (Salvato et al. 2021) rely on establishing the norms of distribution in any given context—from ‘realistic’ driving scenarios (Wayve 2023) to ‘representative’ medical image data (Guo et al. 2024).”
Robust Proximal Adversarial Reinforcement Learning Under Model Mismatch
Zhai, P., Wei, X., Hou, T., Ji, X., Dong, Z., Yi, J., & Zhang, L. (2024) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3472348 · Cited by 3
“These gaps can lead to significant degradation in policy performance and restrict the application of RL in fields with high requirements for reliability and safety, leading researchers to discuss and emphasize the robustness of RL [5].”
BackgroundTowards a Unified Framework for Software-Hardware Integration in Evolutionary Robotics
Buchanan, E., Goff, L. K. L., Hale, M. F., Hart, E., Eiben, A. E., Carlo, M. D., Angus, M., Woolley, R., Timmis, J., Winfield, A. F. T., et al. (2024) · Robotics · doi:10.3390/robotics13110157 · Cited by 1
“Numerous approaches have been proposed in the literature to address the reality gap [20,21,23,28–31].”
Advances in Autonomous Robotic Grasping: An Overview of Reinforcement Learning Approaches
Farkas, B. K., Galambos, P., & Széll, K. (2024) · IEEE International Symposium on Logistics and Industrial Informatics (LINDI) · doi:10.1109/lindi63813.2024.10820398 · Cited by 2
Padalkar, A., Stulp, F., Neumann, G. M., & Silvério, J. (2025) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2025.3566599 · Cited by 7
Safe Resetless Reinforcement Learning: Enhancing Training Autonomy with Risk-Averse Agents
Gottwald, T., Schier, M., & Rosenhahn, B. (2024) · ECCV Workshops · doi:10.1007/978-3-031-92591-7_7 · Cited by 3
“However, most RL agents are trained in simulated environments [4,6,12,20,28,34], which are unable to capture the dynamics and the full complexity of the real world scenario that they are trying to simulate [32].”
BackgroundOn the Impact of Pretraining with Simulated Data on Anomaly Detection in CPS - A Case Study
Jaufmann, R., Widulle, N., Ehrhardt, J., Vranješ, D., & Niggemann, O. (2025) · IEEE International Conference on Emerging Technologies and Factory Automation (ETFA) · doi:10.1109/etfa65518.2025.11205786 · Cited by 1
“This phenomenon has been described as the sim-to-real gap [6].”
InfluentialMa, S., Flanigan, K. A., & Berges, M. E. (2025) · ASCE International Conference on Computing in Civil Engineering (i3CE) · doi:10.1061/9780784486122.057 · Cited by 3
Padalkar, A., Quere, G., Raffin, A., Silvério, J., & Stulp, F. (2023) · Research Square · doi:10.21203/rs.3.rs-3289569/v1 · Cited by 1
A Powered Floor System with Integrated Robot Localization
Seriani, S., Carrato, S., Medvet, E., et al. (2023) · Electronics · doi:10.3390/electronics12010234
Self-citationA perspective on large-scale simulation as an enabler for novel biorobotics applications
Angelidis, E. (2023) · Frontiers in Robotics and AI · doi:10.3389/frobt.2023.1102286 · Cited by 1
“Several factors contribute to the simulation-to-real (sim2real) gap (Salvato et al., 2021).”
BackgroundMotion Planning of 2D Wheeled Mobile Robot using Reinforcement Learning
Yashina, M. V., & Shein, V. A. (2023) · Intelligent Technologies and Electronic Devices in Vehicle and Road Transport Complex (TIRVED) · doi:10.1109/tirved58506.2023.10332610
Pietrosanti, G. M., Nadizar, G., Pigozzi, F., & Medvet, E. (2023) · IEEE Access · doi:10.1109/access.2023.3342912 · Cited by 1
“As already observed in other studies, when working at full control frequency, optimization might end up finding behaviors that are unintuitive and would not be practicable in the real-world—the so called reality gap problem [68], [69].”
BackgroundSelf-citationFernádez, Y. G., Retureta, J. G. L., Franco, R. A., et al. (2023) · doi:10.1109/comrob60035.2023.10349778
“This is often accompanied by a side effect known as the Reality Gap [16], which consists of differences between simulated behavior and that observed in the real environment.”
BackgroundHow does Simulation-based Testing for Self-driving Cars match Human Perception?Preprint
Birchler, C., Mohammed, T. K., Rani, P., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2401.14736
“To that end, in our research, we investigated when and why the safety metrics of simulation-based test cases of SDCs match human perception.”
The Effect of Training Schedules on Morphological Robustness and Generalization
Barba, E., Yaman, A., & Iacca, G. (2024) · Genetic and Evolutionary Computation Conference Companion (GECCO Companion) · doi:10.1145/3638530.3664138
“Robustness and generalizability can also be vital for reducing the simulation-to-reality gap (reality gap for short), which refers to the mismatch between simulated and real-world environments [30].”
Basumatary, H., Sarkar, T. T., & Hazarika, S. M. (2024) · International Conference on Methods and Models in Automation and Robotics · doi:10.1109/mmar62187.2024.10680774
“However, when uncertainties arise , the RL agent may fail.”
BackgroundJi, C.-H., Lim, G., Han, Y., & Moon, S. (2024) · IEEE Access · doi:10.1109/access.2024.3473935
“For this reason, various research has been conducted to achieve stable sim-to-real transfer in RL [7], [8].”
A Cross-domain, Computational Robot Design Flow from Morphology to Fabrication
Chattoraj, A., Lin, R., Martinez, D., et al. (2024) · 2022 IEEE 18th International Conference on Automation Science and Engineering (CASE) · doi:10.1109/case59546.2024.10711390
“Similarly, optimizing hardware configurations alongside control strategies offers a holistic enhancement of the physical robot’s performance capabilities [30], [31].”
BackgroundAn Open-source Sim2Real Approach for Sensor-independent Robot Navigation in a Grid
Abrar, M. M., Mondal, S., & Hickner, M. K. (2024) · International Conference on Robotics and Automation Engineering (ICRAE) · doi:10.1109/icrae64368.2024.10851595 · Cited by 1
“Many studies in the literature introduced various techniques for the effective transfer of simulated results to real hardware [6], [14], [15].”
Dropout MPC: An Ensemble Neural MPC Approach for Systems with Learned Dynamics
Syntakas, S., & Vlachos, K. I. (2025) · IEEE/SICE International Symposium on System Integration (SII) · doi:10.1109/sii59315.2025.10870887 · Cited by 4
“Modern simulators try to tackle the problem of ef cient data gathering, but sim-to-real gap still remains an open problem [4].”
Towards Unified Field-Testing and Monitoring for Safe and Secure Robotic Applications
Stadler, M., Biffl, S., Vierhauser, M., et al. (2025) · doi:10.1109/rose66716.2025.00007
“However, simulations alone are incapable of covering the full spectrum of test scenarios required, and there is still a well-recognized “simulator-to-reality gap” between virtual hardware and how real systems behave in the physical world [8].”
Optimization of Fracture Reduction Robot Controller
An, B., Chen, J., Liu, M., et al. (2024) · International Symposium on Computer Science and Intelligent Control (ISCSIC) · doi:10.1109/iscsic64297.2024.00088
“The optimization search will continue to loop continuously until the requirements for parameter setting are satisfied[17].”
Mehimeh, S., & Tang, X. (2025) · Entropy · doi:10.3390/e27040367 · Cited by 1
“For example, dynamics changes in applications can be seen as a mechanical or an electrical noise in robot sensors, which usually model a Gaussian distribution [7,8], or wind disturbances affecting a drone, where wind typically follows a Weibull distribution [9].”
Evolutionary Robotics: An Efficient Automatic De-Novo Design of RobotsPreprint
Shi, S., Zou, J., & Liu, W. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5255036
Li, H., Luo, M., Luo, W., Li, H., & Cong, S. (2025) · PLoS ONE · doi:10.1371/journal.pone.0324341
“However, the RL methods often simplify the problem, neglecting the dynamics mismatch between pedes-trians and robots [27].”
Cellular plasticity model for self-organized phenotypes in multi-cellular robots
Smith, T. G., Smith, T., Szczecinski, N. S., Yakovenko, S., & Gu, Y. (2025) · npj Robotics · doi:10.1038/s44182-025-00039-y · Cited by 1
“…inward direction of the phase portrait signi fi es a stable non-zero equilibrium. system is stable and maintains biological signi fi cance, the model (23) and (24) must meet these additional conditions: Furthermore, this model displays the fourth cellular plasticity phenomenon, self-regulation,…”
Body-Induced Soft Robotic Locomotion: a Sim-To-Real Approach
Wang, Z. F., Mallikarachchi, S., Arachchige, D. D. K., et al. (2025) · International Conference on Ubiquitous Robots (UR) · doi:10.1109/ur65550.2025.11078082
“By applying sim-to-real techniques such as domain randomization, and characterization, the accuracy between simulation and the real-world can be greatly enhanced [8].”
Reinforcement Learning in Robotic Systems: A Review on Sim-to-Real TransferPreprint
Khapre, S. P., Tiwari, R., & Singh, A. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5387665
Deep Reinforcement Learning for Robotic Grasping: Insights into Learning from Raw Visual Data
Ferrara, J., & Suleiman, W. (2025) · IEEE Workshop on Advanced Robotics and its Social Impacts (ARSO) · doi:10.1109/arso64737.2025.11124980
“[1] One of the key aspects of overcoming this gap is the need for extensive training steps within simulated environments.”
Towards Mitigating Sim2Real Gaps: A Formal Quantitative Approach
Sangeerth, P., Lavaei, A., & Jagtap, P. (2025) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.23919/acc63710.2025.11107534 · Cited by 5
“Several studies, such as [23]–[28], concentrate on bridging the gap between simulation environments and reality within the Sim2Real context.”
Zhang, Z., Ahmad, H. M. S., Sabouni, E., Sun, Y., Huang, F., Li, W.-C., & Miao, F. (2023) · IEEE International Conference on Robotics and Automation · doi:10.1109/icra55743.2025.11128447 · Cited by 9
“Meanwhile, robust RL and robust MARL that only considers to train a policy under state uncertainty or model uncertainty [9], [20], [21], [22], [23] without explicitly considering the safety requirements have been proposed recently.”
Leveraging Analytic Gradients in Provably Safe Reinforcement Learning
Walter, T., Markgraf, H., Külz, J., & Althoff, M. (2025) · IEEE Open Journal of Control Systems · doi:10.1109/ojcsys.2025.3607845
“Otherwise, the unsafeguarded optimisation may converge to a policy that relies on unsafe states or actions.”
Besnard, S., Fadili, J. M., Jurie, F., & Daumont, S. (2025) · IEEE Transactions on Aerospace and Electronic Systems · doi:10.1109/taes.2025.3610819
Learning human-to-robot handovers through 3D scene reconstruction
Wu, Y., Pang, Y. L., Cavallaro, A., & Oh, C. (2025) · IEEE RO-MAN · doi:10.1109/ro-man63969.2025.11217579 · Cited by 1
“However, these methods often require large amounts of data, precise camera calibration, or computationally intensive training [13].”
Enhancing Exploration in Actor-Critic Algorithms: An Approach to Incentivize Plausible Novel States
Banerjee, C., Chen, Z., & Noman, N. (2025) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2025.3637764 · Cited by 1
“However, it faces sim-to-real challenges when applied to real-world systems [34], [35].”
Deep Reinforcement Learning for Autonomous Navigation: Sim-to-Real Transfer on TurtleBots
Nevo, M. D., Turcato, N., Sinigaglia, A., et al. (2025) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2025.12.003
Chenyu, G., & Li, J. (2026) · Mechanisms and machine science · doi:10.1007/978-3-032-01363-7_60
Bi, Z., Zhong, P., & Chen, B. (2026) · Lecture notes in computer science · doi:10.1007/978-981-95-4445-5_23
Mateus, M. G., de Oliveira, G. C., Reichow, L. H. K., Kolling, A. H., Pinheiro, P. M., & Drews, P. (2025) · International Conference on Advanced Robotics (ICAR) · doi:10.1109/icar65334.2025.11338720
“Moreover, with the rapid evolution of artificial intelligence, the generation of realistic synthetic datasets has become a crucial step for training neural network models, as a key for minimizing the sim-to-real gap [6].”
Lazareva, P. A., & Gurko, N. (2026) · doi:10.24108/preprints-3114305
Illy-Net: A GMM Based Performance Evaluation Framework for Sim-to-Real Transfer of RL Policies
Im, S., & Lee, J. (2025) · IEEE International Conference on Intelligent Reality (ICIR) · doi:10.1109/icir68135.2025.11361591
“Salvato et al.[14] analyze multiple techniques for bridging the reality gap and compare their effectiveness.”
Digital Twin Modeling Method Based on the Dynamic Characteristics of Mobile Robots
Ren, B., Wang, S., Zhang, X., et al. (2025) · International Conference on Robotics and Automation Engineering (ICRAE) · doi:10.1109/icrae67496.2025.00013
“Existing simulation platforms, on the other hand, face the "reality gap" problem when migrating from simulated environments to real environments [7].”
Watfa, M. K. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6263107
Asodia, V., Dagda, B., He, Y., et al. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6441775
Wang, Q., Jiang, M., Yu, T., Hu, Y., Dong, Z., Ai, H., Wang, Y., & Búš, P. (2026) · Advanced Engineering Informatics · doi:10.1016/j.aei.2026.104580 · Cited by 1
Optimizing decision-making for human-robot collaboration
Daniel, M. (2022) · theses.fr (ABES) · doi:10.70675/c6fa8163zc855z442bzb6d1z78ba24181b4d · Cited by 1
Learning to Grasp in Robotics with Quality-Diversity Generated Data
Huber, J. B. (2024) · theses.fr (ABES) · doi:10.70675/b98f392dz4df9z4646z8b34z59f447564543
Artificial intelligence in robotics, automation, and predictive maintenance for mechanical systems
Pourrahmani, H. (2026) · Elsevier eBooks · doi:10.1016/b978-0-443-45526-1.00010-6
Swarm Robotics : distributed Online Learning in the realm of Active Matter
Fersula, J. (2023) · theses.fr (ABES) · doi:10.70675/b815b164z178ez4241zb2c8ze110ce204af5
Zhang, X., Li, G., Shokouhi, S., & Thein, M.-W. L. (2026) · Actuators · doi:10.3390/act15050235 · Cited by 1
Cheng, Z., Zhang, Q., & Li, Y. (2025) · China Automation Congress (CAC) · doi:10.1109/cac67268.2025.11486606
“Reference [9] and [10] points out that controllers trained in simulation often fail entirely when transferred to a real robot.”
Guidelines for Safe and Robust Reinforcement Learning: From Definitions to Design
Yamagata, T., & Santos‐Rodríguez, R. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-19099-4_40
Dynamic Grounding Coordination for Sim-to-Real Transfer in Multi-Agent Traffic Control
Wang, Q. (2026) · International Conference on Generative Artificial Intelligence and Information Security (GAIIS) · doi:10.1109/gaiis69281.2026.11519357
“Researchers have explored several categories of solutions [2].”
Hou, L., Wu, Q., Qin, Z., Banerjee, N., Guo, Y., & Laschi, C. (2026) · IEEE International Conference on Soft Robotics (RoboSoft) · doi:10.1109/robosoft67810.2026.11522945 · Cited by 4
Watfa, M. K. (2026) · Applied Soft Computing · doi:10.1016/j.asoc.2026.115566
Sim2Real Policy Transfer in Distributed Systems Using State-Based Potential Games
Yuwono, S., Musthafa, R., Schwung, D., & Schwung, A. (2026) · AI for Engineering · doi:10.3390/aieng1010004
“Policy Transfer in Sim2Real is a huge topic in robotics and machine learning to solve problems related to transferring policies trained in simulated environments to real-world applications [44,47].”
Yuan, T., Huang, T., Lv, X., et al. (2026) · IEEE/ASME Transactions on Mechatronics · doi:10.1109/tmech.2026.3692499
“Given that ASV navigation environments are often highly complex, RL-based methods face sim-to-real challenges [11].”
Xu, W. Q., Liu, Z., Yang, C.-X., Nie, A., & Zhang, H. (2026) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2026.3708974
“…and control delay; accordingly, the field has gradually developed multiple technical routes, including domain randomization, domain adaptation, action transformation, delay-aware modeling, progressive transfer, benchmark-based evaluation, and Real2Sim2Real pipelines [133], [134], [135].”
Murcia, H., & Lacroix, S. (2026) · Precision Agriculture · doi:10.1007/s11119-026-10373-2
“This limita-1 tion reflects the sim-to-real gap in 3D perception, where differences in point density, noise characteristics, and sensor-specific artefacts can affect generalisation (Salvato et al., 2021).”
Kaytan, D. (2026) · Applied Soft Computing · doi:10.1016/j.asoc.2026.115924
Dual variational autoencoders for efficient sim-to-real transfer in low-cost robotic navigation
Díez, Á., & Aznar, F. (2026) · Image and Vision Computing · doi:10.1016/j.imavis.2026.106121
“Similarly, in the scientific literature, the discrepancies specifically be-tween simulation and reality are commonly referred to by the term Reality Gap (RG) [16, 17].”
Yılmaz, T., & Aydoğmuş, Ö. (2026) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2026.105628
Boase, D., Miah, M. S., & Gueaieb, W. (2026) · International Journal of Intelligent Robotics and Applications · doi:10.1007/s41315-026-00560-9
SynCal-Mix: Anchor-Calibrated Synthetic Rare-Event Support for Federated IoT Sensing
Shao, X., & Liu, Z. (2026) · IEEE Internet of Things Journal · doi:10.1109/jiot.2026.3717624
“Yet synthetic support is not automatically beneficial: distributional, semantic, or modality mismatch can cause negative transfer [6].”
Safety-Aware Reinforcement Learning for Lunar Rover Navigation Using Orbital DEMs
Potharaju, A., Ravi, S., Sindoli, M. T., Bellur, P., Rangaswamy, S., & Purohit, M. V. (2026) · IEEE Access · doi:10.1109/access.2026.3717682 · Cited by 1
Baptista, T. A. T. B., Vasques, C. M. A., Castro, P., et al. (2026) · doi:10.3390/engproc2026145008
Sanawar, S., Valerievich, K. V., & Alexandrovich, T. A. (2026) · IEEE International Conference of Young Professionals in Electron Devices and Materials (EDM) · doi:10.1109/edm69524.2026.11632189
“Many studies focus solely on simulation accuracy without validating positioning errors on real robotic platforms, highlighting challenges associated with simulation-to-reality transfer [6].”
Applications of Reinforcement Learning for Autonomous Surgical Robotics: A Systematic Review
Shahid, M., Abdullah, A., Fatima, Z., Feroze, W., Torres-Ruiz, M. J., Saldaña-Pérez, M., Sanchéz-Mejorada, C. G., & Quintero, R. (2026) · Biomimetics · doi:10.3390/biomimetics11080577 · Cited by 2
Raza, M., Nawaz, T., Kumail, Z., Raza, S., Fatima, I., & Li, F.-C. (2026) · Journal of Thermal Analysis and Calorimetry · doi:10.1007/s10973-026-16060-8
Kim, S., Kim, Y., Kim, H., 최동기, & Song, Y.-E. (2026) · IEEE/ASME International Conference on Advanced Intelligent Mechatronics · doi:10.1109/aim65483.2026.11658110
“…conducted to further characterize the resonance-like behavior at 305 mm. Advances in sim-to-real transfer for legged locomotion [17], [18] and analyses of the reality gap [19], [20] suggest that combining domain randomization with system identification can enable deployment of the T-spoke platform.”
Online Real2Sim Calibration of Velocity-Controlled Digital Twins via Reinforcement Learning
Utili, S., Govoni, A., & Palli, G. (2026) · IEEE/ASME International Conference on Advanced Intelligent Mechatronics · doi:10.1109/aim65483.2026.11658146
“…DTs and improve simulator alignment when gradients are unavailable [9], [10] and Sim2Real pipelines often prefer higher-level action spaces for robustness [3], [11], relatively few works have applied RL to achieve accurate, adaptive digital twins with real-time parameter estimation capabilities.”
Jiang, L., Chen, J., Lan, K., et al. (2026) · Biomimetics · doi:10.3390/biomimetics11090610
“The results show that the learned motion patterns can be transferred from simulation to hardware and can produce recognizable coordinated multi-finger piano-playing patterns [20–22].”
InfluentialLee, S., Ju, D., & Lee, Y. S. (2026) · Mathematics · doi:10.3390/math14183319
Baek, J. U., Kim, C. A., Lee, S. H., et al. (2026) · IFIP advances in information and communication technology · doi:10.1007/978-3-032-38614-4_9
Hu, X., Chu, Z., & Shen, X. (2026) · Ocean Engineering · doi:10.1016/j.oceaneng.2026.128492
Metadata
- Authors
- Salvato, E., Fenu, G., Medvet, E., & Pellegrino, F. A.
- Affiliations
- University of Trieste (IT)
- Venue
- IEEE Access
- Published
- 2021
- Volume
- 9
- Pages
- 153171-153187
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W3212580602
- Cited by
- 200
- References
- 173
- Citations by year
- 2026: 45 · 2025: 51 · 2024: 53 · 2023: 39 · 2022: 12
- FWCI
- 9.15
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- IEEE Access (publishedVersion, cc-by)
- ArTS Archivio della ricerca di Trieste (University of Trieste https://www.units.it/) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Italian Ministry for Research in the framework of the 2017 Program for Research Projects of National Interest (2017YKXYXJ)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- sim-to-real transfer, reality gap, robot control, reinforcement learning, robot simulator, robot dynamics modeling
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.