Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 132 of the 173 references it cites were found in a registry or an index; 30 of the 71 sources its claims cite were checked for support.

Crossing the Reality Gap: A Survey on Sim-to-Real Transferability of Robot Controllers in Reinforcement Learning

Open access

Abstract

The growing demand for robots able to act autonomously in complex scenarios has widely accelerated the introduction of Reinforcement Learning (RL) in robots control applications. However, the trial and error intrinsic nature of RL may result in long training time on real robots and, moreover, it may lead to dangerous outcomes. While simulators are useful tools to accelerate RL training and to ensure safety, they often are provided only with an approximated model of robot dynamics and of its interaction with the surrounding environment, thus resulting in what is called the reality gap (RG): a mismatch of simulated and real control-law performances caused by the inaccurate representation of the real environment in simulation. The most undesirable result occurs when the controller learnt in simulation fails the task on the real robot, thus resulting in an unsuccessful sim-to-real transfer. The goal of the present survey is threefold: (1) to identify the main approaches to face the RG problem in the context of robot control with RL, (2) to point out their shortcomings, and (3) to outline new potential research areas.

Overview

CoChat Summary

TL;DR

This survey reviews sim-to-real transferability of RL robot controllers and categorizes reality-gap approaches into domain randomisation, adversarial RL, and transfer learning. It finds no task-independent solution or standard reality-gap metric, and notes many methods lack real-robot validation.

Key findings

  • The survey identifies three main categories for addressing the reality gap: domain randomisation (DR), adversarial RL (ARL), and transfer learning (TL). Across the surveyed articles, 11 use DR, 2 use ARL, and 7 use TL (Tables 2–5).
  • DR and ARL treat sim-to-real transferability as robustness to model perturbations, but the paper argues that robustness to chosen disturbances does not guarantee transferability if those disturbances do not correctly represent simulator inaccuracies.
  • TL approaches appear promising because they involve interaction with the real system during training, but they often lack robustness assessment and may be inefficient due to two successive or simultaneous training phases.
  • Open challenges include the absence of a general task-independent approach, the need for an index to reveal and quantify the reality gap, and the sample efficiency of RL-based methods.

Methods

  • Literature survey: included articles had to explicitly address the reality gap, deal with robot control applications, and employ RL techniques; the survey then categorizes them into DR, ARL, and TL.
  • The paper provides a formal RL/control framework and summarizes each surveyed article by algorithm, task, simulator, and whether evaluation was sim-to-sim and/or sim-to-real.
  • Qualitative synthesis; no primary experiments or quantitative meta-analysis are conducted.

Limitations

  • Stated: task diversity and the lack of a common theoretical framework prevent systematic comparison; many proposed approaches were not experimentally evaluated in an actual sim-to-real scenario; each approach appears tailored to a specific task; a general task-independent approach is still missing; sample efficiency remains a significant open problem.
  • Evident: the survey does not report a systematic search protocol, databases, or quantitative meta-analysis, so its categorization and counts depend on the selected articles.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1109/access.2021.3126658

Cites

132 of 132 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

18 Valid · 11 Not valid · 38 Not checked · 44 Own results and interpretations

33 more claims · 5 with their cited sources checked

Create a free account to see all 111 claims and to verify the 12 references still open.

See all claims — free account
  • It may state two separate claims as one. 37
  • Probably not a substantial claim. 57

Create a free account to see these claims.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

1 of 1

Citations

Cites

Show all 173 references

Research works printed but not linked: 10

  1. tedrake, 2005, Learning to walk in 20 minutes, Proc 14th Yale Workshop Adapt Learn Syst, 95585, 1412
  2. bertsekas, 2019, REINFORCEMENT LEARNING AND OPTIMAL CONTROL
  3. coumans, 2016, Pybullet a python module for physics simulation for games robotics and machine learning
  4. coumans, 2016, Bullet Physics SDK
  5. 2020, Blender Open Source 3D Modeling Suit
  6. 2020, Simspark
  7. 2020, Gazebo-robot simulation made easy
  8. 2001, Vortex Studio Real-time simulation and visualization software for system-level modeling of mechatronics and mechanical equipment
  9. brockman, 2016, OpenAI gym, arXiv 1606 01540 [cs]
  10. 2020, Poppy Project Open Source Robotics Platform

Cited by

See all 200

Metadata

Authors
Salvato, E., Fenu, G., Medvet, E., & Pellegrino, F. A.
Affiliations
University of Trieste (IT)
Venue
IEEE Access
Published
2021
Volume
9
Pages
153171-153187
ISSN
2169-3536 (ISSN-L)
OpenAlex
W3212580602
Cited by
200
References
173
Citations by year
2026: 45 · 2025: 51 · 2024: 53 · 2023: 39 · 2022: 12
FWCI
9.15
Type
journal article
Language
en
Open access
Locations
  • IEEE Access (publishedVersion, cc-by)
  • ArTS Archivio della ricerca di Trieste (University of Trieste https://www.units.it/) (submittedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
Italian Ministry for Research in the framework of the 2017 Program for Research Projects of National Interest (2017YKXYXJ)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
  • Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
Keywords
sim-to-real transfer, reality gap, robot control, reinforcement learning, robot simulator, robot dynamics modeling

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Crossing the Reality Gap: A Survey on Sim-to-Real Transferability of Robot Controllers in Reinforcement Learning. https://app.cochat.ai/papers/10.1109/access.2021.3126658
The paper
Salvato, E., Fenu, G., Medvet, E., & Pellegrino, F. A. (2021). Crossing the Reality Gap: A Survey on Sim-to-Real Transferability of Robot Controllers in Reinforcement Learning. IEEE Access. https://doi.org/10.1109/access.2021.3126658

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.