The DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.
Latent Imagination Facilitates Zero-Shot Transfer in Autonomous Racing
Open accessAbstract
World models learn behaviors in a latent imagination space to enhance the sample-efficiency of deep reinforcement learning (RL) algorithms. While learning world models for high-dimensional observations (e.g., pixel inputs) has become practicable on standard RL benchmarks and some games, their effectiveness in real-world robotics applications has not been explored. In this paper, we investigate how such agents generalize to real-world autonomous vehicle control tasks, where advanced model-free deep RL algorithms fail. In particular, we set up a series of time-lap tasks for an F1TENTH racing robot, equipped with a high-dimensional LiDAR sensor, on a set of test tracks with a gradual increase in their complexity. In this continuous-control setting, we show that model-based agents capable of learning in imagination substantially outperform model-free agents with respect to performance, sample efficiency, successful task completion, and generalization. Moreover, we show that the generalization ability of model-based agents strongly depends on the choice of their observation model. We provide extensive empirical evidence for the effectiveness of world models provided with long enough memory horizons in sim2real tasks.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Model mismatches are witnessed on the physical platform, specifically bang-bang steering behavior limiting maximum speed.
“However, when considering deployment on the physical platform, model mismatches are witnessed by the authors, specifically with the appearance of a bang-bang steering behavior, that limits the maximum speed of the car when compared to simulation.”
From RLPP: A Residual Method for Zero-Shot Real-World Autonomous Racing on Scaled Platforms (2025) · cites this paper as [6]
The Sim2Real transfer section reports deployment on a physical car, and the discussion of the sim-to-real gap says the learned policies often resemble bang-bang control. The Learning performance and sample efficiency section describes on-off steering and unnecessarily curvy trajectories. None of these passages says that this behavior reflects a model mismatch or limits the physical car’s maximum speed relative to simulation. (LLM, conf=0.85)
Experimental — accuracy not yet measuredModel-based RL algorithms generalize well to multiple tracks compared to model-free counterparts in simulation.
“[6] presents an interesting RL example, where the Dreamer architecture [7] is used to show that model-based RL algorithms generalize well to multiple tracks when compared to the model-free counterparts in simulation.”
From RLPP: A Residual Method for Zero-Shot Real-World Autonomous Racing on Scaled Platforms (2025) · cites this paper as [6]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Proximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625
Sim-to-Real Transfer of Robotic Control with Dynamics Randomization
Peng, X. B., Andrychowicz, M., Zaremba, W., & Abbeel, P. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8460528 · Cited by 1,309 · cited 1×
“In order to learn more robust policies [15], other works refined the”
BackgroundSim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey
Zhao, W., Queralta, J. P., & Westerlund, T. (2020) · 2020 IEEE Symposium Series on Computational Intelligence (SSCI) · doi:10.1109/ssci47803.2020.9308468 · Cited by 941
Dyna, an Integrated Architecture for Learning, Planning, and Reacting
Sutton, R. S. (1991) · ACM SIGART Bulletin · doi:10.1145/122344.122377 · Cited by 777 · cited 1×
“by either planning or generating new training experience [20].”
BackgroundKendall, A., Hawke, J., Janz, D., Mazur, P., Reda, D., Allen, J.-M., Lam, V.-D., Bewley, A., & Shah, A. (2019) · 2019 International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2019.8793742 · Cited by 694 · cited 1×
“A large body of research made use of camera images and adopt model-free methods [30], [31], [32].”
Background
Flagged first, then by citations
Research works printed but not linked: 56
- 10.1109/ICRA40945.2020.9196608 · DOI
- vorbach, 2021, Causal navigation by continuous-time neural networks, Advances in neural information processing systems, 34
- fuchs, 2020, Super-Human Performance in Gran Turismo Sport Using Deep Reinforcement Learning, ArXiv Preprint
- 10.1109/FBIT.2007.37 · DOI
- 10.1109/ICRA.2018.8460934 · DOI · Paper page
- ross, 2011, A reduction of imitation learning and structured prediction to no-regret online learning, Proceedings of the fourteenth international conference on artificial intelligence and statistics JMLR Workshop and Conference Proceedings, 627
- 10.1016/S1364-6613(99)01327-3 · DOI
- 10.1109/ICRA.2017.7989202 · DOI
- 10.1109/IROS45743.2020.9341021 · DOI
- 10.2307/j.ctvcm4g0s · DOI
- seyde, 2021, Is bang-bang control all you need? solving continuous control with bernoulli policies, Advances in neural information processing systems, 34
- 10.1109/ICRA48506.2021.9561138 · DOI
- 10.1115/1.4023400 · DOI
- 10.1109/TCST.2014.2377777 · DOI
- 10.1109/IROS45743.2020.9341731 · DOI
- hafner, 2019, Dream to control: Learning behaviors by latent imagination, ArXiv
- sutton, 2018, RL An Introduction
- schwarting, 2021, Deep latent competition: Learning to race using visual control policies in latent space, ArXiv Preprint
- deisenroth, 2011, PILCO: A Model-Based and Data-Efficient Approach to Policy Search, Proceedings of the 28th International Conference on Machine Learning ser ICML'11, 465
- 10.1109/LRA.2019.2926677 · DOI
- kabzan, 2019, Amz driverless The full autonomous racing system
- 10.1109/ISIE.2005.1528936 · DOI
- 10.1109/IROS45743.2020.9341702 · DOI
- 0, The Robot Operating System
- 10.1016/j.compstruc.2007.04.028 · DOI
- 10.1016/j.robot.2012.05.021 · DOI
- akiba, 2019, Optuna A next-generation hyperparameter optimization framework
- hasani, 2021, Closed-form continuous-depth models, ArXiv Preprint
- lechner, 2020, Learning long-term dependencies in irregularly-sampled time series, ArXiv Preprint
- 10.1109/IJCNN.2019.8851954 · DOI
- hasani, 0, Liquid time-constant networks, Proceedings of the AAAI Conference on Artificial Intelligence, 35, 7657, 10.1609/aaai.v35i9.16936 · DOI
- 10.1109/70.954762 · DOI
- hasani, 2020, A natural lottery ticket winner: Reinforcement learning with ordinary neural circuits, International Conference on Machine Learning, 4082
- 10.24963/ijcai.2018/820 · DOI
- ng, 2000, Algorithms for inverse reinforcement learning, ICML, 1, 2
- mnih, 2013, Playing atari with deep reinforcement learning, ArXiv Preprint
- zhu, 2020, The ingredients of real world robotic reinforcement learning, International Conference on Learning Representations
- 10.15607/RSS.2018.XIV.010 · DOI · Paper page
- 10.1109/IROS45743.2020.9341260 · DOI · Paper page
- 10.15607/RSS.2019.XV.011 · DOI · Paper page
- 10.15607/RSS.2019.XV.073 · DOI
- hafner, 2020, Mastering atari with discrete world models, International Conference on Learning Representations
- ha, 0, Recurrent world models facilitate policy evolution, Advances in neural information processing systems, 31
- haarnoja, 2018, Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor, Int Conf on Machine Learning, 1861
- hafner, 2019, Learning latent dynamics for planning from pixels, Int Conf on Machine Learning, 2555
- barth-maron, 2018, Distributed Distributional Deterministic Policy Gradients, International Conference on Learning Representations
- 10.1109/ICRA40945.2020.9197080 · DOI
- abdolmaleki, 2018, Maximum a posteriori policy optimisation, International Conference on Learning Representations
- 10.1109/LRA.2020.2966414 · DOI
- 10.1109/ICRA.2019.8793840 · DOI
- 10.1109/CASE48305.2020.9216949 · DOI
- coumans, 2016, Pybullet a python module for physics simulation for games robotics and machine learning
- 10.29007/21r5 · DOI
- wu, 2019, Imitation learning from imperfect demonstration, International Conference on Machine Learning, 6818
- 10.1038/s42256-020-00237-3 · DOI
- 10.1109/ICRA48506.2021.9561036 · DOI
Cited by
Sorted by citations
RLPP: A Residual Method for Zero-Shot Real-World Autonomous Racing on Scaled Platforms
Ghignone, E., Baumann, N., Hu, C., Wang, J., Xie, L., Carron, A., & Magno, M. (2025) · 2025 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra55743.2025.11127278 · Cited by 1
“[6] presents an interesting RL example, where the Dreamer architecture [7] is used to show that model-based RL algorithms generalize well to multiple tracks when compared to the model-free counterparts in simulation.”
Metadata
- Authors
- Brunnbauer, A., Berducci, L., Brandstatter, A., Lechner, M., Hasani, R., Rus, D., & Grosu, R.
- Affiliations
- TU Wien (AT)
- CPS, Technische Universitát Wien (TU Wien),Austria
- Institute of Science and Technology Austria (AT)
- MIT Computer Science and Artificial Intelligence Laboratory (US)
- Venue
- 2022 International Conference on Robotics and Automation (ICRA)
- Publisher
- IEEE
- Published
- 2022-05-23
- Pages
- 7513-7520
- ISSN
- 1050-4729 (ISSN-L)
- OpenAlex
- W4285102517
- Cited by
- 36
- References
- 6
- Citations by year
- 2026: 2 · 2025: 10 · 2024: 8 · 2023: 10 · 2022: 6
- FWCI
- 4.70
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Proceedings - IEEE International Conference on Robotics and Automation/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- Institute of Science and Technology Austria (submittedVersion, other-oa)
- arXiv (Cornell University) (cc-by)
- Funders
- Austrian Science Fund (FWF) (Z211-N23)
- Boeing Company
- Office of Naval Research (ONR) (N00014-18-1-2830)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- world models, zero-shot transfer, autonomous racing, F1TENTH, sim-to-real transfer, model-based reinforcement learning, lidar observations, sample efficiency, generalization, continuous control
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
