The DOI matches its DataCite record; 6 of the 6 references it cites were found in a registry or an index.
Towards General-Purpose Model-Free Reinforcement Learning
Abstract
Reinforcement learning (RL) promises a framework for near-universal problem-solving. In practice however, RL algorithms are often tailored to specific benchmarks, relying on carefully tuned hyperparameters and algorithmic choices. Recently, powerful model-based RL methods have shown impressive general results across benchmarks but come at the cost of increased complexity and slow run times, limiting their broader applicability. In this paper, we attempt to find a unifying model-free deep RL algorithm that can address a diverse class of domains and problem settings. To achieve this, we leverage model-based representations that approximately linearize the value function, taking advantage of the denser task objectives used by model-based RL while avoiding the costs associated with planning or simulated trajectories. We evaluate our algorithm, MR.Q, on a variety of common RL benchmarks with a single set of hyperparameters and show a competitive performance against domain-specific and general baselines, providing a concrete step towards building general-purpose model-free deep RL algorithms.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Evolution Strategies as a Scalable Alternative to Reinforcement Learning
Salimans, T., Ho, J., Chen, X., Sidor, S., & Sutskever, I. (2017) · arXiv · doi:10.48550/arxiv.1703.03864 · Cited by 1,027 · cited 2×
“To some extent, general-purpose algorithms do exist—policy gradient methods (Williams, 1992; Schulman et al., 2015; 2017) and many evolutionary approaches (Rechenberg, 1978; Back, 1996; Rubinstein, 1997; Salimans et al., 2017) require few assumptions on the underlying problem.”
Tassa, Y., Doron, Y., Muldal, A., Erez, T., Li, Y., Casas, D. D. L., Budden, D., Abdolmaleki, A., Merel, J., Lefrancq, A., et al. (2018) · arXiv · doi:10.48550/arxiv.1801.00690 · Cited by 497 · cited 15×
“Q achieves the highest performance in both DMC benchmarks, showcasing its ability to handle different observation spaces.”
InfluentialDeep Variational Bayes Filters: Unsupervised Learning of State Space Models from Raw Data
Karl, M., Soelch, M., Bayer, J., & van der Smagt, P. (2016) · arXiv · doi:10.48550/arxiv.1605.06432 · Cited by 232 · cited 1×
“…relate to model-based counterparts which learn a latent dynamics model for planning or value estimation (Watter et al., 2015; Finn et al., 2016; Karl et al., 2017; Ha & Schmidhuber, 2018; Schrittwieser et al., 2020; 2021; Ye et al., 2021; Hansen et al., 2022; 2024; Hafner et al., 2019; 2023;…”
Ha, D., & Schmidhuber, J. (2018) · Zenodo · doi:10.5281/zenodo.1207631 · Cited by 187
Applications of Deep Reinforcement Learning in Communications and Networking: A Survey
Luong, N. C., Hoang, D. T., Gong, S., Niyato, D., Wang, P., Liang, Y.-C., & Kim, D. I. (2018) · arXiv · doi:10.48550/arxiv.1810.07862 · Cited by 17
BYOL-Explore: Exploration by Bootstrapped Prediction
Guo, Z. D., Thakoor, S., Pîslar, M., Pires, B. A., Altché, F., Tallec, C., Saade, A., Calandriello, D., Grill, J.-B., Tang, Y., et al. (2022) · arXiv · doi:10.48550/arxiv.2206.08332 · Cited by 12
Cited by
Dynamics-Aligned Latent Imagination in Contextual World Models for Zero-Shot Generalization
Röder, F., Benad, J., Eppe, M., & Banerjee, P. (2025) · Advances in Neural Information Processing Systems 38 · doi:10.52202/085713-2438
Dynamics-Aligned Latent Imagination in Contextual World Models for Zero-Shot Generalization
Röder, F., Benad, J., Eppe, M., & Banerjee, P. K. (2025) · arXiv · doi:10.48550/arxiv.2508.20294
Metadata
- Authors
- Fujimoto, S., D'Oro, P., Zhang, A., Tian, Y., & Rabbat, M.
- Venue
- arXiv
- Published
- 2025
- arXiv
- 2501.16142
- References
- 6
- Type
- Preprint
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Towards General-Purpose Model-Free Reinforcement Learning. https://app.cochat.ai/papers/10.48550/arxiv.2501.16142
- The paper
- Fujimoto, S., D'Oro, P., Zhang, A., Tian, Y., & Rabbat, M. (2025). Towards General-Purpose Model-Free Reinforcement Learning. arXiv. https://doi.org/10.48550/arxiv.2501.16142
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.