Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agents
Open accessAbstract
The Arcade Learning Environment (ALE) is an evaluation platform that poses the challenge of building AI agents with general competency across dozens of Atari 2600 games. It supports a variety of different problem settings and it has been receiving increasing attention from the scientific community, leading to some high-profile success stories such as the much publicized Deep Q-Networks (DQN). In this article we take a big picture look at how the ALE is being used by the research community. We show how diverse the evaluation methodologies in the ALE have become with time, and highlight some key concerns when evaluating agents in the ALE. We use this discussion to present some methodological best practices and provide new benchmark results using these best practices. To further the progress in the field, we introduce a new version of the ALE that supports multiple game modes and provides a form of stochasticity we call sticky actions. We conclude this big picture look by revisiting challenges posed when the ALE was introduced, summarizing the state-of-the-art in various problems and highlighting problems that remain open.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The standard protocol for the Atari benchmark is to use 200M environment interactions per game.
“For instance, the standard protocol for the Atari benchmark (a set of environments that require limited generalisation (Cobbe et al., 2020)) is to use 200M environment interactions per game (Machado et al., 2018)”
From Craftax: A Lightning-Fast Benchmark for Open-Ended Reinforcement Learning (2024) · cites this paper as Machado et al., 2018
Section 6.1 reports benchmark evaluations at 10, 50, 100, and 200 million frames across 60 Atari games, so 200 million is a benchmark threshold in this paper. Section 4.2 presents those thresholds as the authors’ proposed evaluation protocol, not a universal standard. The paper also distinguishes frames from agent steps: with a frame skip of four, 200 million frames correspond to fewer agent decisions, so calling them 200 million environment interactions is imprecise. (LLM, conf=0.87)
Experimental — accuracy not yet measuredThe study abides by Machado et al. (2018)'s recommendations of evaluation protocol, including the use of stochasticity, ignoring the lives signal, and reporting the average performance during training.
“We abide by Machado et al. (2018)'s recommendations of evaluation protocol, including the use of stochasticity, ignoring the lives signal, and reporting the average performance during training.”
From Loss of Plasticity in Continual Deep Reinforcement Learning (2023) · cites this paper as Machado et al. (2018)
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357 · see citing works
Self-citationReinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,263 · see citing works
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612 · see citing works
Bellman, R. E. (2010) · doi:10.1515/9781400835386 · Cited by 14,588 · see citing works
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132 · see citing works
Deep Reinforcement Learning with Double Q-Learning
Van Hasselt, H., Guez, A., & Silver, D. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10295 · Cited by 3,318 · see citing works
Dueling Network Architectures for Deep Reinforcement Learning
Wang, Z., Schaul, T., Hessel, M., van Hasselt, H., Lanctot, M., & de Freitas, N. (2015) · arXiv · doi:10.48550/arxiv.1511.06581 · Cited by 3,300 · see citing works
Schaul, T., Quan, J., Antonoglou, I., & Silver, D. (2015) · arXiv · doi:10.48550/arxiv.1511.05952 · Cited by 3,120 · see citing works
The Arcade Learning Environment: An Evaluation Platform for General Agents
Bellemare, M. G., Naddaf, Y., Veness, J., & Bowling, M. (2013) · Journal of Artificial Intelligence Research · doi:10.1613/jair.3912 · Cited by 2,161 · see citing works
Self-citationTransfer Learning for Reinforcement Learning Domains: A Survey
Taylor, M. E., & Stone, P. (2009) · doi:10.5555/1577069.1755839 · Cited by 1,521 · see citing works
On-line Q-learning Using Connectionist Systems
Rummery, G. A. (1994) · Cited by 1,475 · see citing works
Campbell, M., Hoane, A. J., & Hsu, F.-H. (2002) · Artificial Intelligence · doi:10.1016/s0004-3702(01)00129-1 · Cited by 1,226 · see citing works
Intrinsic Motivation Systems for Autonomous Mental Development
Oudeyer, P.-Y., Kaplan, F., & Hafner, V. V. (2007) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2006.890271 · Cited by 1,126 · see citing works
Deep Q-learning From Demonstrations
Hester, T., Vecerik, M., Pietquin, O., Lanctot, M., Schaul, T., Piot, B., Horgan, D., Quan, J., Sendonaris, A., Osband, I., et al. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11757 · Cited by 889 · see citing works
Deep Exploration via Bootstrapped DQNPreprint
Osband, I., Blundell, C., Pritzel, A., & Roy, B. V. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.04621 · Cited by 837 · see citing works
Reinforcement learning for robots using neural networks
Lin, L.-J. (1992) · Defense Technical Information Center (DTIC) · Cited by 836 · see citing works
Reinforcement Learning with Unsupervised Auxiliary TasksPreprint
Jaderberg, M., Mnih, V., Czarnecki, W. M., Schaul, T., Leibo, J. Z., Silver, D., & Kavukcuoglu, K. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.05397 · Cited by 826 · see citing works
A Distributional Perspective on Reinforcement LearningPreprint
Bellemare, M. G., Dabney, W., & Munos, R. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.06887 · Cited by 724 · see citing works
Self-citationIntrinsically Motivated Reinforcement Learning
Singh, S. P., Barto, A. G., & Chentanez, N. (2005) · doi:10.21236/ada440280 · Cited by 614 · see citing works
FeUdal Networks for Hierarchical Reinforcement LearningPreprint
Vezhnevets, A. S., Osindero, S., Schaul, T., Heess, N., Jaderberg, M., Silver, D., & Kavukcuoglu, K. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.01161 · Cited by 608 · see citing works
Incentivizing Exploration In Reinforcement Learning With Deep Predictive ModelsPreprint
Stadie, B. C., Levine, S., & Abbeel, P. (2015) · arXiv.org · doi:10.48550/arxiv.1507.00814 · Cited by 545 · see citing works
OpenAI GymPreprint
Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., & Zaremba, W. (2016) · arXiv (Cornell University) · Cited by 544 · see citing works
Thrun, S., & Mitchell, T. M. (1995) · Robotics and Autonomous Systems · doi:10.1016/0921-8890(95)00004-y · Cited by 523 · see citing works
Seeger, M. (2000) · Applied Physics Letters · doi:10.1162/153244303765208377 · Cited by 492 · see citing works
Action-Conditional Video Prediction using Deep Networks in Atari GamesPreprint
Oh, J., Guo, X., Lee, H., Lewis, R. L., & Singh, S. P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1507.08750 · Cited by 473 · see citing works
Schaeffer, J., BURCH, N. R., Björnsson, Y., Kishimoto, A., Müller, M., Lake, R., Lu, P., & Sutphen, S. (2007) · Science · doi:10.1126/science.1144079 · Cited by 452 · see citing works
An analysis of model-based Interval Estimation for Markov Decision Processes
Strehl, A. L., & Littman, M. L. (2008) · Journal of Computer and System Sciences · doi:10.1016/j.jcss.2007.08.009 · Cited by 428 · see citing works
A survey of real-time strategy game AI research and competition in StarCraft
Ontanon, S., Synnaeve, G., Uriarte, A., Richoux, F., Churchill, D., & Preuss, M. (2013) · IEEE Transactions on Computational Intelligence and AI in Games · doi:10.1109/tciaig.2013.2286295 · Cited by 423 · see citing works
Massively Parallel Methods for Deep Reinforcement LearningPreprint
Nair, A., Srinivasan, P., Blackwell, S., Alcicek, C., Fearon, R., De Maria, A., Panneershelvam, V., Suleyman, M., Beattie, C., Petersen, S., et al. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1507.04296 · Cited by 390 · see citing works
Count-Based Exploration with Neural Density Models
Ostrovski, G., Bellemare, M. G., Oord, A. V. D., & Munos, R. (2017) · arXiv · doi:10.48550/arxiv.1703.01310 · Cited by 377 · see citing works
Self-citationR-MAX: a general polynomial time algorithm for near-optimal reinforcement learning
Brafman, R. I., & Tennenholtz, M. (2001) · Cited by 366 · see citing works
Racing the beam: the Atari video computer system
(2009) · Choice Reviews Online · doi:10.5860/choice.46-6853 · Cited by 321 · see citing works
Safe and Efficient Off-Policy Reinforcement LearningPreprint
Munos, R., Stepleton, T., Harutyunyan, A., & Bellemare, M. G. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.02647 · Cited by 320 · see citing works
Self-citationSutton, R. S., Modayil, J., Delp, M., Degris, T., Pilarski, P. M., White, A., & Precup, D. (2011) · doi:10.5555/2031678.2031726 · Cited by 301 · see citing works
Knowledge Growth in an Artificial Animal
Wilson, S. W. (1986) · Springer eBooks · doi:10.1007/978-1-4757-1895-9_18 · Cited by 274 · see citing works
Rusu, A. A., Colmenarejo, S. G., Gulcehre, C., Desjardins, G., Kirkpatrick, J., Pascanu, R., Mnih, V., Kavukcuoglu, K., & Hadsell, R. (2015) · arXiv · doi:10.48550/arxiv.1511.06295 · Cited by 270 · see citing works
Deep Learning for Real-Time Atari Game Play Using Offline Monte-Carlo Tree Search Planning
Guo, X., Singh, S. P., Lee, H., Lewis, R. L., & Wang, X. (2014) · Cited by 265 · see citing works
Unifying Count-Based Exploration and Intrinsic Motivation
Bellemare, M. G., Srinivasan, S., Ostrovski, G., Schaul, T., Saxton, D., & Munos, R. (2016) · arXiv · doi:10.48550/arxiv.1606.01868 · Cited by 264 · see citing works
Self-citationIntrinsic Motivation and Reinforcement Learning
Barto, A. G. (2012) · Springer eBooks · doi:10.1007/978-3-642-32375-1_2 · Cited by 234 · see citing works
The Malmo platform for artificial intelligence experimentation
Johnson, M., Hofmann, K., Hutton, T. J., & Bignell, D. E. (2016) · International Joint Conference on Artificial Intelligence · Cited by 205 · see citing works
Actor-Mimic: Deep Multitask and Transfer Reinforcement LearningPreprint
Parisotto, E., Ba, J. L., & Salakhutdinov, R. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.06342 · Cited by 198 · see citing works
Improving Multi-Step Prediction of Learned Time Series Models
Venkatraman, A., Hebert, M., & Bagnell, J. (2015) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v29i1.9590 · Cited by 198 · see citing works
The Arcade Learning Environment: An Evaluation Platform for General Agents
Bellemare, M. G., Naddaf, Y., Veness, J., & Bowling, M. (2012) · arXiv · doi:10.48550/arxiv.1207.4708 · Cited by 187 · see citing works
Self-citationGraying the black box: Understanding DQNsPreprint
Zahavy, T., Ben-Zrihem, N., & Mannor, S. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.02658 · Cited by 181 · see citing works
Reproducibility of Benchmarked Deep Reinforcement Learning Tasks for Continuous Control
Islam, R., Henderson, P., Gomrokchi, M., & Precup, D. (2017) · arXiv · doi:10.48550/arxiv.1708.04133 · Cited by 177 · see citing works
A Neuroevolution Approach to General Atari Game Playing
Hausknecht, M., Lehman, J., Miikkulainen, R. P., & Stone, P. (2014) · IEEE Transactions on Computational Intelligence and AI in Games · doi:10.1109/tciaig.2013.2294713 · Cited by 168 · see citing works
Self-citationSutton, R. S., Maei, H. R., & Szepesvári, C. (2008) · neural information processing systems · Cited by 152 · see citing works
Approximate Planning in Large POMDPs via Reusable Trajectories
Kearns, M. J., Mansour, Y., & Ng, A. Y. (1999) · Cited by 135 · see citing works
Recurrent Environment SimulatorsPreprint
Chiappa, S., Racanière, S., Wierstra, D., & Mohamed, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.02254 · Cited by 130 · see citing works
Increasing the Action Gap: New Operators for Reinforcement Learning
G. Bellemare, M., Ostrovski, G., Guez, A., Thomas, P. S., & Munos, R. (2016) · AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) · doi:10.1609/aaai.v30i1.10303 · Cited by 123 · see citing works
Self-citationCHILD: A First Step Towards Continual Learning
Ring, M. B. (1997) · Machine Learning · doi:10.1023/a:1007331723572 · Cited by 119 · see citing works
Maei, H. R., & Sutton, R. S. (2010) · doi:10.2991/agi.2010.22 · Cited by 104 · see citing works
A Laplacian Framework for Option Discovery in Reinforcement LearningPreprint
Machado, M. C., Bellemare, M. G., & Bowling, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.00956 · Cited by 93 · see citing works
Self-citationLearning from Demonstrations for Real World Reinforcement LearningPreprint
Hester, T., Vecerík, M., Pietquin, O., Lanctot, M., Schaul, T., Piot, B., Sendonaris, A., Dulac-Arnold, G., Osband, I., Agapiou, J., et al. (2017) · arXiv (Cornell University) · Cited by 92 · see citing works
Count-Based Exploration in Feature Space for Reinforcement Learning
Martin, J., Sasikumar, S. N., Everitt, T., & Hütter, M. (2017) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/344 · Cited by 87 · see citing works
Investigating Contingency Awareness Using Atari 2600 Games
Bellemare, M. G., Veness, J., & Bowling, M. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v26i1.8321 · Cited by 75 · see citing works
Self-citationLevine, J., Congdon, C. B., Ebner, M., Kendall, G., Lucas, S. M., Miikkulainen, R. P., Schaul, T., & Thompson, T. (2013) · Open Access at Essex (University of Essex) · doi:10.4230/dfu.vol6.12191.77 · Cited by 74 · see citing works
Beattie, C., Leibo, J. Z., Teplyashin, D., Ward, T., Wainwright, M., Küttler, H., Lefrancq, A., Green, S., Valdés, V., Sadik, A., et al. (2016) · arXiv · doi:10.48550/arxiv.1612.03801 · Cited by 68 · see citing works
Classical planning with simulators: results on the Atari video games
Lipovetzky, N., Ramírez, M., & Geffner, H. (2015) · Cited by 63 · see citing works
State of the Art Control of Atari Games Using Shallow Reinforcement Learning
Liang, Y., Machado, M. C., Talvitie, E., & Bowling, M. (2016) · Adaptive Agents and Multi-Agents Systems · doi:10.5555/2936924.2936996 · Cited by 60 · see citing works
Self-citationModel regularization for stable sample rollouts
Talvitie, E. (2014) · Cited by 51 · see citing works
Self-citationValue-Aware Loss Function for Model-based Reinforcement Learning
Farahmand, A. M., Barreto, A. D. M. S., & Nikovski, D. N. (2017) · PolyPublie (École Polytechnique de Montréal) · Cited by 46 · see citing works
Frame Skip Is a Powerful Parameter for Learning to Play Atari
Braylan, A. (2015) · National Conference on Artificial Intelligence · Cited by 46 · see citing works
Emergent Tangled Graph Representations for Atari Game Playing Agents
Kelly, S., & Heywood, M. I. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-55696-3_5 · Cited by 44 · see citing works
Self-Correcting Models for Model-Based Reinforcement Learning
Talvitie, E. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10850 · Cited by 42 · see citing works
Self-citationReinforcement Learning in POMDPs Without Resets
Even-Dar, E., Kakade, S. M., & Mansour, Y. (2005) · ScholarlyCommons (University of Pennsylvania) · Cited by 39 · see citing works
Game-independent AI agents for playing Atari 2600 console games
Naddaf, Y. (2010) · ERA: Education and Research Archive (University of Alberta) · doi:10.7939/r3134q · Cited by 36 · see citing works
Bellemare, M. G., Veness, J., & Talvitie, E. (2014) · Cited by 32 · see citing works
Self-citationBlind search for Atari-like online planning revisited
Shleyfman, A., Tuisov, A., & Domshlak, C. (2016) · International Joint Conference on Artificial Intelligence · Cited by 25 · see citing works
The Impact of Determinism on Learning Atari 2600 Games
Hausknecht, M., & Stone, P. (2015) · National Conference on Artificial Intelligence · Cited by 24 · see citing works
Self-citationBayesian Learning of Recursively Factored Environments
Bellemare, M. G., Veness, J., & Bowling, M. (2013) · Cited by 21 · see citing works
Self-citationSketch-Based Linear Value Function Approximation
Bellemare, M. G., Veness, J., & Bowling, M. (2012) · Cited by 21 · see citing works
Self-citationDomain-Independent Optimistic Initialization for Reinforcement LearningPreprint
Machado, M. C., Srinivasan, S., & Bowling, M. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1410.4604 · Cited by 21 · see citing works
Self-citationA Comparison of learning algorithms on the Arcade Learning EnvironmentPreprint
Defazio, A., & Graepel, T. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1410.8620 · Cited by 20 · see citing works
Learning to Prune Dominated Action Sequences in Online Black-Box Planning
Jinnai, Y., & Fukunaga, A. S. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10663 · Cited by 12 · see citing works
Reports of the 2016 AAAI Workshop Program
Albrecht, S. V., Bouchard, B., Brownstein, J. S., Buckeridge, D. L., Caragea, C., Carter, K. M., Darwiche, A. Y., Fortuna, B., Francillette, Y., Gaboury, S., et al. (2016) · AI Magazine · doi:10.1609/aimag.v37i3.2680 · Cited by 7 · see citing works
Veness, J., Bellemare, M. G., Hütter, M., Chua, A., & Desjardins, G. (2015) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v29i1.9600 · Cited by 6 · see citing works
Self-citation
Flagged first, then by citations
15 of the 93 references couldn’t be matched to a known paper.
Research works printed but not linked: 8
- Hausknecht, M.; Lehman, J.; Miikkulainen, R.; and Stone, P. 2013. A neuroevolution approach to general atari game playing. In IEEE Transactions on Computational Intelli- gence and AI in Games.
- Mnih, V.; Kavukcuoglu, K.; Silver, D.; Graves, A.; Antonoglou, I.; Wierstra, D.; and Riedmiller, M. 2013. Playing atari with deep reinforcement learning. CoRR abs/1312.5602. · arXiv
- Albrecht, S. V., L., J. C., Buckeridge, D. L., Botea, A., Caragea, C., Chi, C., Damoulas, T., Dilkina, B. N., Eaton, E., Fazli, P., Ganzfried, S., Lindauer, M. T., Machado, M. C., Malitsky, Y., Marcus, G., Meijer, S., Rossi, F., Shaban-Nejad, A., Thiebaux, S., Veloso, M. M., Walsh, T., Wang, C., Zhang, J., & Zheng, Y. (2015). Reports on the 2015 AAAI Workshop Program. AI Magazine, 36 (2), 90-101.
- Campbell, M., Hoane, Jr., A. J., & Hsu, F.-h. (2002). Deep Blue. Artificial Intelligence, 134 (1-2), 57-83.
- Harutyunyan, A., Bellemare, M. G., Stepleton, T., & Munos, R. (2016). Q(λ) with Off- Policy Corrections. In Proceedings of the International Conference on Algorithmic Learning Theory (ALT), pp. 305-320.
- Hutter, M. (2005). Universal Artificial Intelligence: Sequential Decisions based on Algorith- mic Probability. Springer.
- Kearns, M. J., & Singh, S. P. (2002). Near-Optimal Reinforcement Learning in Polynomial Time. Machine Learning, 49 (2-3), 209-232.
- Nair, A., Srinivasan, P., Blackwell, S., Alcicek, C., Fearon, R., Maria, A. D., Panneershelvam, V., Suleyman, M., Beattie, C., Petersen, S., Legg, S., Mnih, V., Kavukcuoglu, K., & Silver, D. (2015). Massively Parallel Methods for Deep Reinforcement Learning. In ICML Deep Learning Workshop.
Cited by
Sorted by citations
Deep Reinforcement Learning That Matters
Henderson, P., Islam, R., Bachman, P., Pineau, J., Precup, D., & Meger, D. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11694 · Cited by 1,685 · see citing works
Mastering Atari, Go, chess and shogi by planning with a learned model
Schrittwieser, J., Antonoglou, I., Hubert, T., Simonyan, K., Sifre, L., Schmitt, S., Guez, A., Lockhart, E., Hassabis, D., Graepel, T., et al. (2020) · RePEc: Research Papers in Economics · Cited by 1,310 · see citing works
Mastering Atari, Go, chess and shogi by planning with a learned model
Schrittwieser, J., Antonoglou, I., Hubert, T., Simonyan, K., Sifre, L., Schmitt, S., Guez, A., Lockhart, E., Hassabis, D., Graepel, T., et al. (2020) · Nature · doi:10.1038/s41586-020-03051-4 · Cited by 1,214 · see citing works
Meta-Learning in Neural Networks: A Survey
Hospedales, T. M., Antoniou, A., Micaelli, P., & Storkey, A. J. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3079209 · Cited by 886 · see citing works
QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation
Kalashnikov, D., Irpan, A., Pastor, P., Ibarz, J., Herzog, A., Jang, E., Quillen, D., Holly, E., Kalakrishnan, M., Vanhoucke, V., et al. (2018) · arXiv · doi:10.48550/arxiv.1806.10293 · Cited by 697 · see citing works
A survey and critique of multiagent deep reinforcement learning
Hernández-Leal, P., Kartal, B., & Taylor, M. E. (2019) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-019-09421-1 · Cited by 747 · see citing works
Unity: A General Platform for Intelligent Agents
Juliani, A., Berges, V.-P., Teng, E., Cohen, A., Harper, J., Elion, C., Goy, C., Gao, Y., Henry, H., Mattar, M., et al. (2018) · arXiv · doi:10.48550/arxiv.1809.02627 · Cited by 626 · see citing works
Model-based Reinforcement Learning: A Survey
Moerland, T. M., Broekens, J., Plaat, A., & Jonker, C. M. (2023) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000086 · Cited by 668 · see citing works
Exploration by Random Network Distillation
Burda, Y., Edwards, H., Storkey, A., & Klimov, O. (2018) · arXiv · doi:10.48550/arxiv.1810.12894 · Cited by 527 · see citing works
Quantifying Generalization in Reinforcement LearningPreprint
Cobbe, K., Klimov, O., Hesse, C., Kim, T., & Schulman, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.02341 · Cited by 348 · see citing works
Deep Learning for Video Game Playing
Justesen, N., Bontrager, P., Togelius, J., & Risi, S. (2019) · IEEE Transactions on Games · doi:10.1109/tg.2019.2896986 · Cited by 284 · see citing works
A Gentle Introduction to Reinforcement Learning and its Application in Different Fields
Naeem, M., Rizvi, S. T. H., & Coronato, A. (2020) · IEEE Access · doi:10.1109/access.2020.3038605 · Cited by 264 · see citing works
A Study on Overfitting in Deep Reinforcement LearningPreprint
Zhang, C., Vinyals, O., Munos, R., & Samy Bengio, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.06893 · Cited by 246 · see citing works
Li, C. (2019) · Chapman & Hall/CRC eBooks · doi:10.1201/9781351006620-6 · Cited by 251 · see citing works
The Hanabi challenge: A new frontier for AI research
Bard, N., Foerster, J., Chandar, S., BURCH, N. R., Lanctot, M., Song, H. F., Parisotto, E., Dumoulin, V., Moitra, S., Hughes, E., et al. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2019.103216 · Cited by 250 · see citing works
Self-citationEcoffet, A., Huizinga, J., Lehman, J., Stanley, K. O., & Clune, J. (2021) · Nature · doi:10.1038/s41586-020-03157-9 · Cited by 235 · see citing works
Towards Continual Reinforcement Learning: A Review and Perspectives
Khetarpal, K., Riemer, M., Rish, I., & Precup, D. (2022) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.13673 · Cited by 224 · see citing works
TextWorld: A Learning Environment for Text-Based Games
Côté, M.-A., Kádár, Á., Yuan, X., Kybartas, B., Barnes, T., Fine, E., Moore, J. A. Y., Hausknecht, M., Asri, L. E., Adada, M., et al. (2019) · Communications in computer and information science · doi:10.1007/978-3-030-24337-1_3 · Cited by 210 · see citing works
Self-citationEcoffet, A., Huizinga, J., Lehman, J., Stanley, K. O., & Clune, J. (2021) · RePEc: Research Papers in Economics · Cited by 201 · see citing works
A Survey of Zero-shot Generalisation in Deep Reinforcement Learning
Kirk, R., Zhang, A., Grefenstette, E., & Rocktäschel, T. (2023) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.14174 · Cited by 193 · see citing works
Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domain
Hao, J., Yang, T., Tang, H., Bai, C., Liu, J., Meng, Z., Liu, P., & Wang, Z. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3236361 · Cited by 209 · see citing works
Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domain
Hao, J., Yang, T., Tang, H., Bai, C., Liu, J., Meng, Z., Liu, P., & Wang, Z. (2021) · arXiv · doi:10.48550/arxiv.2109.06668 · Cited by 192
Self-Supervised Exploration via DisagreementPreprint
Pathak, D., Gandhi, D., & Gupta, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.04161 · Cited by 190 · see citing works
Measuring the Occupational Impact of AI: Tasks, Cognitive Abilities and AI Benchmarks
Tolan, S., Pesole, A., Martínez‐Plumed, F., Fernández‐Macías, E., Hernández‐Orallo, J., & Gómez, E. (2021) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.12647 · Cited by 181 · see citing works
An Optimistic Perspective on Offline Reinforcement LearningPreprint
Agarwal, R., Schuurmans, D., & Norouzi, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.04543 · Cited by 162 · see citing works
A Survey of Deep Reinforcement Learning in Video GamesPreprint
Shao, K., Tang, Z., Zhu, Y., Li, N., & Zhao, D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.10944 · Cited by 224 · see citing works
Mastering diverse control tasks through world models
Hafner, D., Pasukonis, J., Ba, J., & Lillicrap, T. (2025) · Nature · doi:10.1038/s41586-025-08744-2 · Cited by 189 · see citing works
“Atari 35 uses 57 tasks with sticky actions 55 .”
Benchmarking Batch Deep Reinforcement Learning AlgorithmsPreprint
Fujimoto, S., Conti, E., Ghavamzadeh, M., & Pineau, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.01708 · Cited by 166 · see citing works
Learn What Not to Learn: Action Elimination with Deep Reinforcement LearningPreprint
Zahavy, T., Haroush, M., Merlis, N., Mankowitz, D. J., & Mannor, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.02121 · Cited by 132 · see citing works
Futures of artificial intelligence through technology readiness levels
Martínez-Plumed, F., Gómez, E., & Hernández-Orallo, J. (2021) · Telematics and Informatics · doi:10.1016/j.tele.2020.101525 · Cited by 130 · see citing works
Interactive Fiction Games: A Colossal Adventure
Hausknecht, M., Ammanabrolu, P., Côté, M.-A., & Yuan, X. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6297 · Cited by 122 · see citing works
Self-citationAssessing Generalization in Deep Reinforcement Learning
Packer, C., Gao, K., Kos, J., Krähenbühl, P., Koltun, V., & Song, D. (2018) · arXiv · doi:10.48550/arxiv.1810.12282 · Cited by 114 · see citing works
Efficient Exploration Through Bayesian Deep Q-Networks
Azizzadenesheli, K., Brunskill, E., & Anandkumar, A. (2018) · Information Theory and Applications Workshop (ITA) · doi:10.1109/ita.2018.8503252 · Cited by 112 · see citing works
GEP-PG: Decoupling Exploration and Exploitation in Deep Reinforcement Learning AlgorithmsPreprint
Colas, C., Sigaud, O., & Oudeyer, P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.05054 · Cited by 118 · see citing works
Gotta Learn Fast: A New Benchmark for Generalization in RLPreprint
Nichol, A., Pfau, V., Hesse, C., Klimov, O., & Schulman, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.03720 · Cited by 136 · see citing works
OpenSpiel: A Framework for Reinforcement Learning in GamesPreprint
Lanctot, M., Lockhart, E., Lespiau, J.-B., Zambaldi, V., Upadhyay, S., Pérolat, J., Srinivasan, S., Timbers, F., Tuyls, K., Omidshafiei, S., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.09453 · Cited by 104 · see citing works
Data-Efficient Reinforcement Learning with Self-Predictive RepresentationsPreprint
Schwarzer, M., Anand, A., Goel, R., Hjelm, R. D., Courville, A. C., & Bachman, P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.05929 · Cited by 103 · see citing works
Mastering Diverse Domains through World Models
Hafner, D., Pasukonis, J., Ba, J., & Lillicrap, T. (2023) · arXiv · doi:10.48550/arxiv.2301.04104 · Cited by 124 · see citing works
“We use the sticky action simulator setting 39 .”
MethodObstacle Tower: A Generalization Challenge in Vision, Control, and Planning
Juliani, A., Khalifa, A., Berges, V.-P., Harper, J., Teng, E., Henry, H., Crespi, A., Togelius, J., & Lange, D. (2019) · Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/373 · Cited by 93 · see citing works
Generalization and Regularization in DQN
Farebrother, J., Machado, M. C., & Bowling, M. (2018) · arXiv · doi:10.48550/arxiv.1810.00123 · Cited by 92 · see citing works
Self-citationA Dissection of Overfitting and Generalization in Continuous Reinforcement Learning
Zhang, A., Ballas, N., & Pineau, J. (2018) · arXiv · doi:10.48550/arxiv.1806.07937 · Cited by 88 · see citing works
Distributional Reinforcement Learning
Bellemare, M. G., Dabney, W., & Rowland, M. (2023) · The MIT Press eBooks · doi:10.7551/mitpress/14207.001.0001 · Cited by 85 · see citing works
Self-citationLearning Montezuma's Revenge from a Single DemonstrationPreprint
Salimans, T., & Chen, R. J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.03381 · Cited by 102 · see citing works
ChainerRL: A Deep Reinforcement Learning LibraryPreprint
Fujita, Y., Nagarajan, P., Kataoka, T., & Ishikawa, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.03905 · Cited by 81 · see citing works
Model Based Reinforcement Learning for Atari
Kaiser, Ł., Babaeizadeh, M., Miłoś, P., Osiński, B., Campbell, R. H., Czechowski, K., Erhan, D., Finn, C., Kozakowski, P., Levine, S., et al. (2020) · International Conference on Learning Representations · Cited by 75 · see citing works
Time Limits in Reinforcement LearningPreprint
Pardo, F., Tavakoli, A., Levdik, V., & Kormushev, P. S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.00378 · Cited by 80 · see citing works
Evolving simple programs for playing atari games
Wilson, D. G., Cussat‐Blanc, S., Luga, H., & Miller, J. F. (2018) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/3205455.3205578 · Cited by 71 · see citing works
RIDE: Rewarding Impact-Driven Exploration for Procedurally-Generated EnvironmentsPreprint
Raileanu, R., & Rocktäschel, T. (2020) · International Conference on Learning Representations · doi:10.48550/arxiv.2002.12292 · Cited by 224 · see citing works
A Survey of Generalisation in Deep Reinforcement LearningPreprint
Kirk, R., Zhang, A., Grefenstette, E., & Rocktäschel, T. (2021) · arXiv (Cornell University) · Cited by 63 · see citing works
Exploring the Limitations of Behavior Cloning for Autonomous Driving
Codevilla, F., Santana, E., López, A. M., & Gaidon, A. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00942 · Cited by 562 · see citing works
A Review of Uncertainty for Deep Reinforcement Learning
Lockwood, O., & Si, M. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment · doi:10.1609/aiide.v18i1.21959 · Cited by 63 · see citing works
Behaviour Suite for Reinforcement LearningPreprint
Osband, I., Doron, Y., Hessel, M., Aslanides, J., Sezener, E., Saraiva, A., McKinney, K., Lattimore, T., Szepesvári, C., Singh, S. P., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.03568 · Cited by 58 · see citing works
Contingency-Aware Exploration in Reinforcement LearningPreprint
Choi, J., Guo, Y., Moczulski, M., Oh, J., Wu, N., Norouzi, M., & Lee, H. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.01483 · Cited by 57 · see citing works
Is multiagent deep reinforcement learning the answer or the question? A brief surveyPreprint
Hernández-Leal, P., Kartal, B., & Taylor, M. E. (2018) · arXiv (Cornell University) · Cited by 56 · see citing works
Learning with AMIGo: Adversarially Motivated Intrinsic GoalsPreprint
Campero, A., Raileanu, R., Küttler, H., Tenenbaum, J., Rocktäschel, T., & Grefenstette, E. (2020) · International Conference on Learning Representations · doi:10.48550/arxiv.2006.12122 · Cited by 150 · see citing works
Google Research Football: A Novel Reinforcement Learning Environment
Kurach, K., Raichuk, A., Stańczyk, P., Zając, M., Bachem, O., Espeholt, L., Riquelme, C., Vincent, D., Michalski, M., Bousquet, O., et al. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.5878 · Cited by 70 · see citing works
Hyperbolic Discounting and Learning over Multiple HorizonsPreprint
Fedus, W., Gelada, C., Bengio, Y., Bellemare, M. G., & Larochelle, H. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.06865 · Cited by 52 · see citing works
Self-citationDeterministic Implementations for Reproducibility in Deep Reinforcement LearningPreprint
Nagarajan, P., Warnell, G., & Stone, P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.05676 · Cited by 61 · see citing works
Making Efficient Use of Demonstrations to Solve Hard Exploration ProblemsPreprint
Paine, T. L., Gülçehre, Ç., Shahriari, B., Denil, M., Hoffman, M., Soyer, H., Tanburn, R., Kapturowski, S., Rabinowitz, N. C., Williams, D., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.01387 · Cited by 49 · see citing works
Automatic Data Augmentation for Generalization in Deep Reinforcement LearningPreprint
Răileanu, R., Goldstein, M., Yarats, D., Kostrikov, I., & Fergus, R. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.12862 · Cited by 65 · see citing works
Automatic Data Augmentation for Generalization in Reinforcement LearningPreprint
Răileanu, R., Goldstein, M., Yarats, D., Kostrikov, I., & Fergus, R. (2021) · arXiv (Cornell University) · Cited by 44 · see citing works
Learning Abstract OptionsPreprint
Riemer, M., Liu, M., & Tesauro, G. J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.11583 · Cited by 43 · see citing works
Re-evaluating EvaluationPreprint
Balduzzi, D., Tuyls, K., Pérolat, J., & Graepel, T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.02643 · Cited by 46 · see citing works
Learning Purely Tactile In-Hand Manipulation with a Torque-Controlled Hand
Sievers, L., Pitz, J., & Bäuml, B. (2022) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra46639.2022.9812093 · Cited by 41 · see citing works
MinAtar: An Atari-Inspired Testbed for Thorough and Reproducible Reinforcement Learning ExperimentsPreprint
Young, K., & Tian, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.03176 · Cited by 40 · see citing works
Continual Reinforcement Learning in 3D Non-stationary Environments
Lomonaco, V., Desai, K., Culurciello, E., & Maltoni, D. (2020) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00132 · Cited by 36 · see citing works
Distributed Bayesian optimization of deep reinforcement learning algorithms
Young, M. T., Hinkle, J. D., Kannan, R., & Ramanathan, A. (2020) · Journal of Parallel and Distributed Computing · doi:10.1016/j.jpdc.2019.07.008 · Cited by 38 · see citing works
Revisiting Rainbow: Promoting more Insightful and Inclusive Deep Reinforcement Learning ResearchPreprint
Obando-Ceron, J., & Castro, P. S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.14826 · Cited by 37 · see citing works
Canaan, R., Salge, C., Togelius, J., & Nealen, A. (2019) · International Conference on the Foundations of Digital Games (FDG) · doi:10.1145/3337722.3337750 · Cited by 54 · see citing works
The Text-Based Adventure AI Competition
Atkinson, T. J., Baier, H., Copplestone, T., Devlin, S., & Swan, J. (2019) · IEEE Transactions on Games · doi:10.1109/tg.2019.2896017 · Cited by 35 · see citing works
Measuring and characterizing generalization in deep reinforcement learning
Witty, S., Lee, J. K., Tosch, E., Atrey, A., Clary, K., Littman, M. L., & Jensen, D. D. (2021) · Applied AI Letters · doi:10.1002/ail2.45 · Cited by 35 · see citing works
Safe Q-Learning Method Based on Constrained Markov Decision Processes
Ge, Y., Zhu, F., Ling, X., & Liu, Q. (2019) · IEEE Access · doi:10.1109/access.2019.2952651 · Cited by 35 · see citing works
The Animal-AI Environment: Training and Testing Animal-Like Artificial CognitionPreprint
Beyret, B., Hernández-Orallo, J., Cheke, L., Halina, M., Shanahan, M., & Crosby, M. (2019) · arXiv (Cornell University) · doi:10.4230/lipics.time.2024.5 · Cited by 38 · see citing works
Probabilistic Guarantees for Safe Deep Reinforcement Learning
Bacci, E., & Parker, D. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-57628-8_14 · Cited by 33 · see citing works
Dextrous Tactile In-Hand Manipulation Using a Modular Reinforcement Learning Architecture
Pitz, J., Röstel, L., Sievers, L., & Bäuml, B. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10160756 · Cited by 33 · see citing works
An Equivalence between Loss Functions and Non-Uniform Sampling in Experience ReplayPreprint
Fujimoto, S., Meger, D., & Precup, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.06049 · Cited by 33 · see citing works
Online and Offline Reinforcement Learning by Planning with a Learned ModelPreprint
Schrittwieser, J., Hubert, T., Mandhane, A., Barekatain, M., Antonoglou, I., & Silver, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.06294 · Cited by 33 · see citing works
Advances and challenges in learning from experience replay
Neves, D. E., Ishitani, L., & Patrocínio, Z. K. G. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-11062-0 · Cited by 31 · see citing works
Learning Deployable Navigation Policies at Kilometer Scale from a Single TraversalPreprint
Bruce, J., Sünderhauf, N., Mirowski, P., Hadsell, R. T., & Milford, M. J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.05211 · Cited by 32 · see citing works
Using a Team of General AI Algorithms to Assist Game Design and Testing
Guerrero-Romero, C., Lucas, S. M., & Pérez-Liébana, D. (2018) · IEEE Conference on Computational Intelligence and Games (CIG) · doi:10.1109/cig.2018.8490417 · Cited by 30 · see citing works
Do recent advancements in model-based deep reinforcement learning really improve data efficiency?Preprint
Kielak, K. (2019) · arXiv (Cornell University) · Cited by 29 · see citing works
Naturally Interpretable Control Policies via Graph-Based Genetic Programming
Nadizar, G., Medvet, E., & Wilson, D. G. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-56957-9_5 · Cited by 28 · see citing works
Benchmarking Crisis in Social Media Analytics: A Solution for the Data-Sharing Problem
Assenmacher, D., Weber, D., Preuß, M., Valdez, A. S. C., Bradshaw, A., Roß, B., Cresci, S., Trautmann, H., Neumann, F., & Grimme, C. (2021) · Social Science Computer Review · doi:10.1177/08944393211012268 · Cited by 27 · see citing works
Off-Policy Evaluation via Off-Policy ClassificationPreprint
Irpan, A., Rao, K., Bousmalis, K., Harris, C., Ibarz, J., & Levine, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.01624 · Cited by 27 · see citing works
Gossip-based Actor-Learner Architectures for Deep Reinforcement LearningPreprint
Assran, M., Romoff, J., Ballas, N., Pineau, J., & Rabbat, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.04585 · Cited by 26 · see citing works
Optimization of Neuroprosthetic Vision via End-to-end Deep Reinforcement Learning
Küçükoğlu, B., Rueckauer, B., Ahmad, N., de Ruyter van Steveninck, J., Güçlü, U., & van Gerven, M. (2022) · doi:10.1101/2022.02.25.482017 · Cited by 25 · see citing works
Optimization of Neuroprosthetic Vision via End-to-End Deep Reinforcement Learning
Küçükoğlu, B., Rueckauer, B., Ahmad, N., van Steveninck, J. D. R., Güçlü, U., & van Gerven, M. A. J. (2022) · International Journal of Neural Systems · doi:10.1142/s0129065722500526 · Cited by 25 · see citing works
Navigating the landscape of multiplayer games
Omidshafiei, S., Tuyls, K., Czarnecki, W. M., Santos, F. C., Rowland, M., Connor, J., Hennes, D., Muller, P., Pérolat, J., Vylder, B. D., et al. (2020) · Nature Communications · doi:10.1038/s41467-020-19244-4 · Cited by 29 · see citing works
TrojDRL: Trojan Attacks on Deep Reinforcement Learning AgentsPreprint
Kiourti, P., Wardega, K., Jha, S., & Li, W. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.06638 · Cited by 25 · see citing works
Combating the Compounding-Error Problem with a Multi-step ModelPreprint
Asadi, K., Misra, D. K., Kim, S., & Littman, M. L. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.13320 · Cited by 25 · see citing works
Regularized Softmax Deep Multi-Agent $Q$-LearningPreprint
Pan, L., Rashid, T., Peng, B., Huang, L., & Whiteson, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.11883 · Cited by 25 · see citing works
Safe option-critic: learning safety in the option-critic architecture
Jain, A., Khetarpal, K., & Precup, D. (2021) · The Knowledge Engineering Review · doi:10.1017/s0269888921000035 · Cited by 24 · see citing works
Rolling Horizon Evolutionary Algorithms for General Video Game Playing
Gaina, R. D., Devlin, S., Lucas, S. M., & Pérez-Liébana, D. (2021) · IEEE Transactions on Games · doi:10.1109/tg.2021.3060282 · Cited by 24 · see citing works
(2018) · arXiv (Cornell University) · Cited by 24 · see citing works
Task-Agnostic Dynamics Priors for Deep Reinforcement LearningPreprint
Du, Y., & Narasimhan, K. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.04819 · Cited by 27 · see citing works
Decoupling Value and Policy for Generalization in Reinforcement LearningPreprint
Răileanu, R., & Fergus, R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.10330 · Cited by 26 · see citing works
Generalizable Episodic Memory for Deep Reinforcement LearningPreprint
Hu, H., Ye, J., Zhu, G., Ren, Z., & Zhang, C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.06469 · Cited by 28 · see citing works
Exploration by Maximizing Renyi Entropy for Reward-Free RL Framework
Zhang, C., Cai, Y., Huang, L., & Li, J. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i12.17297 · Cited by 24 · see citing works
On Bonus-Based Exploration Methods in the Arcade Learning EnvironmentPreprint
Taïga, A. A., Fedus, W., Machado, M. C., Courville, A. C., & Bellemare, M. G. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.11052 · Cited by 22 · see citing works
Self-citationInvestigation of independent reinforcement learning algorithms in multi-agent environments
Lee, K. M., Subramanian, S. G., & Crowley, M. (2022) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2022.805823 · Cited by 21 · see citing works
- Authors
- Machado, M. C., Bellemare, M. G., Talvitie, E., Veness, J., Hausknecht, M., & Bowling, M.
- Affiliations
- University of Alberta (CA)
- Franklin & Marshall College (US)
- Google DeepMind (United Kingdom) (GB)
- Microsoft (United States) (US)
- Venue
- Journal of Artificial Intelligence Research
- Publisher
- AI Access Foundation
- Published
- 2018-03-19
- Volume
- 61
- Pages
- 523-562
- ISSN
- 1076-9757 (ISSN-L)
- OpenAlex
- W2754879180
- Cited by
- 359
- References
- 93
- Citations by year
- 2026: 9 · 2025: 16 · 2024: 21 · 2023: 27 · 2022: 30 · 2021: 71 · 2020: 71 · 2019: 73 · 2018: 39 · 2017: 2
- FWCI
- 35.74
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- cc-by
- Locations
- Journal of Artificial Intelligence Research (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (1552533)
- Compute Canada
- Alberta Machine Intelligence Institute
- DeepMind
- Alberta Innovates - Technology Futures
- Alberta Innovates
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
- Keywords
- Arcade Learning Environment, evaluation methodology, benchmark results, deep Q-network, game modes, evaluation best practices
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
