The DOI matches its Crossref record; 78 of the 93 references it cites were found in a registry or an index.
Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agents
Open accessAbstract
The Arcade Learning Environment (ALE) is an evaluation platform that poses the challenge of building AI agents with general competency across dozens of Atari 2600 games. It supports a variety of different problem settings and it has been receiving increasing attention from the scientific community, leading to some high-profile success stories such as the much publicized Deep Q-Networks (DQN). In this article we take a big picture look at how the ALE is being used by the research community. We show how diverse the evaluation methodologies in the ALE have become with time, and highlight some key concerns when evaluating agents in the ALE. We use this discussion to present some methodological best practices and provide new benchmark results using these best practices. To further the progress in the field, we introduce a new version of the ALE that supports multiple game modes and provides a form of stochasticity we call sticky actions. We conclude this big picture look by revisiting challenges posed when the ALE was introduced, summarizing the state-of-the-art in various problems and highlighting problems that remain open.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“For instance, the standard protocol for the Atari benchmark (a set of environments that require limited generalisation (Cobbe et al., 2020)) is to use 200M environment interactions per game (Machado et al., 2018)”
From Craftax: A Lightning-Fast Benchmark for Open-Ended Reinforcement Learning (2024) · cites this paper as Machado et al., 2018
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“We abide by Machado et al. (2018)'s recommendations of evaluation protocol, including the use of stochasticity, ignoring the lives signal, and reporting the average performance during training.”
From Loss of Plasticity in Continual Deep Reinforcement Learning (2023) · cites this paper as Machado et al. (2018)
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Self-citationReinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,268
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Bellman, R. E. (2010) · doi:10.1515/9781400835386 · Cited by 14,588
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132
Deep Reinforcement Learning with Double Q-Learning
Van Hasselt, H., Guez, A., & Silver, D. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10295 · Cited by 3,318
Dueling Network Architectures for Deep Reinforcement Learning
Wang, Z., Schaul, T., Hessel, M., van Hasselt, H., Lanctot, M., & de Freitas, N. (2015) · arXiv · doi:10.48550/arxiv.1511.06581 · Cited by 3,300
Schaul, T., Quan, J., Antonoglou, I., & Silver, D. (2015) · arXiv · doi:10.48550/arxiv.1511.05952 · Cited by 3,120
The Arcade Learning Environment: An Evaluation Platform for General Agents
Bellemare, M. G., Naddaf, Y., Veness, J., & Bowling, M. (2013) · Journal of Artificial Intelligence Research · doi:10.1613/jair.3912 · Cited by 2,161
Self-citation
Flagged first, then by citations
15 of the 93 references couldn’t be matched to a known paper.
Research works printed but not linked: 8
- Hausknecht, M.; Lehman, J.; Miikkulainen, R.; and Stone, P. 2013. A neuroevolution approach to general atari game playing. In IEEE Transactions on Computational Intelli- gence and AI in Games.
- Mnih, V.; Kavukcuoglu, K.; Silver, D.; Graves, A.; Antonoglou, I.; Wierstra, D.; and Riedmiller, M. 2013. Playing atari with deep reinforcement learning. CoRR abs/1312.5602. · arXiv
- Albrecht, S. V., L., J. C., Buckeridge, D. L., Botea, A., Caragea, C., Chi, C., Damoulas, T., Dilkina, B. N., Eaton, E., Fazli, P., Ganzfried, S., Lindauer, M. T., Machado, M. C., Malitsky, Y., Marcus, G., Meijer, S., Rossi, F., Shaban-Nejad, A., Thiebaux, S., Veloso, M. M., Walsh, T., Wang, C., Zhang, J., & Zheng, Y. (2015). Reports on the 2015 AAAI Workshop Program. AI Magazine, 36 (2), 90-101.
- Campbell, M., Hoane, Jr., A. J., & Hsu, F.-h. (2002). Deep Blue. Artificial Intelligence, 134 (1-2), 57-83.
- Harutyunyan, A., Bellemare, M. G., Stepleton, T., & Munos, R. (2016). Q(λ) with Off- Policy Corrections. In Proceedings of the International Conference on Algorithmic Learning Theory (ALT), pp. 305-320.
- Hutter, M. (2005). Universal Artificial Intelligence: Sequential Decisions based on Algorith- mic Probability. Springer.
- Kearns, M. J., & Singh, S. P. (2002). Near-Optimal Reinforcement Learning in Polynomial Time. Machine Learning, 49 (2-3), 209-232.
- Nair, A., Srinivasan, P., Blackwell, S., Alcicek, C., Fearon, R., Maria, A. D., Panneershelvam, V., Suleyman, M., Beattie, C., Petersen, S., Legg, S., Mnih, V., Kavukcuoglu, K., & Silver, D. (2015). Massively Parallel Methods for Deep Reinforcement Learning. In ICML Deep Learning Workshop.
Cited by
Sorted by citations
Deep Reinforcement Learning That Matters
Henderson, P., Islam, R., Bachman, P., Pineau, J., Precup, D., & Meger, D. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11694 · Cited by 1,685
Mastering Atari, Go, chess and shogi by planning with a learned model
Schrittwieser, J., Antonoglou, I., Hubert, T., Simonyan, K., Sifre, L., Schmitt, S., Guez, A., Lockhart, E., Hassabis, D., Graepel, T., et al. (2020) · RePEc: Research Papers in Economics · Cited by 1,310
Mastering Atari, Go, chess and shogi by planning with a learned model
Schrittwieser, J., Antonoglou, I., Hubert, T., Simonyan, K., Sifre, L., Schmitt, S., Guez, A., Lockhart, E., Hassabis, D., Graepel, T., et al. (2020) · Nature · doi:10.1038/s41586-020-03051-4 · Cited by 1,214
Meta-Learning in Neural Networks: A Survey
Hospedales, T. M., Antoniou, A., Micaelli, P., & Storkey, A. J. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3079209 · Cited by 886
QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation
Kalashnikov, D., Irpan, A., Pastor, P., Ibarz, J., Herzog, A., Jang, E., Quillen, D., Holly, E., Kalakrishnan, M., Vanhoucke, V., et al. (2018) · arXiv · doi:10.48550/arxiv.1806.10293 · Cited by 697
A survey and critique of multiagent deep reinforcement learning
Hernández-Leal, P., Kartal, B., & Taylor, M. E. (2019) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-019-09421-1 · Cited by 799
Unity: A General Platform for Intelligent Agents
Juliani, A., Berges, V.-P., Teng, E., Cohen, A., Harper, J., Elion, C., Goy, C., Gao, Y., Henry, H., Mattar, M., et al. (2018) · arXiv · doi:10.48550/arxiv.1809.02627 · Cited by 626
Model-based Reinforcement Learning: A Survey
Moerland, T. M., Broekens, J., Plaat, A., & Jonker, C. M. (2023) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000086 · Cited by 871
Exploration by Random Network Distillation
Burda, Y., Edwards, H., Storkey, A., & Klimov, O. (2018) · arXiv · doi:10.48550/arxiv.1810.12894 · Cited by 527
Quantifying Generalization in Reinforcement Learning
Cobbe, K., Klimov, O., Hesse, C., Kim, T., & Schulman, J. (2018) · arXiv · doi:10.48550/arxiv.1812.02341
Deep Learning for Video Game Playing
Justesen, N., Bontrager, P., Togelius, J., & Risi, S. (2019) · IEEE Transactions on Games · doi:10.1109/tg.2019.2896986 · Cited by 304
A Gentle Introduction to Reinforcement Learning and its Application in Different Fields
Naeem, M., Rizvi, S. T. H., & Coronato, A. (2020) · IEEE Access · doi:10.1109/access.2020.3038605 · Cited by 290
A Study on Overfitting in Deep Reinforcement LearningPreprint
Zhang, C., Vinyals, O., Munos, R., & Bengio, S. (2018) · arXiv.org · doi:10.48550/arxiv.1804.06893 · Cited by 447
Li, C. (2019) · Chapman & Hall/CRC eBooks · doi:10.1201/9781351006620-6 · Cited by 257
The Hanabi challenge: A new frontier for AI research
Bard, N., Foerster, J., Chandar, S., BURCH, N. R., Lanctot, M., Song, H. F., Parisotto, E., Dumoulin, V., Moitra, S., Hughes, E., et al. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2019.103216 · Cited by 307
Self-citationEcoffet, A., Huizinga, J., Lehman, J., Stanley, K. O., & Clune, J. (2021) · Nature · doi:10.1038/s41586-020-03157-9 · Cited by 330
Towards Continual Reinforcement Learning: A Review and Perspectives
Khetarpal, K., Riemer, M., Rish, I., & Precup, D. (2022) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.13673 · Cited by 224
TextWorld: A Learning Environment for Text-Based Games
Côté, M.-A., Kádár, Á., Yuan, X., Kybartas, B., Barnes, T., Fine, E., Moore, J. A. Y., Hausknecht, M., Asri, L. E., Adada, M., et al. (2019) · Communications in computer and information science · doi:10.1007/978-3-030-24337-1_3 · Cited by 210
Self-citationEcoffet, A., Huizinga, J., Lehman, J., Stanley, K. O., & Clune, J. (2021) · RePEc: Research Papers in Economics · Cited by 201
A Survey of Zero-shot Generalisation in Deep Reinforcement Learning
Kirk, R., Zhang, A., Grefenstette, E., & Rocktäschel, T. (2023) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.14174 · Cited by 193
Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domain
Hao, J., Yang, T., Tang, H., Bai, C., Liu, J., Meng, Z., Liu, P., & Wang, Z. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3236361 · Cited by 245
Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domain
Hao, J., Yang, T., Tang, H., Bai, C., Liu, J., Meng, Z., Liu, P., & Wang, Z. (2021) · arXiv · doi:10.48550/arxiv.2109.06668 · Cited by 192
Self-Supervised Exploration via DisagreementPreprint
Pathak, D., Gandhi, D., & Gupta, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.04161 · Cited by 190
Measuring the Occupational Impact of AI: Tasks, Cognitive Abilities and AI Benchmarks
Tolan, S., Pesole, A., Martínez‐Plumed, F., Fernández‐Macías, E., Hernández‐Orallo, J., & Gómez, E. (2021) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.12647 · Cited by 211
An Optimistic Perspective on Offline Reinforcement LearningPreprint
Agarwal, R., Schuurmans, D., & Norouzi, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.04543 · Cited by 217
A Survey of Deep Reinforcement Learning in Video GamesPreprint
Shao, K., Tang, Z., Zhu, Y., Li, N., & Zhao, D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.10944 · Cited by 278
Mastering diverse control tasks through world models
Hafner, D., Pasukonis, J., Ba, J., & Lillicrap, T. (2025) · Nature · doi:10.1038/s41586-025-08744-2 · Cited by 189
“Atari 35 uses 57 tasks with sticky actions 55 .”
Benchmarking Batch Deep Reinforcement Learning AlgorithmsPreprint
Fujimoto, S., Conti, E., Ghavamzadeh, M., & Pineau, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.01708 · Cited by 501
Learn What Not to Learn: Action Elimination with Deep Reinforcement LearningPreprint
Zahavy, T., Haroush, M., Merlis, N., Mankowitz, D. J., & Mannor, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.02121 · Cited by 146
Futures of artificial intelligence through technology readiness levels
Martínez-Plumed, F., Gómez, E., & Hernández-Orallo, J. (2021) · Telematics and Informatics · doi:10.1016/j.tele.2020.101525 · Cited by 130
Interactive Fiction Games: A Colossal Adventure
Hausknecht, M., Ammanabrolu, P., Côté, M.-A., & Yuan, X. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6297 · Cited by 158
Self-citationAssessing Generalization in Deep Reinforcement Learning
Packer, C., Gao, K., Kos, J., Krähenbühl, P., Koltun, V., & Song, D. (2018) · arXiv · doi:10.48550/arxiv.1810.12282 · Cited by 114
Efficient Exploration Through Bayesian Deep Q-Networks
Azizzadenesheli, K., Brunskill, E., & Anandkumar, A. (2018) · Information Theory and Applications Workshop (ITA) · doi:10.1109/ita.2018.8503252 · Cited by 137
GEP-PG: Decoupling Exploration and Exploitation in Deep Reinforcement Learning AlgorithmsPreprint
Colas, C., Sigaud, O., & Oudeyer, P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.05054 · Cited by 124
Gotta Learn Fast: A New Benchmark for Generalization in RLPreprint
Nichol, A., Pfau, V., Hesse, C., Klimov, O., & Schulman, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.03720 · Cited by 136
OpenSpiel: A Framework for Reinforcement Learning in GamesPreprint
Lanctot, M., Lockhart, E., Lespiau, J.-B., Zambaldi, V., Upadhyay, S., Pérolat, J., Srinivasan, S., Timbers, F., Tuyls, K., Omidshafiei, S., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.09453 · Cited by 104
Data-Efficient Reinforcement Learning with Self-Predictive RepresentationsPreprint
Schwarzer, M., Anand, A., Goel, R., Hjelm, R. D., Courville, A. C., & Bachman, P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.05929 · Cited by 246
Mastering Diverse Domains through World Models
Hafner, D., Pasukonis, J., Ba, J., & Lillicrap, T. (2023) · arXiv · doi:10.48550/arxiv.2301.04104 · Cited by 124
“We use the sticky action simulator setting 39 .”
MethodObstacle Tower: A Generalization Challenge in Vision, Control, and Planning
Juliani, A., Khalifa, A., Berges, V.-P., Harper, J., Teng, E., Henry, H., Crespi, A., Togelius, J., & Lange, D. (2019) · Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/373 · Cited by 93
Generalization and Regularization in DQN
Farebrother, J., Machado, M. C., & Bowling, M. (2018) · arXiv · doi:10.48550/arxiv.1810.00123 · Cited by 92
Self-citationA Dissection of Overfitting and Generalization in Continuous Reinforcement Learning
Zhang, A., Ballas, N., & Pineau, J. (2018) · arXiv · doi:10.48550/arxiv.1806.07937 · Cited by 88
Distributional Reinforcement Learning
Bellemare, M. G., Dabney, W., & Rowland, M. (2023) · The MIT Press eBooks · doi:10.7551/mitpress/14207.001.0001 · Cited by 102
Self-citationLearning Montezuma's Revenge from a Single DemonstrationPreprint
Salimans, T., & Chen, R. J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.03381 · Cited by 152
ChainerRL: A Deep Reinforcement Learning LibraryPreprint
Fujita, Y., Nagarajan, P., Kataoka, T., & Ishikawa, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.03905 · Cited by 81
Model Based Reinforcement Learning for Atari
Kaiser, Ł., Babaeizadeh, M., Miłoś, P., Osiński, B., Campbell, R. H., Czechowski, K., Erhan, D., Finn, C., Kozakowski, P., Levine, S., et al. (2020) · International Conference on Learning Representations · Cited by 75
Time Limits in Reinforcement LearningPreprint
Pardo, F., Tavakoli, A., Levdik, V., & Kormushev, P. S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.00378 · Cited by 99
Evolving simple programs for playing atari games
Wilson, D. G., Cussat‐Blanc, S., Luga, H., & Miller, J. F. (2018) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/3205455.3205578 · Cited by 71
RIDE: Rewarding Impact-Driven Exploration for Procedurally-Generated EnvironmentsPreprint
Răileanu, R., & Rocktäschel, T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.12292 · Cited by 131
A Survey of Generalisation in Deep Reinforcement LearningPreprint
Kirk, R., Zhang, A., Grefenstette, E., & Rocktäschel, T. (2021) · arXiv (Cornell University) · Cited by 63
Exploring the Limitations of Behavior Cloning for Autonomous Driving
Codevilla, F., Santana, E., López, A. M., & Gaidon, A. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00942 · Cited by 603
A Review of Uncertainty for Deep Reinforcement Learning
Lockwood, O., & Si, M. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment · doi:10.1609/aiide.v18i1.21959 · Cited by 63
Behaviour Suite for Reinforcement LearningPreprint
Osband, I., Doron, Y., Hessel, M., Aslanides, J., Sezener, E., Saraiva, A., McKinney, K., Lattimore, T., Szepesvári, C., Singh, S. P., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.03568 · Cited by 126
Contingency-Aware Exploration in Reinforcement LearningPreprint
Choi, J., Guo, Y., Moczulski, M., Oh, J., Wu, N., Norouzi, M., & Lee, H. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.01483 · Cited by 69
Is multiagent deep reinforcement learning the answer or the question? A brief surveyPreprint
Hernández-Leal, P., Kartal, B., & Taylor, M. E. (2018) · arXiv (Cornell University) · Cited by 56
Learning with AMIGo: Adversarially Motivated Intrinsic GoalsPreprint
Campero, A., Răileanu, R., Küttler, H., Tenenbaum, J. B., Rocktäschel, T., & Grefenstette, E. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.12122 · Cited by 102
Google Research Football: A Novel Reinforcement Learning Environment
Kurach, K., Raichuk, A., Stańczyk, P., Zając, M., Bachem, O., Espeholt, L., Riquelme, C., Vincent, D., Michalski, M., Bousquet, O., et al. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.5878 · Cited by 102
Hyperbolic Discounting and Learning over Multiple HorizonsPreprint
Fedus, W., Gelada, C., Bengio, Y., Bellemare, M. G., & Larochelle, H. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.06865 · Cited by 131
Self-citationDeterministic Implementations for Reproducibility in Deep Reinforcement LearningPreprint
Nagarajan, P., Warnell, G., & Stone, P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.05676 · Cited by 61
Making Efficient Use of Demonstrations to Solve Hard Exploration ProblemsPreprint
Paine, T. L., Gülçehre, Ç., Shahriari, B., Denil, M., Hoffman, M., Soyer, H., Tanburn, R., Kapturowski, S., Rabinowitz, N. C., Williams, D., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.01387 · Cited by 49
Observational Overfitting in Reinforcement Learning
Song, X., Jiang, Y., Tu, S., Du, Y., & Neyshabur, B. (2019) · arXiv · doi:10.48550/arxiv.1912.02975 · Cited by 48
Automatic Data Augmentation for Generalization in Deep Reinforcement LearningPreprint
Răileanu, R., Goldstein, M., Yarats, D., Kostrikov, I., & Fergus, R. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.12862 · Cited by 101
Automatic Data Augmentation for Generalization in Reinforcement LearningPreprint
Răileanu, R., Goldstein, M., Yarats, D., Kostrikov, I., & Fergus, R. (2021) · arXiv (Cornell University) · Cited by 44
Learning Abstract OptionsPreprint
Riemer, M., Liu, M., & Tesauro, G. J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.11583 · Cited by 43
Re-evaluating EvaluationPreprint
Balduzzi, D., Tuyls, K., Pérolat, J., & Graepel, T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.02643 · Cited by 52
Learning Purely Tactile In-Hand Manipulation with a Torque-Controlled Hand
Sievers, L., Pitz, J., & Bäuml, B. (2022) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra46639.2022.9812093 · Cited by 41
MinAtar: An Atari-Inspired Testbed for Thorough and Reproducible Reinforcement Learning ExperimentsPreprint
Young, K., & Tian, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.03176 · Cited by 156
Continual Reinforcement Learning in 3D Non-stationary Environments
Lomonaco, V., Desai, K., Culurciello, E., & Maltoni, D. (2020) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00132 · Cited by 41
Distributed Bayesian optimization of deep reinforcement learning algorithms
Young, M. T., Hinkle, J. D., Kannan, R., & Ramanathan, A. (2020) · Journal of Parallel and Distributed Computing · doi:10.1016/j.jpdc.2019.07.008 · Cited by 40
Revisiting Rainbow: Promoting more Insightful and Inclusive Deep Reinforcement Learning ResearchPreprint
Obando-Ceron, J., & Castro, P. S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.14826 · Cited by 64
Canaan, R., Salge, C., Togelius, J., & Nealen, A. (2019) · International Conference on the Foundations of Digital Games (FDG) · doi:10.1145/3337722.3337750 · Cited by 55
The Text-Based Adventure AI Competition
Atkinson, T. J., Baier, H., Copplestone, T., Devlin, S., & Swan, J. (2019) · IEEE Transactions on Games · doi:10.1109/tg.2019.2896017 · Cited by 36
Measuring and characterizing generalization in deep reinforcement learning
Witty, S., Lee, J. K., Tosch, E., Atrey, A., Clary, K., Littman, M. L., & Jensen, D. D. (2021) · Applied AI Letters · doi:10.1002/ail2.45 · Cited by 35
Safe Q-Learning Method Based on Constrained Markov Decision Processes
Ge, Y., Zhu, F., Ling, X., & Liu, Q. (2019) · IEEE Access · doi:10.1109/access.2019.2952651 · Cited by 35
The Animal-AI Environment: Training and Testing Animal-Like Artificial CognitionPreprint
Beyret, B., Hernández-Orallo, J., Cheke, L., Halina, M., Shanahan, M., & Crosby, M. (2019) · arXiv (Cornell University) · doi:10.4230/lipics.time.2024.5 · Cited by 46
Probabilistic Guarantees for Safe Deep Reinforcement Learning
Bacci, E., & Parker, D. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-57628-8_14 · Cited by 33
Dextrous Tactile In-Hand Manipulation Using a Modular Reinforcement Learning Architecture
Pitz, J., Röstel, L., Sievers, L., & Bäuml, B. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10160756 · Cited by 33
An Equivalence between Loss Functions and Non-Uniform Sampling in Experience ReplayPreprint
Fujimoto, S., Meger, D., & Precup, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.06049 · Cited by 33
Online and Offline Reinforcement Learning by Planning with a Learned ModelPreprint
Schrittwieser, J., Hubert, T., Mandhane, A., Barekatain, M., Antonoglou, I., & Silver, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.06294 · Cited by 33
Advances and challenges in learning from experience replay
Neves, D. E., Ishitani, L., & Patrocínio, Z. K. G. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-11062-0 · Cited by 31
Learning Deployable Navigation Policies at Kilometer Scale from a Single TraversalPreprint
Bruce, J., Sünderhauf, N., Mirowski, P., Hadsell, R. T., & Milford, M. J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.05211 · Cited by 33
Using a Team of General AI Algorithms to Assist Game Design and Testing
Guerrero-Romero, C., Lucas, S. M., & Pérez-Liébana, D. (2018) · IEEE Conference on Computational Intelligence and Games (CIG) · doi:10.1109/cig.2018.8490417 · Cited by 29
Do recent advancements in model-based deep reinforcement learning really improve data efficiency?Preprint
Kielak, K. (2019) · arXiv (Cornell University) · Cited by 29
Naturally Interpretable Control Policies via Graph-Based Genetic Programming
Nadizar, G., Medvet, E., & Wilson, D. G. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-56957-9_5 · Cited by 30
Benchmarking Crisis in Social Media Analytics: A Solution for the Data-Sharing Problem
Assenmacher, D., Weber, D., Preuß, M., Valdez, A. S. C., Bradshaw, A., Roß, B., Cresci, S., Trautmann, H., Neumann, F., & Grimme, C. (2021) · Social Science Computer Review · doi:10.1177/08944393211012268 · Cited by 27
Off-Policy Evaluation via Off-Policy ClassificationPreprint
Irpan, A., Rao, K., Bousmalis, K., Harris, C., Ibarz, J., & Levine, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.01624 · Cited by 34
Gossip-based Actor-Learner Architectures for Deep Reinforcement LearningPreprint
Assran, M., Romoff, J., Ballas, N., Pineau, J., & Rabbat, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.04585 · Cited by 28
Optimization of Neuroprosthetic Vision via End-to-end Deep Reinforcement Learning
Küçükoğlu, B., Rueckauer, B., Ahmad, N., de Ruyter van Steveninck, J., Güçlü, U., & van Gerven, M. (2022) · doi:10.1101/2022.02.25.482017 · Cited by 25
Optimization of Neuroprosthetic Vision via End-to-End Deep Reinforcement Learning
Küçükoğlu, B., Rueckauer, B., Ahmad, N., van Steveninck, J. D. R., Güçlü, U., & van Gerven, M. A. J. (2022) · International Journal of Neural Systems · doi:10.1142/s0129065722500526 · Cited by 25
Navigating the landscape of multiplayer games
Omidshafiei, S., Tuyls, K., Czarnecki, W. M., Santos, F. C., Rowland, M., Connor, J., Hennes, D., Muller, P., Pérolat, J., Vylder, B. D., et al. (2020) · Nature Communications · doi:10.1038/s41467-020-19244-4 · Cited by 34
TrojDRL: Trojan Attacks on Deep Reinforcement Learning AgentsPreprint
Kiourti, P., Wardega, K., Jha, S., & Li, W. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.06638 · Cited by 42
Combating the Compounding-Error Problem with a Multi-step ModelPreprint
Asadi, K., Misra, D. K., Kim, S., & Littman, M. L. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.13320 · Cited by 69
Regularized Softmax Deep Multi-Agent $Q$-LearningPreprint
Pan, L., Rashid, T., Peng, B., Huang, L., & Whiteson, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.11883 · Cited by 29
Safe option-critic: learning safety in the option-critic architecture
Jain, A., Khetarpal, K., & Precup, D. (2021) · The Knowledge Engineering Review · doi:10.1017/s0269888921000035 · Cited by 24
Rolling Horizon Evolutionary Algorithms for General Video Game Playing
Gaina, R. D., Devlin, S., Lucas, S. M., & Pérez-Liébana, D. (2021) · IEEE Transactions on Games · doi:10.1109/tg.2021.3060282 · Cited by 24
(2018) · arXiv (Cornell University) · Cited by 24
Task-Agnostic Dynamics Priors for Deep Reinforcement LearningPreprint
Du, Y., & Narasimhan, K. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.04819 · Cited by 29
Decoupling Value and Policy for Generalization in Reinforcement LearningPreprint
Răileanu, R., & Fergus, R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.10330 · Cited by 39
Generalizable Episodic Memory for Deep Reinforcement LearningPreprint
Hu, H., Ye, J., Zhu, G., Ren, Z., & Zhang, C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.06469 · Cited by 35
Exploration by Maximizing Renyi Entropy for Reward-Free RL Framework
Zhang, C., Cai, Y., Huang, L., & Li, J. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i12.17297 · Cited by 27
On Bonus-Based Exploration Methods in the Arcade Learning EnvironmentPreprint
Taïga, A. A., Fedus, W., Machado, M. C., Courville, A. C., & Bellemare, M. G. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.11052 · Cited by 46
Self-citation
Metadata
- Authors
- Machado, M. C., Bellemare, M. G., Talvitie, E., Veness, J., Hausknecht, M., & Bowling, M.
- Affiliations
- University of Alberta (CA)
- Franklin & Marshall College (US)
- Google DeepMind (United Kingdom) (GB)
- Microsoft (United States) (US)
- Venue
- Journal of Artificial Intelligence Research
- Publisher
- AI Access Foundation
- Published
- 2018-03-19
- Volume
- 61
- Pages
- 523-562
- ISSN
- 1076-9757 (ISSN-L)
- OpenAlex
- W2754879180
- Cited by
- 359
- References
- 93
- Citations by year
- 2026: 9 · 2025: 16 · 2024: 21 · 2023: 27 · 2022: 30 · 2021: 71 · 2020: 71 · 2019: 73 · 2018: 39
- FWCI
- 35.74
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- cc-by
- Locations
- Journal of Artificial Intelligence Research (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (1552533)
- Compute Canada
- Alberta Machine Intelligence Institute
- DeepMind
- Alberta Innovates - Technology Futures
- Alberta Innovates
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
- Keywords
- Arcade Learning Environment, evaluation methodology, benchmark results, deep Q-network, game modes, evaluation best practices
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
