Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agents

Machado, Marlos C., Bellemare, Marc G., Talvitie, Erik, Veness, Joel, Hausknecht, Matthew, Bowling, Michael·2018·Journal of Artificial Intelligence Research
DOI 10.1613/jair.5699

Cited by (358)

Items 101–200 of 358

  1. An Overview of Environmental Features that Impact Deep Reinforcement Learning in Sparse-Reward Domains
    Jim Martin Catacora Ocana, Roberto Capobianco, Daniele Nardi · 2023 · Journal of Artificial Intelligence Research
  2. Variational Dynamic for Self-Supervised Exploration in Deep Reinforcement Learning
    Chenjia Bai, Peng Liu, Kaiyu Liu, Lingxiao Wang, Yingnan Zhao, Lei Han, Zhaoran Wang · 2021 · IEEE Transactions on Neural Networks and Learning Systems
  3. Leveling the Playing Field -- Fairness in AI Versus Human Game Benchmarks
    Rodrigo Canaan, Christoph Salge, Julian Togelius, Andy Nealen · 2019 · arXiv (Cornell University)
  4. RAIDER: Reinforcement-Aided Spear Phishing Detector
    Keelan Evans, Alsharif Abuadbba, Tingmin Wu, Kristen Moore, Mohiuddin Ahmed, Ganna Pogrebna, ‪Surya Nepal‬, Mike Johnstone · 2022 · Lecture notes in computer science
  5. Interpretable pipelines with evolutionary optimized modules for reinforcement learning tasks with visual inputs
    Leonardo Lucio Custode, Giovanni Iacca · 2022 · Genetic and Evolutionary Computation Conference Companion (GECCO Companion)
  6. The Impact of Nondeterminism on Reproducibility in Deep Reinforcement Learning
    Prabhat Nagarajan, Garrett Warnell, Peter Stone · 2018
  7. Rotation, Translation, and Cropping for Zero-Shot Generalization
    Chang Ye, Ahmed Khalifa, Philip Bontrager, Julian Togelius · 2020 · IEEE Conference on Games (CoG)
  8. Evolutionary reinforcement learning via cooperative coevolutionary negatively correlated search
    Peng Yang, Hu Zhang, Yanglong Yu, Mingjia Li, Ke Tang · 2021 · Swarm and Evolutionary Computation
  9. On Inductive Biases in Deep Reinforcement Learning
    Matteo Hessel, Hado P. van Hasselt, Joseph Modayil, David Silver · 2019 · arXiv (Cornell University)
  10. Benchmarking Bonus-Based Exploration Methods on the Arcade Learning Environment
    Adrien Ali Taïga, William Fedus, Marlos C. Machado, Aaron C. Courville, Marc G. Bellemare · 2019 · arXiv (Cornell University)
  11. Is Deep Reinforcement Learning Really Superhuman on Atari? Leveling the playing field
    Marin Toromanoff, Émilie Wirbel, Fabien Moutarde · 2019 · arXiv (Cornell University)
  12. Optimistic Exploration even with a Pessimistic Initialisation
    Tabish Rashid, Peng, Bei, Wendelin Böhmer, Shimon Whiteson · 2020 · arXiv (Cornell University)
  13. Count-based exploration with the successor representation
    Machado, Marlos C., Bellemare, Marc G., Bowling, Michael · 2020 · Proceedings of the AAAI Conference on Artificial Intelligence
  14. Attentive Multi-task Deep Reinforcement Learning
    Timo Bräm, Gino Brunner, Oliver Richter, Roger P. Wattenhofer · 2020 · Lecture notes in computer science
  15. Emergent Policy Discovery for Visual Reinforcement Learning Through Tangled Program Graphs: A Tutorial
    Stephen Kelly, Robert J. Smith, Malcolm Iain Heywood · 2019 · Genetic and evolutionary computation
  16. CLIC: Curriculum Learning and Imitation for Object Control in Nonrewarding Environments
    Pierre Fournier, Cédric Colas, Mohamed Chétouani, Olivier Sigaud · 2019 · IEEE Transactions on Cognitive and Developmental Systems
  17. Targeted Attacks on Deep Reinforcement Learning Agents through Adversarial Observations
    L'eonard Hussenot, M. Geist, O. Pietquin · 2019 · arXiv.org
  18. Ludii as a Competition Platform
    Matthew Stephenson, Éric Piette, Dennis J. N. J. Soemers, Cameron Browne · 2019 · IEEE Conference on Games (CoG)
  19. Continual deep reinforcement learning with task-agnostic policy distillation
    Muhammad Burhan Hafez, Kerim Erekmen · 2024 · Scientific Reports
  20. A Guide for the Design of Benchmark Environments for Building Energy Optimization
    David Wölfle, Arun Vishwanath, Hartmut Schmeck · 2020 · ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation (BuildSys)
  21. Muesli: Combining Improvements in Policy Optimization
    Matteo Hessel, Ivo Danihelka, Fabio Viola, Arthur Guez, Simon Schmitt, Laurent Sifre, Théophane Weber, David Silver, Hado P. van Hasselt · 2021 · arXiv (Cornell University)
  22. Recent advances in leveraging human guidance for sequential decision-making tasks
    Ruohan Zhang, Faraz Torabi, Garrett Warnell, Peter Stone · 2021 · Autonomous Agents and Multi-Agent Systems
  23. Deep Reinforcement Learning based Video Games: A Review
    Kamal A. ElDahshan, Hesham Farouk, Eslam Mofreh · 2022 · International Mobile, Intelligent, and Ubiquitous Computing Conference (MIUCC)
  24. From pixels to planning: scale-free active inference
    Karl John Friston, Conor Heins, Tim Verbelen, Lancelot Da Costa, Tommaso Salvatori, Dimitrije Marković, Alexander Tschantz, Magnus Koudahl, Christopher Buckley, Thomas Parr · 2025 · Frontiers in Network Physiology
  25. Accelerating Reinforcement Learning through GPU Atari Emulation
    Steven Dalton, Iuri Frosio, Michael Garland · 2019 · arXiv (Cornell University)
  26. CORA: Benchmarks, Baselines, and Metrics as a Platform for Continual Reinforcement Learning Agents
    Sam Powers, Eliot Xing, Eric Kolve, Roozbeh Mottaghi, A. Gupta · 2021 · CoLLAs
  27. Examining Average and Discounted Reward Optimality Criteria in Reinforcement Learning
    Vektor Dewanto, Marcus Reginald Gallagher · 2022 · Lecture notes in computer science
  28. Searching for a Diversity of Interpretable Graph Control Policies
    Giorgia Nadizar, Eric Medvet, Dennis G. Wilson · 2024 · Genetic and Evolutionary Computation Conference (GECCO)
  29. Benchmarking the Spectrum of Agent Capabilities
    Hafner, Danijar · 2021 · arXiv
  30. Dynamic Bottleneck for Robust Self-Supervised Exploration
    Chenjia Bai, Lingxiao Wang, Lei Han, Animesh Garg, Jianye Hao, Peng Liu, Zhaoran Wang · 2021 · arXiv (Cornell University)
  31. Metatrace Actor-Critic: Online Step-Size Tuning by Meta-gradient Descent for Reinforcement Learning Control
    Kenny Young, Baoxiang Wang, Matthew Edmund Taylor · 2019 · International Joint Conference on Artificial Intelligence
  32. Interference and Generalization in Temporal Difference Learning
    Bengio, Emmanuel, Pineau, Joelle, Precup, Doina · 2020 · arXiv
  33. Nondeterminism and Instability in Neural Network Optimization
    Cecilia Summers, Michael J. Dinneen · 2021 · arXiv (Cornell University)
  34. Shielding Atari Games with Bounded Prescience
    Mirco Giacobbe, Mohammadhosein Hasanbeig, Daniel Kroening, Hjalmar Wijk · 2021 · Adaptive Agents and Multi-Agents Systems
  35. A Reinforcement Learning Method of Solving Markov Decision Processes: An Adaptive Exploration Model Based on Temporal Difference Error
    Xianjia Wang, Zhipeng Yang, Guici Chen, Yanli Liu · 2023 · Electronics
  36. Solving Task Scheduling Problems in Dew Computing via Deep Reinforcement Learning
    Pablo Sanabria, Tomás Felipe Tapia, Rodrigo Toro Icarte, Andrés Neyem · 2022 · Applied Sciences
  37. Modern Value Based Reinforcement Learning: A Chronological Review
    Mark C. McKenzie, Mark D. McDonnell · 2022 · IEEE Access
  38. Deriving Subgoals Autonomously to Accelerate Learning in Sparse Reward Domains
    Michael Dann, Fabio Zambetta, John Thangarajah · 2019 · Proceedings of the AAAI Conference on Artificial Intelligence
  39. Surprising Negative Results for Generative Adversarial Tree Search
    Kamyar Azizzadenesheli, Brandon Yang, Weitang Liu, Zachary C. Lipton, Animashree Anandkumar · 2018 · arXiv (Cornell University)
  40. Guided Exploration with Proximal Policy Optimization using a Single Demonstration
    Gabriele Libardi, Gianni De Fabritiis · 2020 · arXiv (Cornell University)
  41. Reward Propagation Using Graph Convolutional Networks
    Martin Klissarov, Doina Precup · 2020 · arXiv (Cornell University)
  42. Deep Model-Based Reinforcement Learning for High-Dimensional Problems, a Survey
    Aske Plaat, Walter A. Kosters, Mike Preuß · 2020 · arXiv (Cornell University)
  43. Work-in-Progress: MLGOPerf: An ML Guided Inliner to Optimize Performance
    Amir Hossein Ashouri, Mostafa Elhoushi, Yuzhe Hua, Xiang Wang, Muhammad Asif Manzoor, Bryan K. Chan, Yaoqing Gao · 2022 · International Conference on Compilers, Architecture, and Synthesis for Embedded Systems (CASES)
  44. Rogue-Gym: A New Challenge for Generalization in Reinforcement Learning
    Yuji Kanagawa, Tomoyuki Kaneko · 2019 · IEEE Conference on Games (CoG)
  45. Navigating the landscape of multiplayer games
    Shayegan Omidshafiei, Karl Tuyls, Wojciech Marian Czarnecki, Francisco C. Santos, Mark Rowland, Jerome T. Connor, Daniel Hennes, Paul Friedrich Müller, Julien Pérolat, Bart De Vylder et al. · 2020 · RePEc: Research Papers in Economics
  46. Investigating the properties of neural network representations in reinforcement learning
    Han Wang, Erfan Miahi, Martha White, Marlos C. Machado, Zaheer Abbas, Raksha Kumaraswamy, Vincent Liu, Adam White · 2024 · Artificial Intelligence
  47. SLM Lab: A Comprehensive Benchmark and Modular Software Framework for Reproducible Deep Reinforcement Learning
    Keng Wah Loon, Laura Graesser, Milan W. Cvitkovic · 2019 · arXiv (Cornell University)
  48. Reinforcement Learning Generalization with Surprise Minimization
    Jerry Zikun Chen · 2020 · arXiv (Cornell University)
  49. Spectral Normalisation for Deep Reinforcement Learning: an Optimisation Perspective
    Florin Gogianu, Tudor Berariu, Mihaela Rosca, Claudia Clopath A, Lucian Buşoniu, Razvan Pascanu · 2021 · arXiv (Cornell University)
  50. Revising bus routes to improve access for the transport disadvantaged: A reinforcement learning approach
    Sunhyung Yoo, Jinwoo Brian Lee · 2023 · Journal of Public Transportation
  51. Modeling the Dynamics of Knowledge Potential of Agents in the Educational Social and Communication Environment
    Андрій Ярославович Бомба, Mariia Nazaruk, Nataliia E. Kunanets, Volodymyr V. Pasichnyk · 2019 · Advances in intelligent systems and computing
  52. Enhanced Pub/Sub Communications for Massive IoT Traffic with SARSA Reinforcement Learning
    Carlos E. Arruda, Pedro F. Moraes, Nazim Agoulmine, Joberto S. B. Martins · 2021 · Lecture notes in computer science
  53. Explaining Deep Reinforcement Learning Agents in the Atari Domain through a Surrogate Model
    Alexander Sieusahai, Matthew Guzdial · 2021 · Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment
  54. P3O: Policy-on Policy-off Policy Optimization
    Rasool Fakoor, Pratik Chaudhari, Alexander Johannes Smola · 2019 · arXiv (Cornell University)
  55. Model-Based Deep Reinforcement Learning for High-Dimensional Problems, a Survey.
    Aske Plaat, Walter A. Kosters, Mike Preuß · 2020 · arXiv (Cornell University)
  56. When should agents explore?
    Mîruna Pislar, David Szepesvári, Georg Ostrovski, Diana Borsa, Tom Schaul · 2021 · arXiv (Cornell University)
  57. Difference Based Metrics for Deep Reinforcement Learning Algorithms
    Bernardo Augusto Godinho de Oliveira, Carlos A. P. S. Martins, Flávia Magalhães, Luís F. W. Góes · 2019 · IEEE Access
  58. From Simulation to Real World Maneuver Execution using Deep Reinforcement Learning
    Alessandro Paolo Capasso, Giulio Bacchiani, Alberto Broggi · 2020 · IEEE Intelligent Vehicles Symposium
  59. Adapting Behaviour for Learning Progress
    Tom Schaul, Diana Borsa, Ding, David, David Szepesvári, Georg Ostrovski, Will Dabney, Simon Osindero · 2019 · arXiv (Cornell University)
  60. Measuring Visual Generalization in Continuous Control from Pixels
    Grigsby, Jake, Qi, Yanjun · 2020 · arXiv
  61. FleetRL: Realistic reinforcement learning environments for commercial vehicle fleets
    Enzo Cording, Jagruti Thakur · 2024 · SoftwareX
  62. Planning With Pixels in (Almost) Real Time
    Wilmer Bandres, Blai Bonet, Héctor Geffner · 2018 · Proceedings of the AAAI Conference on Artificial Intelligence
  63. Evolution of Inherently Interpretable Visual Control Policies
    Camilo De La Torre, Giorgia Nadizar, Yuri Cossich Lavinas, Hervé Luga, Dennis G. Wilson, Sylvain Cussat‐Blanc · 2025 · Genetic and Evolutionary Computation Conference (GECCO)
  64. A Review On The Relationship Between Human And Artificial Imagination With Their Implementations In Current Machine Learning Algorithms
    Md. Abdullah-Al-Kafi, Md. Fahad Hossain, Sheak Rashed Haider Noori · 2021 · International Conference on Computing, Communication and Networking Technologies (ICCCNT)
  65. OPtions as REsponses: Grounding behavioural hierarchies in multi-agent reinforcement learning
    Alexander Sasha Vezhnevets, Yuhuai Wu, Maria K. Eckstein, Rémi Leblond, Joel Z. Leibo · 2020 · International Conference on Machine Learning
  66. Research on Action Strategies and Simulations of DRL and MCTS-based Intelligent Round Game
    Yuxiang Sun, Bo Yuan, Yongliang Zhang, Wanwen Zheng, Qingfeng Xia, Bojian Tang, Xianzhong Zhou · 2021 · International Journal of Control Automation and Systems
  67. Adaptive Skip Intervals: Temporal Abstraction for Recurrent Dynamical Models
    Alexander Neitz, Giambattista Parascandolo, Stefan Bauer, Bernhard Schölkopf · 2018 · arXiv (Cornell University)
  68. Toybox: A Suite of Environments for Experimental Evaluation of Deep Reinforcement Learning
    Emma Tosch, Kaleigh Clary, John Foley, David D. Jensen · 2019 · arXiv (Cornell University)
  69. Using a Logarithmic Mapping to Enable Lower Discount Factors in Reinforcement Learning
    Harm van Seijen, Mehdi Fatemi, Arash Tavakoli · 2019 · arXiv (Cornell University)
  70. Options as responses: Grounding behavioural hierarchies in multi-agent RL
    Alexander Sasha Vezhnevets, Yuhuai Wu, Rémi Leblond, Joel Z. Leibo · 2019 · arXiv (Cornell University)
  71. SuperSuit: Simple Microwrappers for Reinforcement Learning Environments
    Justin K. Terry, Benjamin Black, Ananth Hari · 2020 · arXiv (Cornell University)
  72. Bootstrapped Meta-Learning
    Sebastian Flennerhag, Yannick Schroecker, Tom Zahavy, Hado P. van Hasselt, David Silver, Satinder Pal Singh · 2021 · arXiv (Cornell University)
  73. CARL: A Benchmark for Contextual and Adaptive Reinforcement Learning
    Carolin Benjamins, Theresa Eimer, Frederik Schubert, André Biedenkapp, Bodo Rosenhahn, Frank Hutter, Marius Lindauer · 2021 · arXiv (Cornell University)
  74. Game AI Research with Fast Planet Wars Variants
    Simon Mark Lucas · 2018 · IEEE Conference on Computational Intelligence and Games (CIG)
  75. Towards a common environment for learning scheduling algorithms
    Renato L. F. Cunha, Luiz Chaimowicz · 2020 · Proceedings - International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems
  76. Targeted Attacks on Deep Reinforcement Learning Agents through Adversarial Observations.
    Léonard Hussenot, Matthieu Geist, Olivier Pietquin · 2019 · arXiv (Cornell University)
  77. ZPD Teaching Strategies for Deep Reinforcement Learning from Demonstrations
    Daniel Seita, David M. Chan, Roshan Rao, Chen Tang, Zhao Mandi, John Canny · 2019 · arXiv (Cornell University)
  78. Self-Imitation Learning via Trajectory-Conditioned Policy for Hard-Exploration Tasks
    Yijie Guo, Jongwook Choi, Marcin Moczulski, Samy Bengio A, Mohammad Norouzi, Honglak Lee · 2019 · arXiv (Cornell University)
  79. Learning to Track Dynamic Targets in Partially Known Environments
    Heejin Jeong, Hamed Hassani, Manfred Morari, Daniel D. Lee, George J. Pappas · 2020 · arXiv (Cornell University)
  80. Evaluating Agents without Rewards
    Brendon Matusch, Jimmy Lei Ba, Danijar Hafner · 2020 · arXiv (Cornell University)
  81. Systematic Evaluation of Causal Discovery in Visual Model Based Reinforcement Learning
    Nan Rosemary Ke, Aniket Didolkar, Sarthak Mittal, Anirudh Goyal, Guillaume Lajoie, Stefan Bauer, Danilo Jimenez Rezende, Yoshua Bengio, Michael C. Mozer, Christopher Pal · 2021 · arXiv (Cornell University)
  82. AndroidEnv: A Reinforcement Learning Platform for Android
    Daniel Toyama, Hamel, Philippe, Anita Gergely, Gheorghe Comanici, Amelia Glaese, Zafarali Ahmed, Tyler Jackson, Shibl Mourad, Doina Precup · 2021 · arXiv (Cornell University)
  83. Loss of Plasticity in Continual Deep Reinforcement Learning
    Abbas, Zaheer, Zhao, Rosie, Modayil, Joseph, White, Adam, Machado, Marlos C. · 2023 · arXiv
  84. Deep Conservative Policy Iteration
    Nino Vieillard, Olivier Pietquin, Matthieu Geist · 2020 · Proceedings of the AAAI Conference on Artificial Intelligence
  85. Reinforcement learning for fuselage shape control during aircraft assembly
    Tim Lutz, Yinan Wang, Xiaowei Yue, Jaime A. Camelio · 2024 · IISE Transactions
  86. Exploring Teachable Humans and Teachable Agents: Human Strategies Versus Agent Policies and the Basis of Expertise
    John C. Stamper, Steven Moore · 2019 · Lecture notes in computer science
  87. Deep Reinforcement Learning with Decorrelation
    Borislav Mavrin, Hengshuai Yao, Linglong Kong · 2019 · arXiv (Cornell University)
  88. Temporal Difference Uncertainties as a Signal for Exploration
    Sebastian Flennerhag, Jane X. Wang, Pablo Sprechmann, Francesco Visin, Alexandre Galashov, Steven Kapturowski, Diana Borsa, Nicolas Heess, André da Motta Salles Barreto, Razvan Pascanu · 2020 · arXiv (Cornell University)
  89. MiniZero: Comparative Analysis of AlphaZero and MuZero on Go, Othello, and Atari Games
    Ti-Rong Wu, Hung Guei, Pei-Chiun Peng, Po-Wei Huang, Ting-Han Wei, Chung-Chin Shih, Yun-Jui Tsai · 2024 · IEEE Transactions on Games
  90. Learning to Follow Directions in Street View
    Karl Moritz Hermann, Mateusz Malinowski, Piotr Mirowski, Andras Banki-Horvath, Keith Anderson, Raia T. Hadsell · 2020 · Proceedings of the AAAI Conference on Artificial Intelligence
  91. Design and Development of a Benchmark for Dynamic Multi-objective Optimisation Problem in the Context of Deep Reinforcement Learning
    Mahmudul Hasan, Khin T. Lwin, Antesar M. Shabut, Mohammad Alamgir Hossain · 2019 · International Conference on Computer and Information Technology (ICCIT)
  92. Learning key steps to attack deep reinforcement learning agents
    Chien-Min Yu, Ming-Hsin Chen, Hsuan-Tien Lin · 2023 · Machine Learning
  93. Distributional reinforcement learning with unconstrained monotonic neural networks
    Thibaut Théate, Antoine Wehenkel, Adrien Bolland, Gilles Louppe, Damien Ernst · 2023 · Neurocomputing
  94. Microscopic Model-Based RL Approaches for Traffic Signal Control Generalize Better than Model-Free RL Approaches
    Parth Jaggi, Xiaoyu Wang, Nicolas Carrara, Scott Sanner, Baher Abdulhai · 2021 · IEEE International Conference on Intelligent Transportation Systems (ITSC)
  95. Deep Policies for Width-Based Planning in Pixel Domains
    Miquel Junyent, Anders Jönsson, Vicenç Gómez · 2019 · Proceedings of the International Conference on Automated Planning and Scheduling
  96. Temporal Memory Sharing in Visual Reinforcement Learning
    Stephen Kelly, Wolfgang Banzhaf · 2020 · Genetic and evolutionary computation
  97. The Eigenoption-Critic Framework
    Miao Liu, Marlos C. Machado, Gerald J. Tesauro, Murray S. Campbell · 2017 · arXiv (Cornell University)
  98. Towards a Simple Approach to Multi-step Model-based Reinforcement Learning
    Kavosh Asadi, Evan Cater, Dipendra Kumar Misra, Michael L. Littman · 2018 · arXiv (Cornell University)
  99. Importance of using appropriate baselines for evaluation of data-efficiency in deep reinforcement learning for Atari
    Kacper Kielak · 2020 · arXiv (Cornell University)
  100. Measuring Progress in Deep Reinforcement Learning Sample Efficiency
    Dorner, Florian E. · 2021 · arXiv