Procedural Content Generation: Better Benchmarks for Transfer Reinforcement Learning
Open accessAbstract
The idea of transfer in reinforcement learning (TRL) is intriguing: being able to transfer knowledge from one problem to another problem without learning everything from scratch. This promises quicker learning and learning more complex methods. To gain an insight into the field and to detect emerging trends, we performed a database search. We note a surprisingly late adoption of deep learning that starts in 2018. The introduction of deep learning has not yet solved the greatest challenge of TRL: generalization. Transfer between different domains works well when domains have strong similarities (e.g. MountainCar to Cartpole), and most TRL publications focus on different tasks within the same domain that have few differences. Most TRL applications we encountered compare their improvements against self-defined baselines, and the field is still missing unified benchmarks. We consider this to be a disappointing situation. For the future, we note that: (1) A clear measure of task similarity is needed. (2) Generalization needs to improve. Promising approaches merge deep learning with planning via MCTS or introduce memory through LSTMs. (3) The lack of benchmarking tools will be remedied to enable meaningful comparison and measure progress. Already Alchemy and Meta-World are emerging as interesting benchmark suites. We note that another development, the increase in procedural content generation (PCG), can improve both benchmarking and generalization in TRL.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Transfer learning in RL (TRL) generally assumes some form of extra training in the deployment or target environment.
“TRL generally assumes some form of extra training in the deployment or target environment, whereas we are focused on zero-shot generalisation.”
From A Survey of Zero-shot Generalisation in Deep Reinforcement Learning (2023) · cites this paper as Müller-Brockhausen et al. (2021)
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Fast unfolding of communities in large networks
Blondel, V. D., Guillaume, J.-L., Lambiotte, R., & Lefebvre, E. (2008) · Journal of Statistical Mechanics: Theory and Experiment · doi:10.1088/1742-5468/2008/10/p10008 · Cited by 21,424 · see citing works
Weiss, K., Khoshgoftaar, T. M., & Wang, D. (2016) · Journal of Big Data · doi:10.1186/s40537-016-0043-6 · Cited by 6,260 · see citing works
MuJoCo: A physics engine for model-based control
Todorov, E., Erez, T., & Tassa, Y. (2012) · 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2012.6386109 · Cited by 4,767 · see citing works
Grandmaster level in StarCraft II using multi-agent reinforcement learning
Vinyals, O., Babuschkin, I., Czarnecki, W. M., Mathieu, M., Dudzik, A., Chung, J., Choi, D. H., Powell, R., Ewalds, T., Georgiev, P., et al. (2019) · Nature · doi:10.1038/s41586-019-1724-z · Cited by 3,455 · see citing works
Jacomy, M., Venturini, T., Heymann, S., & Bastian, M. (2014) · PLoS ONE · doi:10.1371/journal.pone.0098679 · Cited by 2,909 · see citing works
AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles
Shah, S., Dey, D., Lovett, C., & Kapoor, A. (2018) · Springer Proceedings in Advanced Robotics · doi:10.1007/978-3-319-67361-5_40 · Cited by 2,254 · see citing works
Transfer Learning for Reinforcement Learning Domains: A Survey
Taylor, M. E., & Stone, P. (2009) · doi:10.5555/1577069.1755839 · Cited by 1,521 · see citing works
Hierarchical Reinforcement Learning with the MAXQ Value Function Decomposition
Dietterich, T. G. (2000) · Journal of Artificial Intelligence Research · doi:10.1613/jair.639 · Cited by 1,464 · see citing works
Mastering Atari, Go, chess and shogi by planning with a learned model
Schrittwieser, J., Antonoglou, I., Hubert, T., Simonyan, K., Sifre, L., Schmitt, S., Guez, A., Lockhart, E., Hassabis, D., Graepel, T., et al. (2020) · RePEc: Research Papers in Economics · Cited by 1,310 · see citing works
Mastering Atari, Go, chess and shogi by planning with a learned model
Schrittwieser, J., Antonoglou, I., Hubert, T., Simonyan, K., Sifre, L., Schmitt, S., Guez, A., Lockhart, E., Hassabis, D., Graepel, T., et al. (2020) · Nature · doi:10.1038/s41586-020-03051-4 · Cited by 1,214 · see citing works
Benchmarking Deep Reinforcement Learning for Continuous Control
Duan, Y., Chen, X., Houthooft, R., Schulman, J., & Abbeel, P. (2016) · arXiv · doi:10.48550/arxiv.1604.06778 · Cited by 1,114 · see citing works
Dota 2 with Large Scale Deep Reinforcement Learning
OpenAI, :, Berner, C., Brockman, G., Chan, B., Cheung, V., Dębiak, P., Dennison, C., Farhi, D., Fischer, Q., et al. (2019) · arXiv · doi:10.48550/arxiv.1912.06680 · Cited by 1,065 · see citing works
An Overview of Microsoft Academic Service (MAS) and Applications
Sinha, A., Shen, Z., Song, Y., Ma, H., Eide, D., Hsu, B.-J. P., & Wang, K. (2015) · International World Wide Web Conference (WWW) · doi:10.1145/2740908.2742839 · Cited by 968 · see citing works
Transfer Learning in Deep Reinforcement Learning: A Survey
Zhu, Z., Lin, K., Jain, A. K., & Zhou, J. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3292075 · Cited by 861 · see citing works
Curriculum Learning for Reinforcement Learning Domains: A Framework and SurveyPreprint
Narvekar, S., Peng, B., Leonetti, M., Sinapov, J., Taylor, M. E., & Stone, P. (2020) · Journal of machine learning research · doi:10.48550/arxiv.2003.04960 · Cited by 843 · see citing works
A Hypercube-Based Encoding for Evolving Large-Scale Neural Networks
Stanley, K. O., D’Ambrosio, D. B., & Gauci, J. (2009) · Artificial Life · doi:10.1162/artl.2009.15.2.15202 · Cited by 781 · see citing works
Dyna, an Integrated Architecture for Learning, Planning, and Reacting
Sutton, R. S. (1991) · ACM SIGART Bulletin · doi:10.1145/122344.122377 · Cited by 777 · cited 1× · see citing works
“One trend that could help overcome this problem, at least for navigation-related tasks, is the fusion of learning and planning, as in model-based reinforcement learning [70].”
BackgroundLangley, P. (1995) · Medical Entomology and Zoology · Cited by 512 · see citing works
RL$^2$: Fast Reinforcement Learning via Slow Reinforcement Learning
Duan, Y., Schulman, J., Chen, X., Bartlett, P. L., Sutskever, I., & Abbeel, P. (2016) · arXiv · doi:10.48550/arxiv.1611.02779 · Cited by 503 · see citing works
Carbon Emissions and Large Neural Network Training
Patterson, D., Gonzalez, J., Le, Q., Liang, C., Munguia, L.-M., Rothchild, D., So, D., Texier, M., & Dean, J. (2021) · arXiv · doi:10.48550/arxiv.2104.10350 · Cited by 486 · see citing works
Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning
Yu, T., Quillen, D., He, Z., Julian, R., Narayan, A., Shively, H., Bellathur, A., Hausman, K., Finn, C., & Levine, S. (2019) · arXiv · doi:10.48550/arxiv.1910.10897 · Cited by 450 · see citing works
Emergent Tool Use From Multi-Agent Autocurricula
Baker, B., Kanitscheider, I., Markov, T., Wu, Y., Powell, G., McGrew, B., & Mordatch, I. (2019) · arXiv · doi:10.48550/arxiv.1909.07528 · Cited by 424 · cited 1× · see citing works
“As PCG has proven to be a reliable tool to improve generalization performance in RL [78], [79], the adoption of PCG is the logical next step for TRL.”
MethodA Survey on Transfer Learning for Multiagent Reinforcement Learning Systems
Silva, F. L. D., & Costa, A. H. R. (2019) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.11396 · Cited by 273 · see citing works
Transfer in Reinforcement Learning: A Framework and a Survey
Lazaric, A. (2012) · Adaptation, learning, and optimization · doi:10.1007/978-3-642-27645-3_5 · Cited by 270 · see citing works
Ecoffet, A., Huizinga, J., Lehman, J., Stanley, K. O., & Clune, J. (2021) · Nature · doi:10.1038/s41586-020-03157-9 · Cited by 235 · see citing works
Building portable options: skill transfer in reinforcement learning
Konidaris, G., & Barto, A. G. (2007) · Cited by 220 · see citing works
Darla Improving zero-shot transfer in reinforcement learning
Higgins, I., Pal, A., Rusu, A. A., Matthey, L., Burgess, C. P., Pritzel, A., Botvinick, M., Blundell, C., & Lerchner, A. (2017) · arXiv · doi:10.48550/arxiv.1707.08475 · Cited by 219 · see citing works
Ecoffet, A., Huizinga, J., Lehman, J., Stanley, K. O., & Clune, J. (2021) · RePEc: Research Papers in Economics · Cited by 201 · see citing works
Leveraging Procedural Generation to Benchmark Reinforcement Learning
Cobbe, K., Hesse, C., Hilton, J., & Schulman, J. (2019) · arXiv · doi:10.48550/arxiv.1912.01588 · Cited by 196 · see citing works
Pérez-Liébana, D., Liu, J., Khalifa, A., Gaina, R. D., Togelius, J., & Lucas, S. M. (2019) · IEEE Transactions on Games · doi:10.1109/tg.2019.2901021 · Cited by 185 · see citing works
Finding Structure in Reinforcement Learning
Thrun, S., & Schwartz, A. (1994) · Cited by 176 · see citing works
StarCraft Micromanagement With Reinforcement Learning and Curriculum Transfer Learning
Shao, K., Zhu, Y., & Zhao, D. (2018) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2018.2823329 · Cited by 175 · see citing works
Mocanu, E., Nguyen, P. H., Kling, W. L., & Gibescu, M. (2016) · Energy and Buildings · doi:10.1016/j.enbuild.2016.01.030 · Cited by 171 · see citing works
A Survey of Multi-Task Deep Reinforcement Learning
Varghese, N. V., & Mahmoud, Q. H. (2020) · Electronics · doi:10.3390/electronics9091363 · Cited by 159 · see citing works
Perma: Scoping and Addressing the Problem of Link and Reference Rot in Legal Citations
Zittrain, J. L., Albert, K., & Lessig, L. (2013) · Legal Information Management · doi:10.1017/s1472669614000255 · Cited by 127 · see citing works
Färber, M. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-30796-7_8 · Cited by 126 · see citing works
Illuminating Generalization in Deep Reinforcement Learning through Procedural Level Generation
Justesen, N., Torrado, R. R., Bontrager, P., Khalifa, A., Togelius, J., & Risi, S. (2018) · OAR@UM (University of Malta) · doi:10.48550/arxiv.1806.10729 · Cited by 117 · see citing works
Knowledge Transfer for Deep Reinforcement Learning with Hierarchical Experience Replay
Yin, H., & Pan, S. J. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10733 · Cited by 116 · see citing works
A Beamforming Method Based on Polarization Matching
Yang, X., Gu, J., Jiang, J., & Li, Y. (2020) · IEEE International Conference on Advanced Infocomm Technology · doi:10.1109/icait51223.2020.9315404 · Cited by 107 · see citing works
A Beamforming Method Based on Polarization MatchingPreprint
Yang, X., Gu, J., Jiang, J., et al. (2020) · EasyChair preprint · doi:10.29007/qwpk · Cited by 107 · see citing works
Unsupervised Cross-Domain Transfer in Policy Gradient Reinforcement Learning via Manifold Alignment
Ammar, H. B., Eaton, E., Ruvolo, P., & Taylor, M. E. (2015) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v29i1.9631 · Cited by 93 · see citing works
Sim2real transfer for reinforcement learning without dynamics randomization
Kaspar, M., Munoz Osorio, J. D., & Bock, J. (2020) · 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9341260 · Cited by 87 · see citing works
Anwar, A., & Raychowdhury, A. (2020) · IEEE Access · doi:10.1109/access.2020.2971172 · Cited by 85 · see citing works
An Evolutionary Transfer Reinforcement Learning Framework for Multiagent Systems
Hou, Y., Ong, Y.-S., Feng, L., & Żurada, J. M. (2017) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2017.2664665 · Cited by 85 · see citing works
OpenML: A Collaborative Science Platform
van Rijn, J. N., Bischl, B., Torgo, L., Gao, B., Umaashankar, V., Fischer, S., Winter, P., Wiswedel, B., Berthold, M. R., & Vanschoren, J. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-40994-3_46 · Cited by 77 · see citing works
Adversarial Reinforcement Learning for Procedural Content Generation
Gissl'en, L., Eakins, A., Gordillo, C., Bergdahl, J., & Tollmar, K. (2021) · 2021 IEEE Conference on Games (CoG) · doi:10.1109/cog52621.2021.9619053 · Cited by 73 · see citing works
Transfer Learning for Related Reinforcement Learning Tasks via Image-to-Image Translation
Gamrian, S., & Goldberg, Y. (2018) · arXiv · doi:10.48550/arxiv.1806.07377 · Cited by 70 · see citing works
Assessing Transferability From Simulation to Reality for Reinforcement Learning
Muratore, F., Gienger, M., & Peters, J. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2952353 · Cited by 67 · see citing works
Transfer learning for reinforcement learning on a physical robot
Taylor, M. E. (2010) · Cited by 65 · see citing works
Using Homomorphisms to transfer options across continuous reinforcement learning domains
Soni, V. D., & Singh, S. P. (2006) · Cited by 63 · see citing works
An automated measure of MDP similarity for transfer in reinforcement learning
Ammar, H. B., Eaton, E., Taylor, M. E., Mocanu, D. C., Driessens, K., Weiß, G., & Tuyls, K. (2014) · Cited by 61 · see citing works
Skill Acquisition Via Transfer Learning and Advice Taking
Torrey, L. A., SHAVLIK, J. W., Walker, T., et al. (2006) · Lecture notes in computer science · doi:10.1007/11871842_41 · Cited by 60 · see citing works
An Introduction to Intertask Transfer for Reinforcement Learning
Taylor, M. E., & STONE, P. R. (2011) · AI Magazine · doi:10.1609/aimag.v32i1.2329 · Cited by 55 · see citing works
Transfer Learning Applied to Reinforcement Learning-Based HVAC Control
Lissa, P., Schukat, M., & Barrett, E. (2020) · SN Computer Science · doi:10.1007/s42979-020-00146-7 · Cited by 45 · see citing works
Grounding Language for Transfer in Deep Reinforcement Learning
Narasimhan, K., Barzilay, R., & Jaakkola, T. (2018) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.11263 · Cited by 44 · see citing works
Task similarity measures for transfer in reinforcement learning task libraries
Carroll, J. L., & Seppi, K. D. (2006) · IEEE International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2005.1555955 · Cited by 42 · see citing works
Learning to Search with MCTSnetsPreprint
Guez, A., Weber, T., Antonoglou, I., Simonyan, K., Vinyals, O., Wierstra, D., Munos, R., & Silver, D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.04697 · Cited by 41 · see citing works
Curriculum Learning for Reinforcement Learning Domains: A Framework and Survey
Narvekar, S., Peng, B., Leonetti, M., Sinapov, J., Taylor, M. E., & Stone, P. (2020) · White Rose Research Online (University of Leeds, The University of Sheffield, University of York) · Cited by 41 · see citing works
Efficient Deep Reinforcement Learning via Adaptive Policy Transfer
Yang, T., Hao, J., Meng, Z., Zhang, Z., Hu, Y., Chen, Y., Fan, C., Wang, W., Liu, W., Wang, Z., et al. (2020) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/428 · Cited by 37 · see citing works
Cross-Domain Transfer in Reinforcement Learning Using Target Apprentice
Joshi, G., & Chowdhary, G. V. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8462977 · Cited by 33 · see citing works
Using Cases as Heuristics in Reinforcement Learning: A Transfer Learning Application
Junior, L. A. C., Matsuura, J. P., de Mántaras, R. L., & Bianchi, R. A. C. (2011) · DIGITAL.CSIC (Spanish National Research Council (CSIC)) · Cited by 28 · see citing works
VPE: Variational Policy Embedding for Transfer Reinforcement Learning
Atnekvist, I., Kragić, D., & Stork, J. A. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8793556 · Cited by 27 · see citing works
Single Episode Policy Transfer in Reinforcement LearningPreprint
Yang, J., Petersen, B. K., Hongyuan Zha, B., & Faissol, D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.07719 · Cited by 25 · see citing works
Traoré, R., Caselles-Dupré, H., Lesort, T., Sun, T., Díaz-Rodríguez, N., & Filliat, D. (2019) · arXiv · doi:10.48550/arxiv.1906.04452 · Cited by 24 · see citing works
A Framework for Reinforcement Learning and Planning.Preprint
Moerland, T. M., Broekens, J., & Jonker, C. M. (2020) · arXiv (Cornell University) · Cited by 20 · see citing works
Yoon, I., Anwar, M. A., Joshi, R., Rakshit, T., & Raychowdhury, A. (2019) · IEEE Journal on Emerging and Selected Topics in Circuits and Systems · doi:10.1109/jetcas.2019.2932285 · Cited by 18 · see citing works
Barekatain, M., Yonetani, R., & Hamaya, M. (2020) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/430 · Cited by 17 · see citing works
Automated transfer in reinforcement learning
Ammar, H. B. (2013) · Research Publications (Maastricht University) · doi:10.26481/dis.20130613hb · Cited by 14 · see citing works
ICLR Reproducibility Challenge 2019
Pineau, J., Sinha, K., Fried, G., Ke, R. N., & Larochelle, H. (2019) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.3158243 · Cited by 11 · see citing works
Playing Games in the Dark: An Approach for Cross-Modality Transfer in Reinforcement Learning
Silva, R., Vasco, M., Melo, F. S., Paiva, A., & Veloso, M. (2020) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/znur7157 · Cited by 9 · see citing works
Alchemy: A structured task distribution for meta-reinforcement learning.Preprint
Wang, J. X., King, M. C., Porcel, N., Kurth-Nelson, Z. L., Zhu, T., Deck, C., Choy, P., Cassin, M., Reynolds, M., Song, H., et al. (2021) · arXiv (Cornell University) · Cited by 8 · see citing works
Sonic to knuckles: Evaluations on transfer reinforcement learning
Hamilton, N. P., Schlemmer, L., Menart, C., et al. (2020) · doi:10.1117/12.2559546 · Cited by 7 · see citing works
Evolutionarily-Curated Curriculum Learning for Deep Reinforcement Learning AgentsPreprint
Green, M. C., Sergent, B., Shandilya, P., & Kumar, V. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.05431 · Cited by 7 · see citing works
Xiao, I. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.04058 · Cited by 6 · see citing works
Business Processes Compliance in Partially Observable EnvironmentsPreprint
Silva, I. S., Sousa, P., & Guerreiro, S. L. P. D. (2019) · EasyChair preprint · doi:10.29007/m4db · Cited by 5 · see citing works
Towards Game-Playing AI Benchmarks via Performance Reporting Standards
Volz, V., & Naujoks, B. (2020) · IEEE Conference on Games (CoG) · doi:10.1109/cog47356.2020.9231705 · Cited by 5 · see citing works
Episodic memory transfer for multi-task reinforcement learning
Sorokin, A. Y., & Burtsev, M. (2018) · Biologically Inspired Cognitive Architectures · doi:10.1016/j.bica.2018.09.003 · Cited by 4 · see citing works
A survey of transfer learning methods for reinforcement learning
Bone, N. (2008) · Western CEDAR (Western Washington University) · Cited by 3 · see citing works
Transferring Task Goals via Hierarchical Reinforcement Learning
Xie, S., Galashov, A., Liu, S., et al. (2018) · Cited by 3 · see citing works
Transfer Reinforcement Learning across Environment Dynamics with Multiple Advisors
Plisnier, H. (2019) · VUBIR (Vrije Universiteit Brussel) · Cited by 3 · see citing works
Learning Transfer Automatic through Data Mining in Reinforcement Learning
Arabasadi, Z., & Didkar, N. (2014) · International Journal of Computer Applications · doi:10.5120/15411-3885 · Cited by 1 · see citing works
Transferring experience in reinforcement learning through task decomposition
Partalas, I., Tsoumakas, G., Tzevanidis, K., & Vlahavas, I. P. (2009) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/jrtr4235 · Cited by 1 · see citing works
Miriti, K. A. E. (2014) · University of Nairobi Research Archive (University of Nairobi) · Cited by 1 · see citing works
Flagged first, then by citations
3 of the 86 references couldn’t be matched to a known paper.
Research works printed but not linked: 5
- duan, 2016, R12 Fast reinforcement learning via slow reinforcement learning
- 10.1038/s41586-020-03051-4 · DOI · Paper page
- müller-brockhausen, 2021, Interactive website to explore the data presented in this paper
- pineau, 0, Reproducibility, reusability, and robustness in deep reinforcement learning, International Conference on Learning Representations (ICLR)
- brockman, 2016, OpenAI Gym
Cited by
Sorted by citations
Transfer Learning in Deep Reinforcement Learning: A Survey
Zhu, Z., Lin, K., Jain, A. K., & Zhou, J. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3292075 · Cited by 861 · see citing works
“…for robotics domains, including domain generalization and zero-shot transfer, which is a favored application field of DRL as we discussed in Sec VI. [71] conducted extensive database search and summarized benchmarks for evaluating TL algorithms in RL. [72] surveyed recent progress in multi-task RL.”
MethodA Survey of Zero-shot Generalisation in Deep Reinforcement Learning
Kirk, R., Zhang, A., Grefenstette, E., & Rocktäschel, T. (2023) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.14174 · Cited by 193 · see citing works
“Müller-Brockhausen et al. (2021) and Zhu et al. (2021) survey methods for transfer learning in RL (TRL).”
BackgroundA Survey of Generalisation in Deep Reinforcement LearningPreprint
Kirk, R., Zhang, A., Grefenstette, E., & Rocktäschel, T. (2021) · arXiv (Cornell University) · Cited by 63 · see citing works
Knowledge Transfer for Cross-Domain Reinforcement Learning: A Systematic Review
Serrano, S. A., Martínez-Carranza, J., & Sucar, L. E. (2024) · IEEE Access · doi:10.1109/access.2024.3435558 · Cited by 24 · see citing works
“Moreover, in [47], an analysis of the current state of benchmarks and evaluation methodologies for transfer learning is presented.”
Fukaya, K., Daylamani-Zad, D., & Agius, H. (2024) · ACM Computing Surveys · doi:10.1145/3708499 · Cited by 7 · see citing works
“While existing surveys broadly examine PCG for game content [79, 242, 249], virtual environments [233], buildings [128], deep learning [156], and reinforcement-learning-based PCG [67, 112, 178], none focus on the full range of purely graphical content.”
Mianji, E. M., Muntean, G., & Tal, I. (2025) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2025.3612202 · Cited by 5 · see citing works
Towards verifiable Benchmarks for Reinforcement Learning
Müller-Brockhausen, M., Plaat, A., & Preuß, M. (2022) · IEEE Conference on Games (CoG) · doi:10.1109/cog51982.2022.9893715 · Cited by 3 · see citing works
“However, PCG applied correctly already enables better generalization [44] and more fine-grained training curricula [45], and is thus especially important in benchmarking Transfer in Reinforcement Learning [46].”
MethodSelf-citationLearning state-action features for general game playing
Soemers, D. J. N. J. (2023) · doi:10.26481/dis.20230425ds · Cited by 1 · see citing works
Plaat, A. (2022) · Springer eBooks · doi:10.1007/978-981-19-0638-1_9 · Cited by 1 · see citing works
Self-citationPlaat, A. (2022) · Springer eBooks · doi:10.1007/978-981-19-0638-1_6
Self-citationA Research Agenda for Usability and Generalisation in Reinforcement Learning
Soemers, D. J. N. J., Samothrakis, S., Driessens, K., & Winands, M. (2024) · International Conference on Agents and Artificial Intelligence · doi:10.1007/978-3-032-25032-2_33
“Particularly in the cases of meta RL [137,40,103,119,175,11] and few-shot transfer learning [153,73,102,174], at least a small amount of environment-specific fine-tuning prior to evaluation is also assumed.”
- Authors
- Muller-Brockhausen, M., Preuss, M., & Plaat, A.
- Affiliations
- Leiden Institute of Advanced Computer Science Leiden University,The Netherlands
- Leiden University (NL)
- Venue
- 2021 IEEE Conference on Games (CoG)
- Publisher
- IEEE
- Published
- 2021-08-17
- Pages
- 01-08
- OpenAlex
- W3168730776
- Semantic Scholar
- 5458d8c21735dc40998a6d8077f65fb0afcc5cbb
- Cited by
- 11
- References
- 86
- Citations by year
- 2026: 1 · 2025: 1 · 2024: 2 · 2023: 3 · 2022: 3 · 2021: 1
- FWCI
- 1.04
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Conference on Games (CoG) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Leiden Repository (Leiden University) (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- transfer reinforcement learning, procedural content generation, benchmarking, generalization, task similarity, deep reinforcement learning, Monte Carlo tree search, alchemy, Meta-World, domain transfer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
