Protecting against evaluation overfitting in empirical reinforcement learning
Abstract
Empirical evaluations play an important role in machine learning. However, the usefulness of any evaluation depends on the empirical methodology employed. Designing good empirical methodologies is difficult in part because agents can overfit test evaluations and thereby obtain misleadingly high scores. We argue that reinforcement learning is particularly vulnerable to environment overfitting and propose as a remedy generalized methodologies, in which evaluations are based on multiple environments sampled from a distribution. In addition, we consider how to summarize performance when scores from different environments may not have commensurate values. Finally, we present proof-of-concept results demonstrating how these methodologies can validate an intuitively useful range-adaptive tile coding method.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Evaluating RL policies on exactly the same environment they were trained on is likely to lead to strong evaluation overfitting.
“This is in stark contrast to the standard assumptions of supervised learning where the training and testing sets are disjoint and is likely to lead to strong evaluation overfitting (Whiteson et al., 2011).”
From A Survey of Zero-shot Generalisation in Deep Reinforcement Learning (2023) · cites this paper as Whiteson et al., 2011
Cites
UCI Machine Learning Repository
Asuncion, A. (2007) · Medical Entomology and Zoology · Cited by 24,190 · see citing works
Statistical Comparisons of Classifiers over Multiple Data Sets
Demšar, J. (2006) · doi:10.5555/1248547.1248548 · Cited by 11,966 · see citing works
Bellman, R. (1957) · Indiana University Mathematics Journal · doi:10.1512/iumj.1957.6.56038 · Cited by 2,408 · see citing works
The Lack of A Priori Distinctions Between Learning Algorithms
Wolpert, D. H. (1996) · Neural Computation · doi:10.1162/neco.1996.8.7.1341 · Cited by 1,652 · see citing works
Improving Generalization with Active Learning
Cohn, D. A., Atlas, L., & Ladner, R. E. (1994) · Machine Learning · doi:10.1023/a:1022673506211 · Cited by 1,597 · see citing works
Transfer Learning for Reinforcement Learning Domains: A Survey
Taylor, M. E., & Stone, P. (2009) · doi:10.5555/1577069.1755839 · Cited by 1,521 · see citing works
Self-citationThrun, S., & Pratt, L. (1998) · Springer eBooks · doi:10.1007/978-1-4615-5529-2 · Cited by 1,291 · see citing works
Robust classification for imprecise environments
Provost, F., & Fawcett, T. (2001) · Machine Learning · doi:10.1023/a:1007601015854 · Cited by 1,250 · see citing works
Generalization in Reinforcement Learning: Successful Examples Using Sparse Coarse Coding
Sutton, R. S. (1995) · Cited by 1,073 · see citing works
On Comparing Classifiers: Pitfalls to Avoid and a Recommended Approach
Salzberg, S. L. (1997) · Data Mining and Knowledge Discovery · doi:10.1023/a:1009752403260 · Cited by 822 · see citing works
A Model of Inductive Bias Learning
Baxter, J. (2000) · Cited by 730 · see citing works
MultiBoosting: A Technique for Combining Boosting and Wagging
Webb, G. I. (2000) · Machine Learning · doi:10.1023/a:1007659514849 · Cited by 698 · see citing works
Testing heuristics: We have it all wrong
Hooker, J. (1995) · Journal of Heuristics · doi:10.1007/bf02430364 · Cited by 438 · see citing works
Dixon, W. J., & Mood, A. M. (1946) · Journal of the American Statistical Association · doi:10.1080/01621459.1946.10501898 · Cited by 418 · see citing works
The BellKor Solution to the Netflix Grand Prize
Koren, Y. (2009) · Cited by 340 · see citing works
An analytic solution to discrete Bayesian reinforcement learning
Poupart, P., Vlassis, N. A., Hoey, J., & Regan, K. (2006) · International Conference on Machine Learning · doi:10.1145/1143844.1143932 · Cited by 278 · see citing works
Multi-task reinforcement learning
Wilson, A., Fern, A., Ray, S., & Tadepalli, P. V. (2007) · International Conference on Machine Learning · doi:10.1145/1273496.1273624 · Cited by 262 · see citing works
Artificial Intelligence: An Empirical Science
Simon, H. (1995) · Artificial Intelligence · doi:10.1016/0004-3702(95)00039-h · Cited by 239 · see citing works
Dataset Issues in Object Recognition
Ponce, J., Berg, T. L., Everingham, M., Forsyth, D. A., Hebert, M., Lazebnik, S., Marszałek, M., Schmid, C., Russell, B. C., Torralba, A. B., et al. (2006) · Lecture notes in computer science · doi:10.1007/11957959_2 · Cited by 205 · see citing works
Machine learning as an experimental science
Langley, P. (1988) · Machine Learning · doi:10.1007/bf00115008 · Cited by 187 · see citing works
Konidaris, G., & Barto, A. G. (2006) · International Conference on Machine Learning · doi:10.1145/1143844.1143906 · Cited by 184 · see citing works
The Replacement of General-Purpose Learning Models with Adaptively Specialized Learning Modules
Gallistel, C. R. (2000) · Cited by 173 · see citing works
Saari, D. G., & Merlin, V. R. (1996) · Economic Theory · doi:10.1007/bf01212012 · Cited by 102 · see citing works
RL-Glue: Language-Independent Software for Reinforcement-Learning Experiments
Tanner, B. K., & White, A. (2009) · Cited by 95 · see citing works
Self-citationHow evaluation guides AI research
Coehn, P., & Howe, A. E. (1988) · AI Magazine · doi:10.1609/aimag.v9i4.952 · Cited by 88 · see citing works
The Reinforcement Learning Competitions
Whiteson, S., Tanner, B. K., & White, A. (2010) · AI Magazine · doi:10.1609/aimag.v31i2.2227 · Cited by 44 · see citing works
Self-citationNeuroevolutionary reinforcement learning for generalized helicopter control
Koppejan, R., & Whiteson, S. A. (2009) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/1569901.1569922 · Cited by 38 · see citing works
Self-citationRL-Glue: Language-Independent Software for Reinforcement-Learning Experiments
TannerBrian, & WhiteAdam (2009) · Journal of Machine Learning Research · doi:10.5555/1577069.1755857 · Cited by 35 · see citing works
Warning: statistical benchmarking is addictive. Kicking the habit in machine learning
Drummond, C., & Japkowicz, N. (2009) · Journal of Experimental & Theoretical Artificial Intelligence · doi:10.1080/09528130903010295 · Cited by 31 · see citing works
Chapter 18 Limitations on conclusions using scales of measurement
Roberts, F. S. (1994) · Handbooks in operations research and management science · doi:10.1016/s0927-0507(05)80099-4 · Cited by 19 · see citing works
Falkenauer, E. (1998) · Journal of Heuristics · doi:10.1023/a:1009617801681 · Cited by 15 · see citing works
Multi-task evolutionary shaping without pre-specified representations
Snel, M., & Whiteson, S. (2010) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/1830483.1830671 · Cited by 11 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
The Arcade Learning Environment: An Evaluation Platform for General Agents
Bellemare, M. G., Naddaf, Y., Veness, J., & Bowling, M. (2013) · Journal of Artificial Intelligence Research · doi:10.1613/jair.3912 · Cited by 2,161 · see citing works
Deep Reinforcement Learning That Matters
Henderson, P., Islam, R., Bachman, P., Pineau, J., Precup, D., & Meger, D. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11694 · Cited by 1,685 · see citing works
A survey and critique of multiagent deep reinforcement learning
Hernández-Leal, P., Kartal, B., & Taylor, M. E. (2019) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-019-09421-1 · Cited by 747 · see citing works
Self-citationA Study on Overfitting in Deep Reinforcement LearningPreprint
Zhang, C., Vinyals, O., Munos, R., & Samy Bengio, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.06893 · Cited by 246 · see citing works
AutoCompress: An Automatic DNN Structured Pruning Framework for Ultra-High Compression Rates
Liu, N., Ma, X., Xu, Z., Wang, Y., Tang, J., & Ye, J. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.5924 · Cited by 206 · see citing works
The Malmo platform for artificial intelligence experimentation
Johnson, M., Hofmann, K., Hutton, T. J., & Bignell, D. E. (2016) · International Joint Conference on Artificial Intelligence · Cited by 205 · see citing works
Evaluation in artificial intelligence: from task-oriented to ability-oriented measurement
Hernández‐Orallo, J. (2016) · Artificial Intelligence Review · doi:10.1007/s10462-016-9505-7 · Cited by 208 · see citing works
A Survey of Zero-shot Generalisation in Deep Reinforcement Learning
Kirk, R., Zhang, A., Grefenstette, E., & Rocktäschel, T. (2023) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.14174 · Cited by 193 · see citing works
“This is in stark contrast to the standard assumptions of supervised learning where the training and testing sets are disjoint and is likely to lead to strong evaluation overfitting (Whiteson et al., 2011).”
ResultThe Arcade Learning Environment: An Evaluation Platform for General Agents
Bellemare, M. G., Naddaf, Y., Veness, J., & Bowling, M. (2012) · arXiv · doi:10.48550/arxiv.1207.4708 · Cited by 187 · see citing works
Measuring the Occupational Impact of AI: Tasks, Cognitive Abilities and AI Benchmarks
Tolan, S., Pesole, A., Martínez‐Plumed, F., Fernández‐Macías, E., Hernández‐Orallo, J., & Gómez, E. (2021) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.12647 · Cited by 181 · see citing works
What Makes Reading Comprehension Questions Easier?
Sugawara, S., Inui, K., Sekine, S., & Aizawa, A. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1453 · Cited by 105 · see citing works
A reinforcement learning approach to autonomous decision-making in smart electricity markets
Peters, M., Ketter, W., Saar‐Tsechansky, M., & Collins, J. E. (2013) · Machine Learning · doi:10.1007/s10994-013-5340-0 · Cited by 105 · see citing works
A deep generative model for molecule optimization via one fragment modification
Chen, Z., Min, M. R., Parthasarathy, S., & Ning, X. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00410-2 · Cited by 95 · see citing works
Generalization and Regularization in DQN
Farebrother, J., Machado, M. C., & Bowling, M. (2018) · arXiv · doi:10.48550/arxiv.1810.00123 · Cited by 92 · see citing works
DeepRacer: Autonomous Racing Platform for Experimentation with Sim2Real Reinforcement Learning
Balaji, B., Mallya, S., Genç, Ş., Gupta, S., Dirac, L., Khare, V. R., Roy, G., Sun, T., Tao, Y., Townsend, B., et al. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9197465 · Cited by 122 · see citing works
A Survey of Generalisation in Deep Reinforcement LearningPreprint
Kirk, R., Zhang, A., Grefenstette, E., & Rocktäschel, T. (2021) · arXiv (Cornell University) · Cited by 63 · see citing works
An automated measure of MDP similarity for transfer in reinforcement learning
Ammar, H. B., Eaton, E., Taylor, M. E., Mocanu, D. C., Driessens, K., Weiß, G., & Tuyls, K. (2014) · Cited by 61 · see citing works
Self-citationIs multiagent deep reinforcement learning the answer or the question? A brief surveyPreprint
Hernández-Leal, P., Kartal, B., & Taylor, M. E. (2018) · arXiv (Cornell University) · Cited by 56 · see citing works
Self-citationOn Learning in Collective Self-Adaptive Systems: State of Practice and a 3D Framework
D’Angelo, M., Gerasimou, S., Ghahremani, S., Grohmann, J., Nunes, I., Pournaras, E., & Tomforde, S. (2019) · International Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS) · doi:10.1109/seams.2019.00012 · Cited by 52 · see citing works
CausalWorld: A Robotic Manipulation Benchmark for Causal Structure and Transfer Learning
Ahmed, O., Träuble, F., Goyal, A., Neitz, A., Bengio, Y., Schölkopf, B., Wüthrich, M., & Bauer, S. (2020) · arXiv · doi:10.48550/arxiv.2010.04296 · Cited by 50 · see citing works
Neuroevolutionary reinforcement learning for generalized control of simulated helicopters
Koppejan, R., & Whiteson, S. (2011) · Evolutionary Intelligence · doi:10.1007/s12065-011-0066-z · Cited by 46 · see citing works
Self-citationAn artificial intelligence-based risk prediction model of myocardial infarction
Liu, R., Wang, M., Zheng, T., Zhang, R., Li, N., Chen, Z., Yan, H., & Shi, Q. (2022) · BMC Bioinformatics · doi:10.1186/s12859-022-04761-4 · Cited by 44 · see citing works
Inqiad, W. B., Siddique, M. S., Alarifi, S. S., Butt, M. J., Najeh, T., & Gamil, Y. (2023) · Heliyon · doi:10.1016/j.heliyon.2023.e22036 · Cited by 43 · see citing works
Kuo, C.-H., Chen, C.-T., Lin, S.-J., & Huang, S.-H. (2021) · IEEE Access · doi:10.1109/access.2021.3068269 · Cited by 42 · see citing works
Exploring reinforcement learning in process control: a comprehensive survey
Rajasekhar, N., Radhakrishnan, T. K., & Samsudeen, N. (2025) · International Journal of Systems Science · doi:10.1080/00207721.2025.2469821 · Cited by 45 · see citing works
Selective Unsupervised Learning-Based Wi-Fi Fingerprint System Using Autoencoder and GAN
Seong, J.-H., & Seo, D.-H. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2019.2956986 · Cited by 41 · see citing works
Towards robust car-following based on deep reinforcement learning
Hart, F., Okhrin, O., & Treiber, M. (2024) · Transportation Research Part C Emerging Technologies · doi:10.1016/j.trc.2024.104486 · Cited by 39 · see citing works
Learning potential functions and their representations for multi-task reinforcement learning
Snel, M., & Whiteson, S. (2013) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-013-9235-z · Cited by 36 · see citing works
Self-citationMeasuring and characterizing generalization in deep reinforcement learning
Witty, S., Lee, J. K., Tosch, E., Atrey, A., Clary, K., Littman, M. L., & Jensen, D. D. (2021) · Applied AI Letters · doi:10.1002/ail2.45 · Cited by 35 · see citing works
Hullman, J. R., Kapoor, S., Nanayakkara, P., Gelman, A., & Narayanan, A. (2022) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3514094.3534196 · Cited by 37 · see citing works
Balaji, B., Mallya, S., Genc, S., Gupta, S., Dirac, L., Khare, V., Roy, G., Sun, T., Tao, Y., Townsend, B., et al. (2019) · arXiv · doi:10.48550/arxiv.1911.01562 · Cited by 27 · see citing works
Inqiad, W. B., Siddique, M. S., Ali, M., & Najeh, T. (2024) · Scientific Reports · doi:10.1038/s41598-024-65905-5 · Cited by 25 · see citing works
A Memetic Algorithm With Reinforcement Learning for Sociotechnical Production Scheduling
Grumbach, F., Badr, N. E. A., Reusch, P., & Trojahn, S. (2023) · IEEE Access · doi:10.1109/access.2023.3292548 · Cited by 24 · see citing works
Vulnerability assessment of heat waves within a risk framework using artificial intelligence
Côté, J.-N., Germain, M., Levac, E., & Lavigne, É. (2023) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2023.169355 · Cited by 23 · see citing works
CybORG: An Autonomous Cyber Operations Research GymPreprint
Baillie, C., Standen, M., Schwartz, J., Docking, M., Bowman, D., & Kim, J. (2020) · arXiv.org · doi:10.48550/arxiv.2002.10667 · Cited by 51 · see citing works
Characterizing reinforcement learning methods through parameterized learning problems
Kalyanakrishnan, S., & Stone, P. (2011) · Machine Learning · doi:10.1007/s10994-011-5251-x · Cited by 20 · see citing works
Self-citationField, E. L., Tam, W., Moore, N., & McEntee, M. F. (2023) · Children · doi:10.3390/children10030576 · Cited by 16 · see citing works
hammer: Multi-level coordination of reinforcement learning agents via learned messaging
Gupta, N., Srinivasaraghavan, G., Mohalik, S. K., Kumar, N., & Taylor, M. E. (2023) · Neural Computing and Applications · doi:10.1007/s00521-023-09096-6 · Cited by 15 · see citing works
Self-citationInterference and Generalization in Temporal Difference Learning
Bengio, E., Pineau, J., & Precup, D. (2020) · arXiv · doi:10.48550/arxiv.2003.06350 · Cited by 14 · see citing works
Relative Variational Intrinsic Control
Baumli, K., Warde-Farley, D., Hansen, S., & Mnih, V. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i8.16832 · Cited by 14 · see citing works
AI Evaluation: past, present and futurePreprint
Hernández‐Orallo, J. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1408.6908 · Cited by 18 · see citing works
Inqiad, W. B., Dumitrascu, E. V., & Dobre, R. A. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e32856 · Cited by 12 · see citing works
Difference Based Metrics for Deep Reinforcement Learning Algorithms
de Oliveira, B. A. G., Martins, C. A. P. S., Magalhães, F., & Góes, L. F. W. (2019) · IEEE Access · doi:10.1109/access.2019.2945879 · Cited by 10 · see citing works
Evaluating Domain Randomization in Deep Reinforcement Learning Locomotion Tasks
Ajani, O. S., Hur, S., & Mallipeddi, R. (2023) · Mathematics · doi:10.3390/math11234744 · Cited by 9 · see citing works
Autonomous Data-Driven Decision-Making in Smart Electricity Markets
Peters, M., Ketter, W., Saar‐Tsechansky, M., & Collins, J. E. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33486-3_9 · Cited by 9 · see citing works
Sang, C.-Y., Huang, S.-H., Chen, C.-T., & Chang, H.-T. (2025) · Applied Soft Computing · doi:10.1016/j.asoc.2025.112846 · Cited by 10 · see citing works
Improving Reinforcement Learning-Based Autonomous Agents with Causal Models
Briglia, G., Lippi, M., Mariani, S., & Zambonelli, F. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-77367-9_20 · Cited by 7 · see citing works
Improved empirical methods in reinforcement-learning evaluation
Marivate, V. N. (2015) · doi:10.7282/t34x59h0 · Cited by 7 · see citing works
Toward Nonlinear Local Reinforcement Learning Rules Through Neuroevolution
Vassiliades, V., & Christodoulou, C. C. (2013) · Neural Computation · doi:10.1162/neco_a_00514 · Cited by 6 · see citing works
Closed-Loop Control of Epilepsy Based on Reinforcement Learning
Dan, R., Zhang, H., & Bai, J. (2025) · International Journal of Neural Systems · doi:10.1142/s0129065725500741 · Cited by 5 · see citing works
Mitigating metropolitan carbon emissions with dynamic eco-driving at scale
Jayawardana, V., Freydt, B., Qu, A., Hickert, C., Sanchez, E. N., Tang, C., Taylor, M., Leonard, B., & Wu, C. (2025) · Transportation Research Part C Emerging Technologies · doi:10.1016/j.trc.2025.105146 · Cited by 5 · see citing works
Self-citationMeta-learning from Learning Curves: Challenge Design and Baseline Results
Nguyen, M. H., Sun-Hosoya, L., Grinsztajn, N., & Guyon, I. (2022) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn55064.2022.9892534 · Cited by 5 · see citing works
Regularization in DQN for Parameter-Varying Control Learning Tasks
Li, D., Lei, C., Jin, Q., & Han, M. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-22808-8_4 · Cited by 4 · see citing works
Introduction to the special issue on empirical evaluations in reinforcement learning
Whiteson, S., & Littman, M. L. (2011) · Machine Learning · doi:10.1007/s10994-011-5255-6 · Cited by 5 · see citing works
Self-citationImproving Generalization in Meta-RL with Imaginary Tasks from Latent Dynamics MixturePreprint
Lee, S., & Chung, S.-Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.13524 · Cited by 4 · see citing works
Regularized Parameter Uncertainty for Improving Generalization in Reinforcement Learning
Moure, P., Cheng, L., Ott, J., Wang, Z., & Liu, S. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02247 · Cited by 3 · see citing works
Gatti, C. J. (2014) · Springer theses · doi:10.1007/978-3-319-12197-0_2 · Cited by 4 · see citing works
RAP Vol: Robust Adversary Populations With Volume Diversity Measure
Yang, J., & Zhang, J. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3317145 · Cited by 3 · see citing works
ToyBox: Better Atari Environments for Testing Reinforcement Learning AgentsPreprint
Foley, J., Tosch, E., Clary, K., & Jensen, D. D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.02850 · Cited by 5 · see citing works
Kim, N., Jung, M., Yoon, I., Park, M., & Ahn, C. R. (2022) · Winter Simulation Conference · doi:10.1109/wsc57314.2022.10015415 · Cited by 2 · see citing works
Vajjhala, N. R. (2024) · Smart innovation, systems and technologies · doi:10.1007/978-981-99-7711-6_21 · Cited by 2 · see citing works
A Q-Learning Approach for Investment Decisions
Varela, M., Viera, O., & Robledo, F. (2016) · Springer eBooks · doi:10.1007/978-3-319-32543-9_18 · Cited by 2 · see citing works
Raman, M., Abdallah, N., & Dunoyer, J. (2019) · International Symposium on Quality Electronic Design (ISQED) · doi:10.1109/isqed.2019.8697652 · Cited by 2 · see citing works
Simple Augmentation Goes a Long Way: ADRL for DNN Quantization
Ning, L., Chen, G., Zhang, W., & Shen, X. (2021) · International Conference on Learning Representations · Cited by 2 · see citing works
LISPR: An Options Framework for Policy Reuse with Reinforcement LearningPreprint
Graves, D., Jin, J., & Luo, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.14942 · Cited by 2 · see citing works
Sparse Attention Guided Dynamic Value Estimation for Single-Task Multi-Scene Reinforcement LearningPreprint
Singh, J., & Zheng, L. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.07266 · Cited by 2 · see citing works
Provably Improved Context-Based Offline Meta-RL with Attention and Contrastive LearningPreprint
Li, L., Huang, Y., Chen, M., Luo, S., Luo, D., & Huang, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.10774 · Cited by 4 · see citing works
SmartPlay: A Benchmark for LLMs as Intelligent Agents
Wu, Y., Tang, X., Mitchell, T. M., & Li, Y. (2023) · arXiv · doi:10.48550/arxiv.2310.01557 · Cited by 6 · see citing works
“The idea of using games to evaluate the performance of agents has a long history in A.I. Pell (2011); Schaul et al. (2011); Whiteson et al. (2011) presented early ideas and motivation for using games to measure the general capabilities of an agent, and discussed challenges in measuring A.I. agent…”
BackgroundExperimental Evaluation of Reinforcement Learning Algorithms
Varma, N. S., Sinha, V., & Rahul, K. (2023) · Lecture notes on data engineering and communications technologies · doi:10.1007/978-981-99-0609-3_33 · Cited by 1 · see citing works
Learning robust driving policies without online exploration
Graves, D., Nguyen, N. M., Hassanzadeh, K., Jin, J., & Luo, J. (2021) · IEEE International Conference on Robotics and Automation · doi:10.1109/icra48506.2021.9561450 · Cited by 3 · see citing works
A Methodology for the Development of RL-Based Adaptive Traffic Signal ControllersPreprint
Varela, G. S., Santos, P. P., Sardinha, A., & Melo, F. S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.09614 · Cited by 2 · see citing works
Machine learning applied to prediction, control and planning from dynamic epidemiological models
Bent, O. (2020) · Oxford University Research Archive (ORA) (University of Oxford) · doi:10.5287/ora-b7qkedzkb · Cited by 1 · see citing works
A note about the generalisation of the C-testsPreprint
Hernández‐Orallo, J. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.8529
Gatti, C. J. (2014) · Springer theses · doi:10.1007/978-3-319-12197-0_1
Real-world applicable deep learning techniques
Liu, N. (2020) · doi:10.17760/d20398269
Plaat, A. (2022) · Springer eBooks · doi:10.1007/978-981-19-0638-1_10
Advanced Research in Pediatric Radiology and Nuclear Medicine
Ng, C. K. C. (2023) · MDPI eBooks · doi:10.3390/books978-3-0365-7958-0
Yin, W., Ragab, S. H., Tyshenko, M. G., Arroyo, T. F., & Oraby, T. F. (2025) · PLoS ONE · doi:10.1371/journal.pone.0333536
Automatic Curriculum Learning for Developmental Machine Learners
Portelas, R. (2022) · theses.fr (ABES) · doi:10.70675/947677c7z2ec1z4b61zbae3z6f358c02b050
Controle Seguro e Evasão de Colisões em Tráfego Denso de Drones via Aprendizado por Reforço
dos Santos, H. J. F., Barros, I. D. S., Bittencourt, L. F., Kamienski, C., & de Oliveira, F. M. C. (2026) · Anais do Workshop de Computação Urbana (CoUrb) · doi:10.5753/courb.2026.23126
Evaluating the Robustness of RL-based Autonomous Intersection Control to Data Integrity Attacks
Huszák, Á., Aghabagherloo, A., Yoshizawa, T., & Preneel, B. (2026) · 2026 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops) · doi:10.1109/percomworkshops68308.2026.11585292
Generalizable Graph-based Reinforcement Learning Agents for Automated Cyber Defense
King, I. J., Bowman, B., & Huang, H. H. (2026) · International Conference on Dependable Systems & Networks/Proceedings · doi:10.1109/dsn69566.2026.00079
A Research Agenda for Usability and Generalisation in Reinforcement Learning
Soemers, D. J. N. J., Samothrakis, S., Driessens, K., & Winands, M. (2024) · International Conference on Agents and Artificial Intelligence · doi:10.1007/978-3-032-25032-2_33
Why Generalization in RL is Difficult: Epistemic POMDPs and Implicit Partial Observability
Ghosh, D., Rahme, J., Kumar, A., Zhang, A., Adams, R. P., & Levine, S. (2021) · arXiv · doi:10.48550/arxiv.2107.06277 · Cited by 41 · see citing works
- Authors
- Whiteson, S., Tanner, B., Taylor, M. E., & Stone, P.
- Affiliations
- University of Amsterdam (NL)
- University of Alberta (CA)
- Lafayette College (US)
- The University of Texas at Austin (US)
- Venue
- 2011 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL)
- Publisher
- IEEE
- Published
- 2011-04
- Pages
- 120-127
- OpenAlex
- W2104228245
- Cited by
- 86
- References
- 32
- Citations by year
- 2026: 4 · 2025: 5 · 2024: 6 · 2023: 11 · 2022: 6 · 2021: 13 · 2020: 8 · 2019: 9 · 2018: 6 · 2016: 3 · 2015: 1 · 2014: 5 · 2013: 4 · 2012: 2
- FWCI
- 3.51
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (publishedVersion)
- UvA-DARE (University of Amsterdam) (publishedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
- Keywords
- evaluation methodology, multiple environments
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
