Explainable Reinforcement Learning through a Causal Lens
Open accessAbstract
Prominent theories in cognitive science propose that humans understand and represent the knowledge of the world through causal relationships. In making sense of the world, we build causal models in our mind to encode cause-effect relations of events and use these to explain why new events happen by referring to counterfactuals — things that did not happen. In this paper, we use causal models to derive causal explanations of the behaviour of model-free reinforcement learning agents. We present an approach that learns a structural causal model during reinforcement learning and encodes causal relationships between variables of interest. This model is then used to generate explanations of behaviour based on counterfactual analysis of the causal model. We computationally evaluate the model in 6 domains and measure performance and task prediction accuracy. We report on a study with 120 participants who observe agents playing a real-time strategy game (Starcraft II) and then receive explanations of the agents' behaviour. We investigate: 1) participants' understanding gained by explanations through task prediction; 2) explanation satisfaction and 3) trust. Our results show that causal model explanations perform better on these measures compared to two other baseline explanation models.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Recent research has shown that recommendation-driven explainable/interpretable AI has little effect on decision making, although this is not always the case.
“However, recent research has shown that the recommendation-driven explainable/interpretable AI has little effect on decision making [1, 6-8, 28, 32], although this is not always the case [22,23,41].”
From Explainable AI is Dead, Long Live Explainable AI!: Hypothesis-driven Decision Support using Evaluative AI (2023) · cites this paper as [23]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Amazon's Mechanical Turk: A New Source of Inexpensive, Yet High-Quality, Data?
Buhrmester, M., Kwang, T., & Gosling, S. D. (2011) · Perspectives on Psychological Science · doi:10.1177/1745691610393980 · Cited by 9,996 · see citing works
Explanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302 · see citing works
Self-citationScientific Explanation and the Causal Structure of the World
Salmon, W. C. (2020) · Princeton University Press eBooks · doi:10.2307/j.ctv173f2gh · Cited by 1,992 · see citing works
The Book of Why: The New Science of Cause and Effect
Pearl, J., & Mackenzie, D. N. (2018) · Cited by 1,001 · see citing works
StarCraft II: A New Challenge for Reinforcement LearningPreprint
Vinyals, O., Ewalds, T., Bartunov, S., Georgiev, P., Vezhnevets, A. S., Yeo, M., Makhzani, A., Küttler, H., Agapiou, J., Schrittwieser, J., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.04782 · Cited by 803 · see citing works
Causal Models: How People Think about the World and Its Alternatives
Sloman, S. (2005) · doi:10.1093/acprof:oso/9780195183115.001.0001 · Cited by 677 · see citing works
<i>Why and why not</i> explanations improve the intelligibility of context-aware intelligent systems
Lim, B. Y., Dey, A. K., & Avrahami, D. (2009) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1518701.1519023 · Cited by 552 · see citing works
OpenAI GymPreprint
Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., & Zaremba, W. (2016) · arXiv (Cornell University) · Cited by 544 · see citing works
Metrics for explainable AI: challenges and prospects
Hoffman, R. R., Mueller, S. T., Klein, G., & Litman, J. (2018) · arXiv · doi:10.48550/arxiv.1812.04608 · Cited by 518 · see citing works
Improving Robot Controller Transparency Through Autonomous Policy Explanation
Hayes, B., & Shah, J. A. (2017) · IEEE/ACM International Conference on Human-Robot Interaction · doi:10.1145/2909824.3020233 · Cited by 359 · see citing works
Counterfactuals in Explainable Artificial Intelligence (XAI): Evidence from Human Reasoning
Byrne, R. M. J. (2019) · Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/876 · Cited by 285 · see citing works
Causes and Explanations: A Structural-Model Approach. Part II: Explanations
Halpern, J. Y., & Pearl, J. (2005) · The British Journal for the Philosophy of Science · doi:10.1093/bjps/axi148 · Cited by 242 · see citing works
Fairness in Decision-Making — The Causal Explanation Formula
Zhang, J., & Bareinboim, E. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11564 · Cited by 202 · see citing works
Contrastive explanation: a structural-model approach
Miller, T. (2021) · The Knowledge Engineering Review · doi:10.1017/s0269888921000102 · Cited by 186 · see citing works
Self-citationExplaining control strategies in problem solving
Chandrasekaran, B., Tanner, M. C., & Josephson, J. R. (1989) · IEEE Expert · doi:10.1109/64.21896 · Cited by 137 · see citing works
HIGHLIGHTS: Summarizing Agent Behavior to People
Amir, D., & Amir, O. (2018) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/wizc1785 · Cited by 93 · see citing works
Minimal Sufficient Explanations for Factored Markov Decision Processes
Khan, O., Poupart, P., & Black, J. P. (2009) · Proceedings of the International Conference on Automated Planning and Scheduling · doi:10.1609/icaps.v19i1.13365 · Cited by 77 · see citing works
Counterfactuals and Policy Analysis in Structural ModelsPreprint
Balke, A., & Pearl, J. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1302.4929 · Cited by 71 · see citing works
Contrastive Explanations for Reinforcement Learning in terms of Expected ConsequencesPreprint
van der Waa, J., van Diggelen, J., den Bosch, K. V., & Neerincx, M. A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.08706 · Cited by 66 · see citing works
Distilling Deep Reinforcement Learning Policies in Soft Decision Trees
Coppens, Y., Efthymiadis, K., Lenaerts, T., & Nowé, A. (2019) · Dépôt institutionnel de l'Université libre de Bruxelles (Université Libre de Bruxelles) · Cited by 63 · see citing works
A Grounded Interaction Protocol for Explainable Artificial Intelligence
Madumal, P., Miller, T., Sonenberg, L., & Vetere, F. (2019) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/dgmp4001 · Cited by 60 · see citing works
Self-citationFor you can't always get what you want: When preconditions are better explanations than goals
McClure, J., & Hilton, D. (1997) · British Journal of Social Psychology · doi:10.1111/j.2044-8309.1997.tb01129.x · Cited by 48 · see citing works
Autonomous Self-Explanation of Behavior for Interactive Reinforcement Learning Agents
Fukuchi, Y., Osawa, M., Yamakawa, H., & Imai, M. (2017) · International Conference on Human-Agent Interaction (HAI) · doi:10.1145/3125739.3125746 · Cited by 30 · see citing works
Explaining Explanation, Part 3: The Causal Landscape
Klein, G. A. (2018) · IEEE Intelligent Systems · doi:10.1109/mis.2018.022441353 · Cited by 30 · see citing works
Expert Evaluation of Probabilistic Explanations.
Elizalde, F., & Sucar, L. E. (2009) · Explanation-aware Computing · Cited by 9 · see citing works
Explainable Agency in Intelligent Agents: Doctoral Consortium
Madumal, P. (2019) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/hkau7467 · Cited by 7 · see citing works
Self-citationExplanation Trees for Causal Bayesian NetworksPreprint
Nielsen, U. H., Pellet, J., & Elisseeff, A. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1206.3276 · Cited by 4 · see citing works
Flagged first, then by citations
4 of the 31 references couldn’t be matched to a known paper.
Research works printed but not linked: 5
- De Graaf, M. M. A., and Malle, B. F. 2017. How People Explain Action (and Autonomous Intelligent Systems Should Too). AAAI 2017 Fall Symposium on âĂIJAIHRIâĂİ 19-26.
- Elizalde, F.; Sucar, L. E.; Luque, M.; Diez, J.; and Reyes, A. 2008. Policy explanation in factored markov decision processes. In Proceedings of the 4th European Workshop on Probabilistic Graphical Models (PGM 2008) , 97-104.
- Juozapaitis, Z.; Koul, A.; Ferm, A.; Erwig, M.; and Doshi-Velez, F. 2019. Explainable reinforcement learning via reward decomposition. In Miller, T.; Weber, R.; and Magazzeni, D., eds., Proceedings of the IJCAI 2019 Workshop on Explainable Artificial Intelligence , 47-53.
- Miller, T. 2018a. Contrastive explanation: A structuralmodel approach. arXiv preprint arXiv:1811.03163 . · arXiv
- Mnih, V.; Kavukcuoglu, K.; Silver, D.; Rusu, A. A.; Veness, J.; Bellemare, M. G.; Graves, A.; Riedmiller, M.; Fidjeland, A. K.; Ostrovski, G.; et al. 2015. Humanlevel control through deep reinforcement learning. Nature 518(7540):529.
Cited by
Sorted by citations
Albahri, A. S., Duhaim, A. M., Fadhel, M. A., Alnoor, A., Baqer, N. S., Alzubaidi, L., Albahri, O. S., Alamoodi, A. H., Bai, J., Salhi, A., et al. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.03.008 · Cited by 716 · see citing works
Langer, M., Oster, D., Speith, T., Hermanns, H., Kästner, L., Schmidt, E., Sesing, A., & Baum, K. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2021.103473 · Cited by 652 · see citing works
Explainability in deep reinforcement learning
Heuillet, A., Couthouis, F., & Díaz-Rodríguez, N. (2020) · Knowledge-Based Systems · doi:10.1016/j.knosys.2020.106685 · Cited by 337 · see citing works
Explainable Reinforcement Learning: A Survey
Puiutta, E., & Veith, E. M. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-57321-8_5 · Cited by 278 · see citing works
Causal interpretability for machine learning-problems, methods and evaluation
Moraffah, R., Karami, M., Guo, R., Raglin, A., & Liu, H. (2020) · ACM SIGKDD Explorations Newsletter · doi:10.1145/3400051.3400058 · Cited by 238 · see citing works
Explainability of Deep Vision-Based Autonomous Driving Systems: Review and Challenges
Zablocki, É., Ben-Younes, H., Pérez, P., & Cord, M. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-022-01657-x · Cited by 227 · see citing works
Explainable Deep Reinforcement Learning for UAV autonomous path planning
He, L., Aouf, N., & Song, B. (2021) · Aerospace Science and Technology · doi:10.1016/j.ast.2021.107052 · Cited by 209 · see citing works
Contrastive explanation: a structural-model approach
Miller, T. (2021) · The Knowledge Engineering Review · doi:10.1017/s0269888921000102 · Cited by 186 · see citing works
Self-citationMiller, T. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594001 · Cited by 172 · see citing works
Self-citationA Survey of Algorithmic Recourse: Contrastive Explanations and Consequential Recommendations
Karimi, A.-H., Barthe, G., Schölkopf, B., & Valera, I. (2023) · ACM Computing Surveys · doi:10.1145/3527848 · Cited by 301 · see citing works
Sivaraman, V., Bukowski, L. A., Levin, J., Kahn, J. M., & Perer, A. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581075 · Cited by 141 · see citing works
Do People Engage Cognitively with AI? Impact of AI Assistance on Incidental Learning
Gajos, K. Z., & Mamykina, L. (2022) · 27th International Conference on Intelligent User Interfaces · doi:10.1145/3490099.3511138 · Cited by 127 · see citing works
Levels of explainable artificial intelligence for human-aligned conversational explanations
Dazeley, R., Vamplew, P. W., Foale, C., Young, C., Aryal, S., & Cruz, F. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2021.103525 · Cited by 122 · see citing works
A survey on artificial intelligence assurance
Batarseh, F. A., Freeman, L., & Huang, C. (2021) · Journal Of Big Data · doi:10.1186/s40537-021-00445-7 · Cited by 117 · see citing works
Acquisition of chess knowledge in AlphaZero
McGrath, T., Kapishnikov, A., Tomašev, N., Pearce, A., WATTENBERG, M. P., Hassabis, D., Kim, B., Paquet, U., & Kramnik, V. (2022) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2206625119 · Cited by 113 · see citing works
Sequeira, P., & Gervasio, M. T. (2020) · Artificial Intelligence · doi:10.1016/j.artint.2020.103367 · Cited by 104 · see citing works
Explainable reinforcement learning in production control of job shop manufacturing system
Kuhnle, A., May, M. C., Schäfer, L., & Lanza, G. (2021) · International Journal of Production Research · doi:10.1080/00207543.2021.1972179 · Cited by 87 · see citing works
Explainable deep learning in healthcare: A methodological survey from an attribution view
Jin, D., Sergeeva, E., Weng, W., Chauhan, G., & Szolovits, P. (2022) · WIREs Mechanisms of Disease · doi:10.1002/wsbm.1548 · Cited by 89 · see citing works
A survey of algorithmic recourse: definitions, formulations, solutions, and prospectsPreprint
Karimi, A.-H., Barthe, G., Schölkopf, B., & Valera, I. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.04050 · Cited by 74 · see citing works
The Emerging Landscape of Explainable Automated Planning & Decision Making
Chakraborti, T., Sreedharan, S., & Subbarao Kambhampati, B. (2020) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/669 · Cited by 107 · see citing works
On the Relation of Trust and Explainability: Why to Engineer for Trustworthiness
Kästner, L., Langer, M., Lazar, V., Schomäcker, A., Speith, T., & Sterz, S. (2021) · IEEE International Requirements Engineering Conference Workshops (REW) · doi:10.1109/rew53955.2021.00031 · Cited by 72 · see citing works
The quest of parsimonious XAI: A human-agent architecture for explanation formulation
Mualla, Y., Tchappi, I., Kampik, T., Najjar, A., Calvaresi, D., Abbas‐Turki, A., Galland, S., & Nicolle, C. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2021.103573 · Cited by 61 · see citing works
Explainable reinforcement learning for broad-XAI: a conceptual framework and survey
Dazeley, R., Vamplew, P. W., & Cruz, F. (2023) · Neural Computing and Applications · doi:10.1007/s00521-023-08423-1 · Cited by 68 · see citing works
Requirements for Trustworthy Artificial Intelligence – A Review
Kaur, D., Uslu, S., & Arjan Durresi, A. (2020) · Advances in intelligent systems and computing · doi:10.1007/978-3-030-57811-4_11 · Cited by 56 · see citing works
Local and global explanations of agent behavior: Integrating strategy summaries with saliency maps
Huber, T., Weitz, K., André, E., & Amir, O. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2021.103571 · Cited by 53 · see citing works
Challenges of Reinforcement Learning
Ding, Z., & Dong, H. (2020) · Springer eBooks · doi:10.1007/978-981-15-4095-0_7 · Cited by 52 · see citing works
Causal reasoning: Charting a revolutionary course for next-generation AI-native wireless networks
Thomas, C. K., Chaccour, C., Saad, W., Debbah, M., & Hong, C. S. (2024) · IEEE Vehicular Technology Magazine · doi:10.1109/mvt.2024.3359357 · Cited by 46 · see citing works
Self-Supervised Discovering of Interpretable Features for Reinforcement Learning
Shi, W., Huang, G. M., Song, S., Wang, Z., Lin, T., & Wu, C. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3037898 · Cited by 53 · see citing works
Directive Explanations for Actionable Explainability in Machine Learning Applications
Singh, R., Miller, T., Lyons, H., Sonenberg, L., Velloso, E., Vetere, F., Howe, P. D. L., & Dourish, P. (2023) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3579363 · Cited by 50 · see citing works
Self-citationCausality Learning: A New Perspective for Interpretable Machine LearningPreprint
Xu, G., Duong, T. D., Li, Q., Liu, S., & Wang, X. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.16789 · Cited by 37 · see citing works
Contrastive Explanations of Plans through Model Restrictions
Krarup, B., Krivić, S., Magazzeni, D., Long, D. P., Cashmore, M., & Smith, D. E. (2021) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.12813 · Cited by 36 · see citing works
Singh, G., Nadig, R., Park, J., Bera, R., Hajinazar, N., Novo, D., Gómez-Luna, J., Stuijk, S., Corporaal, H., & Mutlu, O. (2022) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1145/3470496.3527442 · Cited by 36 · see citing works
Explainable deep reinforcement learning for portfolio management: an empirical approach
Guan, M., & Liu, X.-Y. (2021) · Proceedings of the Second ACM International Conference on AI in Finance · doi:10.1145/3490354.3494415 · Cited by 28 · see citing works
Improving Visualization Interpretation Using Counterfactuals
Kaul, S., Borland, D., Cao, N., & Gotz, D. (2021) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2021.3114779 · Cited by 23 · see citing works
Iterative Bounding MDPs: Learning Interpretable Policies via Non-Interpretable Methods
Topin, N., Milani, S., Fang, F., & Veloso, M. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i11.17192 · Cited by 20 · see citing works
A Survey of Collaborative Reinforcement Learning: Interactive Methods and Design Patterns
Li, Z., Shi, L., Cristea, A. I., & Zhou, Y. (2021) · Designing Interactive Systems Conference · doi:10.1145/3461778.3462135 · Cited by 19 · see citing works
Training a Resilient Q-network against Observational Interference
Yang, C.-H. H., Hung, I.-T. D., Ouyang, Y., & Chen, P. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i8.20862 · Cited by 17 · see citing works
Actionable Interpretability through Optimizable Counterfactual Explanations for Tree Ensembles.Preprint
Lučić, A., Oosterhuis, H., Haned, H., & de Rijke, M. (2019) · arXiv (Cornell University) · Cited by 16 · see citing works
Chen, Q., & Heydari, B. (2022) · Journal of Mechanical Design · doi:10.1115/1.4055057 · Cited by 12 · see citing works
Sensor Tasking for Space Situation Awareness: Combining Reinforcement Learning and Causality
Fan, Z., Chang, K., Raz, A. K., Harvey, A., & Chen, G. (2023) · IEEE Aerospace Conference (AERO) · doi:10.1109/aero55745.2023.10115698 · Cited by 13 · see citing works
Explainable Goal-Driven Agents and Robots -- A Comprehensive Review and New FrameworkPreprint
Fatai, S., Loo, C. K., Kerzel, M., & Wermter, S. (2020) · arXiv (Cornell University) · Cited by 11 · see citing works
Explainable Multi-Agent Reinforcement Learning for Temporal Queries
Boggess, K., Kraus, S., & Lu, F. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/7 · Cited by 10 · see citing works
Towards the Deployment of Realistic Autonomous Cyber Network Defence: A Systematic Review
Vyas, S., Mavroudis, V., & Burnap, P. (2025) · ACM Computing Surveys · doi:10.1145/3729213 · Cited by 13 · see citing works
Explaining Reinforcement Learning Agents through Counterfactual Action Outcomes
Amitai, Y., Septon, Y., & Amir, O. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i9.28863 · Cited by 10 · see citing works
A framework of explanation generation toward reliable autonomous robots
Sakai, T., Miyazawa, K., Horii, T., & Nagai, T. (2021) · Advanced Robotics · doi:10.1080/01691864.2021.1946423 · Cited by 8 · see citing works
Incorporating rivalry in reinforcement learning for a competitive game
Barros, P. V. A., Yalçın, Ö. N., Tanevska, A., & Sciutti, A. (2022) · Neural Computing and Applications · doi:10.1007/s00521-022-07746-9 · Cited by 8 · see citing works
Xu, L., Chen, Y., Lu, S., Zhong, K., Li, Y., & Bin, Y. (2023) · IET Image Processing · doi:10.1049/ipr2.12930 · Cited by 5 · see citing works
Discovering User-Interpretable Capabilities of Black-Box Planning Agents
Verma, P., Marpally, S. R., & Srivastava, S. (2022) · Principles of Knowledge Representation and Reasoning · doi:10.24963/kr.2022/36 · Cited by 5 · see citing works
Explaining Deep Reinforcement Learning-Based Methods for Control of Building HVAC Systems
Jiménez-Raboso, J., Manjavacas, A., Campoy-Nieves, A., Molina-Solana, M., & Gómez‐Romero, J. (2023) · Communications in computer and information science · doi:10.1007/978-3-031-44067-0_13 · Cited by 4 · see citing works
Explainable Agency in Reinforcement Learning Agents
Madumal, P. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i10.7134 · Cited by 4 · see citing works
Self-citationDomain-Level Explainability -- A Challenge for Creating Trust in Superhuman AI StrategiesPreprint
Andrulis, J., Meyer, O., Schott, G., Weinbach, S., & Gruhn, V. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.06665 · Cited by 4 · see citing works
Viana, J., Souza, J., Rocha, R. F., Santos, A., & Freitas, A. (2024) · BMC Emergency Medicine · doi:10.1186/s12873-024-01029-3 · Cited by 3 · see citing works
Explainable Agency by Revealing Suboptimality in Child-Robot Learning Scenarios
Tulli, S., Couto, M., Vasco, M., Yadollahi, E., Melo, F. S., & Paiva, A. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-62056-1_3 · Cited by 3 · see citing works
Zahedi, Z., Sengupta, S., & Subbarao Kambhampati, B. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i9.28890 · Cited by 3 · see citing works
Climent, A., Вадимович, Г. Д., & Álvarez-Napagao, S. (2021) · Frontiers in artificial intelligence and applications · doi:10.3233/faia210166 · Cited by 4 · see citing works
Local Explanations for Reinforcement Learning
Luss, R., Dhurandhar, A., & Liu, M. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i7.26081 · Cited by 4 · see citing works
Wang, J., Du, D., Tian, L., Chen, Y., Li, Y., & Li, Y. X. (2025) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v39i26.34981 · Cited by 2 · see citing works
Ingrid Zukerman, A., Partovi, A., & Hohwy, J. (2023) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-023-09357-y · Cited by 2 · see citing works
On Explaining Agent Behaviour via Root Cause Analysis: A Formal Account Grounded in Theory of Mind
Khan, S. M., & Rostamigiv, M. (2023) · Frontiers in artificial intelligence and applications · doi:10.3233/faia230401 · Cited by 2 · see citing works
A Novel Approach to Curiosity and Explainable Reinforcement Learning via Interpretable Sub-GoalsPreprint
van Rossum, C., Feinberg, C., Shumays, A. A., Baxter, K., & Bartha, B. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.06630 · Cited by 2 · see citing works
Explaining Causal Models with Argumentation: the Case of Bi-variate Reinforcement
Rago, A., Baroni, P., & Toni, F. (2022) · Principles of Knowledge Representation and Reasoning · doi:10.24963/kr.2022/52 · Cited by 1 · see citing works
Beyond Value: CheckList for Testing Inferences in Planning-Based RL
Lam, K.-H., Tabatabai, D., Irvine, J., Bertucci, D., Ruangrotsakun, A., Kahng, M., & Fern, A. (2022) · Proceedings of the International Conference on Automated Planning and Scheduling · doi:10.1609/icaps.v32i1.19848 · Cited by 1 · see citing works
Building Human-Autonomy Teaming Aids for Real-Time Strategy Games
Izumigawa, C., Lucero, C., Nans, L., Frederiksen, K., Hui, O., Enriquez, I., Rothman, S., & Iden, R. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-50164-8_8
Black-box model risk in financePreprint
Cohen, S., Snow, D., & Szpruch, Ł. (2021) · arXiv (Cornell University)
Explainability via ResponsibilityPreprint
Khadivpour, F., & Guzdial, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.01676
Practical Guidelines for Building Explainable, Efficient, and Robust Machine Learning SystemsPreprint
Zheng, H., & Ru, Z. (2025) · doi:10.31224/4744
Counterfactual Data Augmentation using Locally Factored Dynamics
Pitis, S., Creager, E., & Garg, A. (2020) · arXiv · doi:10.48550/arxiv.2007.02863 · Cited by 26 · see citing works
- Authors
- Madumal, P., Miller, T., Sonenberg, L., & Vetere, F.
- Affiliations
- The University of Melbourne (AU)
- Venue
- Proceedings of the AAAI Conference on Artificial Intelligence
- Published
- 2020-04-03
- Volume
- 34
- Issue
- 03
- Pages
- 2493-2500
- ISSN
- 2159-5399 (ISSN-L)
- 2374-3468
- OpenAlex
- W2945022240
- Cited by
- 58
- References
- 31
- Citations by year
- 2025: 3 · 2024: 3 · 2023: 10 · 2022: 9 · 2021: 15 · 2020: 17 · 2019: 1
- FWCI
- 4.87
- Type
- conference paper
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- www.aaai.org/ (unspecified)
- Locations
- Proceedings of the AAAI Conference on Artificial Intelligence (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Minerva Access (University of Melbourne) (submittedVersion)
- arXiv (Cornell University)
- Funders
- University of Melbourne
- Australian Research Council (DP190103414)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Bayesian Modeling and Causal Inference
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Keywords
- explainable reinforcement learning, causal models, structural causal models, counterfactual reasoning, model-free reinforcement learning, causal explanation, trust in AI, StarCraft II, real-time strategy games, human-AI interaction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
