Wait, But Why?: Assessing Behavior Explanation Strategies for Real-Time Strategy Games
Open accessThe DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
Abstract
Work in AI-based explanation systems has uncovered an interesting contradiction: people prefer and learn best from why explanations but expert esports commentators primarily answer what questions when explaining complex behavior in real-time strategy games. Three possible explanations for this contradiction are: 1.) broadcast audiences are well-informed and do not need why explanations; 2.) consuming why explanations in real-time is too cognitively demanding for audiences; or 3.) producing live why explanations is too difficult for commentators. We answer this open question by investigating the effects of explanation types and presentation modalities on audience recall and cognitive load in the context of an esports broadcast. We recruit 111 Dota 2 players and split them into three groups: the first group views a Dota 2 broadcast, the second group has the addition of an interactive map that provides what explanations, and the final group receives the interactive map with detailed why explanations. We find that participants who receive short interactive text prompts that provide what explanations outperform the no explanation group on a multiple-choice recall task. We also find that participants who receive detailed why explanations submit reports of cognitive load that are higher than the no explanation group. Our evidence supports the conclusion that informed audiences benefit from explanations but do not have the cognitive resources to process why answers in real-time. It also supports the conclusion that stacked explanation interventions across different modalities, like audio, interactivity, and text, can aid real-time comprehension when attention resources are limited. Together, our results indicate that interactive multimedia interfaces can be leveraged to quickly guide attention and provide low-cost explanations to improve intelligibility when time is too scarce for cognitively demanding why explanations.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Some user groups are more vulnerable to these harms than others, such as AI novices [60], people working in cognitively constrained settings [52], and even those with certain personality traits [17].”
From Designing for Responsible Trust in AI Systems: A Communication Perspective (2022) · cites this paper as [52]
In the citing paper
“For example, recent studies have repeatedly found that XAI features bring less benefit, or even harms (leading to over-trust), to certain user groups such as AI novices or users working in cognitively constrained settings [17, 52, 60].”
From Designing for Responsible Trust in AI Systems: A Communication Perspective (2022) · cites this paper as [52]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 1×
“Machine learning has recently made tremendous advances in many areas, including image recognition [21, 36], language modeling [6, 11, 24], and novel artifact generation [16, 39, 42].”
BackgroundBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173 · cited 2×
“These representations have helped drive the development of modern neural language models, like BERT [11] and GPT-3 [6], which are used for a range of language understanding and generation tasks.”
BackgroundBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 2×
“These representations have helped drive the development of modern neural language models, like BERT [11] and GPT-3 [6], which are used for a range of language understanding and generation tasks.”
BackgroundHuman-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357 · cited 2×
“Deep neural models have learned to play at human or better levels in classic Atari games [26] without hard-coded domain knowledge or supervised training.”
BackgroundDistributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533 · cited 2×
“Word2Vec [24] is an algorithm that produces a word embedding model, which is a set of high-dimensional vector representations of words created by analyzing co-occurrence in large data sets.”
BackgroundMethodUnpaired image-to-image translation using cycle-consistent adversarial networks
Zhu, J.-Y., Park, T., Isola, P., & Efros, A. A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.244 · Cited by 23,602
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 2×
“These representations have helped drive the development of modern neural language models, like BERT [11] and GPT-3 [6], which are used for a range of language understanding and generation tasks.”
BackgroundGAN(Generative Adversarial Nets)
(2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,270
Mastering the game of Go without human knowledge
Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., et al. (2017) · Nature · doi:10.1038/nature24270 · Cited by 9,084
Get To The Point: Summarization with Pointer-Generator Networks
See, A., Liu, P. J., & Manning, C. D. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1099 · Cited by 3,775 · cited 1×
“Work in this area is still developing and deep neural networks have made recent advances [33, 34].”
Background
Research works printed but not linked: 3
- [16] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative Adversarial Nets. In Advances in Neural Information Processing Systems 27 (Montreal, CA) (NIPS 2014) . Neural Information Processing Systems Foundation, San Diego, CA, 2672ś2680. -
- [25] Vincenzo "Skulz" Milella. 2020. Dota Seasonal Rank distribution and Medals - 2020 . Esports Tales. https://www.esportstales.com/dota-2/seasonal-rank-distributionand-mmr-medals -
- [36] Hugo Touvron, Andrea Vedaldi, Matthijs Douze, and Herve Jegou. 2019. Fixing the Train-Test Resolution Discrepancy. In Advances in Neural Information Processing Systems 32 (Vancouver, CA) (NeurIPs 2019) . Neural Information Processing Systems Foundation, San Diego, CA, 8252ś8262. -
Cited by
Human-Centered Explainable AI (XAI): From Algorithms to User Experiences
Liao, Q. V., & Varshney, K. R. (2026) · Cambridge University Press eBooks · doi:10.1017/9781108872225.011 · Cited by 483
Designing for Responsible Trust in AI Systems: A Communication Perspective
Liao, Q. V., & Sundar, S. S. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533182 · Cited by 165
“For example, recent studies have repeatedly found that XAI features bring less benefit, or even harms (leading to over-trust), to certain user groups such as AI novices or users working in cognitively constrained settings [17, 52, 60].”
BackgroundHuman-Centered Explainable AI (XAI): From Algorithms to User Experiences
Liao, Q. V., & Varshney, K. R. (2021) · arXiv · doi:10.48550/arxiv.2110.10790 · Cited by 120
“Other studies suggest that in time and cognitive resource constraint settings people are less able to process explanations effectively [89, 104].”
BackgroundSelective Explanations: Leveraging Human Input to Align Explainable AI
Lai, V., Zhang, Y., Chen, C., Liao, Q. V., & Tan, C. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610206 · Cited by 53
“On the other hand, multiple studies reported that end-users found the explanations generated from popular technical approaches hard to use, distracting, time-consuming, and cognitively demanding [50, 60, 85, 89, 102].”
BackgroundContextualizing User Perceptions about Biases for Human-Centered Explainable Artificial Intelligence
Yuan, C. W., Bi, N., Lin, Y.-F., & Tseng, Y. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580945 · Cited by 47
“Also, some marginalized user groups should be recognized and included in future studies to promote XAI design solutions for empowerment because they may be more vulnerable than others, such as AI novices [41] or those who work with constrained cognition [35].”
BackgroundRole of digital simulation in employee training
Kulkarni, P. M., Appasaba, L. V., Gokhale, P., & Tigadi, B. S. (2022) · Global Transitions Proceedings · doi:10.1016/j.gltp.2022.04.014 · Cited by 42
A Map of Exploring Human Interaction Patterns with LLM: Insights into Collaboration and Creativity
Li, J., Li, J., & Su, Y. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-60615-1_5 · Cited by 44
An Ecosystem Framework for the Meta in Esport Games
Thaicharoen, S., Gow, J., & Drachen, A. (2023) · Journal of Electronic Gaming and Esports · doi:10.1123/jege.2022-0045 · Cited by 19
Self-citationGentile, D., Donmez, B., & Jamieson, G. A. (2023) · Artificial Intelligence · doi:10.1016/j.artint.2023.103945 · Cited by 15
Brachman, M., Pan, Q., Do, H. J., Dugan, C., Chaudhary, A., Johnson, J. M., Rai, P., Chakraborti, T., Gschwind, T., Laredo, J. A., et al. (2023) · Proceedings of the 28th International Conference on Intelligent User Interfaces · doi:10.1145/3581641.3584088 · Cited by 11
“The impact of explanations on task performance has been mixed: explanations improved success on a recall quiz for game events [101], but not for human-AI task assignment [105] or collaborative performance for decision-making [12].”
BackgroundHow Do People Rank Multiple Mutant Agents?
Dodge, J. E., Anderson, A. A., Olson, M., Dikkala, R., & Burnett, M. (2022) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3490099.3511115 · Cited by 10
Predicting the need for XAI from high-granularity interaction data
de Santana, V. F., Fucs, A., Segura, V., de Moraes, D. B., & Cerqueira, R. (2023) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2023.103029 · Cited by 8
Chitayat, A. P., Coates, A., Block, F., Drachen, A., Walker, J. A., Dean, J., Mcconachie, M., & York, P. (2024) · ACM International Conference on Interactive Media Experiences · doi:10.1145/3639701.3656318 · Cited by 8
“Building on our prior work in esports [5, 9, 10, 28, 29, 42, 44, 49, 66], the iterative design process included consultations with senior stakeholders at the company and close co-design with fans.”
Self-citationDasgupta, R., & Kliem, J. (2023) · International Journal of Serious Games · doi:10.17083/ijsg.v10i1.548 · Cited by 5
Gao, E., Zhong, H., Yuan, R., Guo, J., & Chen, Z. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-93733-0_21 · Cited by 6
Ma, R., Zhao, Y., Shao, Y., Yao, Y., & Li, Q. (2025) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3708359.3712088 · Cited by 3
“Data analytics and AI have significantly advanced team strategy analysis in competitive sports, particularly in esports [10, 13, 35, 41, 44, 53, 60, 63].”
Ahn, Y., Lin, Y.-R., Alikhani, M., & Cheon, E. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2502.12354 · Cited by 3
“On the other hand, they tend to be distracting or lead to either overreliance [16, 71] or cognitive overload [51, 57] for lay users without a careful design of how explanations are presented [80].”
Visual Summarisations for Computer-Assisted Live Color Casting and Direction in League of Legends
Margetis, G., Apostolakis, K. C., Ntoa, S., Chatzakis, I., Sykianaki, E., Markopoulos, I., & Stephanidis, C. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-35930-9_10 · Cited by 1
A Plan-Based Formal Model of Character Regret
Martinelli, M., & Robertson, J. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-92300-6_10 · Cited by 1
Self-citationArtificial Intelligence in Esports as a Case Of Digital Transformation in Physical Culture Education
Piatysotska, S., Yefremenko, A., Долгополова, Н., Romanenko, V., & Tropin, Y. (2026) · doi:10.31110/lrdtinedu-3.8
Fiebrink, R., Gillies, M. F. P., & Ramos, G. (2026) · Cambridge University Press eBooks · doi:10.1017/9781108872225.010
Metadata
- Authors
- Robertson, J., Kokkinakis, A. V., Hook, J., Kirman, B., Block, F., Ursu, M. F., Patra, S., Demediuk, S., Drachen, A., & Olarewaju, O.
- Affiliations
- Kennesaw State University (US)
- University of York (GB)
- ORCID
- Venue
- 26th International Conference on Intelligent User Interfaces
- Publisher
- ACM
- Published
- 2021-04-14
- Pages
- 32-42
- OpenAlex
- W3153179514
- Cited by
- 18
- References
- 36
- Citations by year
- 2026: 2 · 2025: 2 · 2024: 2 · 2023: 7 · 2022: 3 · 2021: 2
- FWCI
- 1.62
- Type
- conference paper
- Language
- en
- Open access
- green
- orcid.org
- Locations
- ACM International Conference on Intelligent User Interfaces (IUI) (publishedVersion)
- White Rose Research Online (University of Leeds, The University of Sheffield, University of York) (submittedVersion)
- Funders
- Engineering and Physical Sciences Research Council (EP/R010919/1)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Visual and Cognitive Learning Processes
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
- Keywords
- real-time strategy games, Dota 2, cognitive load, attention guidance
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.