Explainable AI for Designers: A Human-Centered Perspective on Mixed-Initiative Co-Creation
Open accessThe DOI matches its Crossref record; 31 of the 35 references it cites were found in a registry or an index.
Abstract
Growing interest in eXplainable Artificial Intelligence (XAI) aims to make AI and machine learning more understandable to human users. However, most existing work focuses on new algorithms, and not on usability, practical interpretability and efficacy on real users. In this vision paper, we propose a new research area of eXplainable AI for Designers (XAID), specifically for game designers. By focusing on a specific user group, their needs and tasks, we propose a human-centered approach for facilitating game designers to co-create with AI/ML techniques through XAID. We illustrate our initial XAID framework through three use cases, which require an understanding both of the innate properties of the AI techniques and users' needs, and we identify key open challenges.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Furthermore, investigations into mediating factors (e.g., interaction design [140], explainability [209]) typically occur in isolation, limiting the structured mapping of how agency patterns, control mechanism, and contextual factors interrelate.”
From Exploring Collaboration Patterns and Strategies in Human-AI Co-creation through the Lens of Agency: A Scoping Review of the Top-tier HCI Literature (2025) · cites this paper as [209]
In the citing paper
“Transparency and explainability regarding AI actions [70, 187, 209] are crucial for enabling humans to appropriately trust and engage with AI contributions [191, 201].”
From Exploring Collaboration Patterns and Strategies in Human-AI Co-creation through the Lens of Agency: A Scoping Review of the Top-tier HCI Literature (2025) · cites this paper as [209]
In the citing paper
“Zhu et al. [88] noticed that instead of focusing on the practicality and efficacy of current XAI practices, most works are coming up with new methods of explanations.”
From Interpreting Black-Box Models: A Review on Explainable Artificial Intelligence (2024) · cites this paper as [88]
Citations
Cites
Visualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839 · cited 1×
“Techniques that aim to explain relationships between neurons are known as attribution and a variety of different approaches exists [2], [5], [6].”
Background(2011) · Cited by 14,317
Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
Simonyan, K., Vedaldi, A., & Zisserman, A. (2013) · arXiv · doi:10.48550/arxiv.1312.6034 · Cited by 5,370 · cited 1×
“Techniques that aim to explain relationships between neurons are known as attribution and a variety of different approaches exists [2], [5], [6].”
BackgroundDeep neural networks are easily fooled: High confidence predictions for unrecognizable images
Nguyen, A., Yosinski, J., & Clune, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298640 · Cited by 3,258
Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images
Nguyen, A., Yosinski, J., & Clune, J. (2014) · arXiv · doi:10.48550/arxiv.1412.1897 · Cited by 3,258
Interpretable Explanations of Black Boxes by Meaningful Perturbation
Fong, R. C., & Vedaldi, A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.371 · Cited by 1,097 · cited 1×
“Techniques that aim to explain relationships between neurons are known as attribution and a variety of different approaches exists [2], [5], [6].”
BackgroundAbdul, A., Vermeulen, J., Wang, D., Lim, B. Y., & Kankanhalli, M. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174156 · Cited by 743
The Building Blocks of Interpretability
Olah, C., Satyanarayan, A., Johnson, I., Carter, S., Schubert, L., Ye, K., & Mordvintsev, A. (2018) · Distill · doi:10.23915/distill.00010 · Cited by 591 · cited 1×
“In a recent article [8], Olah et al.”
BackgroundPlug & Play Generative Networks: Conditional Iterative Generation of Images in Latent Space
Nguyen, A., Clune, J., Bengio, Y., Dosovitskiy, A., & Yosinski, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.374 · Cited by 578
Common ground at the understanding of demonstrative reference
Clark, H. H., Schreuder, R., & Buttrick, S. (1983) · Journal of Verbal Learning and Verbal Behavior · doi:10.1016/s0022-5371(83)90189-5 · Cited by 445
How the mind explains behavior: folk explanations, meaning, and social interaction
(2005) · Choice Reviews Online · doi:10.5860/choice.42-6170 · Cited by 436
How the Mind Explains Behavior: Folk Explanations, Meaning, and Social Interaction
Malle, B. F. (2004) · The MIT Press eBooks · doi:10.7551/mitpress/3586.001.0001 · Cited by 356
Trust building with explanation interfaces
Pu, P., & Chen, L. (2006) · Proceedings of the 11th international conference on Intelligent user interfaces · doi:10.1145/1111449.1111475 · Cited by 300 · cited 1×
“Reliance on ML for critical tasks and especially those involving human-life requires trust, which is typically gained through some level of transparency that facilitates a comfortable level of model understanding for the person giving that trust [24].”
BackgroundMixed-initiative co-creativity
Yannakakis, G. N., Liapis, A., & Alexopoulos, C. (2014) · OAR@UM (University of Malta) · Cited by 299
Self-citationA review of explanation methods for Bayesian networks
LACAVE, C., & DÍEZ, F. J. (2002) · The Knowledge Engineering Review · doi:10.1017/s026988890200019x · Cited by 237
Sentient sketchbook : computer-assisted game level authoring
Liapis, A., Yannakakis, G. N., & Togelius, J. (2013) · OAR@UM (University of Malta) · Cited by 233
Self-citationTanagra: Reactive Planning and Constraint Solving for Mixed-Initiative Level Design
Smith, G., Whitehead, J., & Mateas, M. (2011) · IEEE Transactions on Computational Intelligence and AI in Games · doi:10.1109/tciaig.2011.2159716 · Cited by 156
A Roadmap for a Rigorous Science of Interpretability.Preprint
Doshi‐Velez, F., & Kim, B. (2017) · arXiv (Cornell University) · Cited by 119
Making Hybrid Plans More Clear to Human Users - A Formal Approach for Generating Sound Explanations
Seegebarth, B., Müller, F., Schattenberg, B., & Biundo, S. (2012) · Proceedings of the International Conference on Automated Planning and Scheduling · doi:10.1609/icaps.v22i1.13503 · Cited by 75
Ropossum: An Authoring Tool for Designing, Optimizing and Solving Cut the Rope Levels
Shaker, N., Shaker, M., & Togelius, J. (2013) · Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment · doi:10.1609/aiide.v9i1.12611 · Cited by 72
Fox, M., Long, D., & Magazzeni, D. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.10256 · Cited by 66
Verbalization: narration of autonomous robot experience
Rosenthal, S. L., Selvaraj, S. P. P., & Veloso, M. (2016) · International Joint Conference on Artificial Intelligence · Cited by 65
What is Mixed-Initiative Interaction?
Novick, D., & Sutton, S. (1997) · Cited by 51
Ludus Ex Machina: Building A 3D Game Designer That Competes Alongside Humans.
Cook, M., & Colton, S. (2014) · ICCC · Cited by 43
Designer Modeling for Personalized Game Content Creation Tools
Liapis, A., Yannakakis, G. N., & Togelius, J. (2013) · Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment · doi:10.1609/aiide.v9i2.12587 · Cited by 41
Self-citationThe Painting Fool Sees! New Projects with the Automated Painter.
Colton, S., Halskov, J., Ventura, D. A., Gouldstone, I. W., Cook, M. L., & Ferrer, B. P. (2015) · ICCC · Cited by 41
Converting a trained neural network to a decision tree dectext - decision tree extractor
Hillman, D. J., & Boz, O. (2000) · International Conference on Machine Learning and Applications · Cited by 23
Verbal Plan Explanations for Hybrid Planning
Bidot, J., Biundo, S., Heinroth, T., Minker, W., Nothdurft, F., & Schattenberg, B. (2010) · Multikonferenz Wirtschaftsinformatik · Cited by 16
Artificial and Computational Intelligence in Games: AI-Driven Game Design (Dagstuhl Seminar 17471)
Spronck, P. H. M., André, E., Cook, M., & Preuß, M. (2018) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/dagrep.7.11.86 · Cited by 10
Towards the automatic optimisation of procedural content generators
Cook, M., Gow, J., & Colton, S. (2016) · IEEE Conference on Computational Intelligence and Games (CIG) · doi:10.1109/cig.2016.7860424 · Cited by 9
The FloWR online platform: automated programming and computational creativity as a service
Charnley, J., Colton, S., Llano, M. T., & Corneli, J. (2016) · Falmouth University Research Repository (FURR) (Falmouth University) · Cited by 6
Research works printed but not linked: 4
- straatman, 2005, Killzones ai: dynamic procedural combat tactics, Game Developers Conference
- miller, 2017, Explainable AI: Beware of inmates running the asylum, Proceedings of the IJCAI Workshop on Workshop on Explainable Artificial Intelligence
- erhan, 2009, Visualizing higher-layer features of a deep network, Universite De Montreal Tech Rep, 1341
- biran, 2017, Explanation and justification in machine learning: A survey, Proceedings of the IJCAI Workshop on Explainable Artificial Intelligence
Cited by
Arrieta, A. B., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., García, S., Gil-López, S., Molina, D., Benjamins, R., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.10045 · Cited by 11,321
Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294
“[157] noted that most existing works focus on new explainingmethods, and not on usability, practical interpretability and efficacy on real users.”
BackgroundInterpreting Black-Box Models: A Review on Explainable Artificial Intelligence
Hassija, V., Chamola, V., Mahapatra, A., Singal, A., Goel, D., Huang, K., Scardapane, S., Spinelli, I., Mahmud, M., & Hussain, A. (2024) · Cognitive Computation · doi:10.1007/s12559-023-10179-8 · Cited by 2,495
“Zhu et al. [88] noticed that instead of focusing on the practicality and efficacy of current XAI practices, most works are coming up with new methods of explanations.”
BackgroundQuestioning the AI: Informing Design Practices for Explainable AI User Experiences
Liao, Q. V., Gruen, D., & Miller, S. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376590 · Cited by 925
Metadata
- Authors
- Zhu, J., Liapis, A., Risi, S., Bidarra, R., & Youngblood, G. M.
- Affiliations
- Drexel University (US)
- University of Malta (MT)
- IT University of Copenhagen (DK)
- Palo Alto Research Center (US)
- Venue
- 2018 IEEE Conference on Computational Intelligence and Games (CIG)
- Publisher
- IEEE
- Published
- 2018-08
- Pages
- 1-8
- OpenAlex
- W2896010852
- Cited by
- 255
- References
- 35
- Citations by year
- 2026: 14 · 2025: 35 · 2024: 38 · 2023: 33 · 2022: 46 · 2021: 41 · 2020: 33 · 2019: 12 · 2018: 3
- FWCI
- 9.05
- Type
- conference paper
- Language
- en
- Open access
- green
- pure.itu.dk
- Licence
- other-oa
- Locations
- IEEE Conference on Computational Intelligence and Games (CIG) (publishedVersion)
- IT University Of Copenhagen (IT University of Copenhagen) (submittedVersion, other-oa)
- Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
- Research Repository (Delft University of Technology) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Keywords
- explainable AI, game designers, human-centered design, interpretability, game design, AI-assisted design, user needs
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.