Explainable AI for Designers: A Human-Centered Perspective on Mixed-Initiative Co-Creation
Open accessThe DOI matches its Crossref record; 31 of the 35 references it cites were found in a registry or an index.
Abstract
Growing interest in eXplainable Artificial Intelligence (XAI) aims to make AI and machine learning more understandable to human users. However, most existing work focuses on new algorithms, and not on usability, practical interpretability and efficacy on real users. In this vision paper, we propose a new research area of eXplainable AI for Designers (XAID), specifically for game designers. By focusing on a specific user group, their needs and tasks, we propose a human-centered approach for facilitating game designers to co-create with AI/ML techniques through XAID. We illustrate our initial XAID framework through three use cases, which require an understanding both of the innate properties of the AI techniques and users' needs, and we identify key open challenges.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Furthermore, investigations into mediating factors (e.g., interaction design [140], explainability [209]) typically occur in isolation, limiting the structured mapping of how agency patterns, control mechanism, and contextual factors interrelate.”
From Exploring Collaboration Patterns and Strategies in Human-AI Co-creation through the Lens of Agency: A Scoping Review of the Top-tier HCI Literature (2025) · cites this paper as [209]
In the citing paper
“Transparency and explainability regarding AI actions [70, 187, 209] are crucial for enabling humans to appropriately trust and engage with AI contributions [191, 201].”
From Exploring Collaboration Patterns and Strategies in Human-AI Co-creation through the Lens of Agency: A Scoping Review of the Top-tier HCI Literature (2025) · cites this paper as [209]
In the citing paper
“Zhu et al. [88] noticed that instead of focusing on the practicality and efficacy of current XAI practices, most works are coming up with new methods of explanations.”
From Interpreting Black-Box Models: A Review on Explainable Artificial Intelligence (2024) · cites this paper as [88]
Citations
Cites
Visualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839 · cited 1×
“Techniques that aim to explain relationships between neurons are known as attribution and a variety of different approaches exists [2], [5], [6].”
Background(2011) · Cited by 14,317
Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
Simonyan, K., Vedaldi, A., & Zisserman, A. (2013) · arXiv · doi:10.48550/arxiv.1312.6034 · Cited by 5,370 · cited 1×
“Techniques that aim to explain relationships between neurons are known as attribution and a variety of different approaches exists [2], [5], [6].”
BackgroundDeep neural networks are easily fooled: High confidence predictions for unrecognizable images
Nguyen, A., Yosinski, J., & Clune, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298640 · Cited by 3,258 · cited 1×
“Visualizing hidden layers’ features [1]–[4] can give insights into what network inputs would cause a certain reaction in an internal neuron or in the network output.”
BackgroundDeep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images
Nguyen, A., Yosinski, J., & Clune, J. (2014) · arXiv · doi:10.48550/arxiv.1412.1897 · Cited by 3,258 · cited 1×
“Visualizing hidden layers’ features [1]–[4] can give insights into what network inputs would cause a certain reaction in an internal neuron or in the network output.”
BackgroundInterpretable Explanations of Black Boxes by Meaningful Perturbation
Fong, R. C., & Vedaldi, A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.371 · Cited by 1,097 · cited 1×
“Techniques that aim to explain relationships between neurons are known as attribution and a variety of different approaches exists [2], [5], [6].”
BackgroundAbdul, A., Vermeulen, J., Wang, D., Lim, B. Y., & Kankanhalli, M. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174156 · Cited by 743 · cited 1×
“However, most existing work focuses on new algorithms of XAI rather than on usability, practical interpretability and efficacy on real users [26]–[28].”
BackgroundThe Building Blocks of Interpretability
Olah, C., Satyanarayan, A., Johnson, I., Carter, S., Schubert, L., Ye, K., & Mordvintsev, A. (2018) · Distill · doi:10.23915/distill.00010 · Cited by 591 · cited 1×
“In a recent article [8], Olah et al.”
BackgroundPlug & Play Generative Networks: Conditional Iterative Generation of Images in Latent Space
Nguyen, A., Clune, J., Bengio, Y., Dosovitskiy, A., & Yosinski, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.374 · Cited by 578
Common ground at the understanding of demonstrative reference
Clark, H. H., Schreuder, R., & Buttrick, S. (1983) · Journal of Verbal Learning and Verbal Behavior · doi:10.1016/s0022-5371(83)90189-5 · Cited by 445 · cited 1×
“Essentially, this is the notion of establishing ‘common ground’ as forwarded and explored by Herb Clark [31].”
Background
Research works printed but not linked: 4
- straatman, 2005, Killzones ai: dynamic procedural combat tactics, Game Developers Conference
- miller, 2017, Explainable AI: Beware of inmates running the asylum, Proceedings of the IJCAI Workshop on Workshop on Explainable Artificial Intelligence
- erhan, 2009, Visualizing higher-layer features of a deep network, Universite De Montreal Tech Rep, 1341
- biran, 2017, Explanation and justification in machine learning: A survey, Proceedings of the IJCAI Workshop on Explainable Artificial Intelligence
Cited by
Items 101–200 of 277
Understanding Mental Models of AI through Player-AI InteractionPreprint
Villareale, J., & Zhu, J. (2021) · IT University Of Copenhagen (IT University of Copenhagen) · doi:10.48550/arxiv.2103.16168 · Cited by 26
“Researchers have argued that the complexity of AI-driven experiences makes them particularly challenging for both users [25, 22, 35] and designers [9, 41].”
BackgroundSelf-citationWu, Z., Weber, T., & Müller, F. (2025) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3708359.3712094 · Cited by 20
Kamnerddee, C., Putjorn, P., & Intarasirisawat, J. (2024) · doi:10.1109/incit63192.2024.10810565 · Cited by 25
Danesh: Interactive Tools for Understanding Procedural Content Generators
Cook, M., Gow, J., Smith, G., & Colton, S. (2022) · IEEE Transactions on Games · doi:10.1109/tg.2021.3078323 · Cited by 19
“In [14], the authors put forward a proposal for XAI for game designers (XAID), which suggests different ways of thinking about and building tools for cocreation in game design, and the challenges inherent in doing so.”
BackgroundLewis, A., Dangol, A., Suh, H., Olszewski, A., Fogarty, J. A., & Kientz, J. A. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714131 · Cited by 22
“Explainability refers to the system’s ability to articulate its reasoning for individual decisions or outputs, particularly for non-experts [8, 90].”
Introduction to Explainable AI and Intelligent Transportation
Gaur, L., & Sahoo, B. M. (2022) · Springer eBooks · doi:10.1007/978-3-031-09644-0_1 · Cited by 26
Open Player Modeling: Empowering Players through Data TransparencyPreprint
Zhu, J., & El‐Nasr, M. S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.05810 · Cited by 20
“With a few exceptions(Zhu et al. 2018; Hooshyar et al. 2020; Zhu and Onta˜n´on 2020), current games research on how to improve the transparency of AI and data is limited.”
Self-citationExplainability increases trust resilience in intelligent agents
Xu, M., & Wang, Y. (2024) · British Journal of Psychology · doi:10.1111/bjop.12740 · Cited by 18
Kim, J., Lee, H., & Lee, H.-C. (2022) · Data Technologies and Applications · doi:10.1108/dta-12-2021-0359 · Cited by 21
Collaborative Agent Gameplay in the Pandemic Board Game
Sfikas, K., & Liapis, A. (2020) · International Conference on the Foundations of Digital Games (FDG) · doi:10.1145/3402942.3402943 · Cited by 22
“For the latter challenge, research in explainable AI [29] can be carried out regarding suggestions of action and cost/benefit visualizations so that human players are convinced of the best course of action, or suggest their own strategies for the AI to follow.”
BackgroundSelf-citationThe Evaluation of the Black Box Problem for AI-Based Recommendations: An Interview-Based Study
Ochmann, J., Zilker, S., & Laumer, S. (2021) · Lecture notes in information systems and organisation · doi:10.1007/978-3-030-86797-3_16 · Cited by 25
“Previous research focuses on the black box problem from a technical perspective (e.g., underlying explanatory structures of an algorithm) [7, 11, 12, 26, 28], in the medical domain [29–32], or in the financial sector [25, 33, 34].”
Backgroundgbt-HIPS: Explaining the Classifications of Gradient Boosted Tree Ensembles
Hatwell, J., Gaber, M. M., & Azad, R. M. A. (2021) · Applied Sciences · doi:10.3390/app11062511 · Cited by 21
“This idea is echoed in the four axioms given in [23], e.”
BackgroundSuSketch: Surrogate Models of Gameplay as a Design Assistant
Migkotzidis, P., & Liapis, A. (2021) · IEEE Transactions on Games · doi:10.1109/tg.2021.3068360 · Cited by 22
“Responding to a general need for explainable AI in design tools [46], both the predictive models and the generated suggestions could enhance their explainability.”
BackgroundSelf-citationOn Two XAI Cultures: A Case Study of Non-technical Explanations in Deployed AI SystemPreprint
Jiang, H., & Senge, E. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2112.01016 · Cited by 23
“We are aware that bridging the approaches, expectations, and mindsets between technical and nontechnical AI explanations, is a tough task, and needs more research and discussions, not only in the XAI community, but also interdisciplinary collaborations from HCI[51, 25, 55, 15, 28], social sciences, and behavioral scie…”
BackgroundExplainable AI for Fighting COVID-19 Pandemic: Opportunities, Challenges, and Future Prospects
Abiodun, M. K., Awotunde, J. B., Aremu, D. R., & Adeniyi, A. E. (2021) · Disruptive technologies and digital transformations for society 5.0 · doi:10.1007/978-981-16-3783-4_15 · Cited by 21
Boosting Mixed-Initiative Co-Creativity in Game Design: A Tutorial
Margarido, S., Machado, P., Roque, L., & Martins, P. (2025) · ACM Computing Surveys · doi:10.1145/3759914 · Cited by 18
“Additional remarks This parameter directly correlates to the XAID (eXplainable AI for Designers) spectrum of domain overlap described by Zhu et al. [66], which distinguishes off-task co-creativity from on-task co-creativity.”
Revealing Neural Network Bias to Non-Experts Through Interactive Counterfactual ExamplesPreprint
Myers, C. M., Freed, E., Pardo, L. L., Furqan, A., Risi, S., & Zhu, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.02271 · Cited by 17
“Projects in XAI develop various UI explanations to aid experts, and occasionally non-experts, in understanding AI decisions [1, 2, 36].”
BackgroundMethodSelf-citationDesigning for Playfulness in Human-AI Authoring Tools
Liapis, A., Guckelsberger, C., Zhu, J., Harteveld, C., Kriglstein, S., Denisova, A., Gow, J., & Preuss, M. (2023) · Proceedings of the 18th International Conference on the Foundations of Digital Games · doi:10.1145/3582437.3587192 · Cited by 14
“Next to engineering for playfulness, we highlight the ability of a tool to explain [43] its AI’s (shifting) process and to verbalise its playfulness as equally important.”
BackgroundSelf-citationOmeiza, D., Bhattacharyya, R., Jirotka, M. D., Hawes, N., & Kunze, L. (2025) · Transportation Research Part F Traffic Psychology and Behaviour · doi:10.1016/j.trf.2025.01.015 · Cited by 14
Does Artificial Intelligence have the Possibility of Taking Over Designers' Jobs in the Future?
Hoque, F. (2023) · International Journal of Science and Business · doi:10.58970/ijsb.2273 · Cited by 17
Game AI Competitions: Motivation for the Imitation Game-Playing Competition
Świechowski, M. (2020) · Annals of Computer Science and Information Systems · doi:10.15439/2020f126 · Cited by 18
“2) Explainability (Research, Industry): As applications of machine learning models such as deep neural networks are growing in numbers, the explainable AI (XAI) is becoming more and more important [47].”
BackgroundHuman-Machine Interactions in Very-Low-Level UAS Operations and Traffic Management
Pongsakornsathien, N., Gardi, A., Sabatini, R., Kistan, T., & Ezer, N. (2020) · IEEE/AIAA Digital Avionics Systems Conference (DASC) · doi:10.1109/dasc50938.2020.9256757 · Cited by 16
“Especially in ATM domain where an experienced human operator tends to be reticent to adopt suggested solutions from decision support system if they are not trustworthy, tractable and interpretable, especially in very complex situations [37].”
BackgroundRezwana, J., & Ford, C. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3719858 · Cited by 14
“AI Communication can range from giving full , to moderate , to the least amount of details supported by Moruzzi and Margarido [70].”
Procedural Generation of 3D Maps With Snappable Meshes
Silva, R. C. E., Fachada, N., de Andrade, D., & Códices, N. (2022) · IEEE Access · doi:10.1109/access.2022.3168832 · Cited by 13
“[1], a common issue with AI techniques in general and PCG methods in particular is that of increasing algorithmic complexity, which hinders the designer’s understanding and trust about what the algorithm is doing.”
BackgroundMethodInfluentialImperfections of XAI: Phenomena Influencing AI-Assisted Decision-Making
Spitzer, P., Morrison, K., Turri, V., Feng, M., Perer, A., & Kühl, N. (2025) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3750052 · Cited by 17
Collaborative Human-AI Sensemaking for Intelligence Analysis
Dorton, S. L., & Hall, R. A. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-77772-2_12 · Cited by 13
Questgram [Qg]: Toward a Mixed-Initiative Quest Generation Tool
Alvarez, A., Grevillius, E., Olsson, E., & Font, J. (2021) · International Conference on the Foundations of Digital Games (FDG) · doi:10.1145/3472538.3472544 · Cited by 15
“Within this, one interesting area is the one of eXplainable AI for Designers [55] where the goal is to achieve system explainability to improve the collaboration and interaction between human and AIs.”
BackgroundOmeiza, D., Anjomshoae, S. T., Webb, H., Jirotka, M. D., & Kunze, L. (2022) · IEEE Intelligent Vehicles Symposium · doi:10.1109/iv51971.2022.9827345 · Cited by 12
“Second, researchers suggested (in [14]) that understanding users’ requirements is”
BackgroundDeepening ideas vs. exploring new ones: AI strategy effects in human-AI creative collaboration
Komura, K., & Yamada, S. (2026) · PLoS ONE · doi:10.1371/journal.pone.0340449 · Cited by 13
Kridalukmana, R., Lu, H., & Naderpour, M. (2022) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2022.3202900 · Cited by 12
Bergström, E., & Wärnestål, P. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-05643-7_1 · Cited by 14
Machine learning explanability method for the multi-label classification model
Singla, K., & Biswas, S. (2021) · IEEE International Conference on Semantic Computing (ICSC) · doi:10.1109/icsc50631.2021.00063 · Cited by 11
“Explainability of machine learning model has found use in recommendation systems [8] and practical use case for game designers [9] and in the medical field for clinical use [10].”
BackgroundToward Automated Game Balance: A Systematic Engineering Design Approach
DeLaurentis, D. A., Panchal, J. H., Raz, A. K., Balasubramani, P., Maheshwari, A., Dachowicz, A. P., & Mall, K. (2021) · IEEE Conference on Games (CoG) · doi:10.1109/cog52621.2021.9619032 · Cited by 14
“XAI has also been recently studied in the game industry to explain phenomena like game cheating [13], for visualization of games played by AI players [14], and for mixed-initiative co-creation [15].”
BackgroundMorrison, K., Jain, M., Hammer, J., & Perer, A. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610064 · Cited by 10
“Previously, most works on explainable AI focused “on new algorithms of XAI rather than on usability, practical interpretability and efficacy on real users” [64].”
BackgroundBasic Issues and Challenges on Explainable Artificial Intelligence (XAI) in Healthcare Systems
Oladipo, I. D., Awotunde, J. B., AbdulRaheem, M., & Salihu, S. A. (2022) · Advances in medical technologies and clinical practice book series · doi:10.4018/978-1-6684-3791-9.ch011 · Cited by 10
Awotunde, J. B., Imoize, A. L., Adeniyi, A. E., Abiodun, M. K., Ayo, F. E., Kavitha, K. V. N., Ajamu, G. J., & Ogundokun, R. O. (2023) · Springer eBooks · doi:10.1007/978-3-031-38036-5_2 · Cited by 13
An Advanced Explainable Belief Rule-Based Framework to Predict the Energy Consumption of Buildings
Kabir, S., Hossain, M. S., & Andersson, K. (2024) · Energies · doi:10.3390/en17081797 · Cited by 11
“However, when the prediction of the buildings’ energy consumption impacts the policymakers’ decisions, they must know the prediction’s rationale [22].”
BackgroundExplainable Artificial Intelligence Methods for Breast Cancer Recognition
Damaševičius, R. (2024) · Innovation Discovery · doi:10.53964/id.2024025 · Cited by 12
Oconitrillo, L. R. R., Vargas, J. J., Camacho, A., Burgos, Á., & Corchado, J. M. (2021) · Electronics · doi:10.3390/electronics10121500 · Cited by 10
“Abbreviations The following abbreviations are used in this manuscript: (IA-AI) Interpretation-Assessment/Assessment-Interpretation (MOE) Mixture of Experts (XAI) Explainable Artificial Intelligence (SN) Semantic Networks (SME) Subject-Matter Experts (AI) Artificial Intelligence (EGI) Explanatory Graphical Interfaces (…”
MethodInfluentialFairness, explainability, privacy, and robustness for trustworthy algorithmic decision-making
Majumdar, S. (2023) · Elsevier eBooks · doi:10.1016/b978-0-323-85713-0.00017-7 · Cited by 10
Interactive Visualizer to Facilitate Game Designers in Understanding Machine Learning
Xie, J., Myers, C. M., & Zhu, J. (2019) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3290607.3312851 · Cited by 10
“[18] propose a new area of research, eXplainable AI for Designers, which focuses on creating explainable AI techniques specifically for game designers (e.g., highlighting transparent and collaborative techniques).”
BackgroundSelf-citationIntegrating Automated Play in Level Co-CreationPreprint
Hoyt, A., Guzdial, M., Kumar, Y., Smith, G., & Riedl, M. O. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.09219 · Cited by 10
Sequential Explanations with Mental Model-Based PoliciesPreprint
Yeung, A. Y., Joshi, S. D., Williams, J. J., & Rudzicz, F. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.09028 · Cited by 14
“…2018; Madumal et al., 2019) and explainee’s local simulatability of the black-box model for each classification possibility (Hoffman et al., 2018; Doshi-Velez & Kim, 2017; Lipton, 2016; Kim et al., 2016; Gilpin et al., 2018; Madumal et al., 2019; Poursabzi-Sangdeh et al., 2018; Zhu et al., 2018).”
BackgroundCo-designing Explainable Robots: A Participatory Design Approach for HRI
Gebellí, F., Ros, R., Lemaignan, S., & Garrell, A. (2024) · IEEE RO-MAN · doi:10.1109/ro-man60168.2024.10731212 · Cited by 14
“Most works in eXplainable HRI (XHRI) focus on ways to generate explanations [27], and then evaluate their effect on the understanding and trust among users.”
BackgroundImpacts of AI on Human Designers: A Systematic Literature Review
Sinlapanuntakul, P., & Zachry, M. R. (2025) · IEEE Transactions on Professional Communication · doi:10.1109/tpc.2025.3588655 · Cited by 11
“The explainability of AI decisions is another crucial factor, as designers need to understand the rationale behind AI-generated designs to effectively iterate and refine them [23], [24].”
TOWARDS USER-CENTRIC EXPLANATIONS FOR EXPLAINABLE MODELS: A REVIEW
Hassan, M. A., Sulaiman, R., Abdulgabber, M. A., & Kahtan, H. (2021) · Journal of Information System and Technology Management · doi:10.35631/jistm.622004 · Cited by 12
“Zhu et al. (2018) proposed an artificial AI (XAID) that can be explained to designers, and hence opened up new areas of research to help game designers better use AI and ML in design work through collaborative creation.”
Fashion Style Generation: Evolutionary Search with Gaussian Mixture Models in the Latent Space
Grabe, I., Zhu, J., & Agirrezabal, M. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-03789-4_6 · Cited by 13
Self-citationTowards a Framework for Human-AI Interaction Patterns in Co-Creative GAN Applications
Grabe, I., Duque, M. G., Risi, S., & Zhu, J. (2022) · IT University Of Copenhagen (IT University of Copenhagen) · Cited by 46
Self-citationFusing Level and Ruleset Features for Multimodal Learning of Gameplay Outcomes
Liapis, A., Karavolos, D., Makantasis, K., Sfikas, K., & Yannakakis, G. N. (2019) · IEEE Conference on Games (CoG) · doi:10.1109/cig.2019.8848015 · Cited by 10
“Finally, the paper suggests how the models can be used for visualization and designer assistance in a possible AI-assisted design tool [38].”
BackgroundMethodSelf-citationVelander, J., Otero, N., Pargman, T. C., & Milrad, M. (2021) · Advances in Analytics for Learning and Teaching · doi:10.1007/978-3-030-81222-5_15 · Cited by 8
Zhang, X., Zhu, S., Zhao, Y., Jia, M., & Zhu, Q. (2024) · Information Research · doi:10.47989/ir292856 · Cited by 9
Exploring Aesthetic Qualities of Deep Generative Models through Technological (Art) Mediation
Sivertsen, C., & Løvlie, A. S. (2024) · Designing Interactive Systems Conference · doi:10.1145/3643834.3661498 · Cited by 8
“Similar concerns have been explored in HCI research relating to Explainable and Interpretable AI, typically aiming to use technical analysis to describe the processes through which machine learning (ML) models derive their output [9, 24, 28, 34, 73].”
BackgroundExplanation Approaches for the Daphne Virtual Assistant
Martin, A. V., & Selva, D. (2020) · AIAA SciTech Forum · doi:10.2514/6.2020-2254 · Cited by 8
Evolutionary Human-Machine Interactions for UAS Traffic Management
Pongsakornsathien, N., Gardi, A., Sabatini, R., & Kistan, T. (2021) · AIAA AVIATION Forum · doi:10.2514/6.2021-2337 · Cited by 8
Explainable Artificial Intelligence (XAI) towards Model Personality in NLP task
Nurdin, N., & Adi, D. (2021) · IPTEK Journal of Engineering · doi:10.12962/j23378557.v7i1.a8989 · Cited by 8
“Furthermore, in an effort to create more explanation techniques that can be interpreted, easily traced and can be trusted, because basically, humans will prefer systems that have these criteria [20].”
BackgroundChen, Z., Xiong, Z., Ruan, X., Jiang, S., Wei, W., & Fang, K. (2024) · 2024 4th International Conference on Educational Technology (ICET) · doi:10.1109/icet62460.2024.10868064 · Cited by 7
“…unauthorized access can lead to distrust among users regarding the confidentiality and integrity of their information [5], and HCI systems that lack transparency in their decision-making processes, algorithms, or data usage practices can engender distrust by creating uncertainty and ambiguity [18].”
Kawabe, W., Nakao, Y., Shitara, A., & Sugano, Y. (2024) · Interacting with Computers · doi:10.1093/iwc/iwae007 · Cited by 9
Assessing the Effects of Interacting with MAP-Elites
Alvarez, A., Font, J., Dahlskog, S., & Togelius, J. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment · doi:10.1609/aiide.v17i1.18899 · Cited by 7
“The area of eXplainable AI for Designers (XAID) (Zhu et al. 2018) strives for achieving system explainability necessarily built on understandings of both algorithmic properties of the underlying AI techniques and the needs of human designers .”
BackgroundLet’s Make Games Together: Explainability in Mixed-initiative Co-creative Game Design
Margarido, S., Machado, P., Roque, L., & Martins, P. (2022) · IEEE Conference on Games (CoG) · doi:10.1109/cog51982.2022.9893636 · Cited by 9
“We drew from XAID [17] and XCC [16] principles and illustrated how these can be applied in various game design approaches, highlighting the different forms of communication that suit each context.”
BackgroundMethodInfluentialWürfel, J., Papenfuß, A., & Wies, M. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-60606-9_17 · Cited by 10
Using neural network tools to accelerate the development of Web interfaces
Петрина, Д., Kornuta, V., & Kornuta, O. (2024) · Information Technology and Computer Engineering · doi:10.31649/1999-9941-2024-60-2-42-50 · Cited by 9
Assessing receptive vocabulary using state‑of‑the‑art natural language processing techniques
Crossley, S. A., & Holmes, L. (2022) · Journal of Second Language Studies · doi:10.1075/jsls.22006.cro · Cited by 7
Select the Unexpected: A Statistical Heuristic for Story Sifting
Kreminski, M., Dickinson, M., Wardrip–Fruin, N., & Mateas, M. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-22298-6_18 · Cited by 7
User centric explanations: a breakthrough for explainable models
Hassan, M. A., Abdulhak, M., bin Sulaiman, R., & Kahtan, H. (2021) · doi:10.1109/icit52682.2021.9491641 · Cited by 7
Algorithmic transparency in path planning: A visual approach to enhancing human understanding
Zou, Y., & Borst, C. (2025) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2025.103573 · Cited by 7
Navigating Decisions in the Cockpit: The Intelligent Pilot Advisory System
Djartov, B., & Würfel, J. (2025) · Progress in IS · doi:10.1007/978-3-031-83512-4_18 · Cited by 6
Tano, S., Kotani, T., Ushida, T., Matsuo, S., Yoshihara, M., Imai, K., Kinoshita, F., Moriyama, Y., Nomoto, M., Yoshida, S., et al. (2024) · Hypertension Research · doi:10.1038/s41440-024-02024-8 · Cited by 6
“Thus, there are concerns about the abundance of research that emphasizes new algorithms without focusing on user-friendliness, practical interpretability, or ef fi cacy for end users [29, 30].”
Perfectly Imperfect: A Speculation about Wabi-Sabi Inspired User Experience Design
Kongot, A., & Matz, A. (2021) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3411763.3450370 · Cited by 6
“Many of the speculations and derived UX concepts imply that there is a co-creation between a machine, software product, and human [15].”
BackgroundExploring the Dynamic Properties of Interaction in Mixed-Initiative Procedural Content Generation
Alvarez, A. (2020) · doi:10.24834/isbn.9789178771400 · Cited by 6
Towards Better User Requirements: How to Involve Human Participants in XAI ResearchPreprint
Nguyen, T., & Zhu, J. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2212.03186 · Cited by 6
“As a result, most explanations produced by XAI still lack usability, practical interpretability, and efficacy for real users [4, 5, 6, 7, 8].”
BackgroundSelf-citationKnow Your Users: Towards Explainable AI in Bangladesh
Islam, F., Mayeesha, T. T., & Ahmed, N. (2024) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/3675094.3679002 · Cited by 5
“The majority of XAI explanations still lack usability, practical interpretability, and effectiveness for actual users [1, 19, 33, 34].”
A typology of explanations to support Explainability-by-Design
Tsakalakis, N., Stalla-Bourdillon, S., Huynh, T. D., & Moreau, L. (2025) · ACM Journal on Responsible Computing · doi:10.1145/3708504 · Cited by 6
“5 Several commentators focus upon specific groups of consumers of explanations, i.e., data subjects [32], developers [62], without directly addressing differing levels of understanding.”
Beyond the Prompt: Community-Oriented Futures for Creative GenAI
Lin, L., Im, J. J., & Long, D. (2025) · Designing Interactive Systems Conference · doi:10.1145/3715336.3735727 · Cited by 5
“This future work could productively build on prior research that has explored how to develop ‘contestable’ AI systems [3] that can be held accountable for their decisions and AI systems that can explain their reasoning processes, especially in shared creative spaces [16, 64, 102].”
RIXA - Explaining Artificial Intelligence in Natural Language
Becker, M., Vishwesh, V., Birnstill, P., Schwall, F., Wu, S., & Beyerer, J. (2023) · IEEE ... International Conference on Data Mining workshops · doi:10.1109/icdmw60847.2023.00118 · Cited by 5
“However, the challenge arises when one notes that most of the XAI implementations lean too heavily on the technical side, often overwhelming end users with intricate metrics and confusing data visualizations [12]– [14].”
BackgroundFrom Metrics to Meaning: Time to Rethink Evaluation in Human–AI Collaborative Design
Walton, S. P., Evans, B. J., Rahat, A., Stovold, J., & Vincalek, J. (2025) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3773292 · Cited by 7
Hand-Crafting Neural Networks for Art-Making.
Ulberg, E., Llach, D. C., & Byrne, D. (2020) · ICCC · Cited by 5
Integrating Players’ Perspectives in AI-Based Games: Case Studies of Player-AI Interaction Design
Villareale, J., Maram, S. S., El‐Nasr, M. S., & Zhu, J. (2023) · International Conference on the Foundations of Digital Games (FDG) · doi:10.1145/3582437.3582451 · Cited by 5
“ing of AI/ML concepts to innovate with AI [27, 49].”
BackgroundSelf-citationCADI — A Conversational Assistive Design Interface for Discovering Pong Variants
Mobramaein, A., Behrooz, M., & Whitehead, J. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment · doi:10.1609/aiide.v14i1.13042 · Cited by 5
Squeezer - A Mixed-Initiative Tool for Designing Juice Effects
Johansen, M., Pichlmair, M., & Risi, S. (2021) · International Conference on the Foundations of Digital Games (FDG) · doi:10.1145/3472538.3472575 · Cited by 6
“Squeezer allows full control and visibility of all the mutated values, providing users with insight into the changes the system is making [37].”
BackgroundSelf-citationThreshold Designer Adaptation: Improved Adaptation for Designers in Co-creative Systems
Halina, E., & Guzdial, M. (2022) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/685 · Cited by 6
Kale, A., & Ma, X. (2023) · Elsevier eBooks · doi:10.1016/b978-0-323-91737-7.00015-3 · Cited by 5
Machine learning methodologies: history and challenges
Loeza-Mejía, C.-I., Sánchez-DelaCruz, E., & Guzmán, I. O. J. (2025) · Evolutionary Intelligence · doi:10.1007/s12065-025-01047-5 · Cited by 6
“Co-creation actively involves these groups in the design process, ensuring their perspectives and needs are prioritized, thereby fostering solutions that are more equitable and responsive to their unique contexts [223, 224].”
Ma, X., Li, Y., & Yang, Y. (2025) · International Conference on Digital Society and Artificial Intelligence (DSAI) · doi:10.1145/3748825.3748837 · Cited by 5
Slide to Explore 'What If': An Analysis of Explainable Interfaces
Meyer, L. S., & Zhu, J. (2024) · Nordic Conference on Human-Computer Interaction · doi:10.1145/3677045.3685416 · Cited by 5
“…methods to define and disseminate explanations [19, 27, 28, 44, 48], principles and protocols for interactions [11, 32], processes for design and analysis of XAI systems [24, 36], a framework for designers to co-create with AI [52], as well as methods for evaluating explanations [14, 30, 37, 42].”
Self-citationTrustworthy Machine Learning Approaches for Cyberattack Detection: A Review
Guembe, B., Azeta, A. A., Misra, S., & Ahuja, R. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-10548-7_20 · Cited by 4
Kim, S., Park, Y. S., Ahn, D., Kwak, J. M., & Kim, J. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3641008 · Cited by 6
“Previous work suggests that users’ perception or reaction to the outcomes of an AI system depends not only on its performance but also on many different factors such as types of errors [37], explainability [29, 62], or user interface of the system [5].”
BackgroundJanson, N., & Kloeden, P. E. (2020) · Journal of Dynamics and Differential Equations · doi:10.1007/s10884-020-09834-7 · Cited by 4
“A popular way to address explainability of a complex system, such as a deep learning neural network, is to construct its model, which would approximate the original system and be explainable, for example to build a decision tree [2,7] or to visualise various stages of data processing [41].”
BackgroundA Machine Learning-Based Micro-World Platform for Condition-Based Maintenance
Guanolusia, D. Q., Rajabiyazdi, F., & Jamieson, G. A. (2020) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc42975.2020.9283448 · Cited by 5
“In the context of explainable AI approaches, ARDAS incorporates visualization-based and post-hoc methods for explaining white-box models [24].”
MethodOconitrillo, L. R. R., Vargas, J. J., Camacho, A., Burgos, A., & Corchado, J. M. (2021) · Studies in computational intelligence · doi:10.1007/978-3-030-64949-4_12 · Cited by 5
Interpretation of SVM to Build an Explainable AI via Granular Computing
Samuel, S. S., Abdullah, N. N. B., & Raj, A. K. (2021) · Studies in computational intelligence · doi:10.1007/978-3-030-64949-4_5 · Cited by 4
Towards the Generation of Musical Explanations with GPT-3
Krol, S. J., Llano, M. T., & McCormack, J. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-03789-4_9 · Cited by 10
“For instance, in [26], different levels of explainability to support game designers are identified, while in [18], a framework for bidirectional human-machine communication is proposed.”
BackgroundOn The Machine Condition and its Creative Expression
2020, 1. I. C. O. C. C., & Colton, S. (2020) · Aaltodoc (Aalto University) · doi:10.48448/29c2-r393 · Cited by 4
Machinic Surrogates: Human-Machine Relationships in Computational CreativityPreprint
Bidgoli, A., Kang, E., & Llach, D. C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.01133 · Cited by 4
“[1, 9, 10] As a broad term, co-creation refers to any act of collective creativity among human agents in different fields, including but not limited to design, public relation, business, and product development.”
BackgroundA Game-Based Approach for Helping Designers Learn Machine Learning ConceptsPreprint
Myers, C. M., Xie, J., & Zhu, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.05605 · Cited by 4
“aterial, and how to implement it purposefully [12]. It is also speculated designers would benefit from improved data literacy, abstracted presentations of ML, and sensitizing concepts [56]. Zhu et al. [57] propose a new area of research, eXplainable AI for Designers, which focuses on creating explainable AI techniques…”
BackgroundInfluentialSelf-citationDigital trust and artificial intelligence
Stawicka, E., & Anderson, T. (2022) · Routledge eBooks · doi:10.4324/9781003266495-14 · Cited by 4
Wu, M., Liu, G., Wang, B., & Ma, Z. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2026.2633738 · Cited by 3
Zhou, W., Karaturhan, P., Weilenmann, A. H., & Zhu, J. (2026) · Designing Interactive Systems Conference · doi:10.1145/3800645.3812954 · Cited by 4
“Despite the technical advancement, however, XAI research has limited success in real-world applications such as MCTAs and other well-being technologies due to the low usability and efficacy of the provided explanations [1, 15, 27, 49, 52, 72].”
Self-citationAn Adaptive and Interpretable Framework for Biomedical Image Analysis
Singh, S., Acton, S. T., Moosa, S., & Sheybani, N. (2023) · Asilomar Conference on Signals, Systems and Computers (ACSSC) · doi:10.1109/ieeeconf59524.2023.10476986 · Cited by 3
“Human beings are more willing to adopt techniques that are directly tractable, interpretable and trustworthy [16].”
BackgroundEthical Challenges in Explainable AI: A Review on Cultural and Social Bias
Thalpage, N. S., Epa, D., & Jayawardhana, S. (2025) · Journal of Digital Art & Humanities · doi:10.33847/2712-8148.6.1_3 · Cited by 4
FUGenerator: multimodal-AI platform for architectural design
Xu, X., Feng, T., Zhang, Y., He, Z., & Yuan, P. F. (2025) · Architectural Intelligence · doi:10.1007/s44223-025-00093-4 · Cited by 3
“It is difficult for designers to evaluate the principles and logic behind the generated content and further propose modifications for the results (Zhu et al., 2018).”
Metadata
- Authors
- Zhu, J., Liapis, A., Risi, S., Bidarra, R., & Youngblood, G. M.
- Affiliations
- Drexel University (US)
- University of Malta (MT)
- IT University of Copenhagen (DK)
- Palo Alto Research Center (US)
- Venue
- 2018 IEEE Conference on Computational Intelligence and Games (CIG)
- Publisher
- IEEE
- Published
- 2018-08
- Pages
- 1-8
- OpenAlex
- W2896010852
- Cited by
- 255
- References
- 35
- Citations by year
- 2026: 14 · 2025: 35 · 2024: 38 · 2023: 33 · 2022: 46 · 2021: 41 · 2020: 33 · 2019: 12 · 2018: 3
- FWCI
- 9.05
- Type
- conference paper
- Language
- en
- Open access
- green
- pure.itu.dk
- Licence
- other-oa
- Locations
- IEEE Conference on Computational Intelligence and Games (CIG) (publishedVersion)
- IT University Of Copenhagen (IT University of Copenhagen) (submittedVersion, other-oa)
- Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
- Research Repository (Delft University of Technology) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Keywords
- explainable AI, game designers, human-centered design, interpretability, game design, AI-assisted design, user needs
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.