Tales from the front lines of a large-scale serious game project
Abstract
Serious games have received much positive attention; correspondingly, many researchers have taken up the challenge of establishing how to best design them. However, the current literature often focuses on best practice design strategies and frameworks. Fine-grained details, contextual descriptions, and organisational factors that are invaluable in helping us to learn from and reflect on project experiences are often overlooked. In this paper, we present five distinct and sometimes competing perspectives that are critical in understanding factors that influence serious game projects: project organisation, technology, domain knowledge, user research, and game design. We explain these perspectives by providing insights from the design and development process of an EU-funded serious game about conflict resolution developed by an interdisciplinary consortium of researchers and industry-based developers. We also point out a set of underlying forces that become evident from viewing the process from different perspectives, to underscore that problems exist in serious game projects and that we should open the conversation about them.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The endeavor to support future development of the practice was messy and complex.
“Instead of painting an overly positive picture, we show the messy and complex character of our endeavor to support the future development of the practice (Khaled and Ingram, 2012).”
From Educational Participatory Design in the Crossroads of Histories and Practices – Aiming for Digital Transformation in Language Pedagogy (2023) · cites this paper as Khaled and Ingram, 2012
Cites
The Reflective Practitioner: How Professionals Think in Action
Checkoway, B. N., & Schön, D. A. (1985) · Journal of Policy Analysis and Management · doi:10.2307/3324262 · Cited by 21,311 · see citing works
The reflective practitioner: How professionals think in action
Stewart, C. J. (1984) · Patient Education and Counseling · doi:10.1016/0738-3991(84)90022-3 · Cited by 19,751 · see citing works
What Video Games Have to Teach Us about Learning and Literacy
(2004) · Education + Training · doi:10.1108/et.2004.00446dae.002 · Cited by 6,674 · see citing works
The Psychology of Human-Computer Interaction
Card, S. K., Moran, T. P., & Newell, A. (2018) · CRC Press eBooks · doi:10.1201/9780203736166 · Cited by 4,103 · see citing works
The Psychology of Everyday Things
Massaro, D. W., & Norman, D. A. (1990) · The American Journal of Psychology · doi:10.2307/1423268 · Cited by 3,475 · see citing works
The psychology of human-computer interaction
Rogers, Y. (1984) · Applied Ergonomics · doi:10.1016/0003-6870(84)90205-9 · Cited by 1,970 · see citing works
Schell, J., & Schell, J. (2008) · CRC Press eBooks · doi:10.1201/9780080919171 · Cited by 1,902 · see citing works
Cockburn, A. (2001) · UCLA - Biblioteca de Ciencias y Tecnologia Felix Morales Bueno · Cited by 1,858 · see citing works
Persuasive Games: The Expressive Power of Videogames
(2008) · Choice Reviews Online · doi:10.5860/choice.46-0096 · Cited by 1,714 · see citing works
From visual simulation to virtual reality to games
Zyda, M. (2005) · Computer · doi:10.1109/mc.2005.297 · Cited by 1,472 · see citing works
The resolution of conflict;: Constructive and destructive processes
Deutsch, M. (1973) · Virtual Defense Library (Ministerio de Defensa) · Cited by 1,441 · see citing works
From Content to Context: Videogames as Designed Experience
Squire, K. (2006) · Educational Researcher · doi:10.3102/0013189x035008019 · Cited by 1,091 · see citing works
Williams, L. A., & Tomayko, J. E. (2002) · Computer Science Education · doi:10.1076/csed.12.3.167.8613 · Cited by 914 · see citing works
What makes things fun to learn? heuristics for designing instructional computer games
Malone, T. W. (1980) · Proceedings of the 3rd ACM SIGSMALL symposium and the first SIGPC symposium on Small systems - SIGSMALL '80 · doi:10.1145/800088.802839 · Cited by 758 · see citing works
Sengers, P., Boehner, K., David, S., & Kaye, J. '. (2005) · Proceedings of the 4th decennial conference on Critical computing: between sense and sensibility · doi:10.1145/1094562.1094569 · Cited by 747 · see citing works
Conflict and conflict management: Reflections and update
Thomas, K. (1992) · Journal of Organizational Behavior · doi:10.1002/job.4030130307 · Cited by 733 · see citing works
The Resolution of Conflict: Constructive and Destructive Processes.
Kochan, T. A., & Deutsch, M. (1974) · Administrative Science Quarterly · doi:10.2307/2393904 · Cited by 725 · see citing works
The Human-Computer Interaction Handbook
Sears, A., & Jacko, J. A. (2007) · CRC Press eBooks · doi:10.1201/9781410615862 · Cited by 725 · see citing works
Salvat, B. M. G. (2007) · Journal of Research on Technology in Education · doi:10.1080/15391523.2007.10782494 · Cited by 682 · see citing works
Hoegl, M., Weinkauf, K., & Gemuenden, H. G. (2004) · Organization Science · doi:10.1287/orsc.1030.0053 · Cited by 611 · see citing works
The Human-Computer Interaction Handbook
(2002) · CRC Press eBooks · doi:10.1201/9781410606723 · Cited by 597 · see citing works
The “I's” Have It: A Framework for Serious Educational Game Design
Annetta, L. A. (2010) · Review of General Psychology · doi:10.1037/a0018985 · Cited by 483 · see citing works
Designing for or designing with? Informant design for interactive learning environments
Scaife, M., Rogers, Y., Aldrich, F. K., & Davies, M. (1997) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/258549.258789 · Cited by 388 · see citing works
Racing the beam: the Atari video computer system
(2009) · Choice Reviews Online · doi:10.5860/choice.46-6853 · Cited by 321 · see citing works
The Design, Play, and Experience Framework
Winn, B. M. (2009) · IGI Global eBooks · doi:10.4018/978-1-59904-808-6.ch058 · Cited by 320 · see citing works
Montfort, N., & Bogost, I. (2009) · The MIT Press eBooks · doi:10.7551/mitpress/7588.001.0001 · Cited by 274 · see citing works
Third generation educational use of computer games
Egenfeldt‐Nielsen, S. (2007) · Journal of educational multimedia and hypermedia · Cited by 230 · see citing works
Practical, appropriate, empirically-validated guidelines for designing educational games
Linehan, C., Kirman, B. J., Lawson, S. W., & Chan, G. (2011) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1978942.1979229 · Cited by 196 · see citing works
Future Directions in the Study of Close Relationships: Conflict Is Bad (Except When It's Not)
Laursen, B. P., & Hafen, C. A. (2009) · Social Development · doi:10.1111/j.1467-9507.2009.00546.x · Cited by 178 · see citing works
Game design workshop a playcentric approach to creating innovative games
Fullerton, T., Swain, C., & Hoffman, S. (2008) · A K Peters/CRC Press eBooks · doi:10.1201/9781003460268 · Cited by 121 · see citing works
Constantine, L. L. (1993) · Communications of the ACM · doi:10.1145/163430.163435 · Cited by 117 · see citing works
Gaver, W. W., Bowers, J., Kerridge, T., Boucher, A., & Jarvis, N. (2009) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1518701.1519040 · Cited by 108 · see citing works
Rankin, Y. A., McNeal, M., Shute, M. W., & Gooch, B. (2008) · ACM SIGGRAPH Symposium on Video Games (Sandbox) · doi:10.1145/1401843.1401851 · Cited by 101 · see citing works
A game design methodology to incorporate social activist themes
Flanagan, M., & Nissenbaum, H. F. (2007) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1240624.1240654 · Cited by 90 · see citing works
Practical barriers in using educational computer games
Egenfeldt‐Nielsen, S. (2004) · On the Horizon The International Journal of Learning Futures · doi:10.1108/10748120410540454 · Cited by 88 · see citing works
Isbister, K., Flanagan, M., & Hash, C. (2010) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1753326.1753637 · Cited by 82 · see citing works
Bodine, R., & Crawford, D. (1998) · Cited by 66 · see citing works
The Psychology of Everyday Things
Friedel, R., & Norman, D. A. (1990) · Technology and Culture · doi:10.2307/3106094 · Cited by 56 · see citing works
Collaboration in serious game development
Tran, M. Q., & Biddle, R. (2008) · Conference on Future Play · doi:10.1145/1496984.1496993 · Cited by 35 · see citing works
Perceptions of Player in Game Design Literature
Sotamaa, O. (2007) · DiGRA Digital Library · doi:10.26503/dl.v2007i1.322 · Cited by 32 · see citing works
Losh, E. (2006) · ACM SIGGRAPH Symposium on Video Games (Sandbox) · doi:10.1145/1183316.1183334 · Cited by 11 · see citing works
Flagged first, then by citations
2 of the 43 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
The Lens of Intrinsic Skill Atoms: A Method for Gameful Design
Deterding, S. (2015) · Human–Computer Interaction · doi:10.1080/07370024.2014.993471 · Cited by 525 · see citing works
Tailoring persuasive health games to gamer type
Orji, R., Mandryk, R. L., Vassileva, J., & Gerling, K. (2013) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2470654.2481341 · Cited by 227 · see citing works
Designing gamification: creating gameful and playful experiences
Deterding, S., Björk, S. L., Nacke, L. E., Dixon, D., & Lawley, E. (2013) · CHI '13 Extended Abstracts on Human Factors in Computing Systems · doi:10.1145/2468356.2479662 · Cited by 192 · see citing works
Tsekleves, E., Cosmas, J., & Aggoun, A. (2014) · British Journal of Educational Technology · doi:10.1111/bjet.12223 · Cited by 149 · see citing works
"A Game that Makes You Question..."
Mekler, E. D., Iacovides, I., & Bopp, J. A. (2018) · Annual Symposium on Computer-Human Interaction in Play · doi:10.1145/3242671.3242691 · Cited by 97 · see citing works
Roussou, M., Katifori, A., Pujol, L., Vayanou, M., & Rennick‐Egglestone, S. (2013) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/2468356.2468453 · Cited by 49 · see citing works
Processes and Models for Serious Game Design and Development
Braad, E., Žavcer, G., & Sandovar, A. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46152-6_5 · Cited by 47 · see citing works
Denisova, A., Bopp, J. A., Nguyen, T. D. T., & Mekler, E. D. (2021) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445286 · Cited by 45 · see citing works
Gee, E. R., & Tran, K. M. (2015) · Advances in media, entertainment and the arts (AMEA) book series · doi:10.4018/978-1-4666-8310-5.ch010 · Cited by 41 · see citing works
Gamification and Serious Games in a Healthcare Informatics Context
Croon, R. D., Wildemeersch, D., Wille, J., Verbert, K., & Abeele, V. V. (2018) · IEEE International Conference on Healthcare Informatics (ICHI) · doi:10.1109/ichi.2018.00014 · Cited by 40 · see citing works
Design metaphors for procedural content generation in games
Khaled, R., Nelson, M. J., & Barr, P. (2013) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2470654.2466201 · Cited by 37 · see citing works
Self-citationState-of-the-art in Business Games
Petridis, P., Hadjicosta, K., Guang, V. S., Dunwell, I., Baines, T. S., Bigdeli, A. Z., Bustinza, Ó. F., & Uren, V. S. (2015) · International Journal of Serious Games · doi:10.17083/ijsg.v2i1.54 · Cited by 31 · see citing works
Adapting Cognitive Task Analysis to Elicit the Skill Chain of a Game
Horn, B., Cooper, S., & Deterding, S. (2017) · Annual Symposium on Computer-Human Interaction in Play · doi:10.1145/3116595.3116640 · Cited by 26 · see citing works
Documenting trajectories in design space
Khaled, R., Lessard, J., & Barr, P. (2018) · International Conference on the Foundations of Digital Games (FDG) · doi:10.1145/3235765.3235767 · Cited by 24 · see citing works
Self-citationFrom Front-End to Back-End and Everything In-Between
Koleva, B. N., Tolmie, P., Brundell, P., Benford, S., & Rennick‐Egglestone, S. (2015) · Annual Symposium on Computer-Human Interaction in Play · doi:10.1145/2793107.2793131 · Cited by 22 · see citing works
Adopting Best Practices from the Games Industry in Development of Serious Games for Health
Ushaw, G., Davison, R., Eyre, J. A., & Morgan, G. (2015) · International Conference on Digital Health · doi:10.1145/2750511.2750513 · Cited by 18 · see citing works
“Her story was complex”: A Twine workshop for ten- to twelve-year-old girls
Tran, K. M. (2016) · E-Learning and Digital Media · doi:10.1177/2042753016689635 · Cited by 18 · see citing works
Altizer, R., Zagal, J. P., Johnson, E. P., Wong, B. G., Anderson, R., Botkin, J. R., & Rothwell, E. (2017) · Annual Symposium on Computer-Human Interaction in Play · doi:10.1145/3130859.3131333 · Cited by 17 · see citing works
Serious Games as Communicative Tools for Attitudinal and Behavioral Change
Muhamad, J. W., & Kim, S. Y. (2020) · Wiley eBooks · doi:10.1002/9781119399926.ch9 · Cited by 14 · see citing works
Using extracted features to inform alignment-driven design ideas in an educational game
Harpstead, E., MacLellan, C. J., Aleven, V., & Myers, B. A. (2014) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2556288.2557393 · Cited by 13 · see citing works
Creating Bridges: The Role of Exploratory Design Research in an Intelligent Tutoring System Project
Gooch, D., Benton, L., Khaled, R., Lukeš, D., & Vasalou, A. (2015) · Interacting with Computers · doi:10.1093/iwc/iwv009 · Cited by 12 · see citing works
Self-citationEnd-User Development for Serious Games
Menestrina, Z., & Angeli, A. D. (2017) · Springer eBooks · doi:10.1007/978-3-319-60291-2_14 · Cited by 11 · see citing works
Codespells: how to design quests to teach java concepts
Esper, S., Wood, S., Foster, S. R., Lerner, S., & Griswold, W. G. (2014) · Journal of computing sciences in colleges · Cited by 10 · see citing works
Cheng, J., Putnam, C., & Guo, J. (2016) · Annual Symposium on Computer-Human Interaction in Play · doi:10.1145/2967934.2968081 · Cited by 10 · see citing works
My Dream Theatre: Putting conflict on center stage.
Campos, J., Martinho, C., Ingram, G., Vasalou, A., & Paiva, A. (2013) · Foundations of Digital Games · Cited by 9 · see citing works
Self-citationEditorial: Learning from failures in game design for children
Moser, C., Tscheligi, M., Zaman, B., Abeele, V. V., Geurts, L., Vandewaetere, M., Markopoulos, P., & Wyeth, P. A. (2014) · International Journal of Child-Computer Interaction · doi:10.1016/j.ijcci.2014.10.001 · Cited by 8 · see citing works
A Serious Game for Promoting Positive Attitudes Towards Nursing Homes Among Youth
Folkins, C., Read, E. A., Mundee, J., Birk, M. V., & Bateman, S. (2020) · Annual Symposium on Computer-Human Interaction in Play · doi:10.1145/3410404.3414253 · Cited by 7 · see citing works
Kellerhals, U., Burgess, N., & Wetzel, R. (2021) · International Conference on the Foundations of Digital Games (FDG) · doi:10.1145/3472538.3472558 · Cited by 7 · see citing works
Kinnula, M., Iivari, N., Kuure, L., & Molin-Juustila, T. (2023) · Computer Supported Cooperative Work (CSCW) · doi:10.1007/s10606-023-09473-8 · Cited by 6 · see citing works
“Instead of painting an overly positive picture, we show the messy Educational Participatory Design in the Crossroads of Histories… and complex character of our endeavor to support the future development of the practice (Khaled and Ingram, 2012).”
BackgroundWetzel, R. D., Bachour, K., & Flintham, M. (2018) · Simulation & Gaming · doi:10.1177/1046878118818867 · Cited by 6 · see citing works
Developing a digital game to support cultural learning amongst immigrants
Dunwell, I., Lameras, P., Stewart, C. D., Petridis, P., Arnab, S., Hendrix, M., de Freitas, S., Gaved, M., Schuller, B. W., & Paletta, L. (2013) · Coventry University Open Collections (Coventry university) · Cited by 6 · see citing works
Detecting Gamification in Breast Cancer Apps: An Automatic Methodology for Screening Purposes
Giunti, G., Giunta, D. H., Hors-Fraile, S., Isomursu, M., & Karosevičiūtė, D. (2017) · Proceedings - IEEE Symposium on Computer-Based Medical Systems · doi:10.1109/cbms.2017.21 · Cited by 6 · see citing works
Sustainable Life Cycle Game Design: Mixing Games and Reality to Transform Education
Harteveld, C., Folajimi, Y., & Sutherland, S. C. (2016) · doi:10.26503/dl.v2016i1.804 · Cited by 4 · see citing works
QR code programming tasks with automated assessment
Hakulinen, L., & Malmi, L. (2014) · ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE) · doi:10.1145/2591708.2591761 · Cited by 4 · see citing works
MASELTOV Deliverable D7.1.1: Incidental Learning Framework
Andrew Brasher, A., Dunwell, I., Akiki, O., & Gaved, M. (2012) · Open Research Online (The Open University) · Cited by 3 · see citing works
Moser, C., Tscheligi, M., Zaman, B., Abeele, V. V., Geurts, L., Vandewaetere, M., Markopoulos, P., Wyeth, P. A., & Hofstätter, J. (2013) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/2468356.2479646 · Cited by 3 · see citing works
Considering Diversity in Collaborative Video Game Design Work
Simons, R. N. (2016) · Annual Symposium on Computer-Human Interaction in Play · doi:10.1145/2968120.2987749 · Cited by 3 · see citing works
Iivari, N. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300626 · Cited by 3 · see citing works
Esa, N. M., Mailok, R., & Yatim, M. H. M. (2017) · Sains Humanika · doi:10.11113/sh.v9n3.932 · Cited by 2 · see citing works
Game Playbooks 2.0: An Updated Strategy for Supporting Game-Based Behavior Change Interventions
Truesdell, E. J. K., Weser, V. U., Murray, T. S., Crocker, A. C., Marks, A. M., Martinello, R. A., & Hieftje, K. D. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3748631 · Cited by 1 · see citing works
Player-Specific Conflict Handling Ontology
Hondrou, C., Tsalapati, E., Raouzaiou, A. T., Karpouzis, K. C., & Kollias, S. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-12157-4_24 · Cited by 1 · see citing works
SUPPORTING THERAPY-CENTERED GAME DESIGN FOR BRAIN INJURY REHABILITATION
Cheng, J. (2017) · Digital Commons-DePaul (DePaul University) · Cited by 1 · see citing works
Ribeiro, C., Antunes, T., Pereira, J. M., & Monteiro, M. (2014) · International Journal of Game-Based Learning · doi:10.4018/ijgbl.2014100105
Fikar, P., Habiger, M., & Tellioğlu, H. (2017) · International Conference on Advances in Mobile Computing and Multimedia (MoMM) · doi:10.1145/3151848.3151876
Spieldesign mit Mixed Reality Game Cards
Wetzel, R. (2019) · Springer eBooks · doi:10.1007/978-3-658-27260-9_10
Incisive Real-Time Biosafety Decision-Making Under Societal Reopening
Hai-Jew, S. (2021) · Advances in educational technologies and instructional design book series · doi:10.4018/978-1-7998-4087-9.ch005
Ribeiro, C., Antunes, T., Pereira, J. M., & Monteiro, M. (2015) · IGI Global eBooks · doi:10.4018/978-1-4666-8200-9.ch080
„Why so serious?” : Komoly játékok az oktatásban
Péter-Szabó, R. (2023) · doi:10.14267/phd.2023030
The Role of Reflection in Educational Games. Developing Skills for This New Millennium
Pacheco-Velazquez, E. A., Glasserman‐Morales, L. D., Ramirez-Echeverri, S., & Carlos-Arroyo, M. (2024) · Lecture notes in educational technology · doi:10.1007/978-981-97-1814-6_4
Experiencing Affective Agents in Simulation Games
Sales, R., Clua, E., Paes, A., & de Oliveira, D. R. C. F. (2014) · Revista de Informática Aplicada · doi:10.13037/ria.vol9n1.2745
- Authors
- Khaled, R., & Ingram, G.
- Affiliations
- IT University of Copenhagen (DK)
- University of Bath (GB)
- Venue
- Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2012-05-05
- Pages
- 69-78
- OpenAlex
- W2158930783
- Cited by
- 51
- References
- 43
- Citations by year
- 2025: 1 · 2024: 1 · 2023: 2 · 2021: 3 · 2020: 2 · 2019: 3 · 2018: 4 · 2017: 7 · 2016: 5 · 2015: 7 · 2014: 7 · 2013: 8 · 2012: 1
- FWCI
- 16.10
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- IT University Of Copenhagen (IT University of Copenhagen) (submittedVersion)
- Research Space (submittedVersion)
- Funders
- European Commission
- FP7 Information and Communication Technologies
- Topics
- Social Sciences › Psychology › Developmental and Educational Psychology › Educational Games and Gamification
- Social Sciences › Sociology and Political Science › Digital Games and Media
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
- Keywords
- serious games, serious game design, project organization, technological factors, domain knowledge, user research, game design, interdisciplinary collaboration, conflict resolution
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
