The DOI matches its Crossref record; 60 of the 60 references it cites were found in a registry or an index.
Assessment in and of Serious Games: An Overview
Open accessAbstract
There is a consensus that serious games have a significant potential as a tool for instruction. However, their effectiveness in terms of learning outcomes is still understudied mainly due to the complexity involved in assessing intangible measures. A systematic approach—based on established principles and guidelines—is necessary to enhance the design of serious games, and many studies lack a rigorous assessment. An important aspect in the evaluation of serious games, like other educational tools, is user performance assessment. This is an important area of exploration because serious games are intended to evaluate the learning progress as well as the outcomes. This also emphasizes the importance of providing appropriate feedback to the player. Moreover, performance assessment enables adaptivity and personalization to meet individual needs in various aspects, such as learning styles, information provision rates, feedback, and so forth. This paper first reviews related literature regarding the educational effectiveness of serious games. It then discusses how to assess the learning impact of serious games and methods for competence and skill assessment. Finally, it suggests two major directions for future research: characterization of the player’s activity and better integration of assessment in games.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Flow: the psychology of optimal experience
(1990) · Choice Reviews Online · doi:10.5860/choice.28-0597 · Cited by 12,637
What Video Games Have to Teach Us about Learning and Literacy
(2004) · Education + Training · doi:10.1108/et.2004.00446dae.002 · Cited by 6,674
A systematic literature review of empirical evidence on computer games and serious games
Connolly, T. M., Boyle, E. A., MacArthur, E., Hainey, T., & Boyle, J. M. (2012) · Computers & Education · doi:10.1016/j.compedu.2012.03.004 · Cited by 2,890
Toward a Theory of Intrinsically Motivating Instruction*
Malone, T. W. (1981) · Cognitive Science · doi:10.1207/s15516709cog0504_2 · Cited by 2,197
Sweetser (2005) · doi:10.1145/1077246.1077253 · Cited by 2,124
Serious Games: Games That Educate, Train, and Inform
Michael, D. R., & Chen, S. L. (2005) · Cited by 2,068
Flow. The Psychology of Optimal Experience. New York (HarperPerennial) 1990.
Csíkszentmihályi, M. (1990) · Cited by 1,864
Digital Game-Based Learning: It's Not Just the Digital Natives Who Are Restless.
Eck, R. N. V. (2006) · UND Scholarly Commons (University of North Dakota) · Cited by 1,168
The development of the Game Engagement Questionnaire: A measure of engagement in video game-playing
Brockmyer, J. H., Fox, C. M., Curtiss, K. A., McBroom, E., Burkhart, K. M., & Pidruzny, J. N. (2009) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2009.02.016 · Cited by 1,010
Serious games : mechanisms and effects
Ritterfeld, U., Cody, M. J., & Vorderer, P. (2009) · MADOC (University of Mannheim) · Cited by 841
Kato, P. M., Cole, S. W., Bradlyn, A. S., & Pollock, B. H. (2008) · Pediatrics · doi:10.1542/peds.2007-3134 · Cited by 802
EGameFlow: A scale to measure learners’ enjoyment of e-learning games
Fu, F.-L., Su, R.-C., & Yu, S.-C. (2009) · Computers & Education · doi:10.1016/j.compedu.2008.07.004 · Cited by 759
Assessing the Thinking Curriculum: New Tools for Educational Reform
Resnick, L. B., & Resnick, D. P. (1992) · Changing Assessments · doi:10.1007/978-94-011-2968-8_3 · Cited by 589
The Effectiveness of Instructional Games: A Literature Review and Discussion
Hays, R. T. (2005) · doi:10.21236/ada441935 · Cited by 581
Classroom Response Systems: A Review of the Literature
Fies, C., & Marshall, J. (2006) · Journal of Science Education and Technology · doi:10.1007/s10956-006-0360-1 · Cited by 525
Don't bother me, Mom, I'm learning!
Prensky, M. (2006) · Cited by 512
Toward an understanding of flow in video games
Cowley, B., Charles, D., Black, M., & Hickey, R. (2008) · Computers in Entertainment · doi:10.1145/1371216.1371223 · Cited by 479
A Review of Scholarship on Assessing Experiential Learning Effectiveness
Gosen, J., & Washbush, J. (2004) · Simulation & Gaming · doi:10.1177/1046878104263544 · Cited by 365
A review of the use of psychophysiological methods in game research
Kivikangas, J. M., Chanel, G., Cowley, B., Ekman, I., Salminen, M., Järvelä, S., & Ravaja, N. (2011) · Journal of Gaming & Virtual Worlds · doi:10.1386/jgvw.3.3.181_1 · Cited by 288
Cognitive science implications for enhancing training effectiveness in a serious gaming context
Greitzer, F. L., Kuchar, O. A., & Huston, K. (2007) · Journal on Educational Resources in Computing · doi:10.1145/1281320.1281322 · Cited by 249
Bergeron, B. P. (2006) · Cited by 248
Designing Effective Serious Games: Opportunities and Challenges for Research
Bellotti, F., Berta, R., & De Gloria, A. (2010) · International Journal of Emerging Technologies in Learning (iJET) · doi:10.3991/ijet.v5s3.1500 · Cited by 245
Self-citationBeatty, I. D., & Gerace, W. J. (2009) · Journal of Science Education and Technology · doi:10.1007/s10956-008-9140-4 · Cited by 239
Analysis of Pre‐test‐Post‐test Control Group Designs in Educational Research
Dugard, P., & Todman, J. (1995) · Educational Psychology · doi:10.1080/0144341950150207 · Cited by 232
(2009) · Serious Games · doi:10.4324/9780203891650-31 · Cited by 226
Dandeneau, S. D., & Baldwin, M. W. (2004) · Journal of Social and Clinical Psychology · doi:10.1521/jscp.23.4.584.40306 · Cited by 204
Kelly, H., Howell, K., Glinert, E., Holding, L., Swain, C., Burrowbridge, A., & Roper, M. (2007) · Communications of the ACM · doi:10.1145/1272516.1272538 · Cited by 176
A serious game model for cultural heritage
Bellotti, F., Berta, R., De Gloria, A., D'ursi, A., & Fiore, V. (2012) · Journal on Computing and Cultural Heritage · doi:10.1145/2399180.2399185 · Cited by 175
Self-citationMeasuring the experience of digital game enjoyment
IJsselsteijn, W. A., van den Hoogen, W. M., Klimmt, C., de Kort, Y. A. W., Lindley, C. A., Mathiak, K., Poels, K., Ravaja, N., Turpeinen, M., & Vorderer, P. (2008) · TU/e Research Portal · Cited by 145
e‐Assessment for learning? The potential of short‐answer free‐text questions with tailored feedback
Jordan, S., & Mitchell, T. (2009) · British Journal of Educational Technology · doi:10.1111/j.1467-8535.2008.00928.x · Cited by 136
Interactivity and reward-related neural activation during a serious videogame
Cole, S. W., Yoo, D. J., & Knutson, B. (2012) · PLoS ONE · doi:10.1371/journal.pone.0033909 · Cited by 130
Capturing Design Space From a User Perspective: The Repertory Grid Technique Revisited
Hassenzahl, M., & Wessler, R. (2000) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2000.9669070 · Cited by 129
Affective Ludology : Scientific Measurement of User Experience in Interactive Entertainment
Nacke, L. E. (2009) · KTH Publication Database DiVA (KTH Royal Institute of Technology) · Cited by 121
Adaptive Experience Engine for Serious Games
Bellotti, F., Berta, R., Gloria, A. D., & Primavera, L. (2009) · IEEE Transactions on Computational Intelligence and AI in Games · doi:10.1109/tciaig.2009.2035923 · Cited by 114
Self-citationEvolving Opponents for Interesting Interactive Computer Games
Yannakakis, G. N., & Hallam, J. C. T. (2004) · The MIT Press eBooks · doi:10.7551/mitpress/3122.003.0062 · Cited by 110
A Metric for Entertainment of Boardgames: Its Implication for Evolution of Chess Variants
Iida, H., Takeshita, N., & Yoshimura, J. (2003) · IFIP advances in information and communication technology · doi:10.1007/978-0-387-35660-0_8 · Cited by 100
Digital Games in eLearning Environments: Current Uses and Emerging Trends
Pablo Moreno-ger, A., Burgos, D., & Torrente, J. (2009) · Simulation & Gaming · doi:10.1177/1046878109340294 · Cited by 100
Kaugars, A. S., & Russ, S. W. (2009) · Early Education and Development · doi:10.1080/10409280802545388 · Cited by 98
Enhancing the educational value of video games
Bellotti, F., Berta, R., Gloria, A. D., & Primavera, L. (2009) · Computers in entertainment · doi:10.1145/1541895.1541903 · Cited by 96
Self-citationDynamic Game Difficulty Scaling Using Adaptive Behavior-Based AI
Tan, C. H., Tan, K. C., & Tay, A. E. (2011) · IEEE Transactions on Computational Intelligence and AI in Games · doi:10.1109/tciaig.2011.2158434 · Cited by 84
Serious Games for Immersive Cultural Training: Creating a Living World
Zielke, M. A., Evans, M., Dufour, F., et al. (2009) · IEEE Computer Graphics and Applications · doi:10.1109/mcg.2009.30 · Cited by 78
The automatic assessment of free text answers using a modified BLEU algorithm
Noorbehbahani, F., & Kardan, A. A. (2010) · Computers & Education · doi:10.1016/j.compedu.2010.07.013 · Cited by 77
Hewson, C. (2012) · Journal of Computer Assisted Learning · doi:10.1111/j.1365-2729.2011.00473.x · Cited by 74
Supporting authors in the development of task‐based learning in serious virtual worlds
Bellotti, F., Berta, R., Gloria, A. D., & Primavera, L. (2009) · British Journal of Educational Technology · doi:10.1111/j.1467-8535.2009.01039.x · Cited by 69
Self-citationRetrospective on educational testing and assessment in the 20th century
Clarke, M., Madaus, G. F., Horn, C. L., & Ramos, M. A. (2000) · Journal of Curriculum Studies · doi:10.1080/002202700182691 · Cited by 64
Improving Student Performance Using Self-Assessment Tests
Guzmán, E., Conejo, R., & Pérez-de-la-Cruz, J.-L. (2007) · IEEE Intelligent Systems · doi:10.1109/mis.2007.71 · Cited by 61
Educational Validity of Business Gaming Simulation: A Research Methodology Framework
Stainton, A., JOHNSON, J. E. V., & Borodzicz, E. P. (2010) · Simulation & Gaming · doi:10.1177/1046878109353467 · Cited by 53
Oscillatory Brain Responses Evoked by Video Game Events: The Case of Super Monkey Ball 2
Salminen, M., & Ravaja, N. (2007) · CyberPsychology & Behavior · doi:10.1089/cpb.2006.9947 · Cited by 48
Web-based self-assessments in pathology with Questionmark Perception
Velan, G. M., Kumar, R. K., Dziegielewski, M., et al. (2002) · Pathology · doi:10.1080/00313020220131372 · Cited by 45
Lee, N. (2005) · Computers in entertainment · doi:10.1145/1077246.1077248 · Cited by 35
The C3 Framework: Evaluating Classroom Response System Interactions in University Classrooms
Fies, C. H., & Marshall, J. A. (2008) · Journal of Science Education and Technology · doi:10.1007/s10956-008-9116-4 · Cited by 30
Designing entertaining educational games using procedural rhetoric
Doucet, L., & Srinivasan, V. (2010) · ACM SIGGRAPH Symposium on Video Games (Sandbox) · doi:10.1145/1836135.1836136 · Cited by 30
Grove, F. D., Mechant, P., & Jan, V. L. (2010) · International Conference on Fun and Games · doi:10.1145/1823818.1823830 · Cited by 23
Enfield, J., Myers, R. D., Lara, M., et al. (2011) · Simulation & Gaming · doi:10.1177/1046878111408024 · Cited by 23
The Use of Question Mark Software for Formative and Summative Assessment in Two Universities
Bull, J., & Stephens, D. (1999) · Innovations in Education and Training International · doi:10.1080/1355800990360205 · Cited by 22
Hattie, J. (2005) · Minerva Access (University of Melbourne) · Cited by 22
The Implementation of Team Based Assessment In Serious Games
Boyle, L., Hancock, F., Seeney, M., & Allen, L. (2009) · International Conference on Games and Virtual Worlds for Serious Applications · doi:10.1109/vs-games.2009.12 · Cited by 20
A task annotation model for Sandbox Serious Games
Bellotti, F., Berta, R., Gloria, A. D., & Primavera, L. (2009) · IEEE Conference on Computational Intelligence and Games (CIG) · doi:10.1109/cig.2009.5286471 · Cited by 15
Self-citationThe Concept of Formative Assessment
Boston, C. (2002) · University of Massachusetts Amherst · doi:10.7275/kmcq-dj31
Cited by
Mapping learning and game mechanics for serious games analysis
Arnab, S., Lim, T., Carvalho, M. B., Bellotti, F., de Freitas, S., Louchart, S., Suttie, N., Berta, R., & De Gloria, A. (2015) · British Journal of Educational Technology · doi:10.1111/bjet.12113 · Cited by 845
Self-citationThe effect of games and simulations on higher education: a systematic literature review
Vlachopoulos, D., & Makri, A. (2017) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-017-0062-1 · Cited by 776
Assessing the effectiveness of digital game-based learning: Best practices
All, A., Nuñez Castellar, E. P., & Van Looy, J. (2016) · Computers & Education · doi:10.1016/j.compedu.2015.10.007 · Cited by 344
The Role of Serious Games, Gamification and Industry 4.0 Tools in the Education 4.0 Paradigm
Almeida, F., & Simoes, J. (2019) · Contemporary Educational Technology · doi:10.30935/cet.554469 · Cited by 300
Serious Games for education and training
De Gloria, A., Bellotti, F., & Berta, R. (2014) · International Journal of Serious Games · doi:10.17083/ijsg.v1i1.11 · Cited by 280
Self-citation
Metadata
- Authors
- Bellotti, F., Kapralos, B., Lee, K., Moreno-Ger, P., & Berta, R.
- Venue
- Advances in Human-Computer Interaction
- Publisher
- Wiley
- Published
- 2013
- Volume
- 2013
- Pages
- 1-11
- ISSN
- 1687-5893 (ISSN-L)
- 1687-5907
- OpenAlex
- W2043238171
- Cited by
- 592
- References
- 60
- FWCI
- 107.95
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.