The DOI matches its Crossref record; 44 of the 44 references it cites were found in a registry or an index.
Measuring problem solving skills via stealth assessment in an engaging video game
Open accessAbstract
We used stealth assessment, embedded in a game called Use Your Brainz (a slightly modified version of Plants vs. Zombies 2), to measure middle-school students' problem solving skills. We began by developing a problem solving competency model based on a review of the relevant literature. We then identified in-game indicators that would provide evidence about students' levels on various problem-solving facets. Our problem solving model was then implemented in the game via Bayesian networks. To validate the stealth assessment, we collected data from students who played the game-based assessment for three hours and completed two external problem solving measures (i.e., Raven's Progressive Matrices and MicroDYN). Results indicated that the problem solving estimates derived from the game significantly correlated with the external measures, which suggests that our stealth assessment is valid. Our next steps include running a larger validation study and developing tools to help educators interpret the results of the assessment, which will subsequently support the development of problem solving skills. We embedded our problem solving competency model in an engaging game.We measured problem solving skills of students via stealth assessment.Two external measures were used for validation: Raven's matrices and MicroDYN.Significant correlations were found between stealth and external measures.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Handbook of educational psychology
(1997) · Choice Reviews Online · doi:10.5860/choice.34-3084 · Cited by 4,825
Cognitive Psychology and Its Implications
Anderson, J. R. (2014) · Cited by 4,121
Toward a design theory of problem solving
Jonassen, D. H. (2000) · Educational Technology Research and Development · doi:10.1007/bf02300500 · Cited by 2,114
How to Solve It. A New Aspect of Mathematical Method.
Bell, E. T., & Polya, G. (1945) · The American Mathematical Monthly · doi:10.2307/2306109 · Cited by 1,806
Immersive Interfaces for Engagement and Learning
Dede, C. (2009) · Science · doi:10.1126/science.1167311 · Cited by 1,707
Robertson, S. I. (2001) · Cited by 1,650
Instructional design models for well-structured and III-structured problem-solving learning outcomes
Jonassen, D. H. (1997) · Educational Technology Research and Development · doi:10.1007/bf02299613 · Cited by 1,608
The Raven's Progressive Matrices: Change and Stability over Culture and Time
Raven, J. (2000) · Cognitive Psychology · doi:10.1006/cogp.1999.0735 · Cited by 1,560
Academically Adrift: Limited Learning on College Campuses
Arum, R., & Roksa, J. (2011) · Cited by 1,457
How to Solve It : A New Aspect of Mathematical Method
Pólya, G. (2014) · Cited by 1,416
Elements of a theory of human problem solving.
Newell, A., Shaw, J. C., & Simon, H. A. (1958) · Psychological Review · doi:10.1037/h0048495 · Cited by 1,380
Academically Adrift: Limited Learning on College Campuses
Conn, S., & Wymore, J. (2012) · Growth: The Journal of the Association for Christians in Student Development · Cited by 1,239
Serious games : mechanisms and effects
Ritterfeld, U., Cody, M. J., & Vorderer, P. (2009) · MADOC (University of Mannheim) · Cited by 841
Cognitive Psychology and Its Implications
West, A. N. (1981) · The Journal of mind and behavior · Cited by 824
Focus Article: On the Structure of Educational Assessments
Mislevy, R. J., Steinberg, L. S., & Almond, R. G. (2003) · Measurement: Interdisciplinary Research & Perspective · doi:10.1207/s15366359mea0101_02 · Cited by 802
Computer Games and Instruction.
Tobias, S., & Fletcher, J. D. (2011) · Cited by 492
Parsons, J., & Taylor, L. (2011) · Current Issues in Education · Cited by 465
(2013) · Handbook of Educational Psychology · doi:10.4324/9780203053874-9 · Cited by 420
Stealth Assessment: Measuring and Supporting Learning in Video Games
Shute, V., & Ventura, M. (2013) · doi:10.7551/mitpress/9589.001.0001 · Cited by 406
Self-citationSTANDARDIZATION OF PROGRESSIVE MATRICES, 1938
RAVEN, J. C. (1941) · British Journal of Medical Psychology · doi:10.1111/j.2044-8341.1941.tb00316.x · Cited by 389
Gick, M. L. (1986) · Educational Psychologist · doi:10.1080/00461520.1986.9653026 · Cited by 387
The ideal problem solver : a guide for improving thinking, learning, and creativity
Bransford, J. D., & Stein, B. S. (1984) · Cited by 340
Foundation for problem‐based gaming
Kiili, K. (2007) · British Journal of Educational Technology · doi:10.1111/j.1467-8535.2007.00704.x · Cited by 290
The power of play: The effects of Portal 2 and Lumosity on cognitive and noncognitive skills
Shute, V. J., Ventura, M., & Ke, F. (2015) · Computers & Education · doi:10.1016/j.compedu.2014.08.013 · Cited by 288
Self-citationUsing Cognitive Tools to Represent Problems
Jonassen, D. (2003) · Journal of Research on Technology in Education · doi:10.1080/15391523.2003.10782391 · Cited by 285
Games, Learning, and Assessment
Shute, V. J., & Ke, F. (2012) · Assessment in Game-Based Learning · doi:10.1007/978-1-4614-3546-4_4 · Cited by 233
Self-citation(2009) · Serious Games · doi:10.4324/9780203891650-31 · Cited by 226
Complex problem solving — More than reasoning?
Wüstenberg, S., Greiff, S., & Funke, J. (2012) · Intelligence · doi:10.1016/j.intell.2011.11.003 · Cited by 219
Self-citationAssessment in Game-Based Learning: Foundations, Innovations, and Perspectives
(2012) · Springer eBooks · doi:10.1007/978-1-4614-3546-4 · Cited by 202
Domain-general problem solving skills and education in the 21st century
Greiff, S., Wüstenberg, S., Csapó, B., Demetriou, A., Hautamäki, J., Graesser, A. C., & Martin, R. (2014) · Educational Research Review · doi:10.1016/j.edurev.2014.10.002 · Cited by 188
Self-citationBayesian Networks in Educational Assessment
Almond, R. G., Mislevy, R. J., Steinberg, L. S., Yan, D., & Williamson, D. M. (2015) · Statistics for Social and Behavioral Sciences · doi:10.1007/978-1-4939-2125-6 · Cited by 185
Assessment and Learning of Qualitative Physics in Newton's Playground
Shute, V. J., Ventura, M., & Kim, Y. J. (2013) · The Journal of Educational Research · doi:10.1080/00220671.2013.832970 · Cited by 162
Self-citationEvaluating multiple aspects of a digital educational problem-solving-based adventure game
Hou, H., & Li, M.-C. (2013) · Computers in Human Behavior · doi:10.1016/j.chb.2013.07.052 · Cited by 151
Advances in the Science of Assessment
Shute, V. J., Leighton, J. P., Jang, E. E., & Chu, M.-W. (2016) · Educational Assessment · doi:10.1080/10627197.2015.1127752 · Cited by 146
Self-citationAssessment and Learning of Qualitative Physics in Newton’s Playground
Ventura, M., Shute, V. J., & Kim, Y. J. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-39112-5_63 · Cited by 110
Self-citationThe influence of video games on executive functions in college students
Buelow, M. T., Okdie, B. M., & Cooper, A. (2015) · Computers in Human Behavior · doi:10.1016/j.chb.2014.12.029 · Cited by 90
Schweizer, F., Wüstenberg, S., & Greiff, S. (2013) · Learning and Individual Differences · doi:10.1016/j.lindif.2012.12.011 · Cited by 88
Self-citationEffects of Instructional Style on Problem-Solving Creativity
Ruscio, A. M., & Amabile, T. M. (1999) · Creativity Research Journal · doi:10.1207/s15326934crj1204_3 · Cited by 81
Bayesian Networks in Educational Assessment: The State of the Field
Culbertson, M. J. (2015) · Applied Psychological Measurement · doi:10.1177/0146621615590401 · Cited by 74
Conceptual Framework for Modeling, Assessing and Supporting Competencies within Game Environments.
Shute, V. J., Masduki, I., & Donmez, O. (2010) · Cited by 58
Self-citationLessons Learned and Best Practices of Stealth Assessment
Wang, L., Shute, V. J., & Moore, G. R. (2015) · International Journal of Gaming and Computer-Mediated Simulations · doi:10.4018/ijgcms.2015100104 · Cited by 46
Self-citationGagné, R. M. (1959) · Annual Review of Psychology · doi:10.1146/annurev.ps.10.020159.001051 · Cited by 38
Assessment With Microworlds using MicroDYN: Measurement Invariance and Latent Mean Comparisons
Greiff, S., & Wüstenberg, S. (2014) · European Journal of Psychological Assessment · doi:10.1027/1015-5759/a000194 · Cited by 25
Self-citationKim, J. S., Vanderwood, M. L., & Lee, C. Y. (2016) · Educational Assessment · doi:10.1080/10627197.2015.1127750 · Cited by 19
Cited by
Review of computer‐based assessment for learning in elementary and secondary education
Shute, V. J., & Rahimi, S. (2017) · Journal of Computer Assisted Learning · doi:10.1111/jcal.12172 · Cited by 282
Self-citationA Scoping Review of Digital Game-Based Technology on English Language Learning
Xu, Z., Chen, Z., Eutsler, L., Geng, Z., & Kogut, A. (2020) · Educational Technology Research and Development · doi:10.1007/s11423-019-09702-2 · Cited by 162
Artificial intelligence moving serious gaming: Presenting reusable game AI components
Westera, W., Prada, R., Mascarenhas, S., Santos, P. A., Dias, J., Guimarães, M., Georgiadis, K., Nyamsuren, E., Bahreini, K., Yumak, Z., et al. (2020) · Education and Information Technologies · doi:10.1007/s10639-019-09968-2 · Cited by 136
Sung, S. H., Li, C., Huang, X., & Xie, C. (2021) · Journal of Computer Assisted Learning · doi:10.1111/jcal.12600 · Cited by 35
Using machine learning to model trace behavioral data from a game‐based assessment
Auer, E. M., Mersy, G., Marin, S., Blaik, J., & Landers, R. N. (2022) · International Journal of Selection and Assessment · doi:10.1111/ijsa.12363 · Cited by 24
Metadata
- Authors
- Shute, V. J., Wang, L., Greiff, S., Zhao, W., & Moore, G.
- Venue
- Computers in Human Behavior
- Publisher
- Elsevier BV
- Published
- 2016-10
- Volume
- 63
- Pages
- 106-117
- ISSN
- 0747-5632 (ISSN-L)
- OpenAlex
- W2398473626
- Cited by
- 271
- References
- 44
- FWCI
- 41.03
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.