CoChat
Sign Up
Cited by 14Educational Games and GamificationDigital Games and MediaMobile Crowdsensing and Crowdsourcing

A Survey of Citizen Science Gaming Experiences

Open access
Miller, Josh Aaron, Gandhi, Kutub, Gander, Anna, Cooper, Seth·2022·Citizen Science: Theory and Practice
DOI 10.5334/cstp.500
Open this paper in CoChat — free accountOpen PDF

Abstract

Citizen science games (CSGs) are a valuable means for motivating citizen science participation. However, many CSGs still suffer from the recruitment and retention issues of traditional citizen science projects, despite much prior literature on what motivates CSG players. In this study, we take a Human-Computer Interaction (HCI) perspective to explore in what ways CSGs are still failing to provide motivating play experiences for players. Using Qualitative Content Analysis, we conducted and analyzed an online survey of 185 players from 9 citizen science games. This survey contributes insights to the current state of CSG experiences and next steps for developers to address these issues. We found that major concerns included scientific communication, instructional design, user interface and controls, task quality, and software issues.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.5334/cstp.500

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The target audience of classical citizen science games focuses on users with prior interest in science, which amplifies demographic skew in citizen science projects.

    “Hence, their target audience focuses on users with prior interest in science15, which eventually amplifies the demographic skew observed in citizen science projects19.”

    From Improving microbial phylogeny with citizen science within a mass-market video game (2025) · cites this paper as [15]

    The Background section says that previous interest in science is one motivation for citizen science game players. It does not establish that these games target users with that interest. The demographic-skew assertion carries a separate citation (19), so it is not assessed against this source. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Still being checked.

Research works printed but not linked: 59

  1. Alexandrovsky, D, Friehs, MA, Birk, MV, Yates, RK and Mandryk, RL. 2019. Game Dynamics that Support Snacking, not Feasting. In: Proceedings of the Annual Symposium on Computer-Human Interaction in Play. Barcelona, Spain: ACM, 573-588. DOI: https://doi.org/10.1145/3311350.3347151 · DOI
  2. Aristeidou, M and Herodotou, C. 2020. Online citizen science: A systematic review of effects on learning and scientific literacy. Citizen Science: Theory and Practice, 5: 1-12. DOI: https://doi.org/10.5334/cstp.224 · DOI
  3. Aristeidou, M, Scanlon, E and Sharples, M. 2017. Profiles of engagement in online communities of citizen science participation. Computers in Human Behavior, 74: 246-256. DOI: https://doi.org/10.1016/j.chb.2017.04.044 · DOI
  4. Becker-Klein, R, Peterman, K and Stylinski, C. 2016. Embedded Assessment as an Essential Method for Understanding Public Engagement in Citizen Science. CSTP, 1: 8. DOI: https://doi. org/10.5334/cstp.15 · DOI
  5. Bowser, A, Hansen, D, He, Y, Boston, C, Reid, M, Gunnell, L and Preece, J. 2013. Using gamification to inspire new citizen science volunteers. In: Proceedings of the First International Conference on Gameful Design, Research, and Applications. Toronto, Ontario, Canada: ACM, 18-25. DOI: https://doi. org/10.1145/2583008.2583011 · DOI
  6. Clark, O. 2014. Games As A Service: How Free to Play Design Can Make Better Games, 1st ed. Burlington, MA, USA: Focal Press.
  7. Clary, EG and Snyder, M. 1999. The Motivations to Volunteer: Theoretical and Practical Considerations. Curr Dir Psychol Sci, 8: 156-159. DOI: https://doi.org/10.1111/1467-8721.00037 · DOI
  8. Cooper, S. 2011. A framework for scientific discovery through video games (Doctoral Dissertation). University of Washington.
  9. Curtis, V. 2018. Who Takes Part in Online Citizen Science?. In: Online Citizen Science and the Widening of Academia. Springer International Publishing. Cham, 45-68. DOI: https:// doi.org/10.1007/978-3-319-77664-4_3 · DOI
  10. Curtis, V. 2015. Motivation to Participate in an Online Citizen Science Game: A Study of Foldit. Science Communication, 37: 723-746. DOI: https://doi.org/10.1177/1075547015609322 · DOI
  11. 'Ornellas, MC, Cargnin, DJ and Prado, ALC. 2015. Evaluating the Impact of Player Experience in the Design of a Serious Game for Upper Extremity Stroke Rehabilitation. In: Proceedings of the 15th World Congress on Health and Biomedical Informatics. IOS Press, 363-367.
  12. de Vries, M, Land-Zandstra, A and Smeets, I. 2019. Citizen scientists' preferences for communication of scientific output: a literature review. Citizen Science: Theory and Practice, 4: 2. DOI: https://doi.org/10.5334/cstp.136 · DOI
  13. Delgado, JCS and Bazán, P. 2019. Educational Serious Games as a Service: Challenges and Solutions. JC&ST, 19: e07. DOI: https://doi.org/10.24215/16666038. · DOI
  14. e07 Desurvire, H and Wiberg, C. 2009. Game Usability Heuristics (PLAY) for Evaluating and Designing Better Games: The Next Iteration. In: Ozok, AA and Zaphiris, P. (Eds.), Online Communities and Social Computing, Lecture Notes in Computer Science. Berlin, Heidelberg: Springer Berlin Heidelberg, 557-566. DOI: https://doi.org/10.1007/978-3- 642-02774-1_60 · DOI
  15. Díaz, C, Ponti, M, Haikka, P, Basaiawmoit, R and Sherson, J. 2020. More than data gatherers: exploring player experience in a citizen science game. Qual User Exp, 5: 1. DOI: https:// doi.org/10.1007/s41233-019-0030-8 · DOI
  16. Elo, S and Kyngäs, H, 2008. The qualitative content analysis process. J Adv Nurs, 62: 107-115. DOI: https://doi. org/10.1111/j.1365-2648.2007.04569.x · DOI · Paper page
  17. Eveleigh, A, Jennett, C, Blandford, A, Brohan, P and Cox, AL. 2014. Designing for dabblers and deterring drop-outs in citizen science. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Toronto, Ontario, Canada: ACM, 2985-2994. DOI: https://doi. org/10.1145/2556288.2557262 · DOI
  18. Eveleigh, A, Jennett, C, Lynn, S and Cox, AL. 2013. "I want to be a captain! I want to be a captain!": gamification in the old weather citizen science project. In: Proceedings of the First International Conference on Gameful Design, Research, and Applications. Toronto, Ontario, Canada: ACM, 79-82. DOI: https://doi.org/10.1145/2583008.2583019 · DOI
  19. Farley, PC. 2013. Using the Computer Game "FoldIt" to Entice Students to Explore External Representations of Protein Structure in a Biochemistry Course for Nonmajors. Biochem. Mol. Biol. Educ, 41: 56-57. DOI: https://doi.org/10.1002/ bmb.20655 · DOI
  20. Forman, J and Damschroder, L, 2007. Qualitative Content Analysis. In: Advances in Bioethics. Elsevier, 39-62. DOI: https://doi.org/10.1016/S1479-3709(07)11003-7 · DOI · Paper page
  21. Gee, JP. 2005. Learning by Design: Good Video Games as Learning Machines. E-Learning and Digital Media, 2: 5-16. DOI: https:// doi.org/10.2304/elea.2005.2.1.5 · DOI
  22. Guest, G and MacQueen, KM. (Eds.), 2008. Handbook for team- based qualitative research. Altamira, Lanham.
  23. Hsieh, H-F and Shannon, SE. 2005. Three Approaches to Qualitative Content Analysis. Qual Health Res, 15: 1277- 1288. DOI: https://doi.org/10.1177/1049732305276687 · DOI · Paper page
  24. Iacovides, I, Jennett, C, Cornish-Trestrail, C and Cox, AL. 2013. Do games attract or sustain engagement in citizen science?: a study of volunteer motivations. In: CHI'13 Extended Abstracts on Human Factors in Computing Systems. ACM, 1101-1106. DOI: https://doi. org/10.1145/2468356.2468553 · DOI
  25. Jackson, C. 2019. Characterizing Novelty as a Motivator in Online Citizen Science (PhD Thesis). Syracuse University.
  26. Jennett, C, Kloetzer, L, Schneider, D, Iacovides, I, Cox, A, Gold, M, Fuchs, B, Eveleigh, A, Mathieu, K, Ajani, Z and Talsi, Y. 2016. Motivations, learning and creativity in online citizen science. JCOM, 15: A05. DOI: https://doi. org/10.22323/2.15030205 · DOI
  27. Kim, S, Robson, C, Zimmerman, T, Pierce, J and Haber, EM. 2011. Creek watch: pairing usefulness and usability for successful citizen science. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Vancouver, BC, Canada: ACM, 2125-2134. DOI: https://doi. org/10.1145/1978942.1979251 · DOI
  28. Korhonen, H and Koivisto, EMI. 2006. Playability heuristics for mobile games. In: Proceedings of the 8th Conference on Human-Computer Interaction with Mobile Devices and Services -MobileHCI '06. Helsinki, Finland: ACM Press, 9. DOI: https://doi.org/10.1145/1152215.1152218 · DOI
  29. Legner, L, Eghtebas, C and Klinker, G. 2019. Persuasive Mobile Game Mechanics For User Retention. In: Extended Abstracts of the Annual Symposium on Computer-Human Interaction in Play Companion Extended Abstracts. Presented at the CHI PLAY '19: The Annual Symposium on Computer-Human Interaction in Play, ACM, Barcelona Spain, 493-500. DOI: https://doi.org/10.1145/3341215.335 6261 · DOI
  30. Lomas, JD, Koedinger, K, Patel, N, Shodhan, S, Poonwala, N and Forlizzi, JL. 2017. Is Difficulty Overrated?: The Effects of Choice, Novelty and Suspense on Intrinsic Motivation in Educational Games. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. Denver, Colorado, USA: ACM, 1028-1039. DOI: https://doi. org/10.1145/3025453.3025638 · DOI · Paper page
  31. Lomas, JD, Patel, K, Forlizzi, JL and Koedinger, KR. 2013. Optimizing challenge in an educational game using large-scale design experiments. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Paris, France: ACM, 89-98. DOI: https://doi. org/10.1145/2470654.2470668 · DOI
  32. MacQueen, KM, McLellan, E, Kay, K and Milstein, B. 1998. Codebook Development for Team-Based Qualitative Analysis. CAM Journal, 10: 31-36. DOI: https://doi.org/10.1177/152582 · DOI · Paper page
  33. 2X980100020301 · DOI · Paper page
  34. Miller, JA and Cooper, S. 2022. Barriers to Expertise in Citizen Science Games. In: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. ACM, p. in press. DOI: https://doi.org/10.1145/3491102.3517541 · DOI
  35. Miller, JA, Horn, B, Guthrie, M, Romano, J, Geva, G, David, C, Sterling, AR and Cooper, S. 2021. How do Players and Developers of Citizen Science Games Conceptualize Skill Chains? In: Proceedings of the Annual Symposium on Computer-Human Interaction in Play. ACM. DOI: https://doi. org/10.1145/3474671 · DOI
  36. Miller, JA, Khatib, F, Hammond, H, Cooper, S and Horowitz, S. 2020. Introducing Foldit Education Mode. Nat Struct Mol Biol, 27, 769-770. DOI: https://doi.org/10.1038/s41594-020-0485-6 · DOI
  37. Newman, G, Wiggins, A, Crall, A, Graham, E, Newman, S and Crowston, K. 2012. The future of citizen science: emerging technologies and shifting paradigms. Frontiers in Ecology and the Environment, 10: 298-304. DOI: https://doi. org/10.1890/110294 · DOI
  38. Olsen, T, Procci, K and Bowers, C. 2011. Serious games usability testing: How to ensure proper usability, playability, and effectiveness. In: International Conference of Design, User Experience, and Usability. Springer, 625-634. DOI: https://doi. org/10.1007/978-3-642-21708-1_70 · DOI
  39. Palacin, V, Gilbert, S, Orchard, S, Eaton, A, Ferrario, MA and Happonen, A. 2020. Drivers of participation in digital citizen science: Case Studies on Järviwiki and safecast. Citizen Science: Theory and Practice, 5. DOI: https://doi. org/10.22323/2.15030205 · DOI
  40. Palacin-Silva, MV, Knutas, A, Ferrario, MA, Porras, J, Ikonen, J and Chea, C. 2018. The Role of Gamification in Participatory Environmental Sensing: A Study In the Wild. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. Montreal, QC, Canada: ACM, 1-13. DOI: https://doi. org/10.1145/3173574.3173795 · DOI
  41. Paleco, C, García Peter, S, Salas Seoane, N, Kaufmann, J and Argyri, P. 2021. Inclusiveness and Diversity in Citizen Science. In: Vohland, K, Land-Zandstra, A, Ceccaroni, L, Lemmens, R, Perelló, J, Ponti, M, Samson, R and Wagenknecht, K. (Eds.), The Science of Citizen Science. Cham: Springer International Publishing, 261-281. DOI: https://doi.org/10.1007/978-3- 030-58278-4_14 · DOI
  42. Ponti, M, Hillman, T, Kullenberg, C and Kasperowski, D. 2018. Getting it Right or Being Top Rank: Games in Citizen Science. CSTP, 3: 1. DOI: https://doi.org/10.5334/cstp.101 · DOI
  43. Prestopnik, N. 2010. Theory, Design and Evaluation -(Don't Just) Pick any Two. THCI, 2: 167-177. DOI: https://doi. org/10.17705/1thci.00021 · DOI
  44. Prestopnik, NR and Tang, J. 2015. Points, stories, worlds, and diegesis: Comparing player experiences in two citizen science games. Computers in Human Behavior, 52: 492-506. DOI: https://doi.org/10.1016/j.chb.2015.05.051 · DOI
  45. Rienzo, A and Cubillos, C. 2020. Playability and player experience in digital games for elderly: A systematic literature review. Sensors, 20: 3958. DOI: https://doi.org/10.3390/s20143958 · DOI
  46. Rotman, D, Hammock, J, Jenny, P, Hansen, D, Boston, C, Bowser, A and He, Y. 2014. Motivations Affecting Initial and Long-Term Participation in Citizen Science Projects in Three Countries. In: IConference 2014 Proceedings. iSchools. DOI: https://doi.org/10.9776/14054 · DOI
  47. Rüfenacht, S, Woods, T, Agnello, G, Gold, M, Hummer, P, Land-Zandstra, A and Sieber, A. 2021. Communication and Dissemination in Citizen Science. In: Vohland, K, Land- Zandstra, A, Ceccaroni, L, Lemmens, R, Perelló, J, Ponti, M, Samson, R and Wagenknecht, K. (Eds.), The Science of Citizen Science. Cham: Springer International Publishing, 475-494. DOI: https://doi.org/10.1007/978-3-030-58278-4_24 · DOI
  48. Schreier, M. 2012. Qualitative content analysis in practice. Los Angeles: SAGE.
  49. Shannon, A, Boyce, A, Gadwal, C and Barnes, DT. 2013. Effective Practices in Game Tutorial Systems. In: Proceedings of the 8th International Conference on the Foundations of Digital Games. ACM, 8.
  50. Skarlatidou, A, Hamilton, A, Vitos, M and Haklay, M. 2019. What do volunteers want from citizen science technologies? A systematic literature review and best practice guidelines. JCOM, 18: A02. DOI: https://doi.org/10.22323/2.18010202 · DOI
  51. Spiers, H, Swanson, A, Fortson, L, Simmons, B, Trouille, L, Blickhan, S and Lintott, C. 2019. Everyone counts? Design considerations in online citizen science. JCOM, 18: A04. DOI: https://doi.org/10.22323/2.18010204 · DOI
  52. Miller, JA, Gandhi, K, Gander, A and Cooper, S. 2022. A Survey of Citizen Science Gaming Experiences. Citizen Science: Theory and Practice, 7(1): 34, pp. 1-12. DOI: https://doi.org/10.5334/cstp.500 Submitted: 19 January 2022 Accepted: 04 October 2022 Published: 25 October 2022
  53. which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. See http://creativecommons.org/licenses/by/4.0/. Citizen Science: Theory and Practice is a peer-reviewed open access journal published by Ubiquity Press. massive-scale citizen science to improve large-scale image classification. Nat Biotechnol, 36: 820-828. DOI: https://doi. org/10.1038/nbt.4225 · DOI
  54. Tasnim, RA and Eishita, FZ. 2021. Analyzing the Distinctive Impact of Personality Traits on Serious Gameplay Experience. In: 2021 IEEE 9th International Conference on Serious Games and Applications for Health (SeGAH). IEEE, 1-8. DOI: https:// doi.org/10.1109/SEGAH52098.2021.9551856 · DOI
  55. Tinati, R, Luczak-Rösch, M, Simperl, E and Hall, W. 2016. Because science is awesome: studying participation in a citizen science game. In: Proceedings of the 8th ACM Conference on Web Science. Hannover, Germany: ACM, 45-54. DOI: https://doi.org/10.1145/2908131.2908151 · DOI
  56. Tuite, K. 2014. GWAPs: Games with a Problem. In: Proceedings of the 9th International Conference on the Foundations of Digital Games. Ft. Lauderdale, FL, USA: ACM, 7.
  57. Waldispühl, J, Szantner, A, Knight, R, Caisse, S and Pitchford, R. 2020. Leveling up citizen science. Nat Biotechnol, 38, 1124- 1126. DOI: https://doi.org/10.1038/s41587-020-0694-x · DOI
  58. Wiemeyer, J, Nacke, L, Moser, C and 'Floyd' Mueller, F, 2016. Player Experience. In: Dörner, R, Göbel, S, Effelsberg, W and Wiemeyer, J. (Eds.), Serious Games. Cham: Springer International Publishing, 243-271. DOI: https://doi. org/10.1007/978-3-319-40612-1_9 · DOI
  59. Wiggins, A and Crowston, K. 2015. Surveying the citizen science landscape. FM 75. DOI: https://doi.org/10.5210/fm.v20i1.5520 · DOI

Cited by

Sorted by citations

Related

Authors
Miller, J. A., Gandhi, K., Gander, A., & Cooper, S.
Affiliations
  • Northeastern University (US)
  • University West (SE)
Corresponding author
Miller, Josh Aaron
ORCID
Venue
Citizen Science: Theory and Practice
Publisher
Ubiquity Press, Ltd.
Published
2022-10-25
Volume
7
Issue
1
Pages
34
ISSN
2057-4991 (ISSN-L)
DOI
10.5334/cstp.500
OpenAlex
W4307368389
Cited by
14
References
0
Citations by year
2026: 1 · 2025: 3 · 2024: 2 · 2023: 8
FWCI
4.96
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • Citizen Science Theory and Practice (publishedVersion, cc-by)
  • KTH Publication Database DiVA (KTH Royal Institute of Technology) (submittedVersion, other-oa)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • Citizen Science: Theory and Practice (Citizen Science Association) (submittedVersion, cc-by)
  • OSF Preprints (OSF Preprints) (submittedVersion)
Topics
  • Social Sciences › Psychology › Developmental and Educational Psychology › Educational Games and Gamification
  • Social Sciences › Sociology and Political Science › Digital Games and Media
  • Physical Sciences › Computer Science › Computer Science Applications › Mobile Crowdsensing and Crowdsourcing
Keywords
gamification, player motivation, game design, science communication, instructional design, user interface design, task quality, recruitment and retention

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review