Competing or aiming to be average?: normification as a means of engaging digital volunteers
Open accessAbstract
Engagement, motivation and active contribution by digital volunteers are key requirements for crowdsourcing and citizen science projects. Many systems use competitive elements, for example point scoring and leaderboards, to achieve these ends. However, while competition may motivate some people, it can have a neutral or demotivating effect on others. In this paper we explore theories of personal and social norms and investigate normification as an alternative approach to engagement, to be used alongside or instead of competitive strategies. We provide a systematic review of existing crowdsourcing and citizen science literature and categorise the ways that theories of norms have been incorporated to date. We then present qualitative interview data from a pro-environmental crowdsourcing study, Close the Door, which reveals normalising attitudes in certain participants. We assess how this links with competitive behaviour and participant performance. Based on our findings and analysis of norm theories, we consider the implications for designers wishing to use normification as an engagement strategy in crowdsourcing and citizen science systems.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Despite concerns that some commonly used game mechanics might reduce participant motivation, we saw no evidence that this was the case.
“Furthermore, despite concerns that some commonly used game mechanics might reduce participant motivation [72] (such as by having a visibly low score [73]), we saw no evidence that this was the case.”
From Gamification of Cognitive Assessment and Cognitive Training: A Systematic Review of Applications and Efficacy (2016) · cites this paper as [73]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Cialdini, R. B., Reno, R. R., & Kallgren, C. A. (1990) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.58.6.1015 · Cited by 5,965 · see citing works
The Constructive, Destructive, and Reconstructive Power of Social Norms
Schultz, P. W., Nolan, J. M., Cialdini, R. B., Goldstein, N. J., & Griskevicius, V. (2007) · Psychological Science · doi:10.1111/j.1467-9280.2007.01917.x · Cited by 3,643 · see citing works
Gamification. using game-design elements in non-gaming contexts
Deterding, S., Sicart, M., Nacke, L., O'Hara, K., & Dixon, D. (2011) · CHI '11 Extended Abstracts on Human Factors in Computing Systems · doi:10.1145/1979742.1979575 · Cited by 2,413 · see citing works
Gamification by Design: Implementing Game Mechanics in Web and Mobile Apps
Zichermann, G., & Cunningham, C. J. L. (2011) · Cited by 2,390 · see citing works
Labeling images with a computer game
von Ahn, L., & Dabbish, L. (2004) · ACM SIGCHI Conference on Human Factors in Computing Systems (CHI) · doi:10.1145/985692.985733 · Cited by 2,230 · see citing works
Normative social influence is underdetected
Nolan, J. M., Schultz, P. W., Cialdini, R. B., Goldstein, N. J., & Griskevicius, V. (2008) · Personality and Social Psychology Bulletin · doi:10.1177/0146167208316691 · Cited by 1,772 · see citing works
Lintott, C. J., Schawinski, K., Slosar, A., Land, K., Bamford, S., Thomas, D., Raddick, M. J., Nichol, R. C., Szalay, A., Andreescu, D., et al. (2008) · Monthly Notices of the Royal Astronomical Society · doi:10.1111/j.1365-2966.2008.13689.x · Cited by 1,488 · see citing works
Crowdsourcing systems on the World-Wide Web
Doan, A., Ramakrishnan, R., & Halevy, A. Y. (2011) · Communications of the ACM · doi:10.1145/1924421.1924442 · Cited by 1,338 · see citing works
Harwood, J., & Ortiz, M. (2006) · Human Communication Research · doi:10.1111/j.1468-2958.2006.00280.x · Cited by 1,291 · see citing works
Huotari, K., & Hamari, J. (2012) · International Academic Mindtrek Conference (Mindtrek) · doi:10.1145/2393132.2393137 · Cited by 1,155 · see citing works
Consolvo, S., McDonald, D. W., Toscos, T., Chen, M. Y., Froehlich, J. E., Harrison, B. L., Klasnja, P., LaMarca, A., LeGrand, L., Libby, R., et al. (2008) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1357054.1357335 · Cited by 1,133 · see citing works
The transsituational influence of social norms.
Reno, R. R., Cialdini, R. B., & Kallgren, C. A. (1993) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.64.1.104 · Cited by 976 · see citing works
How Behaviors are Influenced by Perceived Norms: A Test of the Theory of Normative Social Behavior
Rimal, R. N., & Real, K. (2005) · Communication Research · doi:10.1177/0093650205275385 · Cited by 927 · see citing works
Breaching or Building Social Boundaries?
Postmes, T., Spears, R., & Lea, M. (1998) · Communication Research · doi:10.1177/009365098025006006 · Cited by 904 · see citing works
Harland, P., Staats, H., & Wilke, H. (1999) · Journal of Applied Social Psychology · doi:10.1111/j.1559-1816.1999.tb00123.x · Cited by 884 · see citing works
The design of eco-feedback technology
Froehlich, J., Findlater, L., & Landay, J. (2010) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1753326.1753629 · Cited by 813 · see citing works
Explaining prosocial intentions: Testing causal relationships in the norm activation model
Steg, L., & de Groot, J. I. M. (2009) · British Journal of Social Psychology · doi:10.1348/014466609x477745 · Cited by 597 · see citing works
Algorithm discovery by protein folding game players
Khatib, F., Cooper, S., Tyka, M. D., Xu, K., Makedon, I., Popović, Z., Baker, D. A., & Players, F. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1115898108 · Cited by 583 · see citing works
(2012) · Cited by 563 · see citing works
The formation of group norms in computer-mediated communication
Postmes, T., Spears, R., & Lea, M. (2000) · Human Communication Research · doi:10.1093/hcr/26.3.341 · Cited by 557 · see citing works
Dynamic changes in motivation in collaborative citizen-science projects
Rotman, D., Preece, J., Hammock, J., Procita, K., Hansen, D. L., Parr, C., Lewis, D., & Jacobs, D. (2012) · Conference on Computer Supported Cooperative Work · doi:10.1145/2145204.2145238 · Cited by 547 · see citing works
Starbird, K., & Palen, L. (2011) · ACM SIGCHI Conference on Human Factors in Computing Systems (CHI) · doi:10.1145/1978942.1979102 · Cited by 545 · see citing works
Jordan, R. R. (1997) · doi:10.1017/cbo9780511733062.023 · Cited by 522 · see citing works
Social Comparisons and Contributions to Online Communities: A Field Experiment on MovieLens
Chen, Y., Harper, F. M., Konstan, J. A., & Li, S. X. (2010) · American Economic Review · doi:10.1257/aer.100.4.1358 · Cited by 487 · see citing works
Heuristics for designing enjoyable user interfaces
Malone, T. W. (1982) · ACM Conference on Human Factors in Computing Systems (CHI) · doi:10.1145/800049.801756 · Cited by 453 · see citing works
Using normative social influence to promote conservation among hotel guests
Schultz, W. P., Khazian, A. M., & Zaleski, A. C. (2008) · Social Influence · doi:10.1080/15534510701755614 · Cited by 407 · see citing works
Mind the theoretical gap: interpreting, using, and developing behavioral theory in HCI research
Hekler, E. B., Klasnja, P., Froehlich, J. E., & Buman, M. P. (2013) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2470654.2466452 · Cited by 328 · see citing works
Göckeritz, S., Schultz, P. W., Rendon, T., Cialdini, R. B., Goldstein, N. J., & Griskevicius, V. (2009) · European Journal of Social Psychology · doi:10.1002/ejsp.643 · Cited by 320 · see citing works
He, H. A., Greenberg, S., & Huang, E. M. (2010) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1753326.1753464 · Cited by 313 · see citing works
Heuristics for designing enjoyable user interfaces: lessons from computer games
Malone, T. W. (1984) · Ablex Publishing Corp. eBooks · Cited by 262 · see citing works
Crowdsourcing in the cultural heritage domain
Oomen, J., & Aroyo, L. M. (2011) · International Conference on Communities and Technologies (C&T) · doi:10.1145/2103354.2103373 · Cited by 257 · see citing works
Designing incentives for inexpert human raters
Shaw, A., Horton, J. J., & Chen, D. L. (2011) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/1958824.1958865 · Cited by 239 · see citing works
Stephenson, W. (1981) · Public Relations Review · doi:10.1016/s0363-8111(81)80017-3 · Cited by 221 · see citing works
Quality through flow and immersion
Eickhoff, C., Harris, C. G., de Vries, A. P., & Srinivasan, P. (2012) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/2348283.2348400 · Cited by 190 · see citing works
Kim, S., Robson, C., Zimmerman, T., Pierce, J. S., & Haber, E. M. (2011) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1978942.1979251 · Cited by 185 · see citing works
Leveraging Social Networks To Motivate Individuals to Reduce their Ecological Footprints
Mankoff, J. C., Matthews, D. H., Fussell, S. R., & Johnson, M. P. (2007) · Hawaii International Conference on System Sciences (HICSS) · doi:10.1109/hicss.2007.325 · Cited by 180 · see citing works
Thieme, A., Comber, R., Miebach, J., Weeden, J., Kraemer, N., Lawson, S. W., & Olivier, P. L. (2012) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2207676.2208394 · Cited by 175 · see citing works
Using crowdsourcing to support pro-environmental community activism
Massung, E., Coyle, D., Cater, K. F., Jay, M., & Preist, C. (2013) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2470654.2470708 · Cited by 168 · see citing works
Self-citationNov, O., Arazy, O., & Anderson, D. (2011) · iConference · doi:10.1145/1940761.1940771 · Cited by 154 · see citing works
Petkov, P., Köbler, F., Foth, M., & Krcmar, H. (2011) · International Conference on Communities and Technologies (C&T) · doi:10.1145/2103354.2103358 · Cited by 150 · see citing works
Foster, D., Lawson, S. W., Blythe, M., & Cairns, P. A. (2010) · Nordic Conference on Human-Computer Interaction · doi:10.1145/1868914.1868938 · Cited by 128 · see citing works
Social desirability bias and self-reports of motivation
Antin, J., & Shaw, A. (2012) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2207676.2208699 · Cited by 125 · see citing works
Kim, S., Mankoff, J. C., & Paulos, E. (2013) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2441776.2441940 · Cited by 110 · see citing works
Game design principles in everyday fitness applications
Campbell, T., Ngo, B., & Fogarty, J. A. (2008) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/1460563.1460603 · Cited by 110 · see citing works
Does competition affect giving?
Duffy, J. F., & Kornienko, T. (2010) · Journal of Economic Behavior & Organization · doi:10.1016/j.jebo.2010.02.001 · Cited by 109 · see citing works
Normative influences on thoughtful online participation
Sukumaran, A., Vezich, S., McHugh, M., & Nass, C. (2011) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1978942.1979450 · Cited by 105 · see citing works
Using mobile phones to engage citizen scientists in research
Graham, E. A., Henderson, S., & Schloss, A. L. (2011) · Eos · doi:10.1029/2011eo380002 · Cited by 102 · see citing works
Quality control mechanisms for crowdsourcing
Hansen, D. L., Schone, P., Corey, D. L., Reid, M., & Gehring, J. (2013) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2441776.2441848 · Cited by 52 · see citing works
Uncovering practices of making energy consumption accountable
Schwartz, T., Stevens, G., Ramirez, L., & Wulf, V. (2013) · ACM Transactions on Computer-Human Interaction · doi:10.1145/2463579.2463583 · Cited by 51 · see citing works
Lea, M., Paul, R., & Postmes, T. (2002) · British Journal of Educational Technology · doi:10.1111/1467-8535.00238 · Cited by 48 · see citing works
Free as in puppies: compensating for ict constraints in citizen science
Wiggins, A. (2013) · Proceedings of the 2013 conference on Computer supported cooperative work · doi:10.1145/2441776.2441942 · Cited by 44 · see citing works
Five design challenges for human computation
Reeves, S., & Sherwood, S. (2010) · doi:10.1145/1868914.1868959 · Cited by 12 · see citing works
Theory vs. design-driven approaches for behavior change research
Arriaga, R. I., Miller, A., Mynatt, E. D., Pagliari, C., & Poole, E. S. (2013) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/2468356.2468801 · Cited by 6 · see citing works
Gathering and Presenting Social Feedback to Change Domestic Electricity Consumption
Studley, M., Chambers, S., Rettie, R., & Burchell, K. (2011) · UWE Research Repository (UWE Bristol) · Cited by 6 · see citing works
Massung, E., & Preist, C. (2013) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/2468356.2468615 · Cited by 2 · see citing works
Self-citation(1843) · The London Edinburgh and Dublin Philosophical Magazine and Journal of Science · doi:10.1080/14786444308644705 · Cited by 2 · see citing works
Flagged first, then by citations
1 of the 57 references couldn’t be matched to a known paper.
Research works printed but not linked: 4
- Basarir, M. & Overend, M. Interim Report on the Energy Appraisal of Retail Units: Assessing the effect of open doors on energy consumption and thermal comfort. 2010, University of Cambridge. CUED/D-STRUCT/TR232.
- Kim, S., Mankoff, J., & Paulos, E. Sensr: evaluating a flexible framework for authoring mobile data-collection tools for citizen science. ACM CSCW 2013, 1453-1462.
- Kim, S., Robson, C., Zimmerman, T., Pierce, J., & Haber, E.M. Creek watch: pairing usefulness and usability for successful citizen science. ACM CHI 2011, 2125-2134.
- Sebastopol, CA. O'Reilly Media.
Cited by
Sorted by citations
Lumsden, J., Edwards, E. A., Lawrence, N. S., Coyle, D., & Munafò, M. R. (2016) · JMIR Serious Games · doi:10.2196/games.5888 · Cited by 555 · see citing works
“Furthermore, despite concerns that some commonly used game mechanics might reduce participant motivation [72] (such as by having a visibly low score [73]), we saw no evidence that this was the case.”
BackgroundSelf-citationGamified crowdsourcing: Conceptualization, literature review, and future agenda
Morschheuser, B., Hamari, J., Koivisto, J., & Maedche, A. (2017) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2017.04.005 · Cited by 309 · see citing works
“…that extrinsic rewards (such as money) can potentially decrease intrinsic motivation (Deci, 1971; Deci et al., 1999), Massung et al. (2013) and Preist et al. (2014) found in their experiment that gamification in combination with financial rewards can in fact increase participation when…”
BackgroundInfluentialDoherty, K., & Doherty, G. J. (2018) · ACM Computing Surveys · doi:10.1145/3234149 · Cited by 272 · see citing works
“, for example, propose ‘normification’ as an alternative to gamification, focusing on “ideas of collective engagement,” encouraging “normalising behaviour where community members aim to emulate others in the community and behave similarly to them” (Preist et al. 2014).”
BackgroundMorschheuser, B., Hamari, J., & Maedche, A. (2018) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2018.10.001 · Cited by 222 · see citing works
Gamification in Crowdsourcing: A Review
Morschheuser, B., Hamari, J., & Koivisto, J. (2016) · Hawaii International Conference on System Sciences (HICSS) · doi:10.1109/hicss.2016.543 · Cited by 219 · see citing works
“Psychological [2], [5], [9], [10], [28], [35], [40], [42], [44], [46], [49] 12”
BackgroundResultInfluentialCitizen Science: New Research Challenges for Human–Computer Interaction
Preece, J. (2016) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2016.1194153 · Cited by 135 · see citing works
“…add an element of challenge that may be liked by some contributors, though others may find it daunting (Massung, Coyle, Kater, Jay, & Preist, 2013; Preist, Massung, & Coyle, 2014). iNaturalist has clear policies about how participants should behave, particularly with reference to commenting on…”
BackgroundJahanbakhsh, F., Zhang, A. X., Berinsky, A. J., Pennycook, G., Rand, D. G., & Karger, D. R. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3449092 · Cited by 112 · see citing works
“One concern around the generalizability of our results to online platforms is the possible existence of Hawthorne effect, under which participants change their behavior due to an awareness of being studied [34, 41, 58].”
BackgroundMainstreaming the Blackpill: Understanding the Incel Community on TikTok
Solea, A. I., & Sugiura, L. (2023) · European Journal on Criminal Policy and Research · doi:10.1007/s10610-023-09559-5 · Cited by 105 · see citing works
Naqshbandi, K. Z., Liu, C., Taylor, S., Lim, R., Ahmadpour, N., & Calvo, R. A. (2020) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2020.1746061 · Cited by 85 · see citing works
“Finally, we note that although there is some work on engagement strategies for improving the social engagement aspect for online volunteers (Preist et al., 2014), opportunities to foster genuine interpersonal connections remain underexplored.”
BackgroundIndividualizing gamified systems: The role of trait competitiveness and leaderboard design
Höllig, C. E., Tumasjan, A., & Welpe, I. M. (2018) · Journal of Business Research · doi:10.1016/j.jbusres.2018.10.046 · Cited by 74 · see citing works
Playing (with) Democracy: A Review of Gamified Participation Approaches
Thiel, S.-K., Reisinger, M. R., Röderer, K., & Fröhlich, P. (2016) · JeDEM - eJournal of eDemocracy and Open Government · doi:10.29379/jedem.v8i3.440 · Cited by 63 · see citing works
“…engagement with a service, quality of the output as well as enjoyment of tasks (Itoko et al., 2014; Kawaijiri et al., 2014; Eickhoff et al., 2012) contrasts with negative results of gamifications strategies, especially pertaining to the use of rankings and leaderboards (cf. Preist et al., 2014).”
BackgroundHyrynsalmi, S., Smed, J., & Kimppa, K. K. (2017) · Cited by 61 · see citing works
An interdisciplinary perspective on gamification: Mechanics, psychological mediators and outcomes
Helmefalk, M. (2019) · International Journal of Serious Games · doi:10.17083/ijsg.v6i1.262 · Cited by 51 · see citing works
“Other studies show instead how social motivation and a sense of community influence data collection in the context of crowdsourcing [61, 65].”
BackgroundIncreasing citizen science contribution using a virtual peer
Laut, J., Cappa, F., Nov, O., & Porfiri, M. (2016) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23685 · Cited by 47 · see citing works
“Alternative approaches, such as normification (Preist, Massung, & Coyle, 2014), could be proposed to provide participants with a sense of making a fair contribution by reaching to individuals not motivated by gaming elements.”
BackgroundGamification as Motivation to Engage in Location-Based Public Participation?
Thiel, S.-K., & Fröhlich, P. (2016) · Lecture notes in geoinformation and cartography · doi:10.1007/978-3-319-47289-8_20 · Cited by 48 · see citing works
What Perception Do Students Have About the Gamification Elements?
Iruela, M. G., & Hijón‐Neira, R. (2020) · IEEE Access · doi:10.1109/access.2020.3011222 · Cited by 47 · see citing works
“If we focus on the leaderboards, we find [44] that they show that people perceive the leaderboards differently, which ultimately results in different, and potentially even negative, behavioral results.”
BackgroundSocial Status and Badge Design
Immorlica, N., Stoddard, G., & Syrgkanis, V. (2015) · International World Wide Web Conference (WWW) · doi:10.1145/2736277.2741664 · Cited by 43 · see citing works
“Observational and experimental studies have consistently shown that users alter their behavior in the presence of a badge or leaderboard, or when given information regarding the contributions of other users [26, 28, 17, 12].”
BackgroundKendall, L., Chaudhuri, B., & Bhalla, A. (2020) · Information Systems Frontiers · doi:10.1007/s10796-020-10015-6 · Cited by 37 · see citing works
Ikeda, K., & Bernstein, M. S. (2016) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858327 · Cited by 34 · see citing works
“Competition-based payment works well for skilled workers, whereas norm-based payment motivates novice workers who dislike competition [28].”
BackgroundImproving Productivity in Citizen Science through Controlled Intervention
Segal, A., Gal, K., Simpson, R. J., Homsy, V. V., Hartswood, M., Page, K., & Jirotka, M. D. (2015) · International World Wide Web Conference (WWW) · doi:10.1145/2740908.2743051 · Cited by 33 · see citing works
“A few citizen science projects exhibit badges and leader boards functionalities, although there is evidence that competitive game elements may be counterproductive and work to de-motivate casual contributors and reduce the quality of the work [11, 12, 13].”
BackgroundYang, S., Chen, P., Shih, P. C., Bardzell, J., & Bardzell, S. (2017) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3134750 · Cited by 29 · see citing works
“For example, research has focused on the phenomena of digital volunteers who gather to coordinate, share, and distributed pertinent information in social media during crisis [33,36,49].”
BackgroundMasinde, M., Mwagha, S. M., & Tadesse, T. (2018) · Geosciences · doi:10.3390/geosciences8040135 · Cited by 27 · see citing works
“Using a series of structured interviews [55,56] primary data was collected from five communities— see a sample extract of the questionnaire in the Appendix A.”
BackgroundEducational Functions and Design of Badge Systems: A Conceptual Literature Review
Facey-Shaw, L., Specht, M. M., van Rosmalen, P., Börner, D., & Bartley-Bryan, J. (2017) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2017.2773508 · Cited by 25 · see citing works
“Information on badges 4 Status screen provided information on earning badges [35], [42]; badge ladder presentation of earned badges [71]; badges available the moment received [49]”
BackgroundInfluentialThe Role of Gamification in Participatory Environmental Sensing
Palacin-Silva, M. V., Knutas, A., Ferrario, M. A., Porras, J., Ikonen, J., & Chea, C. (2018) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173795 · Cited by 24 · see citing works
“H.5.m. Information Interfaces and Presentation (e.g. HCI): Miscellaneous”
BackgroundInfluentialSocial signals as design interventions for enhancing citizen science contributions
Diner, D., Nakayama, S., Nov, O., & Porfiri, M. (2017) · Information Communication & Society · doi:10.1080/1369118x.2017.1299779 · Cited by 23 · see citing works
“…that information about the high contribution of the peer or group can elicit increased volunteer contributions while mitigating the risks of alienat-ing volunteers who are averse to competition (Christy & Fox, 2014; Preist et al., 2014) as there is no comparison with a too dissimilar other.”
BackgroundExamining the Effects of Feedback-giving as a Gamification Mechanic in Crowd Rating Systems
Tang, J., Zhao, Y. C., Wang, T., & Zeng, Z. (2021) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2021.1917866 · Cited by 23 · see citing works
“…empirical evidence reported the varied impact of leaderboards on users’ motivation; for example, some studies found a positive impact of leader-boards (Legaki et al., 2020), but others suggested leaderboards were less effective in promoting one’s participation or performance (Preist et al., 2014).”
BackgroundIs Virtual Citizen Science A Game?
Simperl, E. P. B., Reeves, N., Phethean, C., Lynes, T., & Tinati, R. (2018) · ACM Transactions on Social Computing · doi:10.1145/3209960 · Cited by 24 · see citing works
“…themselves through leaderboard and point features, even if not otherwise engaged through competition with other players and these features may also contribute to ’normalising behaviours’, where players feel a (self-driven) social pressure to do their fair share of the work [Preist et al. 2014].”
BackgroundExploring Novice Approaches to Smartphone-based Thermographic Energy Auditing
Mauriello, M. L., Saha, M., Brown, E. B., & Froehlich, J. E. (2017) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025471 · Cited by 19 · see citing works
“Smartphones, our focus, are often used by the public to perform volunteer audits on a range of topics, including noise pollution [59], air conditioning use [57], potholes [48].”
BackgroundNaqshbandi, K. Z., Jeon, Y., & Ahmadpour, N. (2022) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2022.2109246 · Cited by 20 · see citing works
“Preist et al. found that strategies that focus on generating meaning through role identity in a volunteer community encourage long-term engagement of volunteers (Preist et al., 2014).”
BackgroundTo Sign Up, or not to Sign Up?
Jay, C., Dunne, R., Gelsthorpe, D., & Vigo, M. (2016) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858319 · Cited by 18 · see citing works
“While gamification in citizen science has been demonstrated to be effective and motivating [1, 14, 8], concerns have also been raised that encouraging competitive behaviour may reduce altruism [7], and that game interfaces may have a negative effect on intrinsic motivation, and alienate traditional citizen science vol…”
BackgroundResultDesign Considerations for Social Fitness Applications
Chen, Y., Chen, Y., Randriambelonoro, M., Geissbühler, A., & Pu, P. (2017) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2998181.2998350 · Cited by 16 · see citing works
The Gamification of Political Participation
Loh, W. (2019) · Moral Philosophy and Politics · doi:10.1515/mopp-2018-0037 · Cited by 15 · see citing works
Morgan, J. T., & Filippova, A. (2018) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3274321 · Cited by 15 · see citing works
(Un)theorizing citizen science: Investigation of theories applied to citizen science studies
Hajibayova, L. (2019) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.24308 · Cited by 15 · see citing works
“Moreover, they found that injunctive social norms, which invoke the need for social approval, are likely to lead to beneficial social conduct across the greatest number of situations and populations (Preist et al., 2014).”
BackgroundCompetence, collaboration, and time management
Zanatta, A. L., Machado, L. S., & Steinmacher, I. (2018) · doi:10.1145/3195863.3195869 · Cited by 13 · see citing works
“Additionally, Preist et al. [36] suggest that the payment based on competition “works well” for qualified workers.”
BackgroundDoGood: examining gamification, civic engagement, and collective intelligence
Rehm, S., Foth, M., & Mitchell, P. R. (2017) · AI & Society · doi:10.1007/s00146-017-0711-x · Cited by 13 · see citing works
Exploring the invisible college of citizen science: questions, methods and contributions
Hajibayova, L., Coladangelo, L. P., & Soyka, H. A. (2021) · Scientometrics · doi:10.1007/s11192-021-04050-6 · Cited by 12 · see citing works
Self-evaluation of knowledge sharing through the lens of social comparison theory
Heo, M., Toomey, N., & Song, J. S. (2019) · VINE Journal of Information and Knowledge Management Systems · doi:10.1108/vjikms-04-2019-0056 · Cited by 11 · see citing works
Berglund, A., Klompstra, L. V., Orädd, H., Fallström, J., Strömberg, A., Jaarsma, T., & Berglund, E. (2024) · JMIR Serious Games · doi:10.2196/50066 · Cited by 10 · see citing works
“Since low-performing players can be demotivated by a leaderboard when they fall behind and feel that they cannot catch up with others [79], our leaderboard is designed so that players have the same opportunity to be included in it every day, regardless of their performance in the days prior to the…”
GamiDOC: The Importance of Designing Gamification in a Proper Way
Bassanelli, S., Bucchiarone, A., & Gini, F. (2024) · IEEE Transactions on Games · doi:10.1109/tg.2024.3364061 · Cited by 9 · see citing works
Relative Feedback Increases Disparities in Effort and Performance in Crowdsourcing Contests
Tsvetkova, M., Müller, S., Vuculescu, O., Ham, H., & Sergeev, R. A. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555649 · Cited by 8 · see citing works
Current Developments in Big Data and Sustainability Sciences in Mobile Citizen Science Applications
Panchariya, N. S., DeStefano, A. J., Nimbagal, V., Ragupathy, R., Yavuz, S., Herbert, K. G., Hill, E., & Fails, J. A. (2015) · IEEE International Conference on Big Data Computing Service and Applications (BigDataService) · doi:10.1109/bigdataservice.2015.64 · Cited by 8 · see citing works
“community as a method of eliciting participation [24].”
BackgroundMethodHumayun, H., Malik, M. N., & Ghazali, M. (2024) · International Journal of Crowd Science · doi:10.26599/ijcs.2023.9100010 · Cited by 6 · see citing works
IS LEARNING ENGLISH BORING WITH GAMIFICATION?
Armanda, M. L., & Indriani, L. (2022) · ELTR Journal · doi:10.37147/eltr.v6i1.129 · Cited by 6 · see citing works
“Besides, in the leaderboard element, users would view the leaderboard differently, resulting in different, and even potentially adverse, behavioural outcomes (Preist et al., 2014).”
BackgroundHow Do One's Peers on a Leaderboard Affect Oneself?
Leung, W. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300397 · Cited by 6 · see citing works
“[30] interviewed 18 participants who had taken part in a study testing how gamification and financial incentives affected their use of an app designed to encourage shopkeepers to close their doors to save energy during the cold British winter.”
BackgroundRönneberg, M., & Kettunen, P. (2023) · International Journal of Digital Earth · doi:10.1080/17538947.2023.2279673 · Cited by 5 · see citing works
“However, adding more complex a ff ordances can negatively a ff ect barrier of entry and even demotivate citizens to contribute (Preist, Massung, and Coyle 2014).”
BackgroundGraphical Notification to Maintain Good Posture during Visual Display Terminal Work
Murata, K., & Shibuya, Y. (2016) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2016.10.551 · Cited by 5 · see citing works
Is there a European strand of sustainable HCI?
Pargman, D. S., Eriksson, E., Katzeff, C., Preist, C., Håkansson, M., & Knowles, B. (2014) · Nordic Conference on Human-Computer Interaction · doi:10.1145/2639189.2654833 · Cited by 4 · see citing works
“• Chris Preist has published on the role of ICT in supporting environmental community groups, and community engagement with sustainability challenges [13, 16].”
BackgroundSelf-citationSocial norms influence student journalists' perception of wearable technologies
Ware, J. (2014) · ACM International Conference on Design of Communication (SIGDOC) · doi:10.1145/2666216.2666238 · Cited by 4 · see citing works
“Technology use motivating factors were explored through a series of survey questions related to technological proficiency, prior technology use, social norms [4, 5, 9] and a series of open-ended Permission to make digital or hard copies of all or part of this work for”
BackgroundSchiavo, G., Leonardi, C., Pasolli, M., Sarti, S., & Zancanaro, M. (2017) · International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI) · doi:10.1145/3098279.3098551 · Cited by 4 · see citing works
“They investigate the impact of different strategies [32] and motivational factors [22] and report that some “pointsification” and “normification” mechanisms can increase performance and engagement.”
BackgroundCharismatic Data and Material Traces: Monitoring Bird-Building Collisions through Citizen Science
Boone, A., DiSalvo, C. F., & Dantec, C. A. L. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757402 · Cited by 3 · see citing works
'Welcome' Changes? Descriptive and Injunctive Norms in a Wikipedia Sub-CommunityPreprint
Morgan, J. T., & Filippova, A. (2018) · doi:10.31235/osf.io/yr6a3 · Cited by 3 · see citing works
Towards A New Form of Volunteering 2.0: A Survey at Apsopad International
Iraqi, A. (2025) · Journal of Lifestyle and SDGs Review · doi:10.47172/2965-730x.sdgsreview.v5.n01.pe02548 · Cited by 2 · see citing works
“This is consistent with the literature that calls for flexible models of volunteering that cater to individuals' changing circumstances and preferences (Preist et al ., 2014).”
Loh, W. (2018) · J.B. Metzler eBooks · doi:10.1007/978-3-476-04569-0_9 · Cited by 2 · see citing works
A systematic review of the current theory base in the crowdsourcing literature
Amrollahi, A., Ghapanchi, A. H., & Talaei‐Khoei, A. (2014) · Victoria University Research Repository (Victoria University) · Cited by 2 · see citing works
Mulchandani, D. (2021) · DalSpace (Dalhousie University) · Cited by 2 · see citing works
The Crowdrouter Framework: Addressing Issues of Software Design in Support of Crowdwork
Barrenechea, M. (2016) · CU Scholar (University of Colorado Boulder) · Cited by 2 · see citing works
Would You Like to Share? The Influence of Gamification in the Virtual Learning Community
Yang, Y., Yu, A., Li, J., Shafi, M., & Zhong, L. (2024) · Journal of the Knowledge Economy · doi:10.1007/s13132-024-02243-x · Cited by 1 · see citing works
“In the research on gamification, we have noted that gamification mainly builds its own environment through different elements, such as reward badges (Preist et al., 2014), role play (Choi et al., 2014), and competitive rankings (Hanus & Fox, 2015), to bring users a pleasant experience (Pitura &…”
Position Paper: Collaborative Gamification Design for Scientific Software
Queiroz, F., & Spitz, R. (2016) · White Rose Research Online (University of Leeds, The University of Sheffield, University of York)
Data preparation in crowdsourcing for pedagogical purposes
Kuhn, T. Z., Holdt, Š. A., Kosem, I., Tiberius, C., Koppel, K., & Zviel-Girshin, R. (2022) · Slovenščina 2.0: empirical, applied and interdisciplinary research · doi:10.4312/slo2.0.2022.2.62-100
Santos, D., Cardoso, E., & Machado, I. (2023) · Informatics · doi:10.3390/informatics10020036
عمار, ح. م. ا. ص. (2023) · Al-Mağallah Al-Dawliyyaẗ Lil Taʿlim Al-Iliktirūnī · doi:10.21608/ijel.2023.300573
Berglund, E., Klompstra, L. V., Orädd, H., Fallström, J., Strömberg, A., Jaarsma, T., & Berglund, A. (2023) · doi:10.2196/preprints.50066
The Effects of Human Computer Interface on Users
Ngusha, N. C. (2024) · International Conference on Science, Engineering and Business for Driving Sustainable Development Goals (SEB4SDG) · doi:10.1109/seb4sdg60871.2024.10630382
- Authors
- Preist, C., Massung, E., & Coyle, D.
- Affiliations
- University of Bristol (GB)
- Venue
- Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing
- Publisher
- ACM
- Published
- 2014-02-15
- Pages
- 1222-1233
- OpenAlex
- W1998235746
- Cited by
- 65
- References
- 57
- Citations by year
- 2025: 2 · 2024: 5 · 2023: 5 · 2022: 4 · 2021: 5 · 2020: 3 · 2019: 5 · 2018: 10 · 2017: 9 · 2016: 11 · 2015: 3 · 2014: 3
- FWCI
- 5.50
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- ACM Conference on Computer Supported Cooperative Work (CSCW) (publishedVersion)
- Explore Bristol Research (submittedVersion, other-oa)
- Bristol Research (University of Bristol) (submittedVersion, other-oa)
- Funders
- University of Bristol
- Engineering and Physical Sciences Research Council (EP/K012509/1)
- Topics
- Physical Sciences › Computer Science › Computer Science Applications › Mobile Crowdsensing and Crowdsourcing
- Physical Sciences › Computer Science › Computer Science Applications › Open Source Software Innovations
- Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
- Keywords
- personal norms, social norms, crowdsourcing, citizen science, competition, leaderboards, point scoring, participant engagement, volunteer motivation, participant performance
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
