The DOI matches its Crossref record; 163 of the 163 references it cites were found in a registry or an index.
Gamified crowdsourcing: Conceptualization, literature review, and future agenda
Open accessAbstract
Abstract Two parallel phenomena are gaining attention in human–computer interaction research: gamification and crowdsourcing. Because crowdsourcing's success depends on a mass of motivated crowdsourcees, crowdsourcing platforms have increasingly been imbued with motivational design features borrowed from games; a practice often called gamification. While the body of literature and knowledge of the phenomenon have begun to accumulate, we still lack a comprehensive and systematic understanding of conceptual foundations, knowledge of how gamification is used in crowdsourcing, and whether it is effective. We first provide a conceptual framework for gamified crowdsourcing systems in order to understand and conceptualize the key aspects of the phenomenon. The paper's main contributions are derived through a systematic literature review that investigates how gamification has been examined in different types of crowdsourcing in a variety of domains. This meticulous mapping, which focuses on all aspects in our framework, enables us to infer what kinds of gamification efforts are effective in different crowdsourcing approaches as well as to point to a number of research gaps and lay out future research directions for gamified crowdsourcing systems. Overall, the results indicate that gamification has been an effective approach for increasing crowdsourcing participation and the quality of the crowdsourced work; however, differences exist between different types of crowdsourcing: the research conducted in the context of crowdsourcing of homogenous tasks has most commonly used simple gamification implementations, such as points and leaderboards, whereas crowdsourcing implementations that seek diverse and creative contributions employ gamification with a richer set of mechanics.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
1 retracted
Exploring the quantum speed limit with computer games
Sørensen, J. J., Pedersen, M. K., Munch, M. K., Haikka, P., Jensen, J. H., Planke, T., Andreasen, M. G., Gajdacz, M., Mølmer, K., Lieberoth, A., et al. (2016) · Nature · doi:10.1038/nature17620 · Cited by 113
RetractedThe theory of planned behavior
Ajzen, I. (1991) · Organizational Behavior and Human Decision Processes · doi:10.1016/0749-5978(91)90020-t · Cited by 90,587
Ryan, R. M., & Deci, E. L. (2000) · American Psychologist · doi:10.1037//0003-066x.55.1.68 · Cited by 54,599
Self-efficacy: Toward a unifying theory of behavioral change
Bandura, A. (1977) · Psychological Review · doi:10.1037/0033-295x.84.2.191 · Cited by 43,925
Flow: the psychology of optimal experience
(1990) · Choice Reviews Online · doi:10.5860/choice.28-0597 · Cited by 12,637
Analyzing the Past to Prepare for the Future: Writing a Literature Review
Webster, J., & Watson, R. T. (2002) · MIS Quarterly · doi:10.2307/4132319 · Cited by 7,449
Deci, E. L., Koestner, R., & Ryan, R. M. (1999) · Psychological Bulletin · doi:10.1037/0033-2909.125.6.627 · Cited by 7,176 · cited 2×
“that extrinsic rewards (such as money) can potentially decrease intrinsic motivation (Deci, 1971; Deci et al., 1999), Massung et al.”
BackgroundDoes Gamification Work? -- A Literature Review of Empirical Studies on Gamification
Hamari, J., Koivisto, J., & Sarsa, H. (2014) · 2014 47th Hawaii International Conference on System Sciences · doi:10.1109/hicss.2014.377 · Cited by 5,201
Self-citationEffects of externally mediated rewards on intrinsic motivation.
Deci, E. L. (1971) · Journal of Personality and Social Psychology · doi:10.1037/h0030644 · Cited by 4,170
What Constitutes a Theoretical Contribution?
Whetten, D. A. (1989) · The Academy of Management Review · doi:10.2307/258554 · Cited by 4,129
The sharing economy: Why people participate in collaborative consumption
Hamari, J., Sjöklint, M., & Ukkonen, A. (2016) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23552 · Cited by 3,439
Self-citationCommunication and Trust in Global Virtual Teams
Jarvenpaa, S. L., & Leidner, D. E. (1999) · Organization Science · doi:10.1287/orsc.10.6.791 · Cited by 2,903
Gamification in theory and action: a survey
Seaborn, K., & Fels, D. I. (2015) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2014.09.006 · Cited by 2,515 · cited 7×
“We first provide an integrated conceptual framework for gamified crowdsourcing systems (Figure 3), based on the extant literature on crowdsourcing (Geiger and Schader, 2014; Prpić et al., 2015) and gamification (Hamari et al., 2014; Seaborn and Fels 2015).”
BackgroundMethodInfluentialEllis, P. D. (2010) · doi:10.1017/cbo9780511761676 · Cited by 2,461
Motivations for Play in Online Games
Yee, N. (2006) · CyberPsychology & Behavior · doi:10.1089/cpb.2006.9.772 · Cited by 2,173
Gamifying learning experiences: Practical implications and outcomes
Domínguez, A., Saenz-de-Navarrete, J., de-Marcos, L., Fernández-Sanz, L., Pagés, C., & Martínez-Herráiz, J.-J. (2013) · Computers & Education · doi:10.1016/j.compedu.2012.12.020 · Cited by 2,007
Crowdsourcing as a Model for Problem Solving: An Introduction and Cases
Brabham, D. C. (2008) · Convergence: The International Journal of Research into New Media Technologies · doi:10.1177/1354856507084420 · Cited by 1,943
Towards an integrated crowdsourcing definition
Estellés-Arolas, E., & González-Ladrón-de-Guevara, F. (2012) · Journal of Information Science · doi:10.1177/0165551512437638 · Cited by 1,858
Hamari, J., Shernoff, D. J., Rowe, E., Coller, B., Asbell-Clarke, J., & Edwards, T. (2016) · Computers in Human Behavior · doi:10.1016/j.chb.2015.07.045 · Cited by 1,713
Self-citationLintott, C. J., Schawinski, K., Slosar, A., Land, K., Bamford, S., Thomas, D., Raddick, M. J., Nichol, R. C., Szalay, A., Andreescu, D., et al. (2008) · Monthly Notices of the Royal Astronomical Society · doi:10.1111/j.1365-2966.2008.13689.x · Cited by 1,488
Predicting protein structures with a multiplayer online game
Cooper, S., Khatib, F., Treuille, A., Barbero, J., Lee, J., Beenen, M., Leaver-Fay, A., Baker, D., Popović, Z., & players, F. (2010) · Nature · doi:10.1038/nature09304 · Cited by 1,368
Crowdsourcing systems on the World-Wide Web
Doan, A., Ramakrishnan, R., & Halevy, A. Y. (2011) · Communications of the ACM · doi:10.1145/1924421.1924442 · Cited by 1,338
Virtual teams: a review of current literature and directions for future research
Powell, A., Piccoli, G., & Ives, B. (2004) · ACM SIGMIS Database: the DATABASE for Advances in Information Systems · doi:10.1145/968464.968467 · Cited by 1,276
Designing games with a purpose
von Ahn, L., & Dabbish, L. (2008) · Communications of the ACM · doi:10.1145/1378704.1378719 · Cited by 1,155
A definition for gamification: anchoring gamification in the service marketing literature
Huotari, K., & Hamari, J. (2017) · Electronic Markets · doi:10.1007/s12525-015-0212-z · Cited by 1,080
Self-citationCommunication and Trust in Global Virtual Teams
Jarvenpaa, S. L., & Leidner, D. E. (2006) · Journal of Computer-Mediated Communication · doi:10.1111/j.1083-6101.1998.tb00080.x · Cited by 942
Demographic differences in perceived benefits from gamification
Koivisto, J., & Hamari, J. (2014) · Computers in Human Behavior · doi:10.1016/j.chb.2014.03.007 · Cited by 841
Self-citationFundamental Components of the Gameplay Experience: Analysing Immersion
Ermi, L., & Mäyrä, F. (2005) · DiGRA Digital Library · doi:10.26503/dl.v2005i1.119 · Cited by 786
Leveraging Crowdsourcing: Activation-Supporting Components for IT-Based Ideas Competition
Leimeister, J. M., Huber, M., Bretschneider, U., & Krcmar, H. (2009) · Journal of Management Information Systems · doi:10.2753/mis0742-1222260108 · Cited by 745
Hamari, J. (2013) · Electronic Commerce Research and Applications · doi:10.1016/j.elerap.2013.01.004 · Cited by 686
Self-citationProceedings of 48th Hawaii International Conference on System Sciences (HICSS)
Pöyry, E., Parvinen, P. M. T., & McFarland, R. (2015) · Cited by 643
Why do people use gamification services?
Hamari, J., & Koivisto, J. (2015) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2015.04.006 · Cited by 642
Self-citationProceedings of the 24th International Conference on World Wide Web - WWW '15
(2015) · doi:10.1145/2736277 · Cited by 631
An empirical study comparing gamification and social networking on e-learning
de-Marcos, L., Domínguez, A., Saenz-de-Navarrete, J., & Pagés, C. (2014) · Computers & Education · doi:10.1016/j.compedu.2014.01.012 · Cited by 626
(2011) · Choice Reviews Online · doi:10.5860/choice.48-5742 · Cited by 626
More than fun and money. Worker motivation in crowdsourcing - a study on mechanical turk
Kaufmann, N., Schulze, T., & Veit, D. J. (2011) · MADOC (University of Mannheim) · Cited by 577
Evaluation on crowdsourcing research: Current status and future direction
Zhao, Y., & Zhu, Q. (2014) · Information Systems Frontiers · doi:10.1007/s10796-012-9350-4 · Cited by 566
MOVING THE CROWD AT THREADLESS: Motivations for participation in a crowdsourcing application
Brabham, D. C. (2010) · Information, Communication & Society · doi:10.1080/13691181003624090 · Cited by 560
Task Design, Motivation, and Participation in Crowdsourcing Contests
Zheng, H., Li, D., & Hou, W. (2011) · International Journal of Electronic Commerce · doi:10.2753/jec1086-4415150402 · Cited by 538
On being ‘Systematic’ in Literature Reviews in IS
Boell, S. K., & Cecez-Kecmanovic, D. (2015) · Journal of Information Technology · doi:10.1057/jit.2014.26 · Cited by 485
Contribution Behavior in Virtual Communities: Cognitive, Emotional, and Social Influences1
Tsai, H.-T., & Bagozzi, R. P. (2014) · MIS Quarterly · doi:10.25300/misq/2014/38.1.07 · Cited by 479
The effect of virtual achievements on student engagement
Denny, P. (2013) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2470654.2470763 · Cited by 478
“Working out for likes”: An empirical study on social influence in exercise gamification
Hamari, J., & Koivisto, J. (2015) · Computers in Human Behavior · doi:10.1016/j.chb.2015.04.018 · Cited by 473
Self-citationTerlutter, R., & Capella, M. L. (2013) · Journal of Advertising · doi:10.1080/00913367.2013.774610 · Cited by 447
The Effects of Cooperation and Competition on Intrinsic Motivation and Performance.
Tauer, J. M., & Harackiewicz, J. M. (2004) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.86.6.849 · Cited by 436
Player Types: A Meta-synthesis
Hamari, J., & Tuunanen, J. (2014) · Transactions of the Digital Games Research Association · doi:10.26503/todigra.v1i2.13 · Cited by 355
Self-citationTechnical opinionMotivational affordances: reasons for ICT design and use
Zhang, P. (2008) · Communications of the ACM · doi:10.1145/1400214.1400244 · Cited by 349
Measuring flow in gamification: Dispositional Flow Scale-2
Hamari, J., & Koivisto, J. (2014) · Computers in Human Behavior · doi:10.1016/j.chb.2014.07.048 · Cited by 337
Self-citationCommunity‐Based Innovation Contests: Where Competition Meets Cooperation
Bullinger, A. C., Neyer, A., Rass, M., & Moeslein, K. M. (2010) · Creativity and Innovation Management · doi:10.1111/j.1467-8691.2010.00565.x · Cited by 336
Jung, J. H., Schneider, C., & Valacich, J. (2010) · Management Science · doi:10.1287/mnsc.1090.1129 · Cited by 332
Why do people buy virtual goods? Attitude toward virtual good purchases versus game enjoyment
Hamari, J. (2015) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2015.01.007 · Cited by 296
Self-citationBrabham, D. C. (2008) · First Monday · doi:10.5210/fm.v13i6.2159 · Cited by 292
Christy, K. R., & Fox, J. (2014) · Computers & Education · doi:10.1016/j.compedu.2014.05.005 · Cited by 276
Yamagishi, J., Usabaev, B., King, S., Watts, O., Dines, J. D., Tian, J., Hu, R., Guan, Y., Oura, K., Tokuda, K., et al. (2009) · Cited by 270
How to work a crowd: Developing crowd capital through crowdsourcing
Prpić, J., Shukla, P. P., Kietzmann, J. H., & McCarthy, I. P. (2015) · Business Horizons · doi:10.1016/j.bushor.2014.09.005 · Cited by 254
Improving biotech education through gamified laboratory simulations
Bonde, M. T., Makransky, G., Wandall, J., Larsen, M. V., Morsing, M., Jarmer, H., & Sommer, M. O. A. (2014) · Nature Biotechnology · doi:10.1038/nbt.2955 · Cited by 243
Leimeister, J. M. (2010) · Business & Information Systems Engineering · doi:10.1007/s12599-010-0114-8 · Cited by 220
Personalized task recommendation in crowdsourcing information systems — Current state of the art
Geiger, D., & Schader, M. (2014) · Decision Support Systems · doi:10.1016/j.dss.2014.05.007 · Cited by 218
Conceptual Foundations of Crowdsourcing: A Review of IS Research
Pedersen, J., Kocsis, D., Tripathi, A., Tarrell, A., Weerakoon, A., Tahmasbi, N., Xiong, J., Deng, W., Oh, O., & de Vreede, G.-J. (2013) · 2013 46th Hawaii International Conference on System Sciences · doi:10.1109/hicss.2013.143 · Cited by 209
How social Q&A sites are changing knowledge sharing in open source software communities
Vasilescu, B., Serebrenik, A., Devanbu, P., & Filkov, V. (2014) · Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing · doi:10.1145/2531602.2531659 · Cited by 200
Gamifying intelligent environments
Liu, Y., Alexandrova, T., & Nakajima, T. (2011) · Proceedings of the 2011 international ACM workshop on Ubiquitous meta user interfaces · doi:10.1145/2072652.2072655 · Cited by 200
Quality through flow and immersion: gamifying crowdsourced relevance assessments
Eickhoff, C., Harris, C. G., de Vries, A. P., & Srinivasan, P. (2012) · Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/2348283.2348400 · Cited by 190
Proceedings of the 7th International Conference on Weblogs and Social Media, ICWSM 2013
Zhong, C., Shah, S., Sundaravadivelan, K., & Sastry, N. (2013) · International Conference on Weblogs and Social Media · Cited by 188
Chris Zhao, Y., & Zhu, Q. (2014) · Online Information Review · doi:10.1108/oir-08-2014-0188 · Cited by 187
Prakken, H., Kok, E. M., Meyer, J. C., & Vreeswijk, G. A. W. (2012) · Adaptive Agents and Multi-Agents Systems · Cited by 186
Levina, N., & Arriaga, M. (2014) · Information Systems Research · doi:10.1287/isre.2014.0535 · Cited by 183
Analyzing the past to prepare for the future
(2002) · MIS Quarterly · doi:10.5555/2017160.2017162 · Cited by 179
Organizational Learning with Crowdsourcing: The Revelatory Case of LEGO
School, U. A. B., Schlagwein, D., Bjorn-Andersen, N., & School, C. B. (2014) · Journal of the Association for Information Systems · doi:10.17705/1jais.00380 · Cited by 173
GREENIFY: A Real-World Action Game for Climate Change Education
Lee, J. J., Ceyhan, P., Jordan-Cooley, W. C., & Sung, W. (2013) · Simulation & Gaming · doi:10.1177/1046878112470539 · Cited by 169
Using crowdsourcing to support pro-environmental community activism
Massung, E., Coyle, D., Cater, K. F., Jay, M., & Preist, C. (2013) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2470654.2470708 · Cited by 166
Quizz: targeted crowdsourcing with a billion (potential) users
Ipeirotis, P. G., & Gabrilovich, E. (2014) · International World Wide Web Conference (WWW) · doi:10.1145/2566486.2567988 · Cited by 165
Using gamification to inspire new citizen science volunteers
Bowser, A., Hansen, D., He, Y., Boston, C., Reid, M., Gunnell, L., & Preece, J. (2013) · Proceedings of the First International Conference on Gameful Design, Research, and Applications · doi:10.1145/2583008.2583011 · Cited by 159
HealthyTogether: exploring social incentives for mobile fitness applications
Chen, Y., & Pu, P. (2014) · Proceedings of the Second International Symposium of Chinese CHI · doi:10.1145/2592235.2592240 · Cited by 159
von Ahn, L. (2009) · Cited by 156
Jones, B. A., Madden, G. J., & Wengreen, H. J. (2014) · Preventive Medicine · doi:10.1016/j.ypmed.2014.04.015 · Cited by 155
Proceedings of the 8th European Conference on Games Based Learning
Gjedde, L. (2014) · Cited by 153
A taxonomy of crowdsourcing based on task complexity
Nakatsu, R. T., Grossman, E. B., & Iacovou, C. L. (2014) · Journal of Information Science · doi:10.1177/0165551514550140 · Cited by 151
The Ludic Drive as Innovation Driver: Introduction to the Gamification of Innovation
Roth, S., Schneckenberg, D., & Tsai, C. (2015) · Creativity and Innovation Management · doi:10.1111/caim.12124 · Cited by 149
Blohm, I., Bretschneider, U., Leimeister, J. M., & Krcmar, H. (2011) · International Journal of Networking and Virtual Organisations · doi:10.1504/ijnvo.2011.042413 · Cited by 138
Internal Crowdsourcing: Conceptual Framework, Structured Review, and Research Agenda
Zuchowski, O., Posegga, O., Schlagwein, D., & Fischbach, K. (2016) · Journal of Information Technology · doi:10.1057/jit.2016.14 · Cited by 137
A Preliminary Taxonomy of Crowdsourcing
Rouse, A. C. (2010) · Cited by 118
Effects of gamification on participation and data quality in a real-world market research domain
Cechanowicz, J., Gutwin, C., Brownell, B., & Goodfellow, L. (2013) · Proceedings of the First International Conference on Gameful Design, Research, and Applications · doi:10.1145/2583008.2583016 · Cited by 103
Steered crowdsensing: incentive design towards quality-oriented place-centric crowdsensing
Kawajiri, R., Shimosaka, M., & Kashima, H. (2014) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/2632048.2636064 · Cited by 103
Points, stories, worlds, and diegesis: Comparing player experiences in two citizen science games
Prestopnik, N. R., & Tang, J. (2015) · Computers in Human Behavior · doi:10.1016/j.chb.2015.05.051 · Cited by 100
Exploring online social behavior in crowdsourcing communities: A relationship management perspective
Shen, X., Lee, M., & Cheung, C. M. K. (2014) · Computers in Human Behavior · doi:10.1016/j.chb.2014.08.006 · Cited by 83
REfine: A gamified platform for participatory requirements engineering
Snijders, R., Dalpiaz, F., Brinkkemper, S., Hosseini, M., Ali, R., & Atilla, Ö. (2015) · doi:10.1109/crowdre.2015.7367581 · Cited by 83
Gamification for Crowdsourcing Marketing Practices: Applications and Benefits in Tourism
Σιγάλα, Μ. (2015) · Springer eBooks · doi:10.1007/978-3-319-18341-1_11 · Cited by 81
Crowdsourcing innovation and product development: Gamification as a motivational driver
Kavaliova, M., Virjee, F., Mæhle, N., & Kleppe, I. A. (2016) · Cogent Business & Management · doi:10.1080/23311975.2015.1128132 · Cited by 77
Prandi, C., Roccetti, M., Salomoni, P., Nisi, V., & Nunes, N. (2016) · Multimedia Tools and Applications · doi:10.1007/s11042-016-3780-9 · Cited by 75
Gamification for Online Communities: A Case Study for Delivering Government Services
Bista, S. K., Nepal, ., Paris, C. L., & Colineau, N. (2014) · International Journal of Cooperative Information Systems · doi:10.1142/s0218843014410020 · Cited by 74
Intelligent Interactive Multimedia Systems and Services
Damiani, E., Howlett, R. J., Jain, L. C., Gallo, L., & Pietro, G. D. (2015) · Smart innovation, systems and technologies · doi:10.1007/978-3-319-19830-9 · Cited by 73
Improving Paid Microtasks through Gamification and Adaptive Furtherance Incentives
Feyisetan, O., Simperl, E. P. B., Kleek, M. V., et al. (2015) · International World Wide Web Conference (WWW) · doi:10.1145/2736277.2741639 · Cited by 73
Can Distributed Volunteers Accomplish Massive Data Analysis Tasks
Kanefsky, B., Barlow, N. G., & Gulick, V. C. (2001) · Lunar and Planetary Science Conference · Cited by 71
Game of words: tagging places through crowdsourcing on public displays
Gonçalves, J., Hosio, S. J., Ferreira, D., & Kostakos, V. (2014) · Designing Interactive Systems Conference · doi:10.1145/2598510.2598514 · Cited by 67
Competing or aiming to be average?: normification as a means of engaging digital volunteers
Preist, C., Massung, E., & Coyle, D. (2014) · Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing · doi:10.1145/2531602.2531615 · Cited by 65 · cited 18×
“…that extrinsic rewards (such as money) can potentially decrease intrinsic motivation (Deci, 1971; Deci et al., 1999), Massung et al. (2013) and Preist et al. (2014) found in their experiment that gamification in combination with financial rewards can in fact increase participation when…”
BackgroundInfluentialDesigning Cooperative Gamification: Conceptualization and Prototypical Implementation
Morschheuser, B., Maedche, A., & Walter, D. (2017) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2998181.2998272 · Cited by 65
Self-citationMotivating Multi-Generational Crowd Workers in Social-Purpose Work
Kobayashi, M., Arita, S., Itoko, T., Saito, S., & Takagi, H. (2015) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2675133.2675255 · Cited by 60
Crowd-centric Requirements Engineering
Snijders, R., Dalpiaz, F., Hosseini, M., et al. (2014) · IEEE/ACM International Conference on Utility and Cloud Computing (UCC) · doi:10.1109/ucc.2014.96 · Cited by 57
Experiments on Motivational Feedback for Crowdsourced Workers
Lee, T. Y., Dugan, C., Geyer, W., Ratchford, T., Rasmussen, J. C., Shami, N. S., & Lupushor, S. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v7i1.14428 · Cited by 56
Garrigós‐Simón, F. J., Gil-Pechuán, I., & Estellés-Miguel, S. (2015) · Springer eBooks · doi:10.1007/978-3-319-18341-1 · Cited by 53
Lessel, P., Altmeyer, M., & Krüger, A. (2015) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702309 · Cited by 52
Leveraging the Wisdom of the Crowd for Fine-Grained Recognition
Deng, J., Krause, J., Stark, M. M., & Fei-Fei, L. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2439285 · Cited by 52
Tinati, R., Luczak–Roesch, M., Simperl, E. P. B., & Hall, W. (2016) · ACM Web Science Conference (WebSci) · doi:10.1145/2908131.2908151 · Cited by 51
iHEARu-PLAY: Introducing a game for crowdsourced data collection for affective computing
Hantke, S., Eyben, F., Appel, T., & Schuller, B. W. (2015) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii.2015.7344680 · Cited by 50
From gamification to pervasive game in mapping urban accessibility
Prandi, C., Nisi, V., Salomoni, P., & Nunes, N. (2015) · Biannual Conference of the Italian SIGCHI Chapter (CHItaly) · doi:10.1145/2808435.2808449 · Cited by 46
Greenhill, A., Holmes, K., Woodcock, J., Lintott, C. J., Simmons, B. D., Graham, G., Cox, J., Oh, E. Y., & Masters, K. L. (2016) · Aslib Journal of Information Management · doi:10.1108/ajim-11-2015-0182 · Cited by 46
Incentive Mechanisms for Crowdsourcing Platforms
Katmada, A., Σάτσιου, Ά., & Kompatsiaris, I. Y. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-45982-0_1 · Cited by 46
LaToza, T. D., Towne, W. B., van der Hoek, A., & Herbsleb, J. D. (2013) · International Workshop on Cooperative and Human Aspects of Software Engineering (CHASE) · doi:10.1109/chase.2013.6614737 · Cited by 45
How to Use Crowdsourcing Effectively: Guidelines and Examples
Simperl, E. P. B. (2015) · LIBER Quarterly The Journal of the Association of European Research Libraries · doi:10.18352/lq.9948 · Cited by 44
QuaCentive: a quality-aware incentive mechanism in mobile crowdsourced sensing (MCS)
Wang, Y., Jia, X., Jin, Q., & Ma, J. (2015) · The Journal of Supercomputing · doi:10.1007/s11227-015-1395-y · Cited by 44
Tapping the Innovative Business Potential of Innovation Contests
Armisen, A., & Majchrzak, A. (2014) · Business Horizons · doi:10.1016/j.bushor.2015.03.004 · Cited by 41
Morschheuser, B., Henzi, C., & Alt, R. (2015) · Hawaii International Conference on System Sciences (HICSS) · doi:10.1109/hicss.2015.83 · Cited by 36
Self-citationGamifying collective human behavior with gameful digital rhetoric
Sakamoto, M., Nakajima, T., & Akioka, S. (2016) · Multimedia Tools and Applications · doi:10.1007/s11042-016-3665-y · Cited by 36
Tiger Nation: Empowering citizen scientists
Mason, A. D., Michalakidis, G., & Krause, P. J. (2012) · IEEE International Conference on Digital Ecosystems and Technologies · doi:10.1109/dest.2012.6227943 · Cited by 33
Involving Senior Workers in Crowdsourced Proofreading
Itoko, T., Arita, S., Kobayashi, M., & Takagi, H. (2014) · Interacción · doi:10.1007/978-3-319-07446-7_11 · Cited by 32
How (not) to Incent Crowd Workers
Straub, T., Gimpel, H., Teschner, F., & Weinhardt, C. (2015) · Business & Information Systems Engineering · doi:10.1007/s12599-015-0384-2 · Cited by 32
Tolmie, P., Chamberlain, A., & Benford, S. (2013) · Personal and Ubiquitous Computing · doi:10.1007/s00779-013-0755-y · Cited by 31
GAMIFICATION IS ALL ABOUT FUN: THE ROLE OF INCENTIVE TYPE AND COMMUNITY COLLABORATION
Lounis, S., Pramatari, K. C., & Theotokis, A. (2014) · Cited by 30
On Crowd-verification of Biological Networks
Ansari, S., Binder, J., Boué, S., Fabio, A. D., Hayes, W., Hoeng, J., Iskandar, A. R., Kleiman, R. J., Norel, R., O'neel, B., et al. (2013) · Bioinformatics and Biology Insights · doi:10.4137/bbi.s12932 · Cited by 28
A Gamification Framework for Volunteered Geographic Information
Martella, R., Kray, C., & Clementini, E. (2015) · Lecture notes in geoinformation and cartography · doi:10.1007/978-3-319-16787-9_5 · Cited by 25
Gamification of politics : start a new game!
Mahnič, N. (2014) · Teorija in praksa · Cited by 25
EventMask: A Game-Based Framework for Event-Saliency Identification in Images
Rosani, A., Boato, G., & Natale, F. G. B. D. (2015) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2015.2441003 · Cited by 24
Crowd computing: using competitive dynamics to develop and refine highly predictive models
Bentzien, J., Muegge, I., Hamner, B., & Thompson, D. C. (2013) · Drug Discovery Today · doi:10.1016/j.drudis.2013.01.002 · Cited by 22
Crowdsensing in the Wild with Aliens and Micropayments
Talasila, M., Curtmola, R., & Borcea, C. (2016) · IEEE Pervasive Computing · doi:10.1109/mprv.2016.18 · Cited by 22
A collaborative framework for annotating energy datasets
Cao, H.-Â., Wijaya, T. K., Aberer, K., & Nunes, N. (2015) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata.2015.7364072 · Cited by 20
Quizz: Targeted crowdsourcing with a billion (potential) users
Ipeirotis, P. G., & Gabrilovich, E. (2015) · doi:10.48550/arxiv.1506.01062 · Cited by 19
A Study about Designing Reward for Gamified Crowdsourcing System
Choi, J., Choi, H., So, W., Lee, J.-K., & You, J.-J. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-07626-3_64 · Cited by 18
Interaction and Reflection with Quantified Self and Gamification: an Experimental Study
Morschheuser, B., Rivera-Pelayo, V., Mazarakis, A., et al. (2014) · Cited by 18
Self-citationAssessment of crowdsourcing and gamification loss in user-assisted object segmentation
Carlier, A., Salvador, A., Cabezas, F., Giró-i-Nieto, X., Charvillat, V., & Marques, O. (2015) · Multimedia Tools and Applications · doi:10.1007/s11042-015-2897-6 · Cited by 16
How preference markets assist new product idea screening
Lauto, G., & Valentin, F. (2016) · Industrial Management & Data Systems · doi:10.1108/imds-07-2015-0320 · Cited by 16
Modelling learning from Ingress (Google's augmented reality social game)
Sheng, L. Y. (2013) · IEEE Annual Conference International Council for Education Media · doi:10.1109/cicem.2013.6820152 · Cited by 16
Shen, X., Lee, M., Cheung, C. M. K., & Chen, H. (2009) · HKBU Institutional Repository (Hong Kong Baptist University) · Cited by 15
Gamifying Social Media to Encourage Social Activities with Digital-Physical Hybrid Role-Playing
Sakamoto, M., & Nakajima, T. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-07632-4_55 · Cited by 15
Susumpow, P., Pansuwan, P., Sajda, N., & Crawley, A. W. (2014) · International World Wide Web Conference (WWW) · doi:10.1145/2567948.2579273 · Cited by 13
Competing or aiming to be average?: normification as a means of engaging digital volunteers
Preist, C., Massung, E., & Coyle, D. (2014) · Cited by 13
The Effects of Pay-to-Quit Incentives on Crowdworker Task Quality
Harris, C. G. (2015) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2675133.2675185 · Cited by 12
Studying user browsing behavior through gamified search tasks
He, J., Bron, M., Azzopardi, L. A., & de Vries, A. P. (2014) · doi:10.1145/2594776.2594787 · Cited by 12
Brandtner, P., Auinger, A., & Helfert, M. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-07293-7_57 · Cited by 10
Skill Development Framework for Micro-Tasking
Saito, S., Watanabe, T., Kobayashi, M., & Takagi, H. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-07440-5_37 · Cited by 9
Advances in Games Technology: Software, Models, and Intelligence
Prakash, E. C., Brindle, G., Jones, K., Zhou, S., Chaudhari, N. S., & Wong, K. W. (2009) · Simulation & Gaming · doi:10.1177/1046878109335120 · Cited by 9
Current Developments in Big Data and Sustainability Sciences in Mobile Citizen Science Applications
Panchariya, N. S., DeStefano, A. J., Nimbagal, V., Ragupathy, R., Yavuz, S., Herbert, K. G., Hill, E., & Fails, J. A. (2015) · IEEE International Conference on Big Data Computing Service and Applications (BigDataService) · doi:10.1109/bigdataservice.2015.64 · Cited by 8
Toward gamification of knowledge base construction
Roengsamut, B., Kuwabara, K., & Huang, H. (2015) · International Conference on Innovations in Intelligent Systems and Applications (INISTA) · doi:10.1109/inista.2015.7276721 · Cited by 8
Bridging the Utilitarian-Hedonic Divide in Crowdsourcing Applications
Melenhorst, M., Novak, J., Micheel, I., Larson, M., & Böckle, M. (2015) · doi:10.1145/2810188.2810191 · Cited by 8
Harris, C. G. (2014) · doi:10.1145/2594776.2594780 · Cited by 8
People recognition using gamified ambiguous feedback
Brenner, M., Mirza, N., & Ebroul Izquierdo, M. (2014) · doi:10.1145/2594776.2594781 · Cited by 8
Tags You Don’t Forget: Gamified Tagging of Personal Images
Runge, N., Wenig, D., Zitzmann, D., & Malaka, R. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24589-8_23 · Cited by 7
An Investigation of User Interface Features of Crowdsourcing Applications
Nakatsu, R. T., & Iacovou, C. L. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-07293-7_40 · Cited by 7
Inaba, M., Iwata, N., Toriumi, F., Hirayama, T., Enokibori, Y., Takahashi, K., & Mase, K. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-25210-0_8 · Cited by 7
Dumitrache, A., Aroyo, L. M., Welty, C., Sips, R.-J., Levas, A., & Vu (2013) · Digital Academic REpository of VU University Amsterdam (Vrije Universiteit Amsterdam) · Cited by 6
Making Arabic PDF books accessible using gamification
AlRouqi, H., & Al-Khalifa, H. S. (2014) · International Web for All Conference (W4A) · doi:10.1145/2596695.2596712 · Cited by 6
Biegel, B., Beck, F., Lesch, B., & Diehl, S. (2014) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme.2014.64 · Cited by 5
Gamification as a paradigm for the evaluation of visual analytics systems
Ahmed, N., & Mueller, K. D. (2014) · Workshop on Beyond Time and Errors · doi:10.1145/2669557.2669574 · Cited by 5
Collaborative landmark mining with a gamification approach
Bockes, F., Edel, L., Ferstl, M., & Schmid, A. (2015) · International Conference on Mobile and Ubiquitous Multimedia (MUM) · doi:10.1145/2836041.2841209 · Cited by 5
Fava, D. S., Signoles, J., Lemerre, M., et al. (2015) · Lecture notes in computer science · doi:10.1007/978-3-662-48899-7_41 · Cited by 5
CINA - A Crowdsourced Indoor Navigation Assistant
Reinsch, T., Wang, Y., Knechtel, M., Ameling, M., & Herzig, P. (2013) · IEEE/ACM International Conference on Utility and Cloud Computing (UCC) · doi:10.1109/ucc.2013.97 · Cited by 4
Bainbridge, D. (2015) · International Conference on Digital Libraries for Musicology (DLfM) · doi:10.1145/2785527.2785529 · Cited by 4
Crowdpinion: Motivating People to Share Their Momentary Opinion
Machnik, M. F., Riegler, M. A., & Sen, S. (2015) · Cited by 4
Gamification as a Key Enabling Technology for Image Sensing and Content Tagging
Simões, B., & Amicis, R. D. (2016) · Smart innovation, systems and technologies · doi:10.1007/978-3-319-39345-2_44 · Cited by 4
Analysis of self-tagging during conversational chat in multilingual gaming simulation
Nose, T., & Hishiyama, R. (2013) · International Conference on Future Generation Communication Technologies (FGCT) · doi:10.1109/fgct.2013.6767188 · Cited by 3
Cited by
The rise of motivational information systems: A review of gamification research
Koivisto, J., & Hamari, J. (2019) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2018.10.013 · Cited by 1,750
Self-citationKrath, J., Schürmann, L., & von Korflesch, H. F. O. (2021) · Computers in Human Behavior · doi:10.1016/j.chb.2021.106963 · Cited by 943
The state of immersive technology research: A literature analysis
Suh, A., & Prophet, J. (2018) · Computers in Human Behavior · doi:10.1016/j.chb.2018.04.019 · Cited by 841
Xi, N., & Hamari, J. (2019) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2018.12.002 · Cited by 618
Self-citationLegaki, N.-Z., Xi, N., Hamari, J., Karpouzis, K., & Assimakopoulos, V. (2020) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2020.102496 · Cited by 328
Self-citation
Metadata
- Authors
- Morschheuser, B., Hamari, J., Koivisto, J., & Maedche, A.
- Venue
- International Journal of Human-Computer Studies
- Publisher
- Elsevier BV
- Published
- 2017-10
- Volume
- 106
- Pages
- 26-43
- ISSN
- 1071-5819 (ISSN-L)
- OpenAlex
- W2615593621
- Cited by
- 297
- References
- 163
- FWCI
- 39.45
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.