Gamification of Mobile Experience Sampling Improves Data Quality and Quantity
Abstract
The Experience Sampling Method is used to capture high-quality in situ data from study participants. This method has become popular in studies involving smartphones, where it is often adapted to motivate participation through the use of gamification techniques. However, no work to date has evaluated whether gamification actually affects the quality and quantity of data collected through Experience Sampling. Our study systematically investigates the effect of gamification on the quantity and quality of experience sampling responses on smartphones. In a field study, we combine event contingent and interval contingent triggers to ask participants to describe their location. Subsequently, participants rate the quality of these entries by playing a game with a purpose. Our results indicate that participants using the gamified version of our ESM software provided significantly higher quality responses, slightly increased their response rate, and provided significantly more data on their own accord. Our findings suggest that gamifying experience sampling can improve data collection and quality in mobile settings.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
mHealth offers the possibility of incorporating motivational elements such as playfulness and gamification, which is advantageous because humans supposedly learn best by playing.
“mHealth offers the possibility of incorporating motivational elements such as playfulness and gamification, which is advantageous because humans supposedly learn best by playing [13-17].”
From A Smartphone Serious Game for Adolescents (Grow It! App): Development, Feasibility, and Acceptance Study (2022) · cites this paper as [15]
Cites
From game design elements to gamefulness: defining "gamification"
Deterding, S., Dixon, D., Khaled, R., & Nacke, L. (2011) · Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments · doi:10.1145/2181037.2181040 · Cited by 8,402 · see citing works
Does Gamification Work? -- A Literature Review of Empirical Studies on Gamification
Hamari, J., Koivisto, J., & Sarsa, H. (2014) · 2014 47th Hawaii International Conference on System Sciences · doi:10.1109/hicss.2014.377 · Cited by 5,201 · see citing works
Ecological Momentary Assessment (Ema) in Behavioral Medicine
Stone, A. A., & Shiffman, S. (1994) · Annals of Behavioral Medicine · doi:10.1093/abm/16.3.199 · Cited by 1,931 · see citing works
Flow and the Foundations of Positive Psychology: The Collected Works of Mihaly Csikszentmihalyi
Csikszentmihalyi, M. (2014) · doi:10.1007/978-94-017-9088-8 · Cited by 1,653 · see citing works
Knowledge hiding in organizations
Connelly, C. E., Zweig, D., Webster, J., & Trougakos, J. P. (2011) · Journal of Organizational Behavior · doi:10.1002/job.737 · Cited by 1,581 · see citing works
Experience Sampling Method: Measuring the Quality of Everyday Life
Hektner, J. M., Schmidt, J. A., & Csíkszentmihályi, M. (2006) · Cited by 1,255 · see citing works
Hektner, J., Schmidt, J., & Csikszentmihalyi, M. (2007) · doi:10.4135/9781412984201 · Cited by 1,174 · see citing works
Designing games with a purpose
von Ahn, L., & Dabbish, L. (2008) · Communications of the ACM · doi:10.1145/1378704.1378719 · Cited by 1,156 · see citing works
The Experience Sampling Method
Larson, R. W., & Csíkszentmihályi, M. (2014) · Springer eBooks · doi:10.1007/978-94-017-9088-8_2 · Cited by 1,047 · see citing works
Experience Sampling: Promises and Pitfalls, Strengths and Weaknesses
Scollon, C. N., Kim‐Prieto, C., & Diener, E. (2003) · Journal of Happiness Studies · doi:10.1023/a:1023605205115 · Cited by 827 · see citing works
The ecology of adolescent activity and experience
Csikszentmihalyi, M., Larson, R., & Prescott, S. (1977) · Journal of Youth and Adolescence · doi:10.1007/bf02138940 · Cited by 781 · see citing works
Estimating the Effect of Incentives on Mail Survey Response Rates: A Meta-Analysis
Church, A. H. (1993) · Public Opinion Quarterly · doi:10.1086/269355 · Cited by 705 · see citing works
Building Successful Online Communities: Evidence-Based Social Design
Kraut, R. E., Resnick, P., Kiesler, S. B., Ren, Y., Chen, Y., Burke, M., Kittur, N., Riedl, J., & Konstan, J. A. (2012) · Cited by 636 · see citing works
Gamification: What It Is and Why It Matters to Digital Health Behavior Change Developers
Cugelman, B. (2013) · JMIR Serious Games · doi:10.2196/games.3139 · Cited by 531 · cited 1× · see citing works
“Gamification as an element of motivation has been applied more extensively in applications aimed at medical intervention or health behaviour change [9].”
BackgroundMore than fun and money. Worker motivation in crowdsourcing - a study on mechanical turk
Kaufmann, N., Schulze, T., & Veit, D. J. (2011) · MADOC (University of Mannheim) · Cited by 469 · see citing works
Using the experience sampling method to evaluate ubicomp applications
Consolvo, S., & Walker, M. (2003) · IEEE Pervasive Computing · doi:10.1109/mprv.2003.1203750 · Cited by 368 · see citing works
Jung, J. H., Schneider, C., & Valacich, J. S. (2010) · Management Science · doi:10.1287/mnsc.1090.1129 · Cited by 332 · see citing works
Building Successful Online Communities: Evidence-Based Social Design
Kraut, R. E., Resnick, P., Kiesler, S., Burke, M., Chen, Y., Kittur, N., Konstan, J., Ren, Y., & Riedl, J. (2012) · doi:10.7551/mitpress/8472.001.0001 · Cited by 327 · see citing works
A Practical Guide to Experience-Sampling Procedures
Conner, T. S., Barrett, L. F., Bliss‐Moreau, E., Lebo, K., & Kaschub, C. (2003) · Journal of Happiness Studies · doi:10.1023/a:1023609306024 · Cited by 318 · see citing works
The sampling of experiences in situ
Hormuth, S. E. (1986) · Journal of Personality · doi:10.1111/j.1467-6494.1986.tb00395.x · Cited by 294 · see citing works
Measuring Daily Events and Experiences: Decisions for the Researcher
Stone, A. A., Kessler, R. C., & Haythomthwatte, J. A. (1991) · Journal of Personality · doi:10.1111/j.1467-6494.1991.tb00260.x · Cited by 278 · see citing works
Results from deploying a participation incentive mechanism within the enterprise
Farzan, R., DiMicco, J. M., Millen, D. R., Dugan, C., Geyer, W., & Brownholtz, E. A. (2008) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1357054.1357145 · Cited by 258 · see citing works
Voice-mail diary studies for naturalistic data capture under mobile conditions
Palen, L. A., & Salzman, M. (2002) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/587078.587092 · Cited by 255 · see citing works
Gamification in Crowdsourcing: A Review
Morschheuser, B., Hamari, J., & Koivisto, J. (2016) · Hawaii International Conference on System Sciences (HICSS) · doi:10.1109/hicss.2016.543 · Cited by 219 · see citing works
Fischer, J. E., Greenhalgh, C., & Benford, S. (2011) · International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI) · doi:10.1145/2037373.2037402 · Cited by 189 · see citing works
Drawing on Education: Using Student Drawings To Promote Middle School Improvement.
Haney, W., Russell, M., Gulek, C., & Fierros, E. (1998) · Schools in the middle · Cited by 166 · see citing works
Jones, B. A., Madden, G. J., & Wengreen, H. J. (2014) · Preventive Medicine · doi:10.1016/j.ypmed.2014.04.015 · Cited by 155 · see citing works
Fitz-Walter, Z., Tjondronegoro, D., & Wyeth, P. A. (2011) · Australian Computer-Human Interaction Conference (OzCHI) · doi:10.1145/2071536.2071554 · Cited by 153 · see citing works
Reyal, S., Zhai, S., & Kristensson, P. O. (2015) · Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702597 · Cited by 135 · cited 2× · see citing works
“The aforementioned studies have very different study designs (1 daily survey over five successive working days [5], 10 tasks per day over a four-week period [41]), making it difficult to compare and discuss response rates between studies.”
BackgroundResultInfluentialde Oliveira, R. C., Cherubini, M., & Oliver, N. (2010) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/1864349.1864371 · Cited by 114 · see citing works
Hicks, J., Ramanathan, N., Kim, D., Monibi, M., Selsky, J., Hansen, M. H., & Estrin, D. L. (2010) · Wireless Health · doi:10.1145/1921081.1921087 · Cited by 106 · see citing works
Effects of gamification on participation and data quality in a real-world market research domain
Cechanowicz, J., Gutwin, C., Brownell, B., & Goodfellow, L. (2013) · Proceedings of the First International Conference on Gameful Design, Research, and Applications · doi:10.1145/2583008.2583016 · Cited by 103 · see citing works
Exploring micro-incentive strategies for participant compensation in high-burden studies
Musthag, M., Raij, A. B., Ganesan, D., Kumar, S., & Shiffman, S. S. (2011) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/2030112.2030170 · Cited by 103 · see citing works
Beyond designing for motivation
Richards, C., Thompson, C. W., & Graham, N. (2014) · Annual Symposium on Computer-Human Interaction in Play · doi:10.1145/2658537.2658683 · Cited by 99 · see citing works
Gamification-based incentive mechanism for participatory sensing
Ueyama, Y., Tamai, M., Arakawa, Y., & Yasumoto, K. (2014) · IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops) · doi:10.1109/percomw.2014.6815172 · Cited by 82 · see citing works
Using visualizations to increase compliance in experience sampling
Hsieh, G., Li, I., Dey, A. K., Forlizzi, J., & Hudson, S. E. (2008) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/1409635.1409657 · Cited by 75 · see citing works
A Systematic Assessment of Smartphone Usage Gaps
van Berkel, N., Luo, C., Anagnostopoulos, T., Ferreira, D., Gonçalves, J., Hosio, S. J., & Kostakos, V. (2016) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858348 · Cited by 70 · see citing works
Self-citationSituated crowdsourcing using a market model
Hosio, S. J., Gonçalves, J., Lehdonvirta, V., Ferreira, D., & Kostakos, V. (2014) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/2642918.2647362 · Cited by 63 · see citing works
Self-citationGonçalves, J., Hosio, S. J., Ferreira, D., & Kostakos, V. (2014) · Designing Interactive Systems Conference · doi:10.1145/2598510.2598514 · Cited by 63 · see citing works
Self-citationUpton, D., Upton, P., & Taylor, C. (2012) · Public Health Nutrition · doi:10.1017/s1368980012004612 · Cited by 60 · see citing works
Motivating participation and improving quality of contribution in ubiquitous crowdsourcing
Gonçalves, J., Hosio, S. J., Rogstadius, J., Karapanos, E., & Kostakos, V. (2015) · Computer Networks · doi:10.1016/j.comnet.2015.07.002 · Cited by 59 · see citing works
Self-citationParticipation incentive mechanisms in peer-to-peer subscription systems
Lui, S. M., Lang, K. R., & Kwok, S. H. (2003) · Hawaii International Conference on System Sciences (HICSS) · doi:10.1109/hicss.2002.994529 · Cited by 50 · see citing works
Lynn, P. J. (2001) · International Journal for Quality in Health Care · doi:10.1093/ijpor/13.3.326 · Cited by 41 · see citing works
Mehrotra, A., Vermeulen, J., Pejović, V., & Musolesi, M. (2015) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/2800835.2804397 · Cited by 41 · see citing works
Citizen Motivation on the Go: The Role of Psychological Empowerment
Gonçalves, J., Kostakos, V., Karapanos, E., Barreto, M. L., Camacho, T., Tomasic, A., & Zimmerman, J. (2013) · Interacting with Computers · doi:10.1093/iwc/iwt035 · Cited by 40 · see citing works
Self-citationKhan, V.-J., Markopoulos, P., Eggen, B. H., IJsselsteijn, W. A., & de Ruyter, B. (2008) · International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI) · doi:10.1145/1409240.1409316 · Cited by 38 · see citing works
Predicting interruptibility for manual data collection: a cluster-based user model
Visuri, A., van Berkel, N., Luo, C., Goncalves, J., Ferreira, D., & Kostakos, V. (2017) · Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services · doi:10.1145/3098279.3098532 · Cited by 28 · see citing works
Self-citationGonçalves, J., Kostakos, V., Hosio, S. J., Karapanos, E., & Lyra, O. (2013) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/2513383.2517030 · Cited by 27 · see citing works
Self-citationA field study comparing approaches to collecting annotated activity data in real-world settings
Chang, Y.-J., Paruthi, G., & Newman, M. (2015) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/2750858.2807524 · Cited by 26 · see citing works
Photographing information needs
Yue, Z., Litt, E., Cai, C. J., Stern, J., Baxter, K. K., Guan, Z., Sharma, N., & Zhang, G. (. (2014) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2556288.2557192 · Cited by 21 · see citing works
Multiple Intelligence, Learning Styles, Brain-Based Education: Where Do the Messages Overlap?.
Guild, P. B., & Chock-Eng, S. (1998) · Schools in the middle · Cited by 13 · see citing works
Crowdpinion: Motivating People to Share Their Momentary Opinion
Machnik, M. F., Riegler, M. A., & Sen, S. (2015) · Cited by 4 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
The Experience Sampling Method on Mobile Devices
van Berkel, N., Ferreira, D., & Kostakos, V. (2018) · ACM Computing Surveys · doi:10.1145/3123988 · Cited by 432 · see citing works
Self-citationAnnual Research Review: Ecological momentary assessment studies in child psychology and psychiatry
Russell, M. A., & Gajos, J. M. (2020) · Journal of Child Psychology and Psychiatry · doi:10.1111/jcpp.13204 · Cited by 236 · see citing works
The Design of Ecological Momentary Assessment Technologies
Doherty, K., Balaskas, A., & Doherty, G. J. (2020) · Interacting with Computers · doi:10.1093/iwcomp/iwaa019 · Cited by 187 · see citing works
The Impact of Gamification on Learning Outcomes of Computer Science Majors
Ahmad, A., Zeshan, F., Khan, M. S., Marriam, R., Ali, A., & Samreen, A. (2020) · ACM Transactions on Computing Education · doi:10.1145/3383456 · Cited by 124 · see citing works
van Berkel, N., Gonçalves, J., Lovén, L., Ferreira, D., Hosio, S. J., & Kostakos, V. (2018) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2018.12.002 · Cited by 96 · see citing works
Self-citationBülow, A., Neubauer, A. B., Soenens, B., Boele, S., Denissen, J. J. A., & Keijsers, L. (2022) · Scientific Reports · doi:10.1038/s41598-022-21071-0 · Cited by 84 · see citing works
Overcoming compliance bias in self-report studies: A cross-study analysis
van Berkel, N., Gonçalves, J., Hosio, S. J., Sarsenbayeva, Z., Velloso, E., & Kostakos, V. (2019) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2019.10.003 · Cited by 69 · see citing works
Self-citationEngagement with Mental Health Screening on Mobile Devices
Doherty, K., Belisario, J. S. M., Cohn, M., Mastellos, N., Morrison, C. P., Car, J., & Doherty, G. J. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300416 · Cited by 54 · see citing works
Rabbi, M., Philyaw-Kotov, M. L., Cunningham, R. M., Bonar, E. E., Nahum-Shani, I. B., Klasnja, P., Walton, M. A., & Murphy, S. A. (2018) · JMIR Research Protocols · doi:10.2196/resprot.9850 · Cited by 45 · see citing works
Murray, A. L., & Xie, T. (2023) · Journal of Adolescent Health · doi:10.1016/j.jadohealth.2023.06.032 · Cited by 42 · see citing works
Learning-Assisted Optimization in Mobile Crowd Sensing: A Survey
Wang, J., Wang, Y., Zhang, D., Gonçalves, J., Ferreira, D., Visuri, A., & Ma, S. (2018) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2018.2868703 · Cited by 41 · see citing works
Contact Logger: Measuring everyday intergroup contact experiences in near-time
Keil, T. F., Koschate, M., & Levine, M. (2020) · Behavior Research Methods · doi:10.3758/s13428-019-01335-w · Cited by 36 · see citing works
Dietvorst, E., Aukes, M. A., Legerstee, J. S., Vreeker, A., Hrehovcsik, M. M., Keijsers, L., & Hillegers, M. H. J. (2022) · JMIR Formative Research · doi:10.2196/29832 · Cited by 35 · see citing works
“Furthermore, mHealth offers the possibility of incorporating motivational elements such as playfulness and gamification, which is advantageous because humans supposedly learn best by playing [13-17].”
BackgroundDonckt, J. V. D., Vandenbussche, N., Donckt, J. V. D., Chen, S., Stojchevska, M., Brouwer, M. D., Steenwinckel, B., Paemeleire, K., Ongenae, F., & Hoecke, S. V. (2024) · Scientific Reports · doi:10.1038/s41598-024-67767-3 · Cited by 35 · see citing works
Recommendations for Conducting Longitudinal Experience Sampling Studies
van Berkel, N., & Kostakos, V. (2021) · Human-computer interaction series · doi:10.1007/978-3-030-67322-2_4 · Cited by 35 · see citing works
Self-citationContext-Informed Scheduling and Analysis
van Berkel, N., Gonçalves, J., Koval, P., Hosio, S. J., Dingler, T., Ferreira, D., & Kostakos, V. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300281 · Cited by 34 · see citing works
Self-citationHoward, A. L., & Lamb, M. (2023) · Assessment · doi:10.1177/10731911231159937 · Cited by 32 · see citing works
Deep Learning Based-Recommendation System
Ong, K., Haw, S.-C., & Ng, K.-W. (2019) · International Conference on Computational Intelligence and Intelligent Systems (CIIS) · doi:10.1145/3372422.3372444 · Cited by 32 · see citing works
Modeling interaction as a complex system
van Berkel, N., Dennis, S. J., Zyphur, M. J., Li, J., Heathcote, A., & Kostakos, V. (2020) · Human-Computer Interaction · doi:10.1080/07370024.2020.1715221 · Cited by 31 · see citing works
Self-citationKawanaka, S., Matsuda, Y., Suwa, H., Fujimoto, M., Arakawa, Y., & Yasumoto, K. (2020) · Smart Cities · doi:10.3390/smartcities3030037 · Cited by 29 · see citing works
Domoff, S. E., Banga, C. A., Borgen, A. L., Foley, R. P., Robinson, C. A., Avery, K., & Gentile, D. A. (2021) · Human Behavior and Emerging Technologies · doi:10.1002/hbe2.247 · Cited by 26 · see citing works
Andrade, P., Law, E. L., Farah, J. C., & Gillet, D. (2020) · Australian Conference on Human-Computer Interaction (OzCHI) · doi:10.1145/3441000.3441073 · Cited by 25 · see citing works
Rabbi, M., Li, K. T., Yan, H. Y., Dahl, K. L., Klasnja, P., & Murphy, S. A. (2019) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3369806 · Cited by 22 · see citing works
Weermeijer, J. D. M., Kiekens, G., Wampers, M., Kuppens, P., & Myin‐Germeys, I. (2023) · Behaviour and Information Technology · doi:10.1080/0144929x.2023.2178235 · Cited by 22 · see citing works
Labelling Affective States "in the Wild"
Schmidt, P., Reiss, A., Dürichen, R., & Laerhoven, K. V. (2018) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/3267305.3267551 · Cited by 22 · see citing works
Human accuracy in mobile data collection
van Berkel, N., Gonçalves, J., Wac, K., Hosio, S. J., & Cox, A. L. (2020) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2020.102396 · Cited by 21 · see citing works
Self-citationEngaging Participants during Selection Studies in Virtual Reality
Yu, D., Zhou, Q., Tag, B., Dingler, T., Velloso, E., & Gonçalves, J. (2020) · IEEE Conference on Virtual Reality and 3D User Interfaces (VR) · doi:10.1109/vr46266.2020.1581291122479 · Cited by 19 · see citing works
Locating place, crime and the fear of crime
Kronkvist, K. (2022) · doi:10.24834/isbn.9789178773022 · Cited by 19 · see citing works
End-User Development of Experience Sampling Smartphone Apps -Recommendations and Requirements
Rough, D. J., & Quigley, A. J. (2020) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3397307 · Cited by 16 · see citing works
Mairittha, N., Mairittha, T., Lago, P., & Inoue, S. (2021) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3432222 · Cited by 15 · see citing works
Technology-assisted reconstruction: a new alternative to the experience sampling method
Karapanos, E. (2019) · Behaviour and Information Technology · doi:10.1080/0144929x.2019.1608303 · Cited by 14 · see citing works
Elavsky, S., Blahošová, J., Lebedíková, M. Š., Tkaczyk, M., Tancoš, M., Plhák, J., Sotolář, O., & Šmahel, D. (2022) · JMIR Research Protocols · doi:10.2196/35984 · Cited by 14 · see citing works
Dejonckheere, E., Verdonck, S., Andries, J. I., Röhrig, N., Piot, M., kilani, G., & Mestdagh, M. (2024) · Computers in Human Behavior · doi:10.1016/j.chb.2024.108360 · Cited by 14 · see citing works
Blow the Dog Whistle: A Chinese Dataset for Cant Understanding with Common Sense and World Knowledge
Xu, C., Zhou, W., Ge, T., Xu, K., McAuley, J., & Wei, F. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.172 · Cited by 14 · see citing works
Real-time personalized feedback in mHealth for adolescents
Dietvorst, E., Hillegers, M. H. J., Legerstee, J. S., de Vries, L. P., Vreeker, A., & Keijsers, L. (2024) · Digital Health · doi:10.1177/20552076241247937 · Cited by 13 · see citing works
Bülow, A., Janssen, L. H. C., Dietvorst, E., de Vaate, N. A. J. D. B., Hillegers, M. H. J., Valkenburg, P. M., & Keijsers, L. (2025) · Journal of Adolescence · doi:10.1002/jad.12478 · Cited by 13 · see citing works
Berrocal, A., Concepcion, W., Dominicis, S. D., & Wac, K. (2020) · JMIR mhealth and uhealth · doi:10.2196/15947 · Cited by 13 · see citing works
Tangible Self-Report Devices: Accuracy and Resolution of Participant Input
van Berkel, N., Merritt, T. R., Bruun, A., & Skov, M. B. (2022) · International Conference on Tangible, Embedded, and Embodied Interaction (TEI) · doi:10.1145/3490149.3501309 · Cited by 13 · see citing works
Komninos, A. (2018) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2018.12.001 · Cited by 13 · see citing works
Altmeyer, M., Zenuni, B., Spelt, H., Jegen, T., Lessel, P., & Krüger, A. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3549491 · Cited by 12 · see citing works
The Quality Control in Crowdsensing Based on Twice Consensuses of Blockchain
Liang, D., An, J., Cheng, J., He, Y., & Gui, R. (2018) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/3267305.3267547 · Cited by 12 · see citing works
Towards Improving Emotion Self-report Collection using Self-reflection
Ghosh, S., Mitra, B., & De, P. (2020) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3334480.3383019 · Cited by 11 · see citing works
Improving Accuracy in Mobile Human Contributions
van Berkel, N., Budde, M., Wijenayake, S., & Gonçalves, J. (2018) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/3267305.3267541 · Cited by 11 · see citing works
The Experience Sampling Method in the Research on Achievement-Related Emotions and Motivation
Moeller, J., Dietrich, J., & Baars, J. (2023) · Routledge eBooks · doi:10.4324/9781003303473-14 · Cited by 8 · see citing works
Intelligent technologies and applications in Smart Tourism—A systematic review
Si-Tou, C. F. (2024) · Smart Tourism · doi:10.54517/st.v5i1.2643 · Cited by 8 · see citing works
Optimizing activity data collection with gamification points using uncertainty based active learning
Mairittha, N., Mairittha, T., & Inoue, S. (2019) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/3341162.3345585 · Cited by 8 · see citing works
Miyata, A., Murayama, Y., Furuta, A., Okugawa, K., Ochiai, K., & Murayama, Y. (2022) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3491101.3519783 · Cited by 8 · see citing works
Intra-Company Crowdsensing: Datafication with Human-in-the-Loop
Domaszewicz, J., & Parzych, D. (2022) · Sensors · doi:10.3390/s22030943 · Cited by 7 · see citing works
Rabbi, M., Philyaw-Kotov, M. L., Li, J., Li, K. T., Rothman, B., Giragosian, L., Reyes, M., Gadway, H., Cunningham, R. M., Bonar, E. E., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.13545 · Cited by 7 · see citing works
Longitudinal User Engagement with Microinteraction Ecological Momentary Assessment (μEMA)
Ponnada, A., Wang, S. D., Li, J., Wang, W., Dunton, G. F., Hedeker, D., & Intille, S. S. (2025) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3749541 · Cited by 6 · see citing works
Muller, S. C., Baldauf, M., & Seeliger, A. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3546733 · Cited by 6 · see citing works
Budde, M. (2021) · Springer handbooks · doi:10.1007/978-3-030-52171-4_44 · Cited by 6 · see citing works
Trading Accuracy for Enjoyment? Data Quality and Player Experience in Data Collection Games
Gundry, D., & Deterding, S. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3502025 · Cited by 6 · see citing works
Jung, Y., Law, H. Y., Lee, H. S., Lee, J., Lee, B., & Lee, U. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713314 · Cited by 5 · see citing works
Osuna-Acedo, S. (2021) · Springer eBooks · doi:10.1007/978-3-030-67314-7_6 · Cited by 5 · see citing works
Langner, M., Toreini, P., & Maedche, A. (2024) · i-com · doi:10.1515/icom-2023-0035 · Cited by 5 · see citing works
Ecological Momentary Assessment Tools: Lessons Learned from an HCI Perspective
Crovari, P., Catania, F., Spitale, M., & Garzotto, F. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-49059-1_28 · Cited by 4 · see citing works
Case Study: In-the-Field Accessibility Information Collection Using Gamification
Miyata, A., Okugawa, K., Murayama, Y., Furuta, A., Ochiai, K., & Murayama, Y. (2023) · International Web for All Conference (W4A) · doi:10.1145/3587281.3587288 · Cited by 4 · see citing works
Thinking Outside the Lab: VR Size & Depth Perception in the WildPreprint
Arora, R., Li, J., Shi, G., & Singh, K. S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.00584 · Cited by 4 · see citing works
Le, H., Choube, A., Mishra, V., & Intille, S. S. (2026) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3831653 · Cited by 3 · see citing works
Harmon, S., Heitkemper, E., Mamykina, L., & Hwang, M. L. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555593 · Cited by 3 · see citing works
Xu, Z., Zhang, J., Haroutounian, S., Liu, H., Cao, Z., Messner, G. R., Alaverdyan, H., Ahuja, S., Koshy, R., Hanns, J., et al. (2025) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3729488 · Cited by 3 · see citing works
2VT: Visions, Technologies, and Visions of Technologies for Understanding Human Scale Spaces
Paananen, V., Markkanen, P. M. H., Oppenlaender, J., Lee, L., Akmal, H. A., Schieck, A. F. G., Dunham, J., Papangelis, K., LaLone, N. J., van Berkel, N., et al. (2021) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3411763.3441315 · Cited by 3 · see citing works
Self-citationKornacka, M., Jaskulska, A., Skorupska, K., Szastok, M., Nadziejko, M., & Kopeć, W. (2023) · Mental Health and Social Inclusion · doi:10.1108/mhsi-01-2023-0015 · Cited by 3 · see citing works
Oppenlaender, J., Asare, K. O., & Hosio, S. J. (2018) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/3267305.3267550 · Cited by 3 · see citing works
Self-citationGenerating Personalized Games with Sensing Data to Support Self-Reported Data Collection
Kim, J.-S., Le, H., Choube, A., Mishra, V., & Intille, S. S. (2026) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3830398.3830678 · Cited by 2 · see citing works
Gong, Q., He, X., Xie, Q., Lin, S., She, G., Fang, R., Han, R., Chen, Y., Xiao, Y., Fu, X., et al. (2018) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/3267305.3267546 · Cited by 2 · see citing works
Ghosh, S., Mandi, S., Sen, S., Mitra, B., & De, P. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642833 · Cited by 2 · see citing works
van Berkel, N., Hosio, S. J., Tag, B., & Gonçalves, J. (2019) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/3341162.3344846 · Cited by 2 · see citing works
Self-citationBülow, A., Janssen, L. H. C., Dietvorst, E., de Vaate, N. A. J. D. B., Hillegers, M. H. J., Valkenburg, P. M., & Keijsers, L. (2024) · doi:10.31234/osf.io/rnuvz · Cited by 2 · see citing works
WOOL: A gamified, georeferenced mobile application for enhanced tourism exploration
Ladeira, E. F., & Silva, B. M. C. (2026) · Entertainment Computing · doi:10.1016/j.entcom.2026.101108 · Cited by 1 · see citing works
Persuasive Mobile NOW Interactions
Barendregt, R., & Wasson, B. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-05412-9_22 · Cited by 1 · see citing works
Simulating Participant Behavior in Experience Sampling Method Research
Khanshan, A., Gorp, P. M. E. V., & Markopoulos, P. (2023) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3544549.3585586 · Cited by 1 · see citing works
Instruments for Measuring Youth Digital Media Use
Mayen, S., Reinhardt, A., & Wilhelm, C. (2024) · MedienPädagogik Zeitschrift für Theorie und Praxis der Medienbildung · doi:10.21240/mpaed/60/2024.04.25.x · Cited by 1 · see citing works
Exploring the Challenges of Using Food Journaling Apps: A Case-study with Young Adults
Karnavat, T. L., Bhatia, J. S., Ghosh, S., & Sen, S. (2022) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-030-94822-1_4 · Cited by 1 · see citing works
Reinhardt, A., Mayen, S., & Wilhelm, C. (2024) · Social Science Computer Review · doi:10.1177/08944393241235182 · Cited by 1 · see citing works
SpaceMaze: incentivizing correct mobile crowdsourced sensing behaviour with a sensified minigame
Budde, M., Rohe, J. F., Hirschoff, L., Schlosser, P., Beigl, M., Holopainen, J. P., & Schankin, A. (2020) · Behaviour and Information Technology · doi:10.1080/0144929x.2020.1772878
Monitoring everyday life in aging & dementia
Bartels, S. L. (2020) · doi:10.26481/dis.20200917sb
الرحمن, إ. ا. م. أ. ع. (2020) · تکنولوجيا التعليم سلسلة دراسات وبحوث · doi:10.21608/tesr.2020.160500
Inviting Participants’ Peers in a Mobile Assessment Study: An Empirical Investigation
Chang, Y., Lee, H.-P., Chang, Y.-J., & Shen, C.-Y. (2021) · International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI) · doi:10.1145/3447526.3472021
Khanshan, A., Gorp, P. M. E. V., Nuijten, R., & Markopoulos, P. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph182010593
Dietvorst, E., Hillegers, M. H. J., Legerstee, J. S., Vreeker, A., & Keijsers, L. (2023) · doi:10.2196/preprints.47511
Transdiagnostic Emotional Disorder Assessment and Case Formulation in Health Conditions
Rosellini, A. J., Osma, J., Suso, C., & Brown, T. A. (2023) · Oxford University Press eBooks · doi:10.1093/oso/9780197564295.003.0003
Song, S., Kim, H. Y., Kim, J., Kim, K. K., Cho, H., & Lim, D. Y. (2023) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/3594739.3610684
Yu, H., & Chen, S.-T. (2025) · Edelweiss Applied Science and Technology · doi:10.55214/2576-8484.v9i8.9472
Contact Logger: Measuring Everyday Intergroup Contact Experiences in Near-TimePreprint
Koschate, M., Keil, T. F., & Levine, M. (2020) · OSF Preprints (OSF Preprints)
Spatial Experience for the Metaverse
Paananen, V., Oppenlaender, J., van Berkel, N., & Hosio, S. J. (2026) · Springer eBooks · doi:10.1007/978-3-032-03296-6_4
Self-citationSong, S., Kim, H. Y., & Kim, K. K. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2026.2618582
Lane, R., Millard, L. A. C., Salway, R., Brady, S. M., Skinner, A., Stone, C., Mariapun, J., Rajakumar, S., Ramadas, A., Rizal, H., et al. (2026) · Journal of Medical Internet Research · doi:10.2196/73435
Wadenholt, G. Å., Wirebring, L. K., Stillesjö, S., & Andersson, L. (2026) · Acta Psychologica · doi:10.1016/j.actpsy.2026.106548
Oh, J., Seong, E., Lee, J., Jeon, H., & Kim, S. (2026) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2026.103894
Kornacka, M., Barnes, S., Skorupski, J., Skorupski, M. S., Jaskulska, A., Para, M., & Szastok, M. (2026) · Psychiatria i Psychologia Kliniczna · doi:10.15557/pipk.2026.0018
- Authors
- van Berkel, N., Goncalves, J., Hosio, S., & Kostakos, V.
- Affiliations
- The University of Melbourne
- University of Oulu (FI)
- The University of Melbourne (AU)
- Venue
- Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
- Published
- 2017-09-11
- Volume
- 1
- Issue
- 3
- Pages
- 1-21
- ISSN
- 2474-9567 (ISSN-L)
- OpenAlex
- W2754141758
- Cited by
- 95
- References
- 52
- Citations by year
- 2026: 7 · 2025: 6 · 2024: 9 · 2023: 10 · 2022: 12 · 2021: 13 · 2020: 20 · 2019: 8 · 2018: 9 · 2017: 1
- FWCI
- 12.45
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies (publishedVersion)
- UCL Discovery (University College London) (submittedVersion)
- Funders
- Suomalainen Tiedeakatemia (Grants 276786-AWARE, 286386-CPDSS, 285459-iSCIENCE, 304925-CARE)
- Marie SkBodowska-Curie (645706-GRAGE)
- European Commission (Grant, 6AIKA-A71143-AKAI)
- Topics
- Physical Sciences › Computer Science › Computer Science Applications › Mobile Crowdsensing and Crowdsourcing
- Social Sciences › Psychology › Developmental and Educational Psychology › Educational Games and Gamification
- Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
- Keywords
- gamification, ecological momentary assessment, mobile data collection, response rate, in-situ data, location data, games with a purpose, participant motivation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review