Comparing the Similarity of Responses Received from Studies in Amazon’s Mechanical Turk to Studies Conducted Online and with Direct Recruitment
Open accessAbstract
Computer and internet based questionnaires have become a standard tool in Human-Computer Interaction research and other related fields, such as psychology and sociology. Amazon's Mechanical Turk (AMT) service is a new method of recruiting participants and conducting certain types of experiments. This study compares whether participants recruited through AMT give different responses than participants recruited through an online forum or recruited directly on a university campus. Moreover, we compare whether a study conducted within AMT results in different responses compared to a study for which participants are recruited through AMT but which is conducted using an external online questionnaire service. The results of this study show that there is a statistical difference between results obtained from participants recruited through AMT compared to the results from the participant recruited on campus or through online forums. We do, however, argue that this difference is so small that it has no practical consequence. There was no significant difference between running the study within AMT compared to running it with an online questionnaire service. There was no significant difference between results obtained directly from within AMT compared to results obtained in the campus and online forum condition. This may suggest that AMT is a viable and economical option for recruiting participants and for conducting studies as setting up and running a study with AMT generally requires less effort and time compared to other frequently used methods. We discuss our findings as well as limitations of using AMT for empirical studies.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Previous research found a statistically significant difference in results between MTurk participants and participants recruited using more traditional methods, but those differences lacked practical significance.
“Previous research found a statistically significant difference in results between MTurk participants and participants recruited using more traditional methods, but those differences lacked practical significance [63].”
From Internalized Racism and Mental Health: The Moderating Role of Collective Racial Self-Esteem (2024) · cites this paper as [63]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., Vehtari, A., & Rubin, D. B. (2013) · doi:10.1201/b16018 · Cited by 27,276 · see citing works
(1963) · Medical Entomology and Zoology · Cited by 15,085 · see citing works
Amazon's Mechanical Turk: A New Source of Inexpensive, Yet High-Quality, Data?
Buhrmester, M., Kwang, T., & Gosling, S. D. (2011) · Perspectives on Psychological Science · doi:10.1177/1745691610393980 · Cited by 9,996 · see citing works
Constants across cultures in the face and emotion.
Ekman, P., & Friesen, W. V. (1971) · Journal of Personality and Social Psychology · doi:10.1037/h0030377 · Cited by 5,509 · see citing works
WinBUGS - A Bayesian modelling framework: Concepts, structure, and extensibility
Lunn, D. J., Thomas, A., Best, N., & Spiegelhalter, D. J. (2000) · Statistics and Computing · doi:10.1023/a:1008929526011 · Cited by 5,372 · see citing works
Evaluating Online Labor Markets for Experimental Research: Amazon.com's Mechanical Turk
Berinsky, A. J., Huber, G. A., & Lenz, G. S. (2012) · Political Analysis · doi:10.1093/pan/mpr057 · Cited by 4,008 · see citing works
Running experiments on Amazon Mechanical Turk
Paolacci, G., Chandler, J., & Ipeirotis, P. G. (2010) · Judgment and Decision Making · doi:10.1017/s1930297500002205 · Cited by 3,767 · see citing works
Conducting behavioral research on Amazon’s Mechanical Turk
Mason, W., & Suri, S. (2012) · Behavior Research Methods · doi:10.3758/s13428-011-0124-6 · Cited by 3,172 · see citing works
Data collection in a flat world: the strengths and weaknesses of Mechanical Turk samples
Goodman, J. K., Cryder, C. E., & Cheema, A. (2013) · Journal of Behavioral Decision Making · doi:10.1002/bdm.1753 · Cited by 2,242 · see citing works
Crowdsourcing user studies with Mechanical Turk
Kittur, A., Ed, H., & Suh, B. (2008) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1357054.1357127 · Cited by 1,941 · see citing works
Talking about tactile experiences
Obrist, M., Seah, S. A., & Subramanian, S. (2013) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2470654.2466220 · Cited by 1,627 · see citing works
The Online Laboratory: Conducting Experiments in a Real Labor Market∗Preprint
Zeckhauser, R. J. (2016) · Cited by 1,454 · see citing works
Who are the crowdworkers?: shifting demographics in mechanical turk
Ross, J., Irani, L., Silberman, M. S., Zaldivar, A., & Tomlinson, B. (2010) · CHI '10 Extended Abstracts on Human Factors in Computing Systems · doi:10.1145/1753846.1753873 · Cited by 1,004 · see citing works
Crowdsourcing graphical perception
Heer, J., & Bostock, M. (2010) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1753326.1753357 · Cited by 819 · see citing works
Rand, D. G. (2011) · Journal of Theoretical Biology · doi:10.1016/j.jtbi.2011.03.004 · Cited by 686 · see citing works
Methods for Evaluating Interactive Information Retrieval Systems with Users
Kelly, D. (2009) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000012 · Cited by 638 · see citing works
The labor economics of paid crowdsourcing
Horton, J. J., & Chilton, L. B. (2010) · ACM Conference on Electronic Commerce (EC) · doi:10.1145/1807342.1807376 · Cited by 618 · see citing works
Financial incentives and the "performance of crowds"
Mason, W., & Watts, D. J. (2009) · doi:10.1145/1600150.1600175 · Cited by 544 · see citing works
Reips, U. (2000) · Elsevier eBooks · doi:10.1016/b978-012099980-4/50005-8 · Cited by 462 · see citing works
The Online Laboratory: Conducting Experiments in a Real Labor Market
Horton, J. J., Rand, D. G., & Zeckhauser, R. (2010) · National Bureau of Economic Research · doi:10.3386/w15961 · Cited by 402 · see citing works
A mixed‐effects multinomial logistic regression model
Hedeker, D. (2003) · Statistics in Medicine · doi:10.1002/sim.1522 · Cited by 373 · see citing works
Vehtari, A. (2002) · Journal of the Royal Statistical Society Series B (Methodological) · Cited by 333 · see citing works
Sue, V. M., & Ritter, L. A. (2008) · Development in Practice · doi:10.1080/09614520802031274 · Cited by 315 · see citing works
Sue, V. M., & Ritter, L. A. (2007) · Sage eBooks · doi:10.4135/9781412983754 · Cited by 287 · see citing works
USING ELECTRONIC MAIL FOR DATA COLLECTION IN ORGANIZATIONAL RESEARCH.
Sproull, L. S. (1986) · Academy of Management Journal · doi:10.2307/255867 · Cited by 237 · see citing works
Technology Trends in Survey Data Collection
Couper, M. P. (2005) · Social Science Computer Review · doi:10.1177/0894439305278972 · Cited by 200 · see citing works
From Mice to Men - 24 Years of Evaluation in CHI
Barkhuus, L., & Rode, J. A. (2007) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1240624.2180963 · Cited by 180 · see citing works
Bayesian inference for categorical data analysis
Agresti, A., & Hitchcock, D. B. (2005) · Stat. Methods Appl. · doi:10.1007/s10260-005-0121-y · Cited by 174 · see citing works
Crowdsourcing performance evaluations of user interfaces
Komarov, S., Reinecke, K., & Gajos, K. Z. (2013) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2470654.2470684 · Cited by 157 · see citing works
Bayesian Measures of Model Complexity and Fit
Spiegelhalter, D. J., Best, N. G., Carlin, B. P., & Van Der Linde, A. (2002) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/1467-9868.00353 · Cited by 151 · see citing works
Hultsch, D. F., MacDonald, S. W. S., Hunter, M. A., Maitland, S. B., & Dixon, R. A. (2002) · International Journal of Behavioral Development · doi:10.1080/01650250143000247 · Cited by 150 · see citing works
Pixels vs. Paper: Comparing Online and Traditional Survey Methods in Sport Psychology
Lonsdale, C., Hodge, K. P., & Rose, E. A. (2006) · Journal of Sport and Exercise Psychology · doi:10.1123/jsep.28.1.100 · Cited by 107 · see citing works
Computer-Assisted Self-Interviewing Tailored for Special Populations and Topics
de Leeuw, E. D., Hox, J. J., & Kef, S. (2003) · Field Methods · doi:10.1177/1525822x03254714 · Cited by 86 · see citing works
The Web Experiment Method: Advantages, disadvantages, and solutions
Reips, U. (2000) · Zurich Open Repository and Archive (University of Zurich) · doi:10.5167/uzh-19760 · Cited by 85 · see citing works
Do Mechanical Turks dream of square pie charts?
Kosara, R., & Ziemkiewicz, C. (2010) · doi:10.1145/2110192.2110202 · Cited by 75 · see citing works
Computerized Questionnaires and the Quality of Survey Data
Hanscom, B. S., Lurie, J. D., Homa, K. A., & Weinstein, J. N. (2002) · Spine · doi:10.1097/00007632-200208150-00020 · Cited by 68 · see citing works
POTTER, T., & CORNEILLE, O. (2008) · Psychonomic Bulletin & Review · doi:10.3758/pbr.15.3.615 · Cited by 58 · see citing works
Questionnaire mode effects in interactive information retrieval experiments
Kelly, D., Harper, D. J., & Landau, B. J. (2007) · Information Processing & Management · doi:10.1016/j.ipm.2007.02.007 · Cited by 56 · see citing works
EXPLORING TASK PROPERTIES IN CROWDSOURCING – AN EMPIRICAL STUDY ON MECHANICAL TURK
Schulze, T., Seedorf, S., Geiger, D., Kaufmann, N., & Schader, M. (2011) · MADOC (University of Mannheim) · Cited by 46 · see citing works
Behavioral Research and Data Collection via the Internet
Birnbaum, M. H., & Reips, U. (2005) · Cited by 42 · see citing works
USING INTERNET-BASED SURVEYS FOR ACADEMIC RESEARCH: OPPORTUNITIES AND PROBLEMS
Roztocki, N. (2001) · Cited by 40 · see citing works
Employee Reactions to Paper and Electronic Surveys: An Experimental Comparison
Croteau, A., Dyer, L., & Miguel, M. (2010) · IEEE Transactions on Professional Communication · doi:10.1109/tpc.2010.2052852 · Cited by 34 · see citing works
Bulmer, M. (2004) · SAGE Publications Ltd eBooks · doi:10.4135/9781446263204 · Cited by 31 · see citing works
Electronic versus paper surveys: analysis of potential psychometric biases
McCoy, S., Marks, P. V., Carr, C. L., & Mbarika, V. (2004) · Hawaii International Conference on System Sciences (HICSS) · doi:10.1109/hicss.2004.1265634 · Cited by 15 · see citing works
The online laboratory: conducting experiments in a real labor market
Horton, J. J., Rand, D. G., & Zeckhauser, R. (2011) · Experimental Economics · doi:10.1007/s10683-011-9273-9 · Cited by 9 · see citing works
Facial caricature generation using a quadratic deformation model
Obaid, M., Lond, D., Mukundan, R., & Billinghurst, M. (2009) · International Conference on Advances in Computer Entertainment Technology (ACE) · doi:10.1145/1690388.1690437 · Cited by 8 · see citing works
Electronic methods of collecting survey data: a review of 'E-Research'
Tuten, T. L. (1997) · Social Science Open Access Repository (GESIS – Leibniz Institute for the Social Sciences) · Cited by 5 · see citing works
Flagged first, then by citations
2 of the 49 references couldn’t be matched to a known paper.
Research works printed but not linked: 2
- [13]Tourangeau R (1984) Cognitive science and survey methods., Washington D.C.: National Academy Press pp. 73– 100.
- [35]Bartneck C, Obaid M, Zawieska K (2013) Agents with faces - what can we learn from lego minfigures. In: 1st International Conference on Human-Agent Interaction. URL http://bartneck.de/publications/2013/agentsWithFaces/bartneckLEGOAgent.pdf.
Cited by
Sorted by citations
Betsch, C., Schmid, P., Heinemeier, D., Korn, L., Holtmann, C., & Böhm, R. (2018) · PLoS ONE · doi:10.1371/journal.pone.0208601 · Cited by 1,276 · see citing works
Kees, J., Berry, C., Burton, S., & Sheehan, K. (2017) · Journal of Advertising · doi:10.1080/00913367.2016.1269304 · Cited by 902 · see citing works
Americans’ COVID-19 stress, coping, and adherence to CDC guidelines
Park, C. L., Russell, B. S., Fendrich, M., Finkelstein-Fox, L., Hutchison, M., & Becker, J. (2020) · Journal of General Internal Medicine · doi:10.1007/s11606-020-05898-9 · Cited by 728 · see citing works
Russell, B. S., Hutchison, M., Tambling, R. R., Tomkunas, A. J., & Horton, A. L. (2020) · Child Psychiatry & Human Development · doi:10.1007/s10578-020-01037-x · Cited by 715 · see citing works
Can we control it? Autonomous robots threaten human identity, uniqueness, safety, and resources
Złotowski, J. A., Yogeeswaran, K., & Bartneck, C. (2016) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2016.12.008 · Cited by 418 · see citing works
Self-citationThe moral standing of animals: Towards a psychology of speciesism.
Caviola, L., Everett, J. A. C., & Faber, N. S. (2018) · Journal of Personality and Social Psychology · doi:10.1037/pspp0000182 · Cited by 409 · see citing works
Attitudes to in vitro meat: A survey of potential consumers in the United States
Wilks, M., & Phillips, C. J. C. (2017) · PLoS ONE · doi:10.1371/journal.pone.0171904 · Cited by 400 · see citing works
Keith, M. G., Tay, L., & Harms, P. D. (2017) · Frontiers in Psychology · doi:10.3389/fpsyg.2017.01359 · Cited by 319 · see citing works
Hahl, O., Kim, M., & Zuckerman, E. W. (2018) · American Sociological Review · doi:10.1177/0003122417749632 · Cited by 305 · see citing works
The use of crowdsourcing in addiction science research: Amazon Mechanical Turk.
Strickland, J. C., & Stoops, W. W. (2018) · Experimental and Clinical Psychopharmacology · doi:10.1037/pha0000235 · Cited by 249 · see citing works
Human-Robot Interaction: An Introduction
Bartneck, C., Belpaeme, T., Eyssel, F., Kanda, T., Keijsers, M., & Šabanović, S. (2020) · doi:10.1017/9781108676649 · Cited by 211 · see citing works
Self-citationRedmiles, E. M., Kross, S., & Mazurek, M. L. (2019) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp.2019.00014 · Cited by 193 · see citing works
Discovering Smart Home Internet of Things Privacy Norms Using Contextual Integrity
Apthorpe, N., Shvartzshnaider, Y., Mathur, A., Reisman, D., & Feamster, N. (2018) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3214262 · Cited by 166 · see citing works
Building, Hosting and Recruiting: A Brief Introduction to Running Behavioral Experiments Online
Sauter, M., Draschkow, D., & Mack, W. (2020) · Brain Sciences · doi:10.3390/brainsci10040251 · Cited by 164 · see citing works
Papadopoulou, N. M., Ribeiro, M. A., & Prayag, G. (2022) · Journal of Travel Research · doi:10.1177/00472875221089049 · Cited by 181 · see citing works
Saravi, B., Hassel, F., Ülkümen, S., Zink, A., Shavlokhova, V., Couillard‐Després, S., Boeker, M., Obid, P., & Lang, G. M. (2022) · Journal of Personalized Medicine · doi:10.3390/jpm12040509 · Cited by 146 · see citing works
Park, C. L., Finkelstein-Fox, L., Russell, B. S., Fendrich, M., Hutchison, M., & Becker, J. (2021) · American Psychologist · doi:10.1037/amp0000813 · Cited by 127 · see citing works
Lovett, M., Bajaba, S. M., Lovett, M., & Simmering, M. J. (2017) · Applied Psychology · doi:10.1111/apps.12124 · Cited by 112 · see citing works
Do ethical concerns and personal values influence the purchase intention of fast-fashion clothing?
Stringer, T., Mortimer, G., & Payne, A. R. (2020) · Journal of Fashion Marketing and Management · doi:10.1108/jfmm-01-2019-0011 · Cited by 120 · see citing works
AI Technologies for Social Emotional Learning: Recent Research and Future Directions
Sethi, S. S., & Jain, K. (2024) · Journal of Research in Innovative Teaching & Learning · doi:10.1108/jrit-03-2024-0073 · Cited by 111 · see citing works
Schneider-Mayerson, M., Gustafson, A., Leiserowitz, A., Goldberg, M. H., Rosenthal, S., & Ballew, M. T. (2020) · Environmental Communication · doi:10.1080/17524032.2020.1814377 · Cited by 103 · see citing works
Time spent online: Latent profile analyses of emerging adults' social media use
Scott, C. F., Bay‐Cheng, L. Y., Prince, M. A., Nochajski, T. H., & Collins, R. L. (2017) · Computers in Human Behavior · doi:10.1016/j.chb.2017.05.026 · Cited by 108 · see citing works
Keijsers, M., & Bartneck, C. (2018) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1145/3171221.3171266 · Cited by 100 · see citing works
Self-citationCompassion for climate change victims and support for mitigation policy
Lu, H., & Schuldt, J. P. (2016) · Journal of Environmental Psychology · doi:10.1016/j.jenvp.2016.01.007 · Cited by 97 · see citing works
Does telling white lies signal pro-social preferences?
Biziou-van-Pol, L., Haenen, J., Novaro, A., Liberman, A. O., & Capraro, V. (2015) · Judgment and Decision Making · doi:10.1017/s1930297500006987 · Cited by 94 · see citing works
Chatbot Advertising As a Double-Edged Sword: The Roles of Regulatory Focus and Privacy Concerns
Kim, W., Ryoo, Y., Lee, S.-Y., & Lee, J. A. (2022) · Journal of Advertising · doi:10.1080/00913367.2022.2043795 · Cited by 92 · see citing works
Tasci, A. D. A. (2017) · Tourism Management · doi:10.1016/j.tourman.2017.09.020 · Cited by 95 · see citing works
Lu, H. (2016) · Science Communication · doi:10.1177/1075547015619173 · Cited by 80 · see citing works
Hahl, O. (2016) · Organization Science · doi:10.1287/orsc.2016.1072 · Cited by 83 · see citing works
Eco-guilt in tourism: Do tourists intend to behave environmentally friendly and still revisit?
Bahja, F., & Hançer, M. (2021) · Journal of Destination Marketing & Management · doi:10.1016/j.jdmm.2021.100602 · Cited by 77 · see citing works
The COVID-19 Vaccine Is Here—Now Who Is Willing to Get It?
Killgore, W. D. S., Cloonan, S., Taylor, E. C., & Dailey, N. S. (2021) · Vaccines · doi:10.3390/vaccines9040339 · Cited by 68 · see citing works
Improving the Nutritional Impact of the Supplemental Nutrition Assistance Program:
Leung, C. W., Musicus, A. A., Willett, W. C., & Rimm, E. B. (2017) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2016.07.024 · Cited by 68 · see citing works
Russell, B. S., Tomkunas, A. J., Hutchison, M., Tambling, R. R., & Horton, A. L. (2021) · Child Psychiatry & Human Development · doi:10.1007/s10578-021-01243-1 · Cited by 69 · see citing works
Dunn, K. E., Barrett, F. S., Yepez-Laubach, C., Meyer, A. C., Hruska, B. J., Sigmon, S. C., Fingerhood, M., & Bigelow, G. E. (2016) · Journal of Addiction Medicine · doi:10.1097/adm.0000000000000235 · Cited by 63 · see citing works
Bartneck, C., Belpaeme, T., Eyssel, F., Kanda, T., Keijsers, M., & Šabanović, S. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108676649.006 · Cited by 61 · see citing works
Self-citationMeasuring Cumulative Stressfulness: Psychometric Properties of the COVID-19 Stressors Scale
Tambling, R. R., Russell, B. S., Park, C. L., Fendrich, M., Hutchinson, M., Horton, A. L., & Tomkunas, A. J. (2020) · Health Education & Behavior · doi:10.1177/1090198120979912 · Cited by 58 · see citing works
Testing Social Science Network Theories with Online Network Data: An Evaluation of External Validity
Bisbee, J., & Larson, J. M. (2017) · American Political Science Review · doi:10.1017/s0003055417000120 · Cited by 32 · see citing works
Public values and goals for public participation
Bidwell, D., & Schweizer, P. (2020) · Environmental Policy and Governance · doi:10.1002/eet.1913 · Cited by 56 · see citing works
Zhou, X., Krishnan, A., & Dincelli, E. (2021) · Behaviour and Information Technology · doi:10.1080/0144929x.2021.1915383 · Cited by 56 · see citing works
Posttraumatic Stress, Posttraumatic Growth, and Satisfaction With Life in Military Veterans
Morgan, J. K., Desmarais, S. L., Mitchell, R. E., & Simons-Rudolph, J. M. (2017) · Military Psychology · doi:10.1037/mil0000182 · Cited by 56 · see citing works
Development and validation of the Attitudes Towards Police Legitimacy Scale.
Reynolds, J. J., Estrada-Reynolds, V. C., & Nunez, N. L. (2018) · Law and Human Behavior · doi:10.1037/lhb0000281 · Cited by 52 · see citing works
The morality of abusing a robot
Bartneck, C., & Keijsers, M. (2020) · Paladyn Journal of Behavioral Robotics · doi:10.1515/pjbr-2020-0017 · Cited by 52 · see citing works
Self-citationThis Website Uses Nudging: MTurk Workers' Behaviour on Cookie Consent Notices
Bermejo, C., Chatzopoulos, D., Papadopoulos, D., & Hui, P. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3476087 · Cited by 49 · see citing works
Scurich, N., & Monahan, J. T. (2015) · Law and Human Behavior · doi:10.1037/lhb0000161 · Cited by 49 · see citing works
van Dijke, M., Gobena, L. B., & Verboon, P. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.01632 · Cited by 45 · see citing works
Challenges in Recruiting U.S. Farmers for Policy‐Relevant Economic Field Experiments
Weigel, C., Paul, L. A., Ferraro, P. J., & Messer, K. D. (2020) · Applied Economic Perspectives and Policy · doi:10.1002/aepp.13066 · Cited by 45 · see citing works
Kim, Y., & Park, H. (2017) · Corporate Reputation Review · doi:10.1057/s41299-017-0016-5 · Cited by 44 · see citing works
Committed Diversification: Why Authenticity Insulates Against Penalties for Diversification
Hahl, O., & Ha, J. (2019) · Organization Science · doi:10.1287/orsc.2019.1317 · Cited by 44 · see citing works
Morris, K. S., Seaton, E. K., Iida, M., & Johnson, S. L. (2020) · Social Sciences · doi:10.3390/socsci9110191 · Cited by 45 · see citing works
Does telling white lies signal pro-social preferences?Preprint
Biziou-van-Pol, L., Haenen, J., Novaro, A., Liberman, A. O., & Capraro, V. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.03973 · Cited by 43 · see citing works
Zhou, X., & Krishnan, A. (2019) · Health Communication · doi:10.1080/10410236.2018.1428851 · Cited by 41 · see citing works
Teaching Robots a Lesson: Determinants of Robot Punishment
Keijsers, M., Kazmi, H., Eyssel, F., & Bartneck, C. (2019) · International Journal of Social Robotics · doi:10.1007/s12369-019-00608-w · Cited by 42 · see citing works
Self-citationKoroleva, K., & Novak, J. (2020) · Sustainability · doi:10.3390/su12020712 · Cited by 47 · see citing works
Consumer demand for sustainability benchmarks in tourism and hospitality
Tasci, A. D. A. (2017) · Tourism Review · doi:10.1108/tr-05-2017-0087 · Cited by 40 · see citing works
Woosnam, K. M., Ribeiro, M. A., Denley, T. J., Hehir, C., & Boley, B. B. (2021) · Journal of Travel Research · doi:10.1177/00472875211025097 · Cited by 38 · see citing works
Lenient gun laws, perceived risk of gun violence, and attitude towards a destination
Tasci, A. D. A., & Sönmez, S. (2019) · Journal of Destination Marketing & Management · doi:10.1016/j.jdmm.2019.03.007 · Cited by 37 · see citing works
O’Neill, J., Kamper‐DeMarco, K. E., Chen, X., & Orom, H. (2020) · Eating Behaviors · doi:10.1016/j.eatbeh.2020.101417 · Cited by 39 · see citing works
Schiff, D. M., Stoltman, J. J. K., Nielsen, T. C., Myers, S. P., Nolan, M., Terplan, M., Patrick, S. W., Wilens, T. E., & Kelly, J. F. (2021) · Journal of Addiction Medicine · doi:10.1097/adm.0000000000000832 · Cited by 37 · see citing works
Schneider-Mayerson, M. (2020) · ISLE Interdisciplinary Studies in Literature and Environment · doi:10.1093/isle/isaa020 · Cited by 36 · see citing works
Short‐term impacts of COVID‐19 on family caregivers: Emotion regulation, coping, and mental health
Russell, B. S., Hutchison, M., Park, C. L., Fendrich, M., & Finkelstein‐Fox, L. (2021) · Journal of Clinical Psychology · doi:10.1002/jclp.23228 · Cited by 34 · see citing works
Crowd control: Effectively utilizing unscreened crowd workers for biomedical data annotation
Cocos, A., Qian, T., Callison-Burch, C., & Masino, A. J. (2017) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2017.04.003 · Cited by 35 · see citing works
Thematic Analysis of Parent–Child Conversations About COVID-19: “Playing It Safe”
Tambling, R. R., Tomkunas, A. J., Russell, B. S., Horton, A. L., & Hutchison, M. (2021) · Journal of Child and Family Studies · doi:10.1007/s10826-020-01889-w · Cited by 34 · see citing works
Detecting the corruption of online questionnaires by artificial intelligence
Lebrun, B., Temtsin, S., Vonasch, A. J., & Bartneck, C. (2024) · Frontiers in Robotics and AI · doi:10.3389/frobt.2023.1277635 · Cited by 34 · see citing works
Self-citationExploring gamified persuasive system design for energy saving
Böckle, M., Novak, J., & Bick, M. (2020) · Journal of Enterprise Information Management · doi:10.1108/jeim-02-2019-0032 · Cited by 39 · see citing works
An Integrative Theoretical Model of Public Support for Ex-Offender Reentry
Rade, C. B., Desmarais, S. L., & Burnette, J. L. (2017) · International journal of offender therapy and comparative criminology · doi:10.1177/0306624x17714110 · Cited by 30 · see citing works
The psychological implications of believing that everything is one
Diebels, K. J., & Leary, M. R. (2018) · The Journal of Positive Psychology · doi:10.1080/17439760.2018.1484939 · Cited by 32 · see citing works
Feezell, J. T., Glazier, R. A., & Boydstun, A. E. (2019) · Politics Groups and Identities · doi:10.1080/21565503.2019.1584751 · Cited by 31 · see citing works
Embrace the Moment Using Social Media: A Cross-Cultural Study of Mindful Use of Social Media
Shabahang, R., Zsila, Á., Aruguete, M. S., Huynh, H. P., & Orosz, G. (2024) · Mindfulness · doi:10.1007/s12671-023-02271-9 · Cited by 33 · see citing works
Kolp, H. M., Wilder, S. M. J., Andersen, C. V. S., Johnson, E. E. H., Horvath, S. A., Gidycz, C. A., & Shorey, R. C. (2019) · Journal of Clinical Psychology · doi:10.1002/jclp.22880 · Cited by 31 · see citing works
Antecedents of leaders' power sharing: The roles of power instability and distrust
Feenstra, S., Jordan, J., Walter, F. H., & Stoker, J. I. (2020) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2020.01.005 · Cited by 30 · see citing works
Park, C. L., Wilt, J. A., Russell, B. S., & Fendrich, M. (2021) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2021.12.040 · Cited by 31 · see citing works
Whelan, E., Laato, S., Islam, A. K. M. N., & Billieux, J. (2021) · PLoS ONE · doi:10.1371/journal.pone.0246432 · Cited by 29 · see citing works
Dunbar, A., & Hanink, P. A. (2022) · Journal of Experimental Criminology · doi:10.1007/s11292-021-09499-2 · Cited by 29 · see citing works
Impact of User Characteristics on Attitudes Towards Automatic Mobile Application Updates.
Mathur, A., & Chetty, M. (2017) · Symposium On Usable Privacy and Security · Cited by 29 · see citing works
Robots Can Be More Than Black And White
Addison, A., Bartneck, C., & Yogeeswaran, K. (2019) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3306618.3314272 · Cited by 29 · see citing works
Self-citationSurgical Scar Location Preference for Pediatric Kidney and Pelvic Surgery: A Crowdsourced Survey
Garcia-Roig, M. L., Travers, C. D., McCracken, C. M., Cerwinka, W. H., Kirsch, J. M., & Kirsch, A. J. (2016) · The Journal of Urology · doi:10.1016/j.juro.2016.11.033 · Cited by 28 · see citing works
Can You Trust the Black Box? The Effect of Personality Traits on Trust in AI-Enabled User Interfaces
Böckle, M., Yeboah-Antwi, K., & Kouris, I. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-77772-2_1 · Cited by 27 · see citing works
When Is Neglect, Neglect?: It Depends on Who You Ask
Dickerson, K. L., Lindner, S., Scurich, N., & Quas, J. A. (2017) · Child Maltreatment · doi:10.1177/1077559517709558 · Cited by 27 · see citing works
Porter, N. D., Verdery, A. M., & Gaddis, S. M. (2020) · PLoS ONE · doi:10.1371/journal.pone.0233154 · Cited by 26 · see citing works
The Doors of Social Robot Perception: The Influence of Implicit Self-theories
Allan, D. D., Vonasch, A. J., & Bartneck, C. (2021) · International Journal of Social Robotics · doi:10.1007/s12369-021-00767-9 · Cited by 25 · see citing works
Self-citationPersonalized Security Messaging: Nudges for Compliance with Browser Warnings
Malkin, N., Mathur, A., Harbach, M., & Egelman, S. (2017) · European Symposium on Usable Security (EuroUSEC) · doi:10.14722/eurousec.2017.23008 · Cited by 26 · see citing works
Dunbar, A., & Kubrin, C. E. (2018) · Journal of Experimental Criminology · doi:10.1007/s11292-018-9342-6 · Cited by 24 · see citing works
Martinez-Berman, L., McCutcheon, L. E., & Huynh, H. P. (2020) · Psychology Health & Medicine · doi:10.1080/13548506.2020.1778754 · Cited by 24 · see citing works
Associations Between Time Since Event and Posttraumatic Growth Among Military Veterans
Morgan, J. K., & Desmarais, S. L. (2017) · Military Psychology · doi:10.1037/mil0000170 · Cited by 26 · see citing works
Country image and destination image of Brazil in relation to information sources
Hahm, J. J., & Tasci, A. D. A. (2019) · Journal of Hospitality and Tourism Insights · doi:10.1108/jhti-04-2019-0057 · Cited by 24 · see citing works
Fendrich, M., Becker, J., Park, C. L., Russell, B. S., Finkelstein‐Fox, L., & Hutchison, M. (2021) · Substance Abuse · doi:10.1080/08897077.2021.1891603 · Cited by 23 · see citing works
Tauscher, J. S., DePue, M. K., Swank, J. M., & Salloum, R. G. (2023) · Journal of Substance Use and Addiction Treatment · doi:10.1016/j.josat.2022.208938 · Cited by 22 · see citing works
Antecedents of mystical experience and dread in intensive meditation.
Russ, S. L., & Elliott, M. S. (2017) · Psychology of Consciousness Theory Research and Practice · doi:10.1037/cns0000119 · Cited by 23 · see citing works
Krishnan, A., & Zhou, X. (2019) · Journal of Applied Communication Research · doi:10.1080/00909882.2019.1654124 · Cited by 23 · see citing works
Song, H.-Q., Ma, E., & Cheng, M.-M. (2021) · Current Issues in Tourism · doi:10.1080/13683500.2021.1940108 · Cited by 23 · see citing works
Keijsers, M., Bartneck, C., & Eyssel, F. (2021) · International Journal of Social Robotics · doi:10.1007/s12369-021-00799-1 · Cited by 22 · see citing works
Self-citationThe Zoomorphic Miro Robot’s Affective Expression Design and Perceived Appearance
Ghafurian, M., Lakatos, G., & Dautenhahn, K. (2022) · International Journal of Social Robotics · doi:10.1007/s12369-021-00832-3 · Cited by 22 · see citing works
Hanink, P. A., & Dunbar, A. (2022) · Social movement studies · doi:10.1080/14742837.2022.2067842 · Cited by 23 · see citing works
Guillén, L., Jacquart, P., & Hogg, M. A. (2022) · Personality and Social Psychology Bulletin · doi:10.1177/01461672221086771 · Cited by 22 · see citing works
Većkalov, B., Gligorić, V., & Petrović, M. (2024) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2023.104549 · Cited by 14 · see citing works
Hoffmann, D. A., Marx, J. M., Burmeister, J. M., & Musher‐Eizenman, D. R. (2018) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph15040720 · Cited by 21 · see citing works
Godovykh, M., Milman, A., & Tasci, A. D. A. (2019) · Journal of Tourism and Leisure Studies · doi:10.18848/2470-9336/cgp/v04i02/1-21 · Cited by 21 · see citing works
Berryessa, C. M. (2020) · Behavioral Sciences & the Law · doi:10.1002/bsl.2476 · Cited by 22 · see citing works
Algorithmic and HCI Aspects for Explaining Recommendations of Artistic Images
Dominguez, V., Donoso-Guzmán, I., Messina, P., & Parra, D. (2020) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3369396 · Cited by 21 · see citing works
Adkins, A., Normoyle, A., Lin, L., Sun, Y., Ye, Y., Luca, M. D., & Jörg, S. (2023) · ACM Transactions on Graphics · doi:10.1145/3578575 · Cited by 20 · see citing works
- Authors
- Bartneck, C., Duenser, A., Moltchanova, E., & Zawieska, K.
- Affiliations
- University of Canterbury (NZ)
- Commonwealth Scientific and Industrial Research Organisation (AU)
- Łukasiewicz Research Network - Industrial Research Institute for Automation and Measurements (PL)
- Corresponding author
- Bartneck, Christoph
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2015-04-14
- Volume
- 10
- Issue
- 4
- Pages
- e0121595
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC4397064
- OpenAlex
- W2138247806
- Cited by
- 317
- References
- 49
- Citations by year
- 2026: 5 · 2025: 19 · 2024: 22 · 2023: 21 · 2022: 31 · 2021: 43 · 2020: 59 · 2019: 33 · 2018: 28 · 2017: 29 · 2016: 19 · 2015: 7
- FWCI
- 41.51
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- UC Research Repository (University of Canterbury) (submittedVersion, cc-by)
- Topics
- Physical Sciences › Computer Science › Computer Science Applications › Mobile Crowdsensing and Crowdsourcing
- Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
- Physical Sciences › Computer Science › Information Systems › Expert finding and Q&A systems
- Keywords
- Amazon Mechanical Turk, participant recruitment, online surveys, response similarity, online forums, crowdsourcing
- MeSH terms
- Culture, Emotions, Employment, Facial Expression, Humans, Judgment, New Zealand, Personnel Selection, Surveys and Questionnaires, Research Design, Universities, Internet, Remuneration, Crowdsourcing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
