Improving Data Quality Using Amazon Mechanical Turk Through Platform Setup
Abstract
As the use of Amazon’s Mechanical Turk (MTurk) has increased among social science researchers, so, too, has research into the merits and drawbacks of the platform. However, while many endeavors have sought to address issues such as generalizability, the attentiveness of workers, and the quality of the associated data, there has been relatively less effort concentrated on integrating the various strategies that can be used to generate high-quality data using MTurk samples. Accordingly, the purpose of this research is twofold. First, existing studies are integrated into a set of strategies/best practices that can be used to maximize MTurk data quality. Second, focusing on task setup, selected platform-level strategies that have received relatively less attention in previous research are empirically tested to further enhance the contribution of the proposed best practices for MTurk usage.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Research has shown that data from MTurk workers is comparable to data from other pools (e.g., commercial panels, social media, colleges), but recently there has been a decrease in data quality.
“Research has shown data from MTurk workers is comparable to data from other pools (e.g., commercial panels, social media, colleges) [12, 20, 23, 32, 69, 80], but recently there has been a decrease in data quality [28, 49, 66, 68, 104, 108].”
From "I'm Not Sure, But...": Examining the Impact of Large Language Models' Uncertainty Expression on User Reliance and Trust (2024) · cites this paper as [66]
Cites
Podsakoff, P. M., MacKenzie, S. B., Lee, J.-Y., & Podsakoff, N. P. (2003) · Journal of Applied Psychology · doi:10.1037/0021-9010.88.5.879 · Cited by 82,194 · see citing works
Amazon's Mechanical Turk: A New Source of Inexpensive, Yet High-Quality, Data?
Buhrmester, M., Kwang, T., & Gosling, S. D. (2011) · Perspectives on Psychological Science · doi:10.1177/1745691610393980 · Cited by 9,996 · see citing works
Evaluating Online Labor Markets for Experimental Research: Amazon.com's Mechanical Turk
Berinsky, A. J., Huber, G. A., & Lenz, G. S. (2012) · Political Analysis · doi:10.1093/pan/mpr057 · Cited by 4,008 · see citing works
Running experiments on Amazon Mechanical Turk
Paolacci, G., Chandler, J., & Ipeirotis, P. G. (2010) · Judgment and Decision Making · doi:10.1017/s1930297500002205 · Cited by 3,767 · see citing works
Instructional manipulation checks: Detecting satisficing to increase statistical power
Oppenheimer, D. M., Meyvis, T., & Davidenko, N. (2009) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2009.03.009 · Cited by 3,493 · see citing works
Data collection in a flat world: the strengths and weaknesses of Mechanical Turk samples
Goodman, J. K., Cryder, C. E., & Cheema, A. (2013) · Journal of Behavioral Decision Making · doi:10.1002/bdm.1753 · Cited by 2,242 · see citing works
At what sample size do correlations stabilize?
Schönbrodt, F. D., & Perugini, M. (2013) · Journal of Research in Personality · doi:10.1016/j.jrp.2013.05.009 · Cited by 2,232 · see citing works
Inside the Turk: Understanding Mechanical Turk as a Participant Pool
Paolacci, G., & Chandler, J. (2014) · Current Directions in Psychological Science · doi:10.1177/0963721414531598 · Cited by 2,136 · see citing works
Crowdsourcing user studies with Mechanical Turk
Kittur, A., Ed, H., & Suh, B. (2008) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1357054.1357127 · Cited by 1,941 · see citing works
Hauser, D. J., & Schwarz, N. (2016) · Behavior Research Methods · doi:10.3758/s13428-015-0578-z · Cited by 1,739 · see citing works
Reputation as a sufficient condition for data quality on Amazon Mechanical Turk
Peer, E., Vosgerau, J., & Acquisti, A. (2014) · Behavior Research Methods · doi:10.3758/s13428-013-0434-y · Cited by 1,618 · see citing works
The Generalizability of Survey Experiments
Mullinix, K. J., Leeper, T. J., Druckman, J. N., & Jeremy Freese, A. (2015) · Journal of Experimental Political Science · doi:10.1017/xps.2015.19 · Cited by 1,235 · see citing works
Conducting Clinical Research Using Crowdsourced Convenience Samples
Chandler, J., & Shapiro, D. (2016) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-021815-093623 · Cited by 1,072 · see citing works
An MTurk Crisis? Shifts in Data Quality and the Impact on Study Results
Chmielewski, M., & Kucker, S. C. (2020) · Social Psychological and Personality Science · doi:10.1177/1948550619875149 · Cited by 1,027 · see citing works
Kees, J., Berry, C., Burton, S., & Sheehan, K. (2017) · Journal of Advertising · doi:10.1080/00913367.2016.1269304 · Cited by 902 · see citing works
An Evaluation of Amazon’s Mechanical Turk, Its Rapid Rise, and Its Effective Use
Buhrmester, M. D., Talaifar, S., & Gosling, S. D. (2018) · Perspectives on Psychological Science · doi:10.1177/1745691617706516 · Cited by 764 · see citing works
Validity and Mechanical Turk: An assessment of exclusion methods and interactive experiments
Thomas, K. A., & Clifford, S. (2017) · Computers in Human Behavior · doi:10.1016/j.chb.2017.08.038 · Cited by 625 · see citing works
Crowdsourcing Consumer Research
Goodman, J. K., & Paolacci, G. (2017) · Journal of Consumer Research · doi:10.1093/jcr/ucx047 · Cited by 597 · see citing works
Satisfaction with service recovery: Perceived justice and emotional responses
Lanza, A. B. D. R., Vázquez‐Casielles, R., & Díaz‐Martín, A. M. (2008) · Journal of Business Research · doi:10.1016/j.jbusres.2008.09.015 · Cited by 506 · see citing works
The shape of and solutions to the MTurk quality crisis
Kennedy, R., Clifford, S., Burleigh, T., Waggoner, P. D., Jewell, R., & Winter, N. J. G. (2020) · Political Science Research and Methods · doi:10.1017/psrm.2020.6 · Cited by 506 · see citing works
A Data-Driven Analysis of Workers' Earnings on Amazon Mechanical Turk
Hara, K., Adams, A., Milland, K., Savage, S., Callison-Burch, C., & Bigham, J. P. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174023 · Cited by 497 · see citing works
Zhou, H., & Ayelet Fishbach, B. (2016) · Journal of Personality and Social Psychology · doi:10.1037/pspa0000056 · Cited by 462 · see citing works
Smith, S. M., Roster, C. A., Golden, L. L., & Albaum, G. (2016) · Journal of Business Research · doi:10.1016/j.jbusres.2015.12.002 · Cited by 415 · see citing works
A reliability analysis of Mechanical Turk data
Rouse, S. V. (2014) · Computers in Human Behavior · doi:10.1016/j.chb.2014.11.004 · Cited by 414 · see citing works
MTurk Character Misrepresentation: Assessment and Solutions
Sharpe Wessling, K., Huber, J., & Netzer, O. (2017) · Journal of Consumer Research · doi:10.1093/jcr/ucx053 · Cited by 330 · see citing works
Common method bias in hospitality research: A critical review of literature and an empirical study
Min, H. K., Park, J., & Kim, H. J. (2016) · International Journal of Hospitality Management · doi:10.1016/j.ijhm.2016.04.010 · Cited by 281 · see citing works
A convenient solution: Using MTurk to sample from hard-to-reach populations
Smith, N. A., Sabat, I. E., Martinez, L. R., Weaver, K., & Xu, S. (2015) · Industrial and Organizational Psychology · doi:10.1017/iop.2015.29 · Cited by 190 · see citing works
Deng, X. N., & Joshi, K. D. (2016) · Journal of the Association for Information Systems · doi:10.17705/1jais.00441 · Cited by 175 · see citing works
Common Concerns with MTurk as a Participant Pool: Evidence and SolutionsPreprint
Hauser, D., Paolacci, G., & Chandler, J. (2018) · doi:10.31234/osf.io/uq45c · Cited by 156 · see citing works
Consumer Awareness and Use of Product Review Websites
Bailey, A. A. (2005) · Journal of Interactive Advertising · doi:10.1080/15252019.2005.10722109 · Cited by 147 · see citing works
The Role of Perceived Control in Customer Value Cocreation and Service Recovery Evaluation
Guo, L., Lotz, S. L., Tang, C., & Gruen, T. W. (2015) · Journal of Service Research · doi:10.1177/1094670515597213 · Cited by 134 · see citing works
Social desirability bias and self-reports of motivation
Antin, J., & Shaw, A. (2012) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2207676.2208699 · Cited by 125 · see citing works
The Turker Blues: Hidden Factors Behind Increased Depression Rates Among Amazon’s Mechanical Turkers
Ophir, Y., Sisso, I., Asterhan, C. S. C., Tikochinski, R., & Reichart, R. (2019) · Clinical Psychological Science · doi:10.1177/2167702619865973 · Cited by 124 · see citing works
Casey, L. S., Chandler, J., Levine, A. S., Proctor, A., & Strolovitch, D. Z. (2017) · SAGE Open · doi:10.1177/2158244017712774 · Cited by 122 · see citing works
Amazon's Mechanical Turk: A Comment
Ford, J. B. (2017) · Journal of Advertising · doi:10.1080/00913367.2016.1277380 · Cited by 112 · see citing works
Lovett, M., Bajaba, S. M., Lovett, M., & Simmering, M. J. (2017) · Applied Psychology · doi:10.1111/apps.12124 · Cited by 112 · see citing works
Caution! MTurk Workers Ahead—Fines Doubled
Harms, P. D., & DeSimone, J. A. (2015) · Industrial and Organizational Psychology · doi:10.1017/iop.2015.23 · Cited by 109 · see citing works
Arechar, A. A., Kraft‐Todd, G., & Rand, D. G. (2017) · Journal of the Economic Science Association · doi:10.1007/s40881-017-0035-0 · Cited by 90 · see citing works
Loneliness among students with special educational needs in mainstream seventh grade
Bossaert, G., Colpin, H., Pijl, S. J., & Petry, K. (2012) · Research in Developmental Disabilities · doi:10.1016/j.ridd.2012.05.010 · Cited by 84 · see citing works
Social Reward Questionnaire (SRQ): development and validation
Foulkes, L., Viding, E. M., McCrory, E. J. P., & Neumann, C. S. (2014) · Frontiers in Psychology · doi:10.3389/fpsyg.2014.00201 · Cited by 73 · see citing works
From critical design to critical infrastructure
Irani, L. C., & Silberman, M. ‘. (2014) · interactions · doi:10.1145/2627392 · Cited by 60 · see citing works
The Customer Isn’t Always Right: The Implications of Illegitimate Complaints
Kim, K., & Baker, M. A. (2019) · Cornell Hospitality Quarterly · doi:10.1177/1938965519889288 · Cited by 48 · see citing works
Madera, J. M., Taylor, D. C., & Barber, N. (2019) · Cornell Hospitality Quarterly · doi:10.1177/1938965519882315 · Cited by 41 · see citing works
Diversity and Inclusion in Organizations
Stone, D. L., Dulebohn, J. H., & Lukaszewski, K. M. (2020) · Emerald Group Publishing Limited eBooks · doi:10.1108/978-1-64802-006-3 · Cited by 17 · see citing works
Journal of the Midwest Association for Information Systems
(2019) · Journal of the Midwest Association for Information Systems · doi:10.17705/3jmwa · Cited by 13 · see citing works
Cheung, H. K., Hebl, M. R., King, E. B., Markell, H. M., Moreno, C. J., & Nittrouer, C. (2017) · Social Psychological and Personality Science · doi:10.1177/1948550617709113 · Cited by 12 · see citing works
Healthy Taste of High Status: Signaling Status at Restaurants
Shin, J., & Mattila, A. (2020) · Cornell Hospitality Quarterly · doi:10.1177/1938965519877807 · Cited by 10 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Customers’ acceptance of artificially intelligent service robots: The influence of trust and culture
Chi, O. H., Chi, C. G.-Q., Gürsoy, D., & Nunkoo, R. (2023) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2023.102623 · Cited by 293 · see citing works
Comparing attentional disengagement between prolific and MTurk samples
Albert, D. A., & Smilek, D. (2023) · Scientific Reports · doi:10.1038/s41598-023-46048-5 · Cited by 166 · see citing works
Kim, S. S. Y., Liao, Q. V., Vorvoreanu, M., Ballard, S., & Vaughan, J. W. (2024) · The 2024 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3630106.3658941 · Cited by 147 · see citing works
Customers’ perceptions of hotel AI-enabled voice assistants: does brand matter?
Cai, R., Cain, L., & Jeon, H. (2022) · International Journal of Contemporary Hospitality Management · doi:10.1108/ijchm-10-2021-1313 · Cited by 139 · see citing works
Effects of AI ChatGPT on travelers’ travel decision-making
Kim, J. H., Kim, J., Kim, S., & Hailu, T. B. (2023) · Tourism Review · doi:10.1108/tr-07-2023-0489 · Cited by 109 · see citing works
Parvez, M. O., Öztüren, A., Çobanoğlu, C., Araslı, H., & Eluwole, K. K. (2022) · International Journal of Hospitality Management · doi:10.1016/j.ijhm.2022.103336 · Cited by 110 · see citing works
Cai, R., Leung, X. Y., & Chi, C. G.-Q. (2021) · International Journal of Hospitality Management · doi:10.1016/j.ijhm.2021.103110 · Cited by 106 · see citing works
Narrowing the intention-behavior gap: The impact of hotel green certification
Chi, C. G.-Q., Chi, O. H., Xu, X., & Kennedy, I. (2022) · International Journal of Hospitality Management · doi:10.1016/j.ijhm.2022.103305 · Cited by 88 · see citing works
Nostalgic emotions, meaning in life, subjective well-being and revisit intentions
Karagöz, D., & Ramkissoon, H. R. (2023) · Tourism Management Perspectives · doi:10.1016/j.tmp.2023.101159 · Cited by 56 · see citing works
Kim, H., So, K. K. F., & Mihalik, B. J. (2021) · International Journal of Hospitality Management · doi:10.1016/j.ijhm.2021.103086 · Cited by 48 · see citing works
The Burden for High-Quality Online Data Collection Lies With Researchers, Not Recruitment Platforms
Cuskley, C. F., & Sulik, J. (2024) · Perspectives on Psychological Science · doi:10.1177/17456916241242734 · Cited by 48 · see citing works
Cai, R., Wang, Y., & Zhang, T. C. (2024) · International Journal of Contemporary Hospitality Management · doi:10.1108/ijchm-08-2023-1130 · Cited by 40 · see citing works
Wu, A., & Wei, W. (2024) · International Journal of Hospitality Management · doi:10.1016/j.ijhm.2023.103681 · Cited by 40 · see citing works
Foodstagramming as a self-presentational behavior: perspectives of tourists and residents
Lin, B.-N., Fu, X.-X., & Lu, L. (2022) · International Journal of Contemporary Hospitality Management · doi:10.1108/ijchm-01-2022-0042 · Cited by 29 · see citing works
Self-citationHorng, J., Liu, C.-H. S., Chou, S.-F., Yu, T., & Hu, D.-C. (2024) · Journal of Hospitality Marketing & Management · doi:10.1080/19368623.2024.2315947 · Cited by 34 · see citing works
Kim, J., Jhang, J., Kim, S., & Stylidis, D. (2022) · Journal of Travel Research · doi:10.1177/00472875221129255 · Cited by 30 · see citing works
Nanu, L., & Rahman, I. (2023) · International Journal of Hospitality Management · doi:10.1016/j.ijhm.2023.103520 · Cited by 28 · see citing works
What over 1,000,000 participants tell us about online research protocols
Tomczak, J., Gordon, A., Adams, J., Pickering, J. S., Hodges, N., & Evershed, J. K. (2023) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2023.1228365 · Cited by 25 · see citing works
Hyun, H., Hong, E., Ahn, J., & Park, J. (2024) · Journal of Travel & Tourism Marketing · doi:10.1080/10548408.2023.2239837 · Cited by 25 · see citing works
Shin, H. H., Tung, V. W. S., & Jeong, M. (2024) · Journal of Hospitality & Tourism Research · doi:10.1177/10963480241231469 · Cited by 22 · see citing works
Gamification preferences in nutrition apps: Toward healthier diets and food choices
Berger, M., & Jung, C. (2024) · Digital Health · doi:10.1177/20552076241260482 · Cited by 21 · see citing works
Robertson, N., Rotman, J. D., McQuilken, L., & Ringer, A. (2023) · Psychology & Marketing · doi:10.1002/mar.21789 · Cited by 18 · see citing works
Assigning credit and blame: How delivery services affect restaurant EWOM
Line, N. D., Hanks, L., Doğru, T., & Lu, L. (2023) · International Journal of Hospitality Management · doi:10.1016/j.ijhm.2023.103644 · Cited by 19 · see citing works
Self-citationWhen green meets gamification: A winning combo for hotel revisit intentions
Wu, A., Kang, J., Kwun, D. J., & Wei, W. (2024) · International Journal of Hospitality Management · doi:10.1016/j.ijhm.2024.103734 · Cited by 19 · see citing works
Deng, D. S., Seo, S., Harrington, R. J., & MARTIN, D. J. (2024) · International Journal of Wine Business Research · doi:10.1108/ijwbr-04-2024-0019 · Cited by 20 · see citing works
Malibari, M., Bajaba, S. M., Bajaba, A. M., & Basahal, A. S. (2025) · Journal of Innovation & Knowledge · doi:10.1016/j.jik.2025.100701 · Cited by 19 · see citing works
Zelaya, D. G., Rosales, R., García, J. J., Moreno, O., Figuereo, V. J., Kahler, C. W., & DeBlaere, C. (2023) · The American Journal of Drug and Alcohol Abuse · doi:10.1080/00952990.2023.2176235 · Cited by 16 · see citing works
Gamification to Improve Medication Adherence: A Mixed-method Usability Study for MedScrab
Yan, L., Phan, H., Law, A. V., Roosan, M. R., Karim, M., Baskys, A., & Roosan, D. (2023) · Journal of Medical Systems · doi:10.1007/s10916-023-02006-2 · Cited by 16 · see citing works
Why do Experiments Fail? Six Practical Suggestions for Successful Online Experiments
Kim, J. (2023) · Journal of Travel Research · doi:10.1177/00472875231209983 · Cited by 16 · see citing works
Kim, M., & Jang, J. (2022) · Journal of Service Theory and Practice · doi:10.1108/jstp-01-2022-0018 · Cited by 17 · see citing works
Employing the houseless as corporate social responsibility
Smith, N. A., Martinez, L. R., Xu, S. T., Mattila, A. S., & Gao, Y. L. (2023) · International Journal of Contemporary Hospitality Management · doi:10.1108/ijchm-05-2022-0649 · Cited by 14 · see citing works
Negative messages from parents and sisters and Latina college students’ body image shame
Rivero, A., Killoren, S. E., Kline, G. C., & Campione‐Barr, N. (2022) · Body Image · doi:10.1016/j.bodyim.2022.05.011 · Cited by 13 · see citing works
Kim, J., & Moon, J. (2025) · SAGE Open · doi:10.1177/21582440251320173 · Cited by 18 · see citing works
Organizational responses to online employee reviews: A mixed-method research
Zhang, Y. C., Shum, C. W. S., & Belarmino, A. M. (2024) · International Journal of Hospitality Management · doi:10.1016/j.ijhm.2024.104003 · Cited by 11 · see citing works
Medeiros, M., Ozturk, A. B., Okumuş, B., Hançer, M., & Weinland, J. T. (2024) · Journal of Hospitality and Tourism Technology · doi:10.1108/jhtt-07-2023-0208 · Cited by 11 · see citing works
A Behavioral Economic Analysis of Carbon-Neutral Home Energy Enrollment
Gelino, B. W., Kaplan, B. A., & Reed, D. D. (2023) · Behavior and Social Issues · doi:10.1007/s42822-023-00143-4 · Cited by 10 · see citing works
Mistry, T. G., Cain, L., Doğan, S., & Songur, A. G. (2024) · International Journal of Hospitality Management · doi:10.1016/j.ijhm.2024.103962 · Cited by 10 · see citing works
Jang, J., Kang, J., & Huan, C. (2024) · Journal of Product & Brand Management · doi:10.1108/jpbm-09-2023-4704 · Cited by 10 · see citing works
Assessing the quality and reliability of the Amazon Mechanical Turk (MTurk) data in 2024
Shimoni, H., & Axelrod, V. (2025) · Royal Society Open Science · doi:10.1098/rsos.250361 · Cited by 10 · see citing works
The Role of Awe on Booking Intention in Short-Term Rentals
Sun, Z., Yuan, M., Zhang, J., Kou, S., & Hou, Y. (2023) · Journal of Hospitality & Tourism Research · doi:10.1177/10963480231179996 · Cited by 9 · see citing works
Rumination in dementia and its relationship with depression, anxiety, and attentional biases
Greenaway, A.-M., Hwang, F., Nasuto, S. J., & Ho, A. K. (2024) · Aging Neuropsychology and Cognition · doi:10.1080/13825585.2024.2327679 · Cited by 8 · see citing works
Péer, E., Rothschild, D. M., & Gordon, A. (2023) · doi:10.31234/osf.io/buzwn · Cited by 8 · see citing works
Consumer biases in the perception of organizational greed
Arango, L., Singaraju, S., Niininen, O., & D’Souza, C. (2022) · International Journal of Consumer Studies · doi:10.1111/ijcs.12870 · Cited by 8 · see citing works
Hospitality marketing research: bridging the practical gap
Bilgihan, A., Hanks, L., Line, N. D., & Mody, M. A. (2024) · International Journal of Contemporary Hospitality Management · doi:10.1108/ijchm-09-2023-1351 · Cited by 7 · see citing works
Self-citationToloka platform as a source of online survey participants: an experience of assessing data quality
Gavrilov, K. (2022) · Sociology methodology methods mathematical modeling (Sociology 4M) · doi:10.19181/4m.2021.53.5 · Cited by 6 · see citing works
Fear or humor? The effects of negatively framed visual hyperbole in advertising
Huang, Y. T. (2023) · International Journal of Advertising · doi:10.1080/02650487.2023.2262325 · Cited by 6 · see citing works
Kim, J., & Kim, M.-R. (2024) · The Journal of Korean Association of Computer Education · doi:10.32431/kace.2024.27.3.007 · Cited by 7 · see citing works
Kim, J., Jhang, J., Kim, S., Cho, A., & Kim, J. H. (2025) · International Journal of Hospitality Management · doi:10.1016/j.ijhm.2025.104173 · Cited by 5 · see citing works
Moral repair versus punishment: influences on managerial responses
Butterfield, K. D., Neale, N. R., Shin, E., & He, M. (2023) · Organization Management Journal · doi:10.1108/omj-11-2021-1398 · Cited by 5 · see citing works
Self-citationKang, Y. A., Legendre, T. S., Lee, S. A., & Baker, M. A. (2023) · Cornell Hospitality Quarterly · doi:10.1177/19389655231214706 · Cited by 5 · see citing works
Better than Us: The Role of Implicit Self-Theories in Determining Perceived Threat Responses in HRI
Allan, D. D., Vonasch, A. J., & Bartneck, C. (2022) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1109/hri53351.2022.9889520 · Cited by 5 · see citing works
M-StruGAN: An Automatic 2D-Plan Generation System under Mixed Structural Constraints for Homestays
Gao, X., Guo, X., & Lo, T. T. (2023) · Sustainability · doi:10.3390/su15097126 · Cited by 5 · see citing works
Delemere, E., & Liston, P. M. (2024) · Energy Reports · doi:10.1016/j.egyr.2024.05.042 · Cited by 6 · see citing works
Intersection of machine learning and mobile crowdsourcing: a systematic topic-driven review
Chen, W., Hussain, W., Al-Qudah, I., Al‐Naymat, G., & Zhang, X. (2024) · Personal and Ubiquitous Computing · doi:10.1007/s00779-024-01820-w · Cited by 5 · see citing works
Nugraha, T. F., Wibowo, W. S., Genia, V., Fadhil, A., & Ruldeviyani, Y. (2024) · Journal of Information Systems Engineering and Business Intelligence · doi:10.20473/jisebi.10.1.51-69 · Cited by 10 · see citing works
Chagas-Bastos, F. H. (2025) · Cooperation and Conflict · doi:10.1177/00108367241313302 · Cited by 4 · see citing works
Preference reversals in ethicality judgments of medical treatments
Lemli, B. A., & Landy, J. F. (2025) · PLoS ONE · doi:10.1371/journal.pone.0319233 · Cited by 4 · see citing works
The Effect of CAPTCHA Exposure on Traveler's Online Behaviors
Kim, J., Park, S.-Y. I., & Jhang, J. (2022) · Journal of Smart Tourism · doi:10.52255/smarttourism.2022.2.4.2 · Cited by 4 · see citing works
Xu, Y., & Jeong, E. (2025) · International Journal of Hospitality Management · doi:10.1016/j.ijhm.2025.104157 · Cited by 3 · see citing works
Cao, C., Lv, J. H., Xu, H., & Zhang, X. (2025) · Behaviour and Information Technology · doi:10.1080/0144929x.2025.2511736 · Cited by 3 · see citing works
Wu, A., Wei, W., & Zhang, L. (2025) · Journal of Hospitality and Tourism Management · doi:10.1016/j.jhtm.2025.05.011 · Cited by 3 · see citing works
Almeida, I. M., Slavish, D. C., Rafiuddin, H. S., & Contractor, A. A. (2022) · Stress and Health · doi:10.1002/smi.3184 · Cited by 2 · see citing works
The relation between self-reported healthy living and attentional engagement in everyday life
Kruger, T. B., Marty-Dugas, J., Ralph, B. C. W., Dixon, M. J., & Smilek, D. (2022) · Current Research in Behavioral Sciences · doi:10.1016/j.crbeha.2022.100086 · Cited by 3 · see citing works
Borah, P., Xiao, X., Cao, X., & Vishnevskaya, A. (2024) · Communication Quarterly · doi:10.1080/01463373.2024.2302469 · Cited by 2 · see citing works
Transforming spaces, shaping choices: The impact of room aesthetics on guest booking preferences
Nanu, L., Chattaraman, V., & Rahman, I. (2025) · International Journal of Hospitality Management · doi:10.1016/j.ijhm.2025.104100 · Cited by 2 · see citing works
The impact of AI-generated responses on environmental survey data from MTurk
Peterson, T. C. (2025) · Journal of Applied Statistics Environmental Statistics and Data Science · doi:10.1080/29984688.2025.2545754 · Cited by 2 · see citing works
Unmasking survey fraud: investigating data quality issues in an MTurk sample
Maline, M. N., & Polonijo, A. N. (2025) · International Journal of Social Research Methodology · doi:10.1080/13645579.2025.2601994 · Cited by 2 · see citing works
Tone and credibility in voluntary disclosures
Bodoff, D., & Hirsch, I. (2023) · Corporate Communications An International Journal · doi:10.1108/ccij-10-2022-0124 · Cited by 2 · see citing works
Cui, Z., Baraboshkin, V., & Gonçalves, D. (2024) · IEEE Transactions on Engineering Management · doi:10.1109/tem.2024.3459032 · Cited by 2 · see citing works
Salim, S. R., Bhuptani, P. H., Eshelman, L. R., & Messman, T. L. (2024) · Journal of Child Sexual Abuse · doi:10.1080/10538712.2024.2415554 · Cited by 2 · see citing works
Choi, S. E., & Choi, J. (2024) · Foods · doi:10.3390/foods13203276 · Cited by 2 · see citing works
Stringer, T., Alahakoon, T., Mathmann, F., Mortimer, G., & Payne, A. R. (2025) · Journal of Fashion Marketing and Management · doi:10.1108/jfmm-10-2024-0419 · Cited by 2 · see citing works
Montanaro, E. A., Mennicke, A. M., McGonagle, A. K., Bowling, J., Meehan, E., Jules, B. N., Temple, J., Brienzo, M. J., Williams, M., Yoder, A., et al. (2024) · Journal of American College Health · doi:10.1080/07448481.2024.2418534 · Cited by 1 · see citing works
Rashideh, W., Papastathopoulos, A., & Treiblmaier, H. (2025) · ACM SIGMIS Database the DATABASE for Advances in Information Systems · doi:10.1145/3757308.3757315 · Cited by 1 · see citing works
Detecting anger-provoking events with smartphones: a naturalistic driving study
Wang, Y., Liang Ma, A., & Zhang, W. (2025) · Ergonomics · doi:10.1080/00140139.2025.2597507 · Cited by 1 · see citing works
Lin, B.-N. (2025) · Journal of Hospitality and Tourism Insights · doi:10.1108/jhti-07-2025-0859
From Brands to Celebrities: How Para-Social Brand Interactions Relate to Positive WOM
Tran, Q., Huynh, N., Paramita, W., Duong, T. T.-T., Nguyen, M., Northey, G., & Hall, T. (2026) · Australasian Marketing Journal · doi:10.1177/14413582261423011 · Cited by 1 · see citing works
Shah, S. K., Acevedo-Duque, Á., Ardito, L., & Oláh, J. (2026) · British Food Journal · doi:10.1108/bfj-04-2025-0554 · Cited by 1 · see citing works
An experimental approach to training mood for resilience
Mantas, V., Kotoula, V., Zheng, C. Y., Nielson, D. M., & Stringaris, A. K. (2023) · PLoS ONE · doi:10.1371/journal.pone.0290881 · Cited by 1 · see citing works
Chagas-Bastos, F. H. (2023) · Personality and Social Psychology Bulletin · doi:10.1177/01461672231213899 · Cited by 1 · see citing works
Hildenbrand, G. M., & Ammon, M. L. (2024) · Communication Quarterly · doi:10.1080/01463373.2024.2356521 · Cited by 1 · see citing works
From Good Citizens to Bad Apples? The Effect of Boredom Proneness: A Moral Licensing Perspective
Chou, S. Y., Chang, T., & Lakshman, S. (2024) · Psychological Reports · doi:10.1177/00332941241265958 · Cited by 1 · see citing works
Baranes, L., Shimoni, H., & Axelrod, V. (2024) · Current Psychology · doi:10.1007/s12144-024-06541-0 · Cited by 1 · see citing works
Next-Generation Behavioral Economics: Blockchain as the Web3 Infrastructure for Experimental Studies
Zhang, L. (2024) · IntechOpen eBooks · doi:10.5772/intechopen.1006740 · Cited by 1 · see citing works
Patient perceptions of healthcare provider (un)helpful approaches to explaining health information
Hildenbrand, G. M., & Ammon, M. L. (2024) · Communication Research Reports · doi:10.1080/08824096.2024.2370778
Lopez, C., Morrison, M., & Deacon, M. (2024) · European Public & Social Innovation Review · doi:10.31637/epsir-2024-760
Crowdsourcing Task Design Using Multi-agent Systems and an Enhanced Genetic Algorithm
Zou, G., Tang, J., & Yılmaz, L. (2025) · International Journal of Control Automation and Systems · doi:10.1007/s12555-024-0702-x
Morris, D. S. M., & Qirko, H. (2025) · Natural resource management and policy · doi:10.1007/978-3-031-90889-7_6
Ivanov, A., & Kurochkin, I. I. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-13127-0_35
The Role of Educational Background in Dating Between Black and White People
Santana, E. (2026) · Race and Social Problems · doi:10.1007/s12552-025-09467-9
On-the-fly training techniques for improving the readability of parallel coordinates and treemaps
Han, Y., Wang, P.-S., & Tai, C.-C. (2026) · Information Visualization · doi:10.1177/14738716261422150
Xu, Y. (2026) · Tourism and Hospitality · doi:10.3390/tourhosp7090276
Marjanovic, Z., & Comeau, T. (2026) · Acta Psychologica · doi:10.1016/j.actpsy.2026.107818
- Authors
- Lu, L., Neale, N., Line, N. D., & Bonn, M.
- Affiliations
- Temple University (US)
- University of Houston - Downtown (US)
- Florida State University (US)
- Corresponding author
- Lu, Lu
- Venue
- Cornell Hospitality Quarterly
- Publisher
- SAGE Publications
- Published
- 2021-07-01
- Volume
- 63
- Issue
- 2
- Pages
- 231-246
- ISSN
- 1938-9655 (ISSN-L)
- 1938-9663
- OpenAlex
- W3173414067
- Cited by
- 94
- References
- 47
- Citations by year
- 2026: 7 · 2025: 18 · 2024: 31 · 2023: 23 · 2022: 13 · 2021: 2
- FWCI
- 14.60
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Cornell Hospitality Quarterly (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Science Applications › Mobile Crowdsensing and Crowdsourcing
- Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
- Social Sciences › General Social Sciences › Computational and Text Analysis Methods
- Keywords
- Amazon Mechanical Turk, data quality, crowdsourcing, task design, best practices, online labour platforms
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review