The DOI matches its Crossref record; 17 of the 18 references it cites were found in a registry or an index.
Reputation as a sufficient condition for data quality on Amazon Mechanical Turk
Open accessAbstract
Data quality is one of the major concerns of using crowdsourcing websites such as Amazon Mechanical Turk (MTurk) to recruit participants for online behavioral studies. We compared two methods for ensuring data quality on MTurk: attention check questions (ACQs) and restricting participation to MTurk workers with high reputation (above 95% approval ratings). In Experiment 1, we found that high-reputation workers rarely failed ACQs and provided higher-quality data than did low-reputation workers; ACQs improved data quality only for low-reputation workers, and only in some cases. Experiment 2 corroborated these findings and also showed that more productive high-reputation workers produce the highest-quality data. We concluded that sampling high-reputation workers can ensure high-quality data without having to resort to using ACQs, which may lead to selection bias if participants who fail ACQs are excluded post-hoc.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Data obtained from MTurk has been demonstrated to be valid and reliable when appropriate selection criteria and attention checks are used (Casler, Bickel, & Hackett, 2013; Peer, Vosgerau, & Acquisti, 2014), as was the case in this study.”
From The development and evaluation of a brief form of the Normative Male Alexithymia Scale (NMAS-BF). (2019) · cites this paper as Peer, Vosgerau, & Acquisti, 2014
The cited source says what the claim attributes to it.
In the citing paper
“Most of the sites we reviewed have supervision and reputation mechanisms designed to measure the quality of submitted work, and to allow researchers to invite workers with high reputation (similar to MTurk's”
From Beyond the Turk: Alternative platforms for crowdsourcing behavioral research (2017) · cites this paper as Peer et al., 2014
In the citing paper
“Amazon Mechanical Turk is an online sample recruiting platform that, although not nationally representative, yields high data quality through the use of attention checks 44 and worker reputation scores 45 .”
From Understanding, explaining, and utilizing medical artificial intelligence (2021) · cites this paper as [45]
In the citing paper
“Following prior research, we selected participants with an approval rating above 95% on Amazon Mechanical Turk 45 .”
From Understanding, explaining, and utilizing medical artificial intelligence (2021) · cites this paper as [45]
In the citing paper
“which showed that ACQs have low diagnostic ability when data quality is already high.”
From Beyond the Turk: Alternative platforms for crowdsourcing behavioral research (2017) · cites this paper as Peer et al., (2014)
In the citing paper
“they are reliable and validated scales, and b) they have been previously used with success to measure data quality on MTurk (Peer et al., 2014).”
From Beyond the Turk: Alternative platforms for crowdsourcing behavioral research (2017) · cites this paper as Peer et al., 2014
In the citing paper
“they are reliable and validated scales, and b) they have been previously used with success to measure data quality on MTurk (Peer et al., 2014).”
From Beyond the Turk: Alternative platforms for crowdsourcing behavioral research (2017) · cites this paper as Peer et al., 2014
In the citing paper
“Using two attention-check questions (Peer et al., 2014), we checked whether participants read and paid attention to our instructions.”
From Beyond the Turk: Alternative platforms for crowdsourcing behavioral research (2017) · cites this paper as Peer et al., 2014
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Tversky, A., & Kahneman, D. (1974) · Science · doi:10.1126/science.185.4157.1124 · Cited by 28,810
Judgment under Uncertainty: Heuristics and Biases
Tversky, A., & Kahneman, D. (1975) · Utility, Probability, and Human Decision Making · doi:10.1007/978-94-010-1834-0_8 · Cited by 22,650
Amazon's Mechanical Turk: A New Source of Inexpensive, Yet High-Quality, Data?
Buhrmester, M., Kwang, T., & Gosling, S. D. (2011) · Perspectives on Psychological Science · doi:10.1177/1745691610393980 · Cited by 9,996
A very brief measure of the Big-Five personality domains
Gosling, S. D., Rentfrow, P. J., & Swann, W. B. (2003) · Journal of Research in Personality · doi:10.1016/s0092-6566(03)00046-1 · Cited by 8,307
Running experiments on Amazon Mechanical Turk
Paolacci, G., Chandler, J., & Ipeirotis, P. G. (2010) · Judgment and Decision Making · doi:10.1017/s1930297500002205 · Cited by 3,767
Instructional manipulation checks: Detecting satisficing to increase statistical power
Oppenheimer, D. M., Meyvis, T., & Davidenko, N. (2009) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2009.03.009 · Cited by 3,493
Robins, R. W., Hendin, H. M., & Trzesniewski, K. H. (2001) · Personality and Social Psychology Bulletin · doi:10.1177/0146167201272002 · Cited by 3,041
The Efficient Assessment of Need for Cognition
Cacioppo, J. T., Petty, R. E., & Feng Kao, C. (1984) · Journal of Personality Assessment · doi:10.1207/s15327752jpa4803_13 · Cited by 2,913
Data collection in a flat world: the strengths and weaknesses of Mechanical Turk samples
Goodman, J. K., Cryder, C. E., & Cheema, A. (2013) · Journal of Behavioral Decision Making · doi:10.1002/bdm.1753 · Cited by 2,242
Rosenberg, M. (2011) · PsycTESTS Dataset · doi:10.1037/t01038-000 · Cited by 1,226
Cited by
Sorted by citations
Items 401–500 of 1,000
Intermittent Fasting Implementation and Association with Eating Disorder Symptomatology
Cuccolo, K., Kramer, R., Petros, T., & Thoennes, M. (2022) · Eating Disorders · doi:10.1080/10640266.2021.1922145 · Cited by 33
den Kamp, E. M. O., Tims, M., Bakker, A. B., & Demerouti, E. (2022) · Applied Psychology · doi:10.1111/apps.12399 · Cited by 40
Fergus, T. A., & Bardeen, J. R. (2015) · Personality and Individual Differences · doi:10.1016/j.paid.2015.08.010 · Cited by 33
Castille, C. M., Kuyumcu, D., & Bennett, R. J. (2016) · Personality and Individual Differences · doi:10.1016/j.paid.2016.07.026 · Cited by 34
Two-Faced Morality: Distrust Promotes Divergent Moral Standards for the Self Versus Others
Weiß, A., Burgmer, P., & Mussweiler, T. (2018) · Personality and Social Psychology Bulletin · doi:10.1177/0146167218775693 · Cited by 35
Dong, M., van Prooijen, J., & Lange, P. A. M. V. (2019) · PLoS ONE · doi:10.1371/journal.pone.0219382 · Cited by 42
We stand by our brand: Consumers’ post-food safety crisis purchase intention and moral reasoning
Yu, H. C., Legendre, T. S., & Ma, J. (2021) · Journal of Business Research · doi:10.1016/j.jbusres.2021.04.020 · Cited by 44
Does prospect theory explain ethical decision making? Evidence from tax compliance
Austin, C. R., Bobek, D. D., & Jackson, S. B. (2021) · Accounting Organizations and Society · doi:10.1016/j.aos.2021.101251 · Cited by 38
Designing With Pictographs: Envision Topics Without Sacrificing Understanding
Burns, A., Xiong, C., Franconeri, S., Cairo, A., & Mahyar, N. (2021) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2021.3092680 · Cited by 33
Comparing Data Collected on Amazon's Mechanical Turk to National Surveys
Qureshi, N., Edelen, M. O., Hilton, L. G., Rodriguez, A., Hays, R. D., & Herman, P. M. (2022) · American Journal of Health Behavior · doi:10.5993/ajhb.46.5.1 · Cited by 33
Johnson, J. D., & Lecci, L. B. (2019) · Psychology of Violence · doi:10.1037/vio0000228 · Cited by 33
When and Why Test Generators for Deep Learning Produce Invalid Inputs: an Empirical Study
Riccio, V., & Tonella, P. (2023) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse48619.2023.00104 · Cited by 42
Concerned protesters: from compassion to retaliation
Antonetti, P., & Maklan, S. (2017) · European Journal of Marketing · doi:10.1108/ejm-03-2016-0145 · Cited by 29
Risky riders: A comparison of personality theories on motorcyclist riding behaviour
Antoniazzi, D., & Klein, R. G. (2018) · Transportation Research Part F Traffic Psychology and Behaviour · doi:10.1016/j.trf.2018.12.008 · Cited by 33
Privacy in Crowdsourcing: a Review of the Threats and Challenges
Xia, H., & McKernan, B. (2020) · Computer Supported Cooperative Work (CSCW) · doi:10.1007/s10606-020-09374-0 · Cited by 38
Petit, J., Li, C., Millet, B., Ali, K. I., & Sun, R. (2021) · Science Communication · doi:10.1177/10755470211009887 · Cited by 39
A self-determination theory approach to predicting daily prosocial behavior
Peetz, J., & Milyavskaya, M. (2021) · Motivation and Emotion · doi:10.1007/s11031-021-09902-5 · Cited by 32
Jang, W. W., Byon, K. K., & Song, H. (2021) · Sustainability · doi:10.3390/su13148019 · Cited by 32
Angerbauer, K., Rodrigues, N., Cutura, R., Öney, S., Pathmanathan, N., Morariu, C., Weiskopf, D., & Sedlmair, M. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3502133 · Cited by 32
Kim, B., Yu, H. C., Huang, Y., & Lee, S. (2023) · International Journal of Hospitality Management · doi:10.1016/j.ijhm.2023.103522 · Cited by 41
Using behavioral economic variables to predict future alcohol use in a crowdsourced sample
Strickland, J. C., Alcorn, J. L., & Stoops, W. W. (2019) · Journal of Psychopharmacology · doi:10.1177/0269881119827800 · Cited by 33
Investigating the Effects of (Empty) Promises on Human-Automation Interaction and Trust Repair
Albayram, Y., Jensen, T., Khan, M. M. H., Fahim, M. A. A., Buck, R., & Coman, E. (2020) · Proceedings of the 8th International Conference on Human-Agent Interaction · doi:10.1145/3406499.3415064 · Cited by 31
Please participate in Part 2: Maximizing response rates in longitudinal MTurk designs
Stoycheff, E. (2016) · Methodological Innovations · doi:10.1177/2059799116672879 · Cited by 32
Gender, leadership, and the display of empathic anger
Keck, S. (2019) · Journal of Occupational and Organizational Psychology · doi:10.1111/joop.12264 · Cited by 33
(Smart)Watch Out! encouraging privacy-protective behavior through interactive games
Williams, M., Nurse, J. R. C., & Creese, S. (2019) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2019.07.012 · Cited by 31
Exclusion Criteria as Measurements I: Identifying Invalid Responses
Dewitt, B., Fischhoff, B., Davis, A. L., Broomell, S. B., Roberts, M. S., & Hanmer, J. (2019) · Medical Decision Making · doi:10.1177/0272989x19856617 · Cited by 31
Effects of Nonstereotyped OccupationalGender Role Portrayal in Advertising
Liljedal, K. T., Berg, H., & Dahlén, M. (2020) · Journal of Advertising Research · doi:10.2501/jar-2020-008 · Cited by 31
Ouden, L. D., Tiego, J., Lee, R. S. C., Albertella, L., Greenwood, L., Fontenelle, L. F., Yücel, M., & Segrave, R. A. (2020) · Addictive Behaviors · doi:10.1016/j.addbeh.2020.106464 · Cited by 40
Wörtler, B., Yperen, N. W. V., & Barelds, D. P. H. (2020) · European Journal of Work and Organizational Psychology · doi:10.1080/1359432x.2020.1844663 · Cited by 37
Insomnia during the COVID-19 pandemic: the role of depression and COVID-19-related risk factors
Pizzonia, K. L., Koscinski, B., Suhr, J. A., Accorso, C., Allan, D. M., & Allan, N. P. (2021) · Cognitive Behaviour Therapy · doi:10.1080/16506073.2021.1879241 · Cited by 31
Emotional contagion as a trigger for moral disengagement: Their effects on workplace injuries
Петитта, Л., Probst, T. M., Ghezzi, V., & Barbaranelli, C. (2021) · Safety Science · doi:10.1016/j.ssci.2021.105317 · Cited by 32
McKee, L. G., DiMarzio, K., Parent, J. M., Dale, C. F., Acosta, J., & O’Leary, J. L. (2021) · Research on Child and Adolescent Psychopathology · doi:10.1007/s10802-021-00829-6 · Cited by 36
Abbreviated San Diego Wisdom Scale (SD-WISE-7) and Jeste-Thomas Wisdom Index (JTWI)
Thomas, M. L., Palmer, B. W., Lee, E., Liu, J., Daly, R., Tu, X. M., & Jeste, D. V. (2021) · International Psychogeriatrics · doi:10.1017/s1041610221002684 · Cited by 36
Multiracial Children’s and Adults’ Categorizations of Multiracial Individuals
Roberts, S. O., & Gelman, S. A. (2016) · Journal of Cognition and Development · doi:10.1080/15248372.2015.1086772 · Cited by 31
The role of dialecticism and reviewer expertise in consumer responses to mixed reviews
Hwang, Y., Choi, S., & Mattila, A. S. (2017) · International Journal of Hospitality Management · doi:10.1016/j.ijhm.2017.10.009 · Cited by 31
The Relationship between Purity Culture and Rape Myth Acceptance
Owens, B. C., Hall, M. E. L., & Anderson, T. L. (2021) · Journal of Psychology and Theology · doi:10.1177/0091647120974992 · Cited by 33
Privacy Concerns and Continued Use Intention of Telemedicine During COVID-19
Kato-Lin, Y.-C., & Thelen, S. T. (2022) · Telemedicine Journal and e-Health · doi:10.1089/tmj.2021.0603 · Cited by 31
The Bilingual Code-Switching Profile (BCSP)
Olson, D. J. (2022) · Linguistic Approaches to Bilingualism · doi:10.1075/lab.21039.ols · Cited by 32
Snooping on Mobile Phones: Prevalence and Trends
Marques, D., Muslukhov, I., Guerreiro, T., Beznosov, K., & Carriço, L. M. (2016) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.3264715 · Cited by 30
Financial Infidelity in Couple Relationships
Jeanfreau, M. M., Noguchi, K., Mong, M. D., & Stadthagen, H. (2018) · Journal of Financial Therapy · doi:10.4148/1944-9771.1159 · Cited by 30
What Crowdsourcing Can Offer to Cross-Cultural Psychological Science
Cuccolo, K., Irgens, M. S., Zlokovich, M. S., Grahe, J. E., & Edlund, J. E. (2020) · Cross-Cultural Research · doi:10.1177/1069397120950628 · Cited by 38
Analyzing Biases in Perception of Truth in News Stories and Their Implications for Fact Checking
Babaei, M., Kulshrestha, J., Chakraborty, A., Redmiles, E. M., Cha, M., & Gummadi, K. P. (2021) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2021.3096038 · Cited by 31
Kreitzer, R., Maltby, E., & Smith, C. W. (2022) · Journal of Public Policy · doi:10.1017/s0143814x21000222 · Cited by 35
Identifying Mental Disorder from the Faces of Women with Borderline Personality Disorder
Daros, A. R., Ruocco, A. C., & Rule, N. O. (2016) · Journal of Nonverbal Behavior · doi:10.1007/s10919-016-0237-9 · Cited by 31
Typologies of PTSD clusters and reckless/self-destructive behaviors: A latent profile analysis
Contractor, A. A., & Weiss, N. H. (2018) · Psychiatry Research · doi:10.1016/j.psychres.2018.12.124 · Cited by 33
Narcissism and romantic commitment: The role of attitudes toward alternative romantic partners
Zeigler‐Hill, V., Seidman, G., Trombly, D. R. C., Ishaq, J., Shango, R., & Roberts, A. (2019) · Self and Identity · doi:10.1080/15298868.2019.1635521 · Cited by 30
Who is the Average Individual Investor? Numerical Skills and Implications for Accounting ResearchPreprint
Krische, S. D. (2014) · SSRN Electronic Journal · doi:10.2139/ssrn.2426570 · Cited by 31
Fergus, T. A. (2015) · Journal of Clinical Psychology · doi:10.1002/jclp.22194 · Cited by 29
Distinguishing moral hazard from access for high-cost healthcare under insurance
Robertson, C. T., Yuan, A., Zhang, W., & Joiner, K. A. (2020) · PLoS ONE · doi:10.1371/journal.pone.0231768 · Cited by 31
Dunbar, A., & Hanink, P. A. (2022) · Journal of Experimental Criminology · doi:10.1007/s11292-021-09499-2 · Cited by 29
Shin, I., Riforgiate, S. E., Coker, M. C., & Godager, E. A. (2022) · Management Communication Quarterly · doi:10.1177/08933189221144996 · Cited by 30
Chu, C.-C., & Chou, C.-Y. (2024) · Journal of Vocational Behavior · doi:10.1016/j.jvb.2024.104053 · Cited by 34
Nothing compares to me: How narcissism shapes comparative thinking
Ohmann, K., & Burgmer, P. (2016) · Personality and Individual Differences · doi:10.1016/j.paid.2016.03.069 · Cited by 31
Hapenny, J. E., & Fergus, T. A. (2016) · Journal of Contextual Behavioral Science · doi:10.1016/j.jcbs.2016.10.004 · Cited by 31
Impact of User Characteristics on Attitudes Towards Automatic Mobile Application Updates.
Mathur, A., & Chetty, M. (2017) · Symposium On Usable Privacy and Security · Cited by 29
Biasing and debiasing health decisions with bar graphs: Costs and benefits of graph literacy
Okan, Y., García‐Retamero, R., Cokely, E. T., & Maldonado, A. (2018) · Quarterly Journal of Experimental Psychology · doi:10.1177/1747021817744546 · Cited by 36
Nagle, C. (2019) · Journal of Second Language Pronunciation · doi:10.1075/jslp.18016.nag · Cited by 29
Asada, A., Ko, Y., & Jang, W. (2020) · Journal of Sport Management · doi:10.1123/jsm.2019-0055 · Cited by 26
Theistic intellectual humility and well-being: does ideological context matter?
Hill, P. C., Hall, M. E. L., Wang, D., & Decker, L. A. (2019) · The Journal of Positive Psychology · doi:10.1080/17439760.2019.1689424 · Cited by 32
The impact of customer compassion on face-to-face and online complaints
Hwang, Y., & Mattila, A. S. (2020) · Journal of Hospitality Marketing & Management · doi:10.1080/19368623.2020.1711546 · Cited by 29
Self-Service Technology Recovery: The Effect of Recovery Initiation and Locus of Responsibility
Dao, H. M., & Theotokis, A. (2020) · Journal of Interactive Marketing · doi:10.1016/j.intmar.2020.09.001 · Cited by 34
The Hidden Cost of Using Amazon Mechanical Turk for Research
Saravanos, A., Zervoudakis, S., Zheng, D., Stott, N., Hawryluk, B., & Delfino, D. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-90238-4_12 · Cited by 29
Hwang, Y., Wang, X., & Guchait, P. (2022) · International Journal of Contemporary Hospitality Management · doi:10.1108/ijchm-12-2021-1490 · Cited by 29
Revenue Management in a Refurbishing Duopoly with Cannibalization
Kurdhi, N. A., Dabadghao, S. S., & Fransoo, J. C. (2020) · Journal of Operations Management · doi:10.1002/joom.1208 · Cited by 29
Scale Mean and Variance Differences in MTurk and Non-MTurk Samples
Keith, M. G., Stevenor, B. A., & McAbee, S. T. (2022) · Journal of Personnel Psychology · doi:10.1027/1866-5888/a000309 · Cited by 33
Public support for second look sentencing: Is there a Shawshank redemption effect?
Hannan, K. R., Cullen, F. T., Graham, A., Jonson, C. L., Pickett, J. T., Haner, M., & Sloan, M. M. (2023) · Criminology & Public Policy · doi:10.1111/1745-9133.12616 · Cited by 29
Children’s and Adults’ Predictions of Black, White, and Multiracial Friendship Patterns
Roberts, S. O., Williams, A. D., & Gelman, S. A. (2016) · Journal of Cognition and Development · doi:10.1080/15248372.2016.1262374 · Cited by 29
Support for the Confederate battle flag in the Southern United States: Racism or Southern pride?
Wright, J. D., & Esses, V. M. (2017) · Journal of Social and Political Psychology · doi:10.5964/jspp.v5i1.687 · Cited by 29
Are manipulation checks necessary?Preprint
Hauser, D., Ellsworth, P. C., & Gonzalez, R. (2018) · doi:10.31234/osf.io/afmbg · Cited by 33
Headphone screening to facilitate web-based auditory experimentsPreprint
Woods, K. J. P., Siegel, M., Traer, J. A., & McDermott, J. H. (2016) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/aez6f · Cited by 42
The importance of belonging for well-being in college students
Romeo, I., Harold Stanislaw, A., McCreary, J., & Hawley, M. (2024) · PLOS mental health. · doi:10.1371/journal.pmen.0000057 · Cited by 28
Beyond Anthropomorphism: Social Presence in Human– AI Collaboration Processes
Siemon, D., Elshan, E., de Vreede, T., Ebel, P., & de Vreede, G. (2025) · Journal of Management Studies · doi:10.1111/joms.70000 · Cited by 36
Mays, D. M., & Tercyak, K. P. (2015) · American Journal of Public Health · doi:10.2105/ajph.2015.302665 · Cited by 28
Lac, A., & Luk, J. W. (2019) · Prevention Science · doi:10.1007/s11121-019-0986-x · Cited by 29
Steiner, M. D., Seitz, F. I., & Frey, R. (2021) · Decision · doi:10.1037/dec0000127 · Cited by 27
A New Frontier: The Development and Validation of the Intimate Partner Cyber Abuse Instrument
Fissel, E. R., Graham, A., Butler, L. C., & Fisher, B. S. (2021) · Social Science Computer Review · doi:10.1177/0894439321994618 · Cited by 28
Bazzoli, A., Probst, T. M., & Lee, H. J. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18052338 · Cited by 25
Work and Non-Work Sickness Presenteeism
Probst, T. M., Lee, H. J., Bazzoli, A., Jenkins, M. R., & Bettac, E. L. (2021) · Journal of Occupational and Environmental Medicine · doi:10.1097/jom.0000000000002240 · Cited by 28
Measurement Invariance of the Scale of Positive and Negative Experience Across 13 Countries
Jovanović, V., Joshanloo, M., Carbonell, M. M., Caudek, C., Espejo, B., Checa, I., Krasko, J., Kyriazos, T. A., Piotrowski, J. P., Rice, S. P. M., et al. (2021) · Assessment · doi:10.1177/10731911211021494 · Cited by 28
Jackl, H. M. (2021) · Punishment & Society · doi:10.1177/14624745211059318 · Cited by 28
Data-driven prototyping via natural-language-based GUI retrieval
Kolthoff, K., Bartelt, C., & Ponzetto, S. P. (2023) · Automated Software Engineering · doi:10.1007/s10515-023-00377-x · Cited by 34
Hellali, W., Koraï, B., & Lambert, R. (2023) · Food Quality and Preference · doi:10.1016/j.foodqual.2023.104945 · Cited by 28
Meredith, S. E., Sweeney, M. M., Johnson, P. S., Johnson, M. W., & Griffiths, R. R. (2015) · Journal of Caffeine Research · doi:10.1089/jcr.2015.0024 · Cited by 28
Mothers and Fathers Perform More Mate Retention Behaviors than Individuals without Children
Barbaro, N., Shackelford, T. K., & Weekes‐Shackelford, V. A. (2016) · Human Nature · doi:10.1007/s12110-016-9261-z · Cited by 28
Williamson, D., & Johnston, C. (2017) · Journal of Attention Disorders · doi:10.1177/1087054717693373 · Cited by 28
Weiss, N. H., Schick, M. R., Contractor, A. A., & Dixon–Gordon, K. L. (2019) · Addictive Behaviors · doi:10.1016/j.addbeh.2019.04.029 · Cited by 28
Reducing perceived vulnerability to increase utilization of nonprofit services
Tanner, E. C., & Su, L. (2019) · Journal of Services Marketing · doi:10.1108/jsm-12-2017-0434 · Cited by 28
Fergus, T. A. (2014) · Journal of Obsessive-Compulsive and Related Disorders · doi:10.1016/j.jocrd.2014.05.004 · Cited by 27
Motta, M., Callaghan, T., & Smith, B. (2016) · International Journal of Public Opinion Research · doi:10.1093/ijpor/edw027 · Cited by 27
Relevance insensitivity: A new look at some old biases
Hsee, C. K., Yang, Y., & Li, X. (2019) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2019.05.002 · Cited by 28
Do Difficulties Regulating Positive Emotions Contribute to Risky Sexual Behavior? A Path Analysis
Weiss, N. H., Forkus, S. R., Contractor, A. A., Darosh, A. G., Goncharenko, S., & Dixon–Gordon, K. L. (2019) · Archives of Sexual Behavior · doi:10.1007/s10508-019-1410-0 · Cited by 30
Job insecurity and impression management
Probst, T. M., Jiang, L., & Bohle, S. L. (2019) · Career Development International · doi:10.1108/cdi-04-2018-0119 · Cited by 32
Liu, Y., Huang, Z., Jiang, L., & Messier, W. F. (2019) · The Accounting Review · doi:10.2308/tar-2017-0284 · Cited by 28
Jiang, L., Hu, S., Näswall, K., Bohle, S. L., & Wang, H. (2020) · European Journal of Work and Organizational Psychology · doi:10.1080/1359432x.2020.1758669 · Cited by 30
Caton, N. R., Pearson, S. G., & Dixson, B. J. W. (2022) · Evolution and Human Behavior · doi:10.1016/j.evolhumbehav.2022.04.002 · Cited by 27
Resilient Agility: A Necessary Condition for Employee and Organizational Sustainability
Prieto, L., & Talukder, M. F. (2023) · Sustainability · doi:10.3390/su15021552 · Cited by 63
Mousavi, R., & Gu, B. (2023) · Information Systems Research · doi:10.1287/isre.2021.0599 · Cited by 30
Age of First Exposure to Football Is Not Associated with Midlife Brain Health Problems
Iverson, G. L., Terry, D. P., Caccese, J. B., Büttner, F., & Merz, Z. C. (2020) · Journal of Neurotrauma · doi:10.1089/neu.2020.7041 · Cited by 27
Unemployment and Online Labor: Evidence from Microtasking
Laitenberger, U., Viete, S., Slivko, O., Kummer, M. E., Borchert, K., & Hirth, M. (2023) · MIS Quarterly · doi:10.25300/misq/2022/17347 · Cited by 37
What’s the Appeal? Perceptions of Review Processes for Algorithmic Decisions
Lyons, H., Wijenayake, S., Miller, T., & Velloso, E. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517606 · Cited by 27
Metadata
- Authors
- Peer, E., Vosgerau, J., & Acquisti, A.
- Affiliations
- Tilburg University (NL)
- Carnegie Mellon University (US)
- Venue
- Behavior Research Methods
- Published
- 2013-12-20
- Volume
- 46
- Issue
- 4
- Pages
- 1023-1031
- ISSN
- 1554-351X (ISSN-L)
- 1554-3528
- OpenAlex
- W2105753174
- Cited by
- 1,618
- References
- 18
- Citations by year
- 2026: 68 · 2025: 96 · 2024: 128 · 2023: 167 · 2022: 175 · 2021: 212 · 2020: 210 · 2019: 147 · 2018: 119 · 2017: 132 · 2016: 91 · 2015: 49 · 2014: 17 · 2013: 1
- FWCI
- 79.80
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Behavior Research Methods (publishedVersion)
- Research portal (Tilburg University) (publishedVersion)
- PubMed (publishedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- SSRN Electronic Journal (submittedVersion)
- Figshare (submittedVersion, cc-by)
- Archivio istituzionale della Ricerca - Bocconi (submittedVersion)
- KiltHub Repository
- Topics
- Physical Sciences › Computer Science › Computer Science Applications › Mobile Crowdsensing and Crowdsourcing
- Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Keywords
- data quality, Amazon Mechanical Turk, worker reputation, crowdsourcing, selection bias, labor productivity, participant recruitment
- MeSH terms
- Data Collection, Humans, Research Design, Patient Selection, Internet, Behavioral Research, Crowdsourcing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
