The DOI matches its Crossref record; 17 of the 18 references it cites were found in a registry or an index.
Reputation as a sufficient condition for data quality on Amazon Mechanical Turk
Open accessAbstract
Data quality is one of the major concerns of using crowdsourcing websites such as Amazon Mechanical Turk (MTurk) to recruit participants for online behavioral studies. We compared two methods for ensuring data quality on MTurk: attention check questions (ACQs) and restricting participation to MTurk workers with high reputation (above 95% approval ratings). In Experiment 1, we found that high-reputation workers rarely failed ACQs and provided higher-quality data than did low-reputation workers; ACQs improved data quality only for low-reputation workers, and only in some cases. Experiment 2 corroborated these findings and also showed that more productive high-reputation workers produce the highest-quality data. We concluded that sampling high-reputation workers can ensure high-quality data without having to resort to using ACQs, which may lead to selection bias if participants who fail ACQs are excluded post-hoc.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Data obtained from MTurk has been demonstrated to be valid and reliable when appropriate selection criteria and attention checks are used (Casler, Bickel, & Hackett, 2013; Peer, Vosgerau, & Acquisti, 2014), as was the case in this study.”
From The development and evaluation of a brief form of the Normative Male Alexithymia Scale (NMAS-BF). (2019) · cites this paper as Peer, Vosgerau, & Acquisti, 2014
The cited source says what the claim attributes to it.
In the citing paper
“Most of the sites we reviewed have supervision and reputation mechanisms designed to measure the quality of submitted work, and to allow researchers to invite workers with high reputation (similar to MTurk's”
From Beyond the Turk: Alternative platforms for crowdsourcing behavioral research (2017) · cites this paper as Peer et al., 2014
In the citing paper
“Amazon Mechanical Turk is an online sample recruiting platform that, although not nationally representative, yields high data quality through the use of attention checks 44 and worker reputation scores 45 .”
From Understanding, explaining, and utilizing medical artificial intelligence (2021) · cites this paper as [45]
In the citing paper
“Following prior research, we selected participants with an approval rating above 95% on Amazon Mechanical Turk 45 .”
From Understanding, explaining, and utilizing medical artificial intelligence (2021) · cites this paper as [45]
In the citing paper
“which showed that ACQs have low diagnostic ability when data quality is already high.”
From Beyond the Turk: Alternative platforms for crowdsourcing behavioral research (2017) · cites this paper as Peer et al., (2014)
In the citing paper
“they are reliable and validated scales, and b) they have been previously used with success to measure data quality on MTurk (Peer et al., 2014).”
From Beyond the Turk: Alternative platforms for crowdsourcing behavioral research (2017) · cites this paper as Peer et al., 2014
In the citing paper
“they are reliable and validated scales, and b) they have been previously used with success to measure data quality on MTurk (Peer et al., 2014).”
From Beyond the Turk: Alternative platforms for crowdsourcing behavioral research (2017) · cites this paper as Peer et al., 2014
In the citing paper
“Using two attention-check questions (Peer et al., 2014), we checked whether participants read and paid attention to our instructions.”
From Beyond the Turk: Alternative platforms for crowdsourcing behavioral research (2017) · cites this paper as Peer et al., 2014
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Tversky, A., & Kahneman, D. (1974) · Science · doi:10.1126/science.185.4157.1124 · Cited by 28,810
Judgment under Uncertainty: Heuristics and Biases
Tversky, A., & Kahneman, D. (1975) · Utility, Probability, and Human Decision Making · doi:10.1007/978-94-010-1834-0_8 · Cited by 22,650
Amazon's Mechanical Turk: A New Source of Inexpensive, Yet High-Quality, Data?
Buhrmester, M., Kwang, T., & Gosling, S. D. (2011) · Perspectives on Psychological Science · doi:10.1177/1745691610393980 · Cited by 9,996
A very brief measure of the Big-Five personality domains
Gosling, S. D., Rentfrow, P. J., & Swann, W. B. (2003) · Journal of Research in Personality · doi:10.1016/s0092-6566(03)00046-1 · Cited by 8,307
Running experiments on Amazon Mechanical Turk
Paolacci, G., Chandler, J., & Ipeirotis, P. G. (2010) · Judgment and Decision Making · doi:10.1017/s1930297500002205 · Cited by 3,767
Instructional manipulation checks: Detecting satisficing to increase statistical power
Oppenheimer, D. M., Meyvis, T., & Davidenko, N. (2009) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2009.03.009 · Cited by 3,493
Robins, R. W., Hendin, H. M., & Trzesniewski, K. H. (2001) · Personality and Social Psychology Bulletin · doi:10.1177/0146167201272002 · Cited by 3,041
The Efficient Assessment of Need for Cognition
Cacioppo, J. T., Petty, R. E., & Feng Kao, C. (1984) · Journal of Personality Assessment · doi:10.1207/s15327752jpa4803_13 · Cited by 2,913
Data collection in a flat world: the strengths and weaknesses of Mechanical Turk samples
Goodman, J. K., Cryder, C. E., & Cheema, A. (2013) · Journal of Behavioral Decision Making · doi:10.1002/bdm.1753 · Cited by 2,242
Rosenberg, M. (2011) · PsycTESTS Dataset · doi:10.1037/t01038-000 · Cited by 1,226
Cited by
Sorted by citations
Items 501–600 of 1,000
Can “Focused Deterrence” Produce More Effective Ethics Codes? An Experimental Study
Rorie, M. L., & West, M. P. (2020) · Journal of White Collar and Corporate Crime · doi:10.1177/2631309x20940664 · Cited by 27
Jin, D. (2024) · International Journal of Consumer Studies · doi:10.1111/ijcs.13025 · Cited by 30
Maeder, E. M., Yamamoto, S., & McManus, L. A. (2017) · Psychology Crime and Law · doi:10.1080/1068316x.2017.1409895 · Cited by 26
Does Being Verified Make You More Credible?
Vaidya, T., Votipka, D., Mazurek, M. L., & Sherr, M. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300755 · Cited by 28
Exploring protective decision‐making in the context of impact‐based flood warnings
Landaverde, E. R. M., Werner, M., & Verkade, J. S. (2019) · Journal of Flood Risk Management · doi:10.1111/jfr3.12587 · Cited by 27
Meier, E., Aziz, S., Wuensch, K. L., & Dolbier, C. L. (2020) · Journal of Leisure Research · doi:10.1080/00222216.2020.1778589 · Cited by 35
Götz, M., Donzallaz, M. C., & Jonas, K. W. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.01788 · Cited by 36
A theorisation of discrete emotion spillovers: an empirical test for anger
Antonetti, P., & Valor, C. (2020) · Journal of Marketing Management · doi:10.1080/0267257x.2020.1817767 · Cited by 27
Attribution of Responsibility after Failures within Platform Ecosystems
Dunn, B. K., Jensen, M. L., & Ralston, R. (2021) · Journal of Management Information Systems · doi:10.1080/07421222.2021.1912937 · Cited by 28
Usability inspection: Novice crowd inspectors versus expert
Nasir, M., Ikram, N., & Jalil, Z. (2021) · Journal of Systems and Software · doi:10.1016/j.jss.2021.111122 · Cited by 26
Clercq, D. D., Shu, C., & Gu, M. (2022) · Personnel Review · doi:10.1108/pr-09-2020-0699 · Cited by 28
Kaufmann, L., Schreiner, M., & Reimann, F. (2022) · Journal of Supply Chain Management · doi:10.1111/jscm.12280 · Cited by 29
Lee, J., Kim, J. W., & Lee, H. Y. (2022) · Health Communication · doi:10.1080/10410236.2022.2031452 · Cited by 31
Farooq, A., Dahabiyeh, L., & Javed, Y. (2023) · Online Information Review · doi:10.1108/oir-04-2022-0232 · Cited by 29
Parker, M. A., & Borrie, S. A. (2017) · Journal of Voice · doi:10.1016/j.jvoice.2017.08.002 · Cited by 26
Digital booking services: comparing online with phone reservation services
Schaarschmidt, M., & Höber, B. (2017) · Journal of Services Marketing · doi:10.1108/jsm-04-2016-0145 · Cited by 26
Kim, D.-Y., Kim, S., & Kim, K. J. (2018) · Service Business · doi:10.1007/s11628-018-0386-5 · Cited by 28
Is More Always Merrier? Intersectionality as an Antecedent of Job Insecurity
Lavaysse, L. M., Probst, T. M., & Arena, D. F. (2018) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph15112559 · Cited by 31
COVID-related fear maintains controlling parenting behaviors during the pandemic
Wissemann, K., Mathes, B. M., Meyer, A. M., & Schmidt, N. B. (2021) · Cognitive Behaviour Therapy · doi:10.1080/16506073.2021.1878274 · Cited by 28
Munzel, A. (2015) · Recherche et Applications en Marketing (English Edition) · doi:10.1177/2051570715604155 · Cited by 24
Rogge, R. D., Lin, Y.-Y., Swanson, D. P., & Amaro, A. (2024) · Mindfulness · doi:10.1007/s12671-022-01886-8 · Cited by 25
“I Don't Agree with Giving Cash”: A Survey Experiment Examining Support for Public Assistance
Campbell, C., & Gaddis, S. M. (2016) · Social Science Quarterly · doi:10.1111/ssqu.12338 · Cited by 25
Linking Phubbing Behavior to Self-Reported Attentional Failures and Media Multitasking
Sansevere, K. S., & Ward, N. (2021) · Future Internet · doi:10.3390/fi13040100 · Cited by 33
How Do Parents Evaluate and Select Schools? Evidence From a Survey Experiment
Haderlein, S. A. K. (2021) · American Educational Research Journal · doi:10.3102/00028312211046360 · Cited by 27
Khoa, D. T., Gip, H., Guchait, P., Wang, C., & Baaklini, E. S. (2022) · Journal of service management · doi:10.1108/josm-09-2021-0352 · Cited by 31
The effects of knowledge mechanisms on employees' information security threat construal
Mady, A., Gupta, S., & Warkentin, M. E. (2023) · Information Systems Journal · doi:10.1111/isj.12424 · Cited by 25
Lorona, R. T., & Fergus, T. A. (2015) · Journal of Obsessive-Compulsive and Related Disorders · doi:10.1016/j.jocrd.2015.06.007 · Cited by 25
Examining the Specific Facets of Distress Tolerance That Are Relevant to Health Anxiety
Fergus, T. A., Bardeen, J. R., & Orcutt, H. K. (2015) · Journal of Cognitive Psychotherapy · doi:10.1891/0889-8391.29.1.32 · Cited by 25
User Studies of Principled Model Finder Output
Danas, N., Nelson, T., Harrison, L., Krishnamurthi, S., & Dougherty, D. J. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-66197-1_11 · Cited by 25
Lac, A., & Brack, N. (2017) · Addictive Behaviors · doi:10.1016/j.addbeh.2017.10.006 · Cited by 32
You Talk Too Much: Limiting Privacy Exposure Via Voice Input
Vaidya, T., & Sherr, M. (2019) · IEEE Security and Privacy Workshops (SPW) · doi:10.1109/spw.2019.00026 · Cited by 25
The condom purchase task: A hypothetical demand method for evaluating sexual health decision‐making
Strickland, J. C., Marks, K. R., & Bolin, B. L. (2020) · Journal of the Experimental Analysis of Behavior · doi:10.1002/jeab.585 · Cited by 27
Zheng, Y. (2020) · International Journal of Strategic Communication · doi:10.1080/1553118x.2020.1720026 · Cited by 25
Reducing the number of non-naïve participants in Mechanical Turk samples
Meyers, E. A., Walker, A. C., Fugelsang, J. A., & Koehler, D. J. (2020) · Methods in Psychology · doi:10.1016/j.metip.2020.100032 · Cited by 29
Survey-Software Implicit Association Tests: A Methodological and Empirical AnalysisPreprint
Carpenter, T., Pogacar, R., Pullig, C., Kouril, M., Aguilar, S. J., LaBouff, J. P., Isenberg, N., & Chakroff, A. (2018) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/hgy3z · Cited by 38
Conspicuous political brand interactions on social network sites
Marder, B., Marchant, C., Archer-Brown, C., Yau, A., & Colliander, J. (2018) · European Journal of Marketing · doi:10.1108/ejm-01-2017-0059 · Cited by 19
Crowdsourcing Public Perceptions of Plastic Surgeons: Is There a Gender Bias?
Bucknor, A. E. M., Christensen, J. M., Kamali, P., Egeler, S. A., van Veldhuisen, C. L., Rakhorst, H. A., Mathijssen, I. M. J., Lin, S. J., & Furnas, H. J. (2018) · Plastic & Reconstructive Surgery Global Open · doi:10.1097/gox.0000000000001728 · Cited by 24
Schaarschmidt, M., & Könsgen, R. (2019) · Journal of Business Research · doi:10.1016/j.jbusres.2019.04.004 · Cited by 28
Alt-White? A Gendered Look at “Victim” Ideology and the Alt-Right
Boehme, H. M., & Isom, D. A. (2019) · Victims & Offenders · doi:10.1080/15564886.2019.1679308 · Cited by 24
Contractor, A. A., Weiss, N. H., & Forkus, S. R. (2020) · Journal of Clinical Psychology · doi:10.1002/jclp.23046 · Cited by 25
Wang, Q., & Jeon, H. J. (2020) · PLoS ONE · doi:10.1371/journal.pone.0240232 · Cited by 24
What Does “Mind‐Wandering” Mean to the Folk? An Empirical Investigation
Irving, Z. C., Glasser, A., Alison Gopnik, C., Pinter, V., & Sripada, C. (2020) · Cognitive Science · doi:10.1111/cogs.12908 · Cited by 24
Are sexist beliefs related to mental health problems?
Borgogna, N. C., & Aita, S. L. (2020) · The Social Science Journal · doi:10.1080/03623319.2020.1809902 · Cited by 25
A Cheating Mood: The Emotional and Psychological Benefits of Cheating in Single-Player Games
Passmore, C. J., Miller, M. K., Liu, J., Phillips, C. J., & Mandryk, R. L. (2020) · Annual Symposium on Computer-Human Interaction in Play · doi:10.1145/3410404.3414252 · Cited by 27
Sewall, C. J. R., & Parry, D. A. (2021) · Technology Mind and Behavior · doi:10.1037/tmb0000036 · Cited by 30
The Denier in Chief: Faith in Trump and Techniques of Neutralization in a Pandemic
Cullen, F. T., Graham, A., Jonson, C. L., Pickett, J. T., Sloan, M. M., & Haner, M. (2021) · Deviant Behavior · doi:10.1080/01639625.2021.1918035 · Cited by 25
Ambiguity and legal compliance
Barnum, T. C., & Nagin, D. S. (2021) · Criminology & Public Policy · doi:10.1111/1745-9133.12565 · Cited by 26
Denver, M., & DeWitt, S. E. (2022) · Criminology · doi:10.1111/1745-9125.12319 · Cited by 27
Targeting Type 2: Linguistic Agency Assignment in Diabetes Prevention Policy Messaging
Glowacki, E. M., McGlone, M. S., & Bell, R. A. (2016) · Journal of Health Communication · doi:10.1080/10810730.2015.1095821 · Cited by 24
Rupprecht, E. A., Kueny, C. R., Shoss, M. K., & Metzger, A. J. (2016) · Journal of Occupational Health Psychology · doi:10.1037/a0040074 · Cited by 24
Narcissism and perceived power in romantic relationships
Vrabel, J. K., Zeigler‐Hill, V., Lehtman, M. J., & Hernández, K. D. P. (2019) · Journal of Social and Personal Relationships · doi:10.1177/0265407519858685 · Cited by 33
The sound of soft alcohol: Crossmodal associations between interjections and liquor
Winter, B., Sobrino, P. P., & Brown, L. (2019) · PLoS ONE · doi:10.1371/journal.pone.0220449 · Cited by 27
Contractor, A. A., Weiss, N. H., Kearns, N. T., Caldas, S. V., & Dixon–Gordon, K. L. (2019) · International Journal of Stress Management · doi:10.1037/str0000145 · Cited by 24
Strickland, J. C., & Victor, G. A. (2019) · International Journal of Drug Policy · doi:10.1016/j.drugpo.2019.10.013 · Cited by 24
L2 self-paced reading data collection across three contexts: In-person, online, and crowdsourcing
Patterson, A. S., & Nicklin, C. (2023) · Research Methods in Applied Linguistics · doi:10.1016/j.rmal.2023.100045 · Cited by 23
Hyun, H., Hong, E., Ahn, J., & Park, J. (2024) · Journal of Travel & Tourism Marketing · doi:10.1080/10548408.2023.2239837 · Cited by 25
Barbaro, N., Boutwell, B. B., & Shackelford, T. K. (2019) · Personality and Individual Differences · doi:10.1016/j.paid.2019.05.008 · Cited by 25
Every ending is a new beginning: Poignancy increases consumer preferences for self-made products
Septianto, F. (2020) · International Journal of Research in Marketing · doi:10.1016/j.ijresmar.2020.11.004 · Cited by 24
Rounding up for a cause: The joint effect of donation type and crowding on donation likelihood
Hwang, Y., Choi, S., & Mattila, A. S. (2020) · International Journal of Hospitality Management · doi:10.1016/j.ijhm.2020.102779 · Cited by 25
Bursell, M., & Olsson, F. (2021) · Poetics · doi:10.1016/j.poetic.2021.101549 · Cited by 23
Cognitive Effects of Inattentive Responding in an MTurk Sample
Pyo, J., & Maxfield, M. G. (2021) · Social Science Quarterly · doi:10.1111/ssqu.12954 · Cited by 23
Situational Peer Dynamics and Crime Decisions
Barnum, T. C., & Pogarsky, G. (2022) · Journal of Research in Crime and Delinquency · doi:10.1177/00224278211070498 · Cited by 23
Chen, C., & Gonzales, L. (2022) · Journal of Health Psychology · doi:10.1177/13591053221079177 · Cited by 26
Digital and computational demography
Kashyap, R., Rinderknecht, R. G., Akbaritabar, A., Alburez‐Gutierrez, D., Gil‐Clavel, S., Grow, A., Kim, J., Leasure, D. R., Lohmann, S., Negraia, D. V., et al. (2023) · Edward Elgar Publishing eBooks · doi:10.4337/9781789906769.00010 · Cited by 26
Using the Beck Depression Inventory to Assess Anhedonia: A Scale Validation Study
Cogan, A. B., Persons, J. B., & Kring, A. M. (2023) · Assessment · doi:10.1177/10731911231164628 · Cited by 25
Using surveytainment to counter declining survey data quality
Kostyk, A., Zhou, W., & Hyman, M. R. (2018) · Journal of Business Research · doi:10.1016/j.jbusres.2018.10.024 · Cited by 25
Can Gossip Buffer the Effect of Job Insecurity on Workplace Friendships?
Jiang, L., Xu, X. V., & Hu, X. (2019) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph16071285 · Cited by 27
Gödöllei, A. F., & Beck, J. W. (2023) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2023.100342 · Cited by 28
Lacerenza, C. N., Johnson, S. K., Lambert, B. K., & Wagoner, H. P. V. (2024) · Journal of Organizational Behavior · doi:10.1002/job.2768 · Cited by 29
Why you shouldn’t trust data collected on MTurk
Kay, C. S. (2025) · Behavior Research Methods · doi:10.3758/s13428-025-02852-7 · Cited by 23
Fergus, T. A., & Bardeen, J. R. (2015) · Journal of Psychopathology and Behavioral Assessment · doi:10.1007/s10862-015-9511-0 · Cited by 25
Exploring the clinical profile of problematic pornography use
Ince, C., Yücel, M., Albertella, L., & Fontenelle, L. F. (2020) · CNS Spectrums · doi:10.1017/s1092852920001686 · Cited by 32
Should the EFQM Excellence Model be Adapted for Specific Industries? A Restaurant Sector Example
Liu, Y.-L., Ko, P.-F., Chiang, J.-T., & Shyr, W. (2020) · International Journal of Hospitality Management · doi:10.1016/j.ijhm.2020.102694 · Cited by 22
Bandyopadhyay, A., Septianto, F., & Nallaperuma, K. (2021) · Journal of Retailing and Consumer Services · doi:10.1016/j.jretconser.2021.102671 · Cited by 23
Jansson‐Knodell, C. L., White, M. K., Lockett, C. J., Xu, H., & Shin, A. (2021) · Clinical Gastroenterology and Hepatology · doi:10.1016/j.cgh.2021.12.021 · Cited by 26
Employees’ Death Awareness and Organizational Citizenship Behavior: A Moderated Mediation Model
Jacobsen, A., & Beehr, T. A. (2021) · Journal of Business and Psychology · doi:10.1007/s10869-021-09772-1 · Cited by 24
Hanink, P. A., & Dunbar, A. (2022) · Social movement studies · doi:10.1080/14742837.2022.2067842 · Cited by 23
How and when Narcissism and faith in humanity drive sustainable consumption
Bowen, K. T., Musarra, G., & Ou, Y. (2022) · Psychology and Marketing · doi:10.1002/mar.21694 · Cited by 27
Kamran, K., Azam, A., & Atif, M. M. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.801283 · Cited by 25
The Ethical, Societal, and Global Implications of Crowdsourcing Research
Du, S., Babalola, M. T., D’Cruz, P., Dóci, E., Garcia‐Lorenzo, L., Hassan, L. M., Islam, G., Newman, A., Noronha, E., & van Gils, S. (2024) · Journal of Business Ethics · doi:10.1007/s10551-023-05604-9 · Cited by 27
Dahling, J. J., Wiley, S., Fishman, Z. A., & Loihle, A. (2015) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2015.11.003 · Cited by 22
Bauhoff, S., Montero, A., & Scharf, D. M. (2016) · The American Journal of Drug and Alcohol Abuse · doi:10.1080/00952990.2016.1207654 · Cited by 22
Yelderman, L. A., & Miller, M. K. (2016) · Psychology Crime and Law · doi:10.1080/1068316x.2016.1239097 · Cited by 22
Stadthagen-González, H., Couto, M. C. P., Párraga, C. A., & Damian, M. F. (2017) · International Journal of Bilingualism · doi:10.1177/1367006917728390 · Cited by 22
Sex Differences in Distress From Infidelity in Early Adulthood and in Later Life
IJzerman, H., Blanken, I., Brandt, M. J., Oerlemans, J. M., van den Hoogenhof, M., Franken, S. J. M., & Oerlemans, M. W. G. (2014) · Social Psychology · doi:10.1027/1864-9335/a000185 · Cited by 21
Quantifying Users' Beliefs about Software Updates
Mathur, A., Malkin, N., Harbach, M., Peer, E., & Egelman, S. (2018) · Workshop on Usable Security (USEC) · doi:10.14722/usec.2018.23036 · Cited by 21
Self-citationTornquist, M., & Miles, E. (2019) · European Journal of Social Psychology · doi:10.1002/ejsp.2581 · Cited by 24
Wicaksono, A., Hendley, R., & Beale, R. (2019) · Interacting with Computers · doi:10.1093/iwc/iwz012 · Cited by 26
Black, J., Belicki, K., Piro, R., & Hughes, H. (2020) · OMEGA - Journal of Death and Dying · doi:10.1177/0030222820903850 · Cited by 22
Benfer, N., Rogers, T. A., & Bardeen, J. R. (2020) · Journal of Contextual Behavioral Science · doi:10.1016/j.jcbs.2020.06.003 · Cited by 21
Motivating language and employee outcomes: a multinational investigation
Nguyen, C. N., Ning, W., Alikaj, A., & Tran, Q. N. (2020) · Management Research Review · doi:10.1108/mrr-02-2020-0071 · Cited by 21
Jang, W., Wu, L., & Wen, J. (2020) · Sport Management Review · doi:10.1016/j.smr.2020.07.002 · Cited by 21
Extracting Agency and Communion From the Big Five: A Four-Way Competition
Entringer, T. M., Gebauer, J. E., & Paulhus, D. L. (2021) · Assessment · doi:10.1177/10731911211003978 · Cited by 22
Followership at a distance: Follower adjustment to distal leadership during COVID‐19
Carsten, M. K., Goswami, A., Shepard, A. K., & Donnelly, L. I. (2021) · Applied Psychology · doi:10.1111/apps.12337 · Cited by 24
Overconfident and unaware: Intellectual humility and the calibration of metacognition
Leman, J. C., Kurinec, C. A., & Rowatt, W. C. (2021) · The Journal of Positive Psychology · doi:10.1080/17439760.2021.1975155 · Cited by 24
How racialized policy contact shapes the social constructions of policy targets
Maltby, E., & Kreitzer, R. (2022) · Policy Studies Journal · doi:10.1111/psj.12481 · Cited by 21
Good Intentions, Bad Inventions: How Employees Judge Pervasive Technologies in the Workplace
Constantinides, M., & Quercia, D. (2022) · IEEE Pervasive Computing · doi:10.1109/mprv.2022.3217408 · Cited by 21
Customer engagement in idea contests: Emotional and behavioral consequences of idea rejection
Schaarschmidt, M., & Dose, D. B. (2023) · Psychology and Marketing · doi:10.1002/mar.21794 · Cited by 26
Fergus, T. A., & Rowatt, W. C. (2014) · Journal of Behavior Therapy and Experimental Psychiatry · doi:10.1016/j.jbtep.2014.09.003 · Cited by 21
Ewanation, L. A., & Maeder, E. M. (2018) · Canadian Journal of Criminology and Criminal Justice/La Revue canadienne de criminologie et de justice pénale · doi:10.3138/cjccj.2017-0047.r2 · Cited by 21
Metadata
- Authors
- Peer, E., Vosgerau, J., & Acquisti, A.
- Affiliations
- Tilburg University (NL)
- Carnegie Mellon University (US)
- Venue
- Behavior Research Methods
- Published
- 2013-12-20
- Volume
- 46
- Issue
- 4
- Pages
- 1023-1031
- ISSN
- 1554-351X (ISSN-L)
- 1554-3528
- OpenAlex
- W2105753174
- Cited by
- 1,618
- References
- 18
- Citations by year
- 2026: 68 · 2025: 96 · 2024: 128 · 2023: 167 · 2022: 175 · 2021: 212 · 2020: 210 · 2019: 147 · 2018: 119 · 2017: 132 · 2016: 91 · 2015: 49 · 2014: 17 · 2013: 1
- FWCI
- 79.80
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Behavior Research Methods (publishedVersion)
- Research portal (Tilburg University) (publishedVersion)
- PubMed (publishedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- SSRN Electronic Journal (submittedVersion)
- Figshare (submittedVersion, cc-by)
- Archivio istituzionale della Ricerca - Bocconi (submittedVersion)
- KiltHub Repository
- Topics
- Physical Sciences › Computer Science › Computer Science Applications › Mobile Crowdsensing and Crowdsourcing
- Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Keywords
- data quality, Amazon Mechanical Turk, worker reputation, crowdsourcing, selection bias, labor productivity, participant recruitment
- MeSH terms
- Data Collection, Humans, Research Design, Patient Selection, Internet, Behavioral Research, Crowdsourcing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
