The DOI matches its Crossref record; 17 of the 18 references it cites were found in a registry or an index.
Reputation as a sufficient condition for data quality on Amazon Mechanical Turk
Open accessAbstract
Data quality is one of the major concerns of using crowdsourcing websites such as Amazon Mechanical Turk (MTurk) to recruit participants for online behavioral studies. We compared two methods for ensuring data quality on MTurk: attention check questions (ACQs) and restricting participation to MTurk workers with high reputation (above 95% approval ratings). In Experiment 1, we found that high-reputation workers rarely failed ACQs and provided higher-quality data than did low-reputation workers; ACQs improved data quality only for low-reputation workers, and only in some cases. Experiment 2 corroborated these findings and also showed that more productive high-reputation workers produce the highest-quality data. We concluded that sampling high-reputation workers can ensure high-quality data without having to resort to using ACQs, which may lead to selection bias if participants who fail ACQs are excluded post-hoc.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Data obtained from MTurk has been demonstrated to be valid and reliable when appropriate selection criteria and attention checks are used (Casler, Bickel, & Hackett, 2013; Peer, Vosgerau, & Acquisti, 2014), as was the case in this study.”
From The development and evaluation of a brief form of the Normative Male Alexithymia Scale (NMAS-BF). (2019) · cites this paper as Peer, Vosgerau, & Acquisti, 2014
The cited source says what the claim attributes to it.
In the citing paper
“Most of the sites we reviewed have supervision and reputation mechanisms designed to measure the quality of submitted work, and to allow researchers to invite workers with high reputation (similar to MTurk's”
From Beyond the Turk: Alternative platforms for crowdsourcing behavioral research (2017) · cites this paper as Peer et al., 2014
In the citing paper
“Amazon Mechanical Turk is an online sample recruiting platform that, although not nationally representative, yields high data quality through the use of attention checks 44 and worker reputation scores 45 .”
From Understanding, explaining, and utilizing medical artificial intelligence (2021) · cites this paper as [45]
In the citing paper
“Following prior research, we selected participants with an approval rating above 95% on Amazon Mechanical Turk 45 .”
From Understanding, explaining, and utilizing medical artificial intelligence (2021) · cites this paper as [45]
In the citing paper
“which showed that ACQs have low diagnostic ability when data quality is already high.”
From Beyond the Turk: Alternative platforms for crowdsourcing behavioral research (2017) · cites this paper as Peer et al., (2014)
In the citing paper
“they are reliable and validated scales, and b) they have been previously used with success to measure data quality on MTurk (Peer et al., 2014).”
From Beyond the Turk: Alternative platforms for crowdsourcing behavioral research (2017) · cites this paper as Peer et al., 2014
In the citing paper
“they are reliable and validated scales, and b) they have been previously used with success to measure data quality on MTurk (Peer et al., 2014).”
From Beyond the Turk: Alternative platforms for crowdsourcing behavioral research (2017) · cites this paper as Peer et al., 2014
In the citing paper
“Using two attention-check questions (Peer et al., 2014), we checked whether participants read and paid attention to our instructions.”
From Beyond the Turk: Alternative platforms for crowdsourcing behavioral research (2017) · cites this paper as Peer et al., 2014
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Tversky, A., & Kahneman, D. (1974) · Science · doi:10.1126/science.185.4157.1124 · Cited by 28,810
Judgment under Uncertainty: Heuristics and Biases
Tversky, A., & Kahneman, D. (1975) · Utility, Probability, and Human Decision Making · doi:10.1007/978-94-010-1834-0_8 · Cited by 22,650
Amazon's Mechanical Turk: A New Source of Inexpensive, Yet High-Quality, Data?
Buhrmester, M., Kwang, T., & Gosling, S. D. (2011) · Perspectives on Psychological Science · doi:10.1177/1745691610393980 · Cited by 9,996
A very brief measure of the Big-Five personality domains
Gosling, S. D., Rentfrow, P. J., & Swann, W. B. (2003) · Journal of Research in Personality · doi:10.1016/s0092-6566(03)00046-1 · Cited by 8,307
Running experiments on Amazon Mechanical Turk
Paolacci, G., Chandler, J., & Ipeirotis, P. G. (2010) · Judgment and Decision Making · doi:10.1017/s1930297500002205 · Cited by 3,767
Instructional manipulation checks: Detecting satisficing to increase statistical power
Oppenheimer, D. M., Meyvis, T., & Davidenko, N. (2009) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2009.03.009 · Cited by 3,493
Robins, R. W., Hendin, H. M., & Trzesniewski, K. H. (2001) · Personality and Social Psychology Bulletin · doi:10.1177/0146167201272002 · Cited by 3,041
The Efficient Assessment of Need for Cognition
Cacioppo, J. T., Petty, R. E., & Feng Kao, C. (1984) · Journal of Personality Assessment · doi:10.1207/s15327752jpa4803_13 · Cited by 2,913
Data collection in a flat world: the strengths and weaknesses of Mechanical Turk samples
Goodman, J. K., Cryder, C. E., & Cheema, A. (2013) · Journal of Behavioral Decision Making · doi:10.1002/bdm.1753 · Cited by 2,242
Rosenberg, M. (2011) · PsycTESTS Dataset · doi:10.1037/t01038-000 · Cited by 1,226
Cited by
Sorted by citations
Items 701–800 of 1,000
Trump Trumps Baldwin? How Trump’s Tweets Transform SNL into Trump’s Strategic Advantage
Becker, A. B. (2017) · Journal of Political Marketing · doi:10.1080/15377857.2017.1411860 · Cited by 16
Fogel, R. S., Comerford, M., Chilukuri, P., Orman, E. S., Chalasani, N., & Lammert, C. S. (2018) · Interactive Journal of Medical Research · doi:10.2196/ijmr.9625 · Cited by 16
Fiset, J. E., Robinson, M. A., & Robertson, M. C. S. (2019) · European Journal of Work and Organizational Psychology · doi:10.1080/1359432x.2019.1637853 · Cited by 16
Can Expert Testimony Sensitize Jurors to Coercive Interrogation Tactics?
Jones, A. M., & Penrod, S. (2016) · Journal of Forensic Psychology Practice · doi:10.1080/15228932.2016.1232029 · Cited by 16
Crowdsourcing Analysis of Twitter Data on Climate Change: Paid Workers vs. Volunteers
Kirilenko, A., Desell, T., Kim, H., & Stepchenkova, S. O. (2017) · Sustainability · doi:10.3390/su9112019 · Cited by 16
Malivoire, B. L., Stewart, K. E., Tallon, K., Ovanessian, M. M., Pawluk, E. J., & Koerner, N. (2019) · Personality and Individual Differences · doi:10.1016/j.paid.2019.03.021 · Cited by 18
Work–family conflict, sleep, and health
Bettac, E. L., & Probst, T. M. (2019) · International Journal of Manpower · doi:10.1108/ijm-02-2019-0106 · Cited by 18
Woike, J. K., Collard, P., & Hood, B. M. (2020) · PLoS ONE · doi:10.1371/journal.pone.0228271 · Cited by 16
Pursuit of Harmony: Relational and Individual Predictors of Sexual Passion Expression
Leonhardt, N. D., Busby, D. M., & Valdez, C. M. (2020) · Archives of Sexual Behavior · doi:10.1007/s10508-020-01769-w · Cited by 15
Mechanical Turk and the “Don’t Know” Option
Brown, A. R., & Pope, J. C. (2021) · PS Political Science & Politics · doi:10.1017/s1049096520001651 · Cited by 15
Psychosocial determinants of anxiety about the COVID-19 pandemic
Wright, R. N., Faul, L., Graner, J. L., Stewart, G. W., & LaBar, K. S. (2021) · Journal of Health Psychology · doi:10.1177/13591053211030981 · Cited by 15
Vakhitova, Z., Mawby, R. I., Helps, N., & Alston‐Knox, C. (2022) · Crime & Delinquency · doi:10.1177/00111287221106962 · Cited by 15
Engaging employees through transformational leadership: the mediating role of emotional energy
Bao, S., & Jolly, P. M. (2023) · Journal of Hospitality and Tourism Insights · doi:10.1108/jhti-03-2023-0152 · Cited by 20
Sumner, R. (2016) · Applied Research in Quality of Life · doi:10.1007/s11482-016-9448-9
Patterns of relational responding and a healthy self in older adolescents
Moran, O., & McHugh, L. (2019) · Journal of Contextual Behavioral Science · doi:10.1016/j.jcbs.2019.02.002 · Cited by 15
Johnson, J. M., & Lecci, L. B. (2019) · British Journal of Social Psychology · doi:10.1111/bjso.12347 · Cited by 15
Smartphones within Psychological Science
Ellis, D. A. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108671408 · Cited by 14
Russell, A. M., Barry, A. E., & Patterson, M. S. (2020) · Drug and Alcohol Review · doi:10.1111/dar.13140 · Cited by 15
Telemedicine for Acute Conditions During COVID-19: A Nationwide Survey Using Crowdsourcing
Kato-Lin, Y.-C., & Thelen, S. T. (2020) · Telemedicine Journal and e-Health · doi:10.1089/tmj.2020.0351 · Cited by 15
Capturing women’s voices: lived experiences of incivility during childbirth in the United States
Vargas, E. A., Marshall, R. A., & Mahalingam, R. (2021) · Women & Health · doi:10.1080/03630242.2021.1957745 · Cited by 15
Sedentary Behaviors, Sleep, and Health-related Quality of Life in Middle-aged Adults
Chen, Y. T., Holahan, C. K., & Castelli, D. M. (2021) · American Journal of Health Behavior · doi:10.5993/ajhb.45.4.16 · Cited by 15
Coping with Digital Extortion: An Experimental Study of Benefit Appeals and Normative Appeals
Chen, K., Wang, J., & Lang, Y. (2021) · Management Science · doi:10.1287/mnsc.2021.4154 · Cited by 15
Do political actions affect country image and travel intentions? The case of COVID-19
Gohary, A., Shah, A., & Chan, E. Y. (2022) · Tourism Recreation Research · doi:10.1080/02508281.2022.2034104 · Cited by 15
Bazzoli, A., & Probst, T. M. (2022) · Applied Psychology · doi:10.1111/apps.12397 · Cited by 15
Moral foundations and obsessive-compulsive symptoms: A preliminary examination
Kang, L. L., Rowatt, W. C., & Fergus, T. A. (2016) · Journal of Obsessive-Compulsive and Related Disorders · doi:10.1016/j.jocrd.2016.06.004 · Cited by 17
Opposing Affective and Cognitive Effects of Prevention Focus on Counterproductive Work Behavior
Lin, S., & Johnson, R. E. (2017) · Journal of Business and Psychology · doi:10.1007/s10869-017-9493-x · Cited by 15
da Costa, H. P., Vrabel, J. K., Zeigler‐Hill, V., & Vonk, J. M. (2018) · Journal of Research in Personality · doi:10.1016/j.jrp.2018.05.001 · Cited by 16
Effects of Power and Implicit Theories on Donation
Khalil, M., Khan, S. R., & Septianto, F. (2020) · Australasian Marketing Journal (AMJ) · doi:10.1016/j.ausmj.2020.07.002 · Cited by 17
Lee, B., Liu, J., Choung, H., & McLeod, D. M. (2020) · Journalism & Mass Communication Quarterly · doi:10.1177/1077699020934195 · Cited by 15
A survey on participant recruitment in crowdsensing systems
Davari, M., & Amintoosi, H. (2016) · International Conference on Computer and Knowledge Engineering · doi:10.1109/iccke.2016.7802154 · Cited by 14
Improving juror sensitivity to specific eyewitness factors: judicial instructions fail the test
Jones, A. M., Bergold, A. N., & Penrod, S. (2020) · Psychiatry Psychology and Law · doi:10.1080/13218719.2020.1719379 · Cited by 15
Clauss, K. E., Witte, T. K., & Bardeen, J. R. (2021) · Journal of Personality Assessment · doi:10.1080/00223891.2021.1887879 · Cited by 14
Korman, B. A., Tröster, C., & Giessner, S. R. (2021) · Journal of Leadership & Organizational Studies · doi:10.1177/15480518211005463 · Cited by 15
Liu, S., Wei, C., Kalgotra, P., & Çobanoğlu, C. (2022) · International Journal of Hospitality Management · doi:10.1016/j.ijhm.2022.103248 · Cited by 16
A Latent Profile Analysis of Precarity and Its Associated Outcomes: The Haves and the Have-Nots
Bazzoli, A., Probst, T. M., & Tomas, J. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph19137582 · Cited by 16
Butler, L. C., Fissel, E. R., Gildea, B., & Fisher, B. S. (2023) · Victims & Offenders · doi:10.1080/15564886.2022.2139032 · Cited by 14
Public attitudes and preferences for green rooftop technologies in the US: a choice experiment
Meyer, N., & Trandafir, S. (2023) · Agricultural and Resource Economics Review · doi:10.1017/age.2023.17 · Cited by 15
Keeping warm: racial disparities of warmth judgments in the service industry
Lee, L., Yu, H. C., & Jolly, P. M. (2023) · International Journal of Contemporary Hospitality Management · doi:10.1108/ijchm-12-2022-1510 · Cited by 14
Priming through Persuasion: Towards Secure Password Behavior
Paudel, R., & Al-Ameen, M. N. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3637387 · Cited by 14
Caton, N. R., Brown, L. M., Zhao, A. A. Z., & Dixson, B. J. W. (2024) · Human Nature · doi:10.1007/s12110-024-09473-7 · Cited by 14
Wu, S., Zheng, X., Ni, D., Chen, C., Wang, M., Wu, W., & Zhai, H. (2024) · Journal of Organizational Behavior · doi:10.1002/job.2843 · Cited by 17
DQ2S – A framework for data quality-aware information management
Sampaio, S., Dong, C., & Sampaio, P. (2015) · Expert Systems with Applications · doi:10.1016/j.eswa.2015.06.050 · Cited by 14
Martin, K. D., & Fowle, M. Z. (2020) · Sociological perspectives · doi:10.1177/0731121420970599 · Cited by 12
Jiang, Q., Cheng, Y., & Cho, S. K. (2021) · Chinese Journal of Communication · doi:10.1080/17544750.2021.1902360 · Cited by 14
The Christian Sanctification of Suffering Scale: measure development and relationship to well-being
Hall, M. E. L., McMartin, J., Wang, D., Shannonhouse, L., Aten, J. D., Silverman, E. J., & Decker, L. A. (2021) · Mental Health Religion & Culture · doi:10.1080/13674676.2021.1884670 · Cited by 15
Women’s attractiveness contingent self-esteem, romantic rejection, and body dissatisfaction
Harrington, A. G., & Overall, N. C. (2021) · Body Image · doi:10.1016/j.bodyim.2021.06.004 · Cited by 17
Investigating the impact of ransomware splash screens
Yilmaz, Y., Çetin, O., Arief, B., & Hernandez-Castro, J. C. (2021) · Journal of Information Security and Applications · doi:10.1016/j.jisa.2021.102934 · Cited by 16
Parent, M. C., & Wille, L. (2021) · Archives of Sexual Behavior · doi:10.1007/s10508-021-01968-z · Cited by 14
COVID-19 Impact Battery: Development and Validation
Schmidt, N. B., Allan, N. P., Koscinski, B., Mathes, B. M., Eackles, K., Accorso, C., Saulnier, K. G., Allan, D. M., Potter, K., Garey, L., et al. (2021) · Journal of Psychopathology and Behavioral Assessment · doi:10.1007/s10862-021-09919-7 · Cited by 14
Albayram, Y., Liu, J., & Cangonj, S. (2021) · European Symposium on Usable Security (EuroUSEC) · doi:10.1145/3481357.3481519 · Cited by 14
Dunlop, P. D., Holtrop, D., Ashby, L. M., Jolly, A. A., & Donovan, J. J. (2022) · Journal of Business and Psychology · doi:10.1007/s10869-022-09797-0 · Cited by 14
Promoting HPV vaccination among Latinx: an application of the extended parallel processing model
Reno, J. E., & Dempsey, A. F. (2022) · Journal of Behavioral Medicine · doi:10.1007/s10865-022-00293-7 · Cited by 18
Tornquist, M., & Miles, E. (2022) · Self and Identity · doi:10.1080/15298868.2022.2146183 · Cited by 18
Public Perceptions of Community Pharmacy-Based Naloxone Services: A National Cross-Sectional Survey
Hohmann, L. A., Krauss, Z., Patel, J., & Marley, G. T. (2022) · Pharmacy · doi:10.3390/pharmacy10060171 · Cited by 16
Ready to forget: American attitudes toward the right to be forgotten
Bode, L., & Jones, M. L. (2017) · The Information Society · doi:10.1080/01972243.2016.1271071 · Cited by 15
I heard it through the grapevine: Where and what parents learn about youth mental health treatments
O’Connor, E., & Langer, D. A. (2018) · Journal of Clinical Psychology · doi:10.1002/jclp.22706 · Cited by 14
Identification performance from multiple lineups: Should eyewitnesses who pick fillers be burned?
Smalarz, L., Kornell, N., Vaughn, K. E., & Palmer, M. A. (2019) · Journal of Applied Research in Memory and Cognition · doi:10.1016/j.jarmac.2019.03.001 · Cited by 14
When “decoy effect” meets gender bias: The role of choice set composition in hiring decisions
Keck, S., & Tang, W. (2019) · Journal of Behavioral Decision Making · doi:10.1002/bdm.2157 · Cited by 14
Identifying Statistical Bias in Dataset ReplicationPreprint
Engstrom, L., Ilyas, A., Santurkar, S., Tsipras, D., Steinhardt, J., & Mądry, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.09619 · Cited by 14
The development and evaluation of a brief form of the Normative Male Alexithymia Scale (NMAS-BF).
Levant, R. F., & Parent, M. C. (2019) · Journal of Counseling Psychology · doi:10.1037/cou0000312 · Cited by 12
Do Americans Want a Right to be Forgotten? Estimating Public Support for Digital Erasure Legislation
Bode, L., & Jones, M. L. (2018) · Policy & Internet · doi:10.1002/poi3.174 · Cited by 13
'Surprised, Shocked, Worried'}: User Reactions to Facebook Data Collection from Third Parties
Arias-Cabarcos, P., Khalili, S., & Strufe, T. (2023) · Proceedings on Privacy Enhancing Technologies · doi:10.56553/popets-2023-0023 · Cited by 20
Why Casting Older Female Models Is Good for Advertising
Berg, H., & Liljedal, K. T. (2023) · Journal of Advertising Research · doi:10.2501/jar-2023-001 · Cited by 17
Explaining technology migration against the change in terms of use: an fsQCA approach
Dahabiyeh, L., Farooq, A., Ahmad, F., & Javed, Y. (2023) · Information Technology and People · doi:10.1108/itp-07-2022-0498 · Cited by 13
Kolc, K. L., Tan, Y. X. K., Lo, A. Z. Y., Shvetcov, A., Mitchell, P. B., & Perkes, I. E. (2023) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2023.05.027 · Cited by 13
Henderson, K. E., & Welsh, E. T. (2023) · International Journal of Selection and Assessment · doi:10.1111/ijsa.12454 · Cited by 16
Wang, W., Kang, S., Choi, S. B., & Jeung, W. (2024) · Leadership & Organization Development Journal · doi:10.1108/lodj-01-2023-0001 · Cited by 14
Shrestha, A., Behfar, A., & Al-Ameen, M. N. (2024) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2024.2323248 · Cited by 13
Assessing quality of unmet user needs: Effects of need statement characteristics
Schaffhausen, C. R., & Kowalewski, T. M. (2016) · Design Studies · doi:10.1016/j.destud.2016.01.002 · Cited by 14
Sahakyan, L., Kwapil, T. R., & Jiang, L. (2019) · Journal of Experimental Psychology General · doi:10.1037/xge0000645 · Cited by 13
Exploring the Effect of Incorporating Danmaku into Advertising
Wei, L., Sun, T., & Liu, B. (2019) · Journal of Interactive Advertising · doi:10.1080/15252019.2019.1691094 · Cited by 13
Masson, G., Mills, K., Griffin, S. J., Sharp, S. J., Klein, W. M. P., Sutton, S. R., & Usher‐Smith, J. A. (2020) · Preventive Medicine · doi:10.1016/j.ypmed.2020.106154 · Cited by 13
Neuman, E. L., & Sheu, R. J. (2021) · Journal of the American Taxation Association · doi:10.2308/jata-2020-038 · Cited by 13
Forward-Looking Belief Elicitation Enhances Inter-Generational Beneficence
Bosetti, V., Dennig, F., Liu, N., Tavoni, M., & Weber, E. U. (2020) · Environmental and Resource Economics · doi:10.1007/s10640-022-00648-3 · Cited by 13
Public Opinion about America’s Opioid Crisis: Severity, Sources, and Solutions in Context
Sun, D. K., Graham, A., Feldmeyer, B., Cullen, F. T., & Kulig, T. C. (2022) · Deviant Behavior · doi:10.1080/01639625.2022.2071656 · Cited by 13
Jones, A. M., & Brimbal, L. (2017) · Journal of Investigative Psychology and Offender Profiling · doi:10.1002/jip.1476 · Cited by 13
Lorona, R. T., Rowatt, W. C., & Fergus, T. A. (2017) · Journal of Personality Assessment · doi:10.1080/00223891.2017.1303774 · Cited by 13
Mental Contrasting of a Negative Future with a Positive Reality Regulates State Anxiety
Brodersen, G., & Oettingen, G. (2017) · Frontiers in Psychology · doi:10.3389/fpsyg.2017.01596 · Cited by 13
The impact of financial strain on medication non-adherence: Influence of psychiatric medication use
Strickland, J. C., Stoops, W. W., Kincer, M. A., & Rush, C. R. (2018) · Psychiatry Research · doi:10.1016/j.psychres.2018.11.055 · Cited by 13
Cassino, D., & Besen‐Cassino, Y. (2020) · European Journal of Politics and Gender · doi:10.1332/251510820x15912551895078 · Cited by 13
Diners’ responses to talent vs. effort of restaurant employees
Hwang, Y., & Shin, J. (2021) · International Journal of Hospitality Management · doi:10.1016/j.ijhm.2021.102958 · Cited by 13
Visual vignettes for cross-national research
Facciani, M. J., Brashears, M. E., & Zhong, J. (2020) · International Journal of Social Research Methodology · doi:10.1080/13645579.2020.1844897 · Cited by 12
An Examination of the Link between Leader Motivating Language and Follower Interactional Justice
Alikaj, A., & Hanke, D. (2021) · International Journal of Business Communication · doi:10.1177/23294884211011240 · Cited by 13
Carnahan, D., Ulusoy, E., Barry, R. C., McGraw, J., Virtue, I., & Bergan, D. E. (2022) · Journal of Communication · doi:10.1093/joc/jqac023 · Cited by 16
Making the call: how does perceived race affect desire to call the police?
Sola, J. L., & Kubrin, C. E. (2023) · Journal of Experimental Criminology · doi:10.1007/s11292-023-09571-z · Cited by 13
Jang, W. W., Choi, W., Kim, M. J., Song, H., & Byon, K. K. (2024) · International Journal of Sports Marketing and Sponsorship · doi:10.1108/ijsms-07-2023-0132 · Cited by 12
Kim, H., Park, S., Gim, J., & Kim, S. I. (2024) · International Journal of Hospitality Management · doi:10.1016/j.ijhm.2024.103849 · Cited by 15
Supplying Cash-Constrained Retailers: Understanding Shopkeeper Behavior at the Bottom of the Pyramid
Villa, S., Escamilla, R., & Fransoo, J. C. (2021) · Manufacturing & Service Operations Management · doi:10.1287/msom.2022.0362 · Cited by 12
Measuring Privacy Concern and the Right to Be Forgotten
Steinbart, P. J., Keith, M. J., & Babb, J. S. (2017) · Hawaii International Conference on System Sciences (HICSS) · doi:10.24251/hicss.2017.603 · Cited by 13
Who Is Most Likely to Wear Rose-Colored Glasses? How Regulatory Mode Moderates Self-Flattery
Komissarouk, S., Chernikova, M., Kruglanski, A. W., & Higgins, E. T. (2018) · Personality and Social Psychology Bulletin · doi:10.1177/0146167218783194 · Cited by 12
Drug Legalization and Decriminalization Beliefs Among Substance-using and Nonusing Individuals
Hammond, A. S., Dunn, K. E., & Strain, E. C. (2019) · Journal of Addiction Medicine · doi:10.1097/adm.0000000000000542 · Cited by 13
Shin, I., & Hampton, K. N. (2021) · Information Communication & Society · doi:10.1080/1369118x.2021.1950804 · Cited by 12
Transmitting Desire: An Experiment on a Novel Measure of Gun Desirability in a Pandemic
Sola, J. L. (2021) · Sociological Perspectives · doi:10.1177/07311214211007179 · Cited by 12
Abad, Z. S. H., Butler, G., Thompson, W., & Lee, J. (2021) · Journal of Medical Internet Research · doi:10.2196/28749 · Cited by 12
Does social class matter in recovering self-service technology failures?
Kim, H., & Jang, S. S. (2021) · International Journal of Contemporary Hospitality Management · doi:10.1108/ijchm-06-2021-0741 · Cited by 12
Lemos, R. F., Tegge, A. N., Craft, W. H., Tomlinson, D. C., Stein, J. S., & Bickel, W. K. (2022) · Experimental and Clinical Psychopharmacology · doi:10.1037/pha0000579 · Cited by 12
Arai, A., Ko, Y. J., Asada, A., & Connaughton, D. P. (2023) · Sport Marketing Quarterly · doi:10.32731/smq.321.032023.05 · Cited by 12
The Importance of Worker Reputation Information in Microtask-Based Crowd Work Systems
Tarable, A., Nordio, A., Leonardi, E., & Marsan, M. A. (2016) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2016.2572078 · Cited by 12
Presidential Appointments and Public Trust
Hollibaugh, G. E. (2016) · Presidential Studies Quarterly · doi:10.1111/psq.12298 · Cited by 12
Dyadic power, marital status, face concerns, and politeness in the context of relational complaining
Worley, T. R., & Samp, J. A. (2018) · Journal of Social and Personal Relationships · doi:10.1177/0265407518765988 · Cited by 12
Metadata
- Authors
- Peer, E., Vosgerau, J., & Acquisti, A.
- Affiliations
- Tilburg University (NL)
- Carnegie Mellon University (US)
- Venue
- Behavior Research Methods
- Published
- 2013-12-20
- Volume
- 46
- Issue
- 4
- Pages
- 1023-1031
- ISSN
- 1554-351X (ISSN-L)
- 1554-3528
- OpenAlex
- W2105753174
- Cited by
- 1,618
- References
- 18
- Citations by year
- 2026: 68 · 2025: 96 · 2024: 128 · 2023: 167 · 2022: 175 · 2021: 212 · 2020: 210 · 2019: 147 · 2018: 119 · 2017: 132 · 2016: 91 · 2015: 49 · 2014: 17 · 2013: 1
- FWCI
- 79.80
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Behavior Research Methods (publishedVersion)
- Research portal (Tilburg University) (publishedVersion)
- PubMed (publishedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- SSRN Electronic Journal (submittedVersion)
- Figshare (submittedVersion, cc-by)
- Archivio istituzionale della Ricerca - Bocconi (submittedVersion)
- KiltHub Repository
- Topics
- Physical Sciences › Computer Science › Computer Science Applications › Mobile Crowdsensing and Crowdsourcing
- Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Keywords
- data quality, Amazon Mechanical Turk, worker reputation, crowdsourcing, selection bias, labor productivity, participant recruitment
- MeSH terms
- Data Collection, Humans, Research Design, Patient Selection, Internet, Behavioral Research, Crowdsourcing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
