The DOI matches its Crossref record; 17 of the 18 references it cites were found in a registry or an index.
Reputation as a sufficient condition for data quality on Amazon Mechanical Turk
Open accessAbstract
Data quality is one of the major concerns of using crowdsourcing websites such as Amazon Mechanical Turk (MTurk) to recruit participants for online behavioral studies. We compared two methods for ensuring data quality on MTurk: attention check questions (ACQs) and restricting participation to MTurk workers with high reputation (above 95% approval ratings). In Experiment 1, we found that high-reputation workers rarely failed ACQs and provided higher-quality data than did low-reputation workers; ACQs improved data quality only for low-reputation workers, and only in some cases. Experiment 2 corroborated these findings and also showed that more productive high-reputation workers produce the highest-quality data. We concluded that sampling high-reputation workers can ensure high-quality data without having to resort to using ACQs, which may lead to selection bias if participants who fail ACQs are excluded post-hoc.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Data obtained from MTurk has been demonstrated to be valid and reliable when appropriate selection criteria and attention checks are used (Casler, Bickel, & Hackett, 2013; Peer, Vosgerau, & Acquisti, 2014), as was the case in this study.”
From The development and evaluation of a brief form of the Normative Male Alexithymia Scale (NMAS-BF). (2019) · cites this paper as Peer, Vosgerau, & Acquisti, 2014
The cited source says what the claim attributes to it.
In the citing paper
“Most of the sites we reviewed have supervision and reputation mechanisms designed to measure the quality of submitted work, and to allow researchers to invite workers with high reputation (similar to MTurk's”
From Beyond the Turk: Alternative platforms for crowdsourcing behavioral research (2017) · cites this paper as Peer et al., 2014
In the citing paper
“Amazon Mechanical Turk is an online sample recruiting platform that, although not nationally representative, yields high data quality through the use of attention checks 44 and worker reputation scores 45 .”
From Understanding, explaining, and utilizing medical artificial intelligence (2021) · cites this paper as [45]
In the citing paper
“Following prior research, we selected participants with an approval rating above 95% on Amazon Mechanical Turk 45 .”
From Understanding, explaining, and utilizing medical artificial intelligence (2021) · cites this paper as [45]
In the citing paper
“which showed that ACQs have low diagnostic ability when data quality is already high.”
From Beyond the Turk: Alternative platforms for crowdsourcing behavioral research (2017) · cites this paper as Peer et al., (2014)
In the citing paper
“they are reliable and validated scales, and b) they have been previously used with success to measure data quality on MTurk (Peer et al., 2014).”
From Beyond the Turk: Alternative platforms for crowdsourcing behavioral research (2017) · cites this paper as Peer et al., 2014
In the citing paper
“they are reliable and validated scales, and b) they have been previously used with success to measure data quality on MTurk (Peer et al., 2014).”
From Beyond the Turk: Alternative platforms for crowdsourcing behavioral research (2017) · cites this paper as Peer et al., 2014
In the citing paper
“Using two attention-check questions (Peer et al., 2014), we checked whether participants read and paid attention to our instructions.”
From Beyond the Turk: Alternative platforms for crowdsourcing behavioral research (2017) · cites this paper as Peer et al., 2014
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Tversky, A., & Kahneman, D. (1974) · Science · doi:10.1126/science.185.4157.1124 · Cited by 28,810
Judgment under Uncertainty: Heuristics and Biases
Tversky, A., & Kahneman, D. (1975) · Utility, Probability, and Human Decision Making · doi:10.1007/978-94-010-1834-0_8 · Cited by 22,650
Amazon's Mechanical Turk: A New Source of Inexpensive, Yet High-Quality, Data?
Buhrmester, M., Kwang, T., & Gosling, S. D. (2011) · Perspectives on Psychological Science · doi:10.1177/1745691610393980 · Cited by 9,996
A very brief measure of the Big-Five personality domains
Gosling, S. D., Rentfrow, P. J., & Swann, W. B. (2003) · Journal of Research in Personality · doi:10.1016/s0092-6566(03)00046-1 · Cited by 8,307
Running experiments on Amazon Mechanical Turk
Paolacci, G., Chandler, J., & Ipeirotis, P. G. (2010) · Judgment and Decision Making · doi:10.1017/s1930297500002205 · Cited by 3,767
Instructional manipulation checks: Detecting satisficing to increase statistical power
Oppenheimer, D. M., Meyvis, T., & Davidenko, N. (2009) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2009.03.009 · Cited by 3,493
Robins, R. W., Hendin, H. M., & Trzesniewski, K. H. (2001) · Personality and Social Psychology Bulletin · doi:10.1177/0146167201272002 · Cited by 3,041
The Efficient Assessment of Need for Cognition
Cacioppo, J. T., Petty, R. E., & Feng Kao, C. (1984) · Journal of Personality Assessment · doi:10.1207/s15327752jpa4803_13 · Cited by 2,913
Data collection in a flat world: the strengths and weaknesses of Mechanical Turk samples
Goodman, J. K., Cryder, C. E., & Cheema, A. (2013) · Journal of Behavioral Decision Making · doi:10.1002/bdm.1753 · Cited by 2,242
Rosenberg, M. (2011) · PsycTESTS Dataset · doi:10.1037/t01038-000 · Cited by 1,226
Cited by
Sorted by citations
Items 101–200 of 1,000
The micro-task market for lemons: data quality on Amazon's Mechanical Turk
Ahler, D. J., Roush, C. E., & Sood, G. (2021) · Political Science Research and Methods · doi:10.1017/psrm.2021.57 · Cited by 121
Investment Experience, Financial Literacy, and Investment‐Related Judgments
Krische, S. D. (2018) · Contemporary Accounting Research · doi:10.1111/1911-3846.12469 · Cited by 126
Fergus, T. A., & Russell, L. H. (2016) · Journal of Anxiety Disorders · doi:10.1016/j.janxdis.2016.01.009 · Cited by 116
Chen, M., Bell, R., & Taylor, L. (2016) · Journal of health communication · doi:10.1080/10810730.2016.1177147 · Cited by 102
Levant, R. F., McDermott, R. C., Parent, M. C., Alshabani, N., Mahalik, J. R., & Hammer, J. H. (2020) · Journal of Counseling Psychology · doi:10.1037/cou0000414 · Cited by 135
Cooksey Stowers, K., Jiang, Q., Atoloye, A., Lucan, S., & Gans, K. (2020) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph17197143 · Cited by 117
Andel, S. A., Shen, W., & Arvan, M. L. (2021) · Journal of Occupational Health Psychology · doi:10.1037/ocp0000271 · Cited by 132
Graham, A., Pickett, J. T., & Cullen, F. T. (2021) · Crime & Delinquency · doi:10.1177/0011128720977439 · Cited by 114
Using Mechanical Turk to Study Family Processes and Youth Mental Health: A Test of Feasibility
Schleider, J. L., & Weisz, J. R. (2015) · Journal of Child and Family Studies · doi:10.1007/s10826-015-0126-6 · Cited by 112
Lovett, M., Bajaba, S. M., Lovett, M., & Simmering, M. J. (2017) · Applied Psychology · doi:10.1111/apps.12124 · Cited by 125
How Guests Develop Trust in Hosts: An Investigation of Trust Formation in P2P Accommodation
Park, S., & Tussyadiah, I. P. (2019) · Journal of Travel Research · doi:10.1177/0047287519884654 · Cited by 122
Problematic internet use and internet searches for medical information: the role of health anxiety
Fergus, T. A., & Dolan, S. L. (2014) · Cyberpsychology Behavior and Social Networking · doi:10.1089/cyber.2014.0169 · Cited by 112
Does the dark side of a calling exist? Examining potential negative effects
Duffy, R. D., Douglass, R. P., Autin, K. L., England, J. W., & Dik, B. J. (2016) · The Journal of Positive Psychology · doi:10.1080/17439760.2015.1137626 · Cited by 111
Toward A Bifurcated Theory of Emotional Deterrence
Pickett, J. T., Roche, S. P., & Pogarsky, G. (2018) · Criminology · doi:10.1111/1745-9125.12153 · Cited by 94
A Technical Guide to Using Amazon's Mechanical Turk in Behavioral Accounting Research
Buchheit, S., Doxey, M. M., Pollard, T., & Stinson, S. R. (2017) · Behavioral Research in Accounting · doi:10.2308/bria-51977 · Cited by 115
Kim, M., & Beehr, T. A. (2019) · The International Journal of Human Resource Management · doi:10.1080/09585192.2019.1657166 · Cited by 126
Is the Convenience Worth the Risk? An Investigation of Mobile Payment Usage
Pal, A., Herath, T. C., Dé, R., & Rao, H. R. (2020) · Information Systems Frontiers · doi:10.1007/s10796-020-10070-z · Cited by 120
Moss, A. J., Rosenzweig, C., Robinson, J., Jaffe, S. N., & Litman, L. (2023) · Behavior Research Methods · doi:10.3758/s13428-022-02005-0 · Cited by 129
HEURISTICS AND BIASES, RATIONAL CHOICE, AND SANCTION PERCEPTIONS*
Pogarsky, G., Roche, S. P., & Pickett, J. T. (2017) · Criminology · doi:10.1111/1745-9125.12129 · Cited by 107
Sirota, M., & Juanchich, M. (2018) · Behavior Research Methods · doi:10.3758/s13428-018-1029-4 · Cited by 120
Racial/Ethnic Perceptions from Hispanic Names: Selecting Names to Test for Discrimination
Gaddis, S. M. (2017) · Socius Sociological Research for a Dynamic World · doi:10.1177/2378023117737193 · Cited by 106
The scales of general well-being (SGWB)
Longo, Y., Coyne, I. J., & Joseph, S. (2017) · Personality and Individual Differences · doi:10.1016/j.paid.2017.01.005 · Cited by 110
Chun, J. W., & Lee, M. J. (2017) · Computers in Human Behavior · doi:10.1016/j.chb.2017.04.010 · Cited by 113
Trap questions in online surveys: Results from three web survey experiments
Liu, M., & Wronski, L. (2018) · International Journal of Market Research · doi:10.1177/1470785317744856 · Cited by 103
Kim, M., & Beehr, T. A. (2019) · International Journal of Stress Management · doi:10.1037/str0000135 · Cited by 117
Kay, A. A., & Skarlicki, D. P. (2020) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2020.02.005 · Cited by 124
Consumers care and firms should too: On the benefits of disclosing supplier monitoring activities
Duan, Y., Hofer, C., & Aloysius, J. A. (2020) · Journal of Operations Management · doi:10.1002/joom.1129 · Cited by 104
Becker, A. B., & Bode, L. (2017) · Information Communication & Society · doi:10.1080/1369118x.2017.1301517 · Cited by 102
Mitra, T., Hutto, C. J., & Gilbert, E. (2015) · Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702553 · Cited by 102
Proactive vitality management in the work context: development and validation of a new instrument
Op den Kamp, E. M., Tims, M., Bakker, A. B., & Demerouti, E. (2018) · European Journal of Work and Organizational Psychology · doi:10.1080/1359432x.2018.1483915 · Cited by 85
Li, Y., Vishwamitra, N., Knijnenburg, B. P., Hu, H., & Caine, K. E. (2017) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3134702 · Cited by 104
Kim, M., & Beehr, T. A. (2019) · European Journal of Work and Organizational Psychology · doi:10.1080/1359432x.2019.1697237 · Cited by 110
Probst, T. M., Lee, H. J., & Bazzoli, A. (2020) · Journal of Applied Psychology · doi:10.1037/apl0000797 · Cited by 106
A Comparison of Three Online Recruitment Strategies for Engaging Parents
Dworkin, J., Hessel, H. M., Gliske, K., & Rudi, J. H. (2016) · Family Relations · doi:10.1111/fare.12206 · Cited by 99
Shin, Y. H., Im, J., Jung, S. E., & Severt, K. (2018) · Journal of Quality Assurance in Hospitality & Tourism · doi:10.1080/1528008x.2018.1483288 · Cited by 105
The Shape of and Solutions to the MTurk Quality CrisisPreprint
Kennedy, R., Clifford, S., Burleigh, T. J., Jewell, R., & Waggoner, P. (2018) · SSRN Electronic Journal · doi:10.2139/ssrn.3272468 · Cited by 99
What does the public want police to do during pandemics? A national experiment
Nix, J., Ivanov, S., & Pickett, J. T. (2021) · Criminology & Public Policy · doi:10.1111/1745-9133.12535 · Cited by 99
Mechanical Turk and the gig economy: Exploring differences between gig workers
Keith, M. G., Harms, P., & Tay, L. (2019) · Journal of Managerial Psychology · doi:10.1108/jmp-06-2018-0228 · Cited by 123
Rodd, J. M. (2023) · Journal of Memory and Language · doi:10.1016/j.jml.2023.104472 · Cited by 117
Can we turn shirkers into workers?
Berinsky, A. J., Margolis, M. F., & Sances, M. (2016) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2015.09.010 · Cited by 96
Gender-Inclusive Bathrooms Signal Fairness Across Identity Dimensions
Chaney, K. E., & Sanchez, D. T. (2017) · Social Psychological and Personality Science · doi:10.1177/1948550617737601 · Cited by 100
Diversity of artists in major U.S. museums
Topaz, C. M., Klingenberg, B., Turek, D., Heggeseth, B., Harris, P. E., Blackwood, J. C., Chavoya, C. O., Nelson, S., & Murphy, K. (2019) · PLoS ONE · doi:10.1371/journal.pone.0212852 · Cited by 99
Managing consumer privacy concerns and defensive behaviours in the digital marketplace
Bandara, R. J., Fernando, M., & Akter, S. (2020) · European Journal of Marketing · doi:10.1108/ejm-06-2019-0515 · Cited by 104
Improving Data Quality Using Amazon Mechanical Turk Through Platform Setup
Lu, L., Neale, N., Line, N. D., & Bonn, M. (2022) · Cornell Hospitality Quarterly · doi:10.1177/19389655211025475 · Cited by 94
Faith in Trump, Moral Foundations, and Social Distancing Defiance during the Coronavirus Pandemic
Graham, A., Cullen, F. T., Pickett, J. T., Jonson, C. L., Haner, M., & Sloan, M. M. (2020) · Socius Sociological Research for a Dynamic World · doi:10.1177/2378023120956815 · Cited by 100
Chen, M., Bell, R. A., & Taylor, L. D. (2017) · Journal of Health Communication · doi:10.1080/10810730.2017.1341568 · Cited by 92
Fergus, T. A. (2015) · Journal of Affective Disorders · doi:10.1016/j.jad.2015.06.017 · Cited by 90
Gleibs, I. H. (2016) · Behavior Research Methods · doi:10.3758/s13428-016-0789-y · Cited by 93
Spillover effects of information leakages in buyer–supplier–supplier triads
Ried, L., Eckerd, S., Kaufmann, L., & Carter, C. R. (2020) · Journal of Operations Management · doi:10.1002/joom.1116 · Cited by 94
Emotion regulation and obsessive–compulsive symptoms: A further examination of associations
Fergus, T. A., & Bardeen, J. R. (2014) · Journal of Obsessive-Compulsive and Related Disorders · doi:10.1016/j.jocrd.2014.06.001 · Cited by 93
Kim, M., & Beehr, T. A. (2022) · The International Journal of Human Resource Management · doi:10.1080/09585192.2022.2054281 · Cited by 115
A brief measure of reactance to health warnings
Hall, M. G., Sheeran, P., Noar, S. M., Ribisl, K. M., Boynton, M. H., & Brewer, N. T. (2017) · Journal of Behavioral Medicine · doi:10.1007/s10865-016-9821-z · Cited by 90
Biased Humans, (Un)Biased Algorithms?
Pethig, F., & Kroenung, J. (2022) · Journal of Business Ethics · doi:10.1007/s10551-022-05071-8 · Cited by 118
Berry, C., Kees, J., & Burton, S. (2022) · Journal of Advertising · doi:10.1080/00913367.2022.2079026 · Cited by 90
Using Mechanical Turk Data in IS Research: Risks, Rewards, and Recommendations
Jia, R., Steelman, Z., & Reich, B. H. (2017) · Communications of the Association for Information Systems · doi:10.17705/1cais.04114 · Cited by 94
Andonopoulos, V., Lee, J., & Mathies, C. (2023) · Journal of Retailing and Consumer Services · doi:10.1016/j.jretconser.2023.103521 · Cited by 114
Antecedents and consequences of user engagement in smartphone travel apps
Ali, F., Terrah, A., Wu, C., Ali, L., & Wu, H. (2021) · Journal of Hospitality and Tourism Technology · doi:10.1108/jhtt-09-2020-0221 · Cited by 99
Life Interrupted: Family Routines Buffer Stress during the COVID-19 Pandemic
Bates, C. R., Nicholson, L. M., Rea, E. M., Hagy, H. A., & Bohnert, A. M. (2021) · Journal of Child and Family Studies · doi:10.1007/s10826-021-02063-6 · Cited by 88
Reactance to Health Warnings Scale: Development and Validation
Hall, M. G., Sheeran, P., Noar, S. M., Ribisl, K. M., Bach, L., & Brewer, N. T. (2016) · Annals of Behavioral Medicine · doi:10.1007/s12160-016-9799-3 · Cited by 89
Arechar, A. A., & Rand, D. G. (2021) · Behavior Research Methods · doi:10.3758/s13428-021-01588-4 · Cited by 105
Examining aging sexual stigma attitudes among adults by gender, age, and generational status
Syme, M. L., & Cohn, T. J. (2015) · Aging & Mental Health · doi:10.1080/13607863.2015.1012044 · Cited by 80
Suppressing the Search Engine Manipulation Effect (SEME)
Epstein, R., Robertson, R. E., Lazer, D. M. J., & Wilson, C. (2017) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3134677 · Cited by 108
Sadiq, M., Bharti, K., Adil, M., & Singh, R. P. (2021) · Journal of Retailing and Consumer Services · doi:10.1016/j.jretconser.2021.102643 · Cited by 98
Photo Privacy Conflicts in Social Media
Such, J. M., Porter, J., Preibusch, S., & Joinson, A. (2017) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025668 · Cited by 90
Rothausen, T. J., & Henderson, K. E. (2018) · Journal of Business and Psychology · doi:10.1007/s10869-018-9545-x · Cited by 99
Wu, Z., & Liu, Y. (2022) · International Journal of Bank Marketing · doi:10.1108/ijbm-02-2022-0052 · Cited by 93
Characterizing Sleep Issues Using Twitter
McIver, D. J., Hawkins, J. B., Chunara, R., Chatterjee, A. K., Bhandari, A., Fitzgerald, T. P., JAIN, S. H., & Brownstein, J. S. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.4476 · Cited by 83
Loyal rebels? A test of the normative conflict model of constructive deviance
Dahling, J. J., & Gutworth, M. B. (2017) · Journal of Organizational Behavior · doi:10.1002/job.2194 · Cited by 96
Ert, E., & Fleischer, A. (2019) · Psychology and Marketing · doi:10.1002/mar.21297 · Cited by 83
Development of the short version of the Scales of General Well-Being: The 14-item SGWB
Longo, Y., Coyne, I. J., & Joseph, S. (2017) · Personality and Individual Differences · doi:10.1016/j.paid.2017.11.042 · Cited by 82
Shafieizadeh, K., & Tao, C.-W. (2020) · Journal of Hospitality and Tourism Management · doi:10.1016/j.jhtm.2020.04.007 · Cited by 95
Survey Satisficing Inflates Reliability and Validity Measures
Hamby, T., & Taylor, W. (2016) · Educational and Psychological Measurement · doi:10.1177/0013164415627349 · Cited by 94
Fergus, T. A., & Spada, M. M. (2018) · Journal of Anxiety Disorders · doi:10.1016/j.janxdis.2018.09.003 · Cited by 80
The effect of ad appeals and message framing on consumer responses to plant-based menu items
Ye, T., & Mattila, A. S. (2021) · International Journal of Hospitality Management · doi:10.1016/j.ijhm.2021.102917 · Cited by 80
Consumer values and service quality perceptions of food truck experiences
Shin, Y. H., Kim, H., & Severt, K. (2018) · International Journal of Hospitality Management · doi:10.1016/j.ijhm.2018.12.008 · Cited by 80
The Positive Valence Systems Scale: Development and Validation
Khazanov, G. K., Ruscio, A. M., & Forbes, C. N. (2019) · Assessment · doi:10.1177/1073191119869836 · Cited by 83
The Effects of Agency Assignment and Reference Point on Responses to COVID-19 Messages
Ma, H., & Miller, C. H. (2020) · Health Communication · doi:10.1080/10410236.2020.1848066 · Cited by 85
A Survey on Task Assignment in Crowdsourcing
Hettiachchi, D., Kostakos, V., & Gonçalves, J. (2022) · ACM Computing Surveys · doi:10.1145/3494522 · Cited by 85
Weiss, N. H., Darosh, A. G., Contractor, A. A., Schick, M. M., & Dixon‐Gordon, K. L. (2019) · Journal of Clinical Psychology · doi:10.1002/jclp.22768 · Cited by 78
Who and how many can work from home? Evidence from task descriptions
Holgersen, H., Jia, Z., & Svenkerud, S. (2021) · Journal for Labour Market Research · doi:10.1186/s12651-021-00287-z · Cited by 76
The negative effect of red tape on procedural satisfaction
Kaufmann, W., & Tummers, L. G. (2016) · Public Management Review · doi:10.1080/14719037.2016.1210907 · Cited by 76
Testing a Social Schematic Model of Police Procedural Justice
Pickett, J. T., Nix, J., & Roche, S. P. (2018) · Social Psychology Quarterly · doi:10.1177/0190272518765134 · Cited by 77
Bardeen, J. R., & Fergus, T. A. (2014) · Journal of Anxiety Disorders · doi:10.1016/j.janxdis.2014.03.002 · Cited by 75
Social Identification and Corporate Irresponsibility: A Model of Stakeholder Punitive Intentions
Antonetti, P., & Maklan, S. (2016) · British Journal of Management · doi:10.1111/1467-8551.12168 · Cited by 76
Measuring the Perceived Social Intelligence of Robots
Barchard, K. A., Lapping-Carr, L. R., Westfall, R. S., Fink-Armold, A., Banisetty, S. B., & Feil-Seifer, D. (2020) · ACM Transactions on Human-Robot Interaction · doi:10.1145/3415139 · Cited by 75
Woo, S. E., Keith, M. G., & Thornton, M. A. (2015) · Industrial and Organizational Psychology · doi:10.1017/iop.2015.21 · Cited by 75
Shin, Y. H., Im, J., Jung, S. E., & Severt, K. (2017) · Journal of Hospitality Marketing & Management · doi:10.1080/19368623.2017.1263821 · Cited by 74
A panel for lemons? Positivity bias, reputation systems and data quality on MTurk
Matherly, T. (2019) · European Journal of Marketing · doi:10.1108/ejm-07-2017-0491 · Cited by 79
Understanding typologies of feedback environment perceptions: A latent profile investigation
Dahling, J. J., Gabriel, A. S., & MacGowan, R. L. (2017) · Journal of Vocational Behavior · doi:10.1016/j.jvb.2017.05.007 · Cited by 83
Probst, T. M., Петитта, Л., Barbaranelli, C., & Austin, C. (2018) · Journal of Business Ethics · doi:10.1007/s10551-018-4002-3 · Cited by 81
Collecting Online Survey Data: A Comparison of Data Quality among a Commercial Panel & MTurk
Zhang, B., & Gearhart, S. (2020) · Survey Practice · doi:10.29115/sp-2020-0015 · Cited by 82
Uittenhove, K. L., Jeanneret, S., & Vergauwe, E. (2023) · Journal of Cognition · doi:10.5334/joc.259 · Cited by 73
Könsgen, R., Schaarschmidt, M., Ivens, S., & Munzel, A. (2018) · Journal of Interactive Marketing · doi:10.1016/j.intmar.2018.05.001 · Cited by 77
Israelashvili, J., Sauter, D. A., & Fischer, A. H. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.02475 · Cited by 80
Katt, F., & Meixner, O. (2020) · Foods · doi:10.3390/foods9040458 · Cited by 90
A first look into users’ perceptions of facial recognition in the physical world
Seng, S., Al-Ameen, M. N., & Wright, M. (2021) · Computers & Security · doi:10.1016/j.cose.2021.102227 · Cited by 75
Pickett, J. T., & Roche, S. P. (2016) · Science and Engineering Ethics · doi:10.1007/s11948-017-9886-2 · Cited by 77
Imagining a sustainable world: Measuring cognitive alternatives to the environmental status quo
Wright, J. D., Schmitt, M. T., Mackay, C. M. L., & Neufeld, S. D. (2020) · Journal of Environmental Psychology · doi:10.1016/j.jenvp.2020.101523 · Cited by 75
Towards a better understanding of memorable wellness tourism experience
Sthapit, E., Björk, P., & Coudounaris, D. N. (2022) · International Journal of Spa and Wellness · doi:10.1080/24721735.2022.2136120 · Cited by 90
Barbaro, N., & Shackelford, T. K. (2016) · Journal of Interpersonal Violence · doi:10.1177/0886260516640548 · Cited by 71
Metadata
- Authors
- Peer, E., Vosgerau, J., & Acquisti, A.
- Affiliations
- Tilburg University (NL)
- Carnegie Mellon University (US)
- Venue
- Behavior Research Methods
- Published
- 2013-12-20
- Volume
- 46
- Issue
- 4
- Pages
- 1023-1031
- ISSN
- 1554-351X (ISSN-L)
- 1554-3528
- OpenAlex
- W2105753174
- Cited by
- 1,618
- References
- 18
- Citations by year
- 2026: 68 · 2025: 96 · 2024: 128 · 2023: 167 · 2022: 175 · 2021: 212 · 2020: 210 · 2019: 147 · 2018: 119 · 2017: 132 · 2016: 91 · 2015: 49 · 2014: 17 · 2013: 1
- FWCI
- 79.80
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Behavior Research Methods (publishedVersion)
- Research portal (Tilburg University) (publishedVersion)
- PubMed (publishedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- SSRN Electronic Journal (submittedVersion)
- Figshare (submittedVersion, cc-by)
- Archivio istituzionale della Ricerca - Bocconi (submittedVersion)
- KiltHub Repository
- Topics
- Physical Sciences › Computer Science › Computer Science Applications › Mobile Crowdsensing and Crowdsourcing
- Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Keywords
- data quality, Amazon Mechanical Turk, worker reputation, crowdsourcing, selection bias, labor productivity, participant recruitment
- MeSH terms
- Data Collection, Humans, Research Design, Patient Selection, Internet, Behavioral Research, Crowdsourcing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
