Attentive Turkers: MTurk participants perform better on online attention checks than do subject pool participants
Open accessAbstract
Participant attentiveness is a concern for many researchers using Amazon's Mechanical Turk (MTurk). Although studies comparing the attentiveness of participants on MTurk versus traditional subject pool samples have provided mixed support for this concern, attention check questions and other methods of ensuring participant attention have become prolific in MTurk studies. Because MTurk is a population that learns, we hypothesized that MTurkers would be more attentive to instructions than are traditional subject pool samples. In three online studies, participants from MTurk and collegiate populations participated in a task that included a measure of attentiveness to instructions (an instructional manipulation check: IMC). In all studies, MTurkers were more attentive to the instructions than were college students, even on novel IMCs (Studies 2 and 3), and MTurkers showed larger effects in response to a minute text manipulation. These results have implications for the sustainable use of MTurk samples for social science research and for the conclusions drawn from research with MTurk and college subject pool samples.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In crowd-sourced settings such as MTurk, respondents are often incentivized to respond as attentively, compliantly, and quickly as possible to both open higher-paying opportunities and prevent being rejected from future employment.
“In crowd-sourced settings such as MTurk, respondents are often incentivized to respond as attentively, compliantly, and quickly as possible to both open higher-paying opportunities and prevent being rejected from future employment [90, 100].”
From Not so terrifying after all? A set of failed replications of the mortality salience effects of Terror Management Theory (2023) · cites this paper as [100]
The MTurk discussion says incentives to pay close attention to instructions can make workers more attentive, and the Abstract describes compensation tied to attention checks and restrictions to high-reputation workers. The paper does not establish that workers are incentivized to respond quickly, that doing so opens higher-paying opportunities, or that they seek to avoid rejection from future employment. (LLM, conf=0.85)
Experimental — accuracy not yet measuredComparable attention checks have been shown to improve the quality of data obtained through web-based surveys above that of classic paper and pencil questionnaires.
“Comparable attention checks have been shown to improve the quality of data obtained through web-based surveys above that of classic paper and pencil questionnaires [57].”
From Dispositional factors in the explanation of symptoms of depression, anxiety, health anxiety and COVID-19 Phobia (2024) · cites this paper as [57]
The paper describes attention checks as filters intended to prevent inattentive MTurk participants from introducing error into online studies (introductory text indexed under Abstract). It does not compare web-based surveys with paper-and-pencil questionnaires or show that attention checks make web-based data better than paper-and-pencil data. (LLM, conf=0.85)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007) · Behavior Research Methods · doi:10.3758/bf03193146 · Cited by 65,971 · see citing works
Advances In Experimental Social Psychology
(2022) · Advances in experimental social psychology · doi:10.1016/s0065-2601(22)x0002-6 · Cited by 22,559 · see citing works
Mental Accounting and Consumer Choice
Thaler, R. (1985) · Marketing science (Providence, R.I.) · doi:10.1287/mksc.1070.0330 · Cited by 5,298 · see citing works
Mental Accounting and Consumer Choice
Thaler, R. (1985) · Marketing Science · doi:10.1287/mksc.4.3.199 · Cited by 4,885 · see citing works
Evaluating Online Labor Markets for Experimental Research: Amazon.com's Mechanical Turk
Berinsky, A. J., Huber, G. A., & Lenz, G. S. (2012) · Political Analysis · doi:10.1093/pan/mpr057 · Cited by 4,008 · see citing works
Running experiments on Amazon Mechanical Turk
Paolacci, G., Chandler, J., & Ipeirotis, P. G. (2010) · Judgment and Decision Making · doi:10.1017/s1930297500002205 · Cited by 3,767 · see citing works
Instructional manipulation checks: Detecting satisficing to increase statistical power
Oppenheimer, D. M., Meyvis, T., & Davidenko, N. (2009) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2009.03.009 · Cited by 3,493 · see citing works
Conducting behavioral research on Amazon’s Mechanical Turk
Mason, W., & Suri, S. (2012) · Behavior Research Methods · doi:10.3758/s13428-011-0124-6 · Cited by 3,172 · see citing works
Data collection in a flat world: the strengths and weaknesses of Mechanical Turk samples
Goodman, J. K., Cryder, C. E., & Cheema, A. (2013) · Journal of Behavioral Decision Making · doi:10.1002/bdm.1753 · Cited by 2,242 · see citing works
Inside the Turk: Understanding Mechanical Turk as a Participant Pool
Paolacci, G., & Chandler, J. (2014) · Current Directions in Psychological Science · doi:10.1177/0963721414531598 · Cited by 2,136 · see citing works
Crowdsourcing user studies with Mechanical Turk
Kittur, A., Ed, H., & Suh, B. (2008) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1357054.1357127 · Cited by 1,941 · see citing works
Reputation as a sufficient condition for data quality on Amazon Mechanical Turk
Peer, E., Vosgerau, J., & Acquisti, A. (2014) · Behavior Research Methods · doi:10.3758/s13428-013-0434-y · Cited by 1,618 · see citing works
Evaluating amazon’s mechanical turk as a tool for experimental behavioral research
Crump, M. J. C., McDonnell, J. V., & Gureckis, T. M. (2013) · PLoS ONE · doi:10.1371/journal.pone.0057410 · Cited by 1,617 · see citing works
Mental Accounting and Consumer Choice / Теория ментального учета и потребительский выбор
Thaler, R. H. (1985) · Marketing Science · Cited by 1,257 · see citing works
Investigating Variation in Replicability: A “Many Labs” Replication Project
Klein, R. A., Ratliff, K. A., Vianello, M., Adams, R. B., Bahník, Š., Bernstein, M. J., Bocian, K., Brandt, M. J., Brooks, B., Brumbaugh, C. C., et al. (2014) · Social Psychology · doi:10.1027/1864-9335/a000178 · Cited by 1,208 · see citing works
Chandler, J., Mueller, P., & Paolacci, G. (2014) · Behavior Research Methods · doi:10.3758/s13428-013-0365-7 · Cited by 950 · see citing works
Judgment in a Social Context: Biases, Shortcomings, and the Logic of Conversation
Schwarz, N. (1994) · Advances in experimental social psychology · doi:10.1016/s0065-2601(08)60153-7 · Cited by 366 · see citing works
Self-citationEmotion rating from short blog texts
Gill, A. J., Gergle, D., French, R. M., & Oberlander, J. (2008) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1357054.1357229 · Cited by 232 · see citing works
Clifford, S., & Jerit, J. (2014) · Journal of Experimental Political Science · doi:10.1017/xps.2014.5 · Cited by 206 · see citing works
Encoding Under Trust and Distrust: The Spontaneous Activation of Incongruent Cognitions.
Schul, Y., Mayo, R., & Burnstein, E. (2004) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.86.5.668 · Cited by 167 · see citing works
Mayo, R., Alfasi, D., & Schwarz, N. (2014) · Journal of Experimental Psychology: General · doi:10.1037/a0035127 · Cited by 43 · see citing works
Self-citation
Flagged first, then by citations
2 of the 23 references couldn’t be matched to a known paper.
Research works printed but not linked: 1
- Kapelner, A., & Chandler, D. (2010). Preventing satisficing in online surveys: A Bkapcha^to ensure higher data quality. In Proceedings of CrowdConf 2010. Available at www.academia.edu/2788541/
Cited by
Sorted by citations
Gorilla in our midst: An online behavioral experiment builder
Anwyl-Irvine, A. L., Massonnié, J., Flitton, A., Kirkham, N. Z., & Evershed, J. K. (2019) · Behavior Research Methods · doi:10.3758/s13428-019-01237-x · Cited by 1,878 · see citing works
Do Investors Value Sustainability? A Natural Experiment Examining Ranking and Fund Flows
HARTZMARK, S. M., & SUSSMAN, A. B. (2019) · The Journal of Finance · doi:10.1111/jofi.12841 · Cited by 1,607 · see citing works
Methods for the detection of carelessly invalid responses in survey data
Curran, P. G. (2016) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2015.07.006 · Cited by 1,356 · see citing works
Conducting Clinical Research Using Crowdsourced Convenience Samples
Chandler, J., & Shapiro, D. (2016) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-021815-093623 · Cited by 1,072 · see citing works
Brysbaert, M. (2019) · Journal of Cognition · doi:10.5334/joc.72 · Cited by 996 · see citing works
Kees, J., Berry, C., Burton, S., & Sheehan, K. (2017) · Journal of Advertising · doi:10.1080/00913367.2016.1269304 · Cited by 902 · see citing works
An Evaluation of Amazon’s Mechanical Turk, Its Rapid Rise, and Its Effective Use
Buhrmester, M. D., Talaifar, S., & Gosling, S. D. (2018) · Perspectives on Psychological Science · doi:10.1177/1745691617706516 · Cited by 768 · see citing works
Are samples drawn from Mechanical Turk valid for research on political ideology?
Clifford, S., Jewell, R., & Waggoner, P. (2015) · Research & Politics · doi:10.1177/2053168015622072 · Cited by 680 · see citing works
Online panels in social science research: Expanding sampling methods beyond Mechanical Turk
Chandler, J., Rosenzweig, C., Moss, A. J., Robinson, J., & Litman, L. (2019) · Behavior Research Methods · doi:10.3758/s13428-019-01273-7 · Cited by 668 · see citing works
Validity and Mechanical Turk: An assessment of exclusion methods and interactive experiments
Thomas, K. A., & Clifford, S. (2017) · Computers in Human Behavior · doi:10.1016/j.chb.2017.08.038 · Cited by 625 · see citing works
People are averse to machines making moral decisions.
Bigman, Y. E., & Gray, K. (2018) · Cognition · doi:10.1016/j.cognition.2018.08.003 · Cited by 738 · see citing works
Crowdsourcing Consumer Research
Goodman, J. K., & Paolacci, G. (2017) · Journal of Consumer Research · doi:10.1093/jcr/ucx047 · Cited by 594 · see citing works
Measuring and Bounding Experimenter Demand
de Quidt, J., Haushofer, J., & Roth, C. (2018) · American Economic Review · doi:10.1257/aer.20171330 · Cited by 573 · see citing works
Abbey, J. D., & Meloy, M. G. (2017) · Journal of Operations Management · doi:10.1016/j.jom.2017.06.001 · Cited by 522 · see citing works
“Participant” Perceptions of Twitter Research Ethics
Fiesler, C. L., & Proferes, N. (2018) · Social Media + Society · doi:10.1177/2056305118763366 · Cited by 545 · see citing works
The shape of and solutions to the MTurk quality crisis
Kennedy, R., Clifford, S., Burleigh, T., Waggoner, P. D., Jewell, R., & Winter, N. J. G. (2020) · Political Science Research and Methods · doi:10.1017/psrm.2020.6 · Cited by 506 · see citing works
Why the COVID-19 pandemic is a traumatic stressor
Bridgland, V. M. E., Moeck, E. K., Green, D. M., Swain, T. L., Nayda, D. M., Matson, L. A., Hutchison, N. P., & Takarangi, M. K. T. (2021) · PLOS ONE · doi:10.1371/journal.pone.0240146 · Cited by 501 · see citing works
Are Manipulation Checks Necessary?
Hauser, D., Ellsworth, P. C., & Gonzalez, R. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.00998 · Cited by 501 · see citing works
Self-citationAmazon mechanical Turk in organizational psychology: an evaluation and practical recommendations
Cheung, J. H., Burns, D. K., Sinclair, R. R., & Sliter, M. (2017) · Journal of Business and Psychology · doi:10.1007/s10869-016-9458-5 · Cited by 474 · see citing works
Zhou, H., & Ayelet Fishbach, B. (2016) · Journal of Personality and Social Psychology · doi:10.1037/pspa0000056 · Cited by 462 · see citing works
Enhancing user engagement: The role of gamification in mobile apps
Bitrián, P., Buil, I., & Catalán, S. (2021) · Journal of Business Research · doi:10.1016/j.jbusres.2021.04.028 · Cited by 442 · see citing works
Crowdsourcing research: Data collection with Amazon’s Mechanical Turk
Sheehan, K. B. (2018) · Communication Monographs · doi:10.1080/03637751.2017.1342043 · Cited by 408 · see citing works
Does Counter-Attitudinal Information Cause Backlash? Results from Three Large Survey Experiments
Guess, A., & Coppock, A. (2020) · British Journal of Political Science · doi:10.1017/s0007123418000327 · Cited by 372 · see citing works
Marketing AI recruitment: The next phase in job application and selection
van Esch, P., Black, J. S., & Ferolie, J. (2019) · Computers in Human Behavior · doi:10.1016/j.chb.2018.09.009 · Cited by 391 · see citing works
Are Attention Check Questions a Threat to Scale Validity?
Kung, F. Y. H., Kwok, N., & Brown, D. J. (2017) · Applied Psychology · doi:10.1111/apps.12108 · Cited by 392 · see citing works
Empathy is hard work: People choose to avoid empathy because of its cognitive costs.
Cameron, C. D., Hutcherson, C. A. C., Ferguson, A. M., Scheffer, J. A., Hadjiandreou, E., & Inzlicht, M. (2019) · Journal of Experimental Psychology General · doi:10.1037/xge0000595 · Cited by 380 · see citing works
Shin, Y. H., Im, J., Jung, S. E., & Severt, K. (2018) · International Journal of Hospitality Management · doi:10.1016/j.ijhm.2017.10.011 · Cited by 358 · see citing works
The Development and Validation of the Compassion Scale
Pommier, E., Neff, K. D., & Tóth-Király, I. (2020) · Assessment · doi:10.1177/1073191119874108 · Cited by 345 · see citing works
The Use of Online Panel Data in Management Research: A Review and Recommendations
Porter, C. O. L. H., Outlaw, R., Gale, J., & Cho, T. S. (2018) · Journal of Management · doi:10.1177/0149206318811569 · Cited by 346 · see citing works
Do travelers trust intelligent service robots?
Tussyadiah, I. P., Zach, F., & Wang, J. (2020) · Annals of Tourism Research · doi:10.1016/j.annals.2020.102886 · Cited by 305 · see citing works
The average laboratory samples a population of 7,300 Amazon Mechanical Turk workers
Stewart, N., Ungemach, C., Harris, A. J. L., Bartels, D. M., Newell, B. R., Paolacci, G., & Chandler, J. (2015) · Judgment and Decision Making · doi:10.1017/s1930297500005611 · Cited by 304 · see citing works
Conducting perception research over the internet: A tutorial review
Woods, A. T., Velasco, C., Levitan, C. A., Wan, X., & Spence, C. (2015) · PeerJ · doi:10.7717/peerj.1058 · Cited by 301 · see citing works
Data Collection via Online Platforms: Challenges and Recommendations for Future Research
Newman, A., Bavik, Y. L., Mount, M. P., & Shao, B. (2020) · Applied Psychology · doi:10.1111/apps.12302 · Cited by 307 · see citing works
MTurk Character Misrepresentation: Assessment and Solutions
Sharpe Wessling, K., Huber, J., & Netzer, O. (2017) · Journal of Consumer Research · doi:10.1093/jcr/ucx053 · Cited by 330 · see citing works
A Note on Dropping Experimental Subjects who Fail a Manipulation Check
Aronow, P. M., Baron, J., & Pinson, L. E. (2019) · Political Analysis · doi:10.1017/pan.2019.5 · Cited by 280 · see citing works
No Harm in Checking: Using Factual Manipulation Checks to Assess Attentiveness in Experiments
Kane, J., & Barabas, J. (2018) · American Journal of Political Science · doi:10.1111/ajps.12396 · Cited by 277 · see citing works
Evaluating CloudResearch’s Approved Group as a solution for problematic data quality on MTurk
Hauser, D., Moss, A. J., Rosenzweig, C., Jaffe, S. N., Robinson, J., & Litman, L. (2022) · Behavior Research Methods · doi:10.3758/s13428-022-01999-x · Cited by 276 · see citing works
Self-citationFostering Intrinsic Motivation through Avatar Identification in Digital Games
Birk, M. V., Atkins, C., Bowey, J. T., & Mandryk, R. L. (2016) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858062 · Cited by 269 · see citing works
How psychological and contextual factors contribute to travelers’ propensity to choose green hotels?
Yadav, R., Balaji, M. S., & Jebarajakirthy, C. (2018) · International Journal of Hospitality Management · doi:10.1016/j.ijhm.2018.08.002 · Cited by 262 · see citing works
Does information change attitudes toward immigrants?
Grigorieff, A., Roth, C., & Ubfal, D. (2020) · Demography · doi:10.1007/s13524-020-00882-8 · Cited by 254 · see citing works
The use of crowdsourcing in addiction science research: Amazon Mechanical Turk.
Strickland, J. C., & Stoops, W. W. (2018) · Experimental and Clinical Psychopharmacology · doi:10.1037/pha0000235 · Cited by 249 · see citing works
Self-Injurious Thoughts and Behaviors Interview—Revised: Development, reliability, and validity.
Fox, K. R., Harris, J. A., Wang, S. B., Millner, A. J., Deming, C. A., & Nock, M. K. (2020) · Psychological Assessment · doi:10.1037/pas0000819 · Cited by 249 · see citing works
Who Are the Turkers? A Characterization of MTurk Workers Using the Personality Assessment Inventory
McCredie, M. N., & Morey, L. C. (2018) · Assessment · doi:10.1177/1073191118760709 · Cited by 235 · see citing works
Molyneux, L. (2017) · Digital Journalism · doi:10.1080/21670811.2017.1334567 · Cited by 227 · see citing works
Crowdsourcing Samples in Cognitive Science.
Stewart, N., Chandler, J. J., & Paolacci, G. (2017) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2017.06.007 · Cited by 210 · see citing works
Franklin, J. C., Fox, K. R., Franklin, C. R., Kleiman, E. M., Ribeiro, J. D., Jaroszewski, A. C., Hooley, J. M., & Nock, M. K. (2016) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000093 · Cited by 217 · see citing works
Robinson, J., Rosenzweig, C., Moss, A. J., & Litman, L. (2019) · PLoS ONE · doi:10.1371/journal.pone.0226394 · Cited by 203 · see citing works
Harris, L. M., Huang, X., Linthicum, K. P., Bryen, C. P., & Ribeiro, J. D. (2020) · Scientific Reports · doi:10.1038/s41598-020-70866-6 · Cited by 212 · see citing works
The effect of credit card versus mobile payment on convenience and consumers’ willingness to pay
Boden, J., Maier, E., & Wilken, R. (2019) · Journal of Retailing and Consumer Services · doi:10.1016/j.jretconser.2019.101910 · Cited by 204 · see citing works
Reh, S., Tröster, C., & Quaquebeke, N. V. (2017) · Journal of Applied Psychology · doi:10.1037/apl0000281 · Cited by 184 · see citing works
Brands as Rivals: Consumer Pursuit of Distinctiveness and the Role of Brand Anthropomorphism
Puzakova, M., & Aggarwal, P. (2018) · Journal of Consumer Research · doi:10.1093/jcr/ucy035 · Cited by 195 · see citing works
Validating Machine Learning Algorithms for Twitter Data Against Established Measures of Suicidality
Braithwaite, S., Giraud-Carrier, C. G., West, J. H., Barnes, M. D., & Hanson, C. L. (2016) · JMIR Mental Health · doi:10.2196/mental.4822 · Cited by 193 · see citing works
The Role of MTurk in Education Research: Advantages, Issues, and Future Directions
Follmer, D. J., Sperling, R. A., & Suen, H. K. (2017) · Educational Researcher · doi:10.3102/0013189x17725519 · Cited by 184 · see citing works
Miller, J. D., Crowe, M., Weiss, B., Maples-Keller, J. L., & Lynam, D. R. (2017) · Personality Disorders Theory Research and Treatment · doi:10.1037/per0000191 · Cited by 179 · see citing works
Self-Compassion: A Potential Path to Adolescent Resilience and Positive Exploration
Bluth, K., Mullarkey, M. C., & Lathren, C. (2018) · Journal of Child and Family Studies · doi:10.1007/s10826-018-1125-1 · Cited by 207 · see citing works
Eckerd, S., DuHadway, S., Bendoly, E., Carter, C. R., & Kaufmann, L. (2020) · Journal of Operations Management · doi:10.1002/joom.1128 · Cited by 172 · see citing works
It’s a Trap! Instructional Manipulation Checks Prompt Systematic Thinking on “Tricky” Tasks
Hauser, D., & Schwarz, N. (2015) · SAGE Open · doi:10.1177/2158244015584617 · Cited by 173 · see citing works
Self-citationThe critical role of second-order normative beliefs in predicting energy conservation
Jachimowicz, J., Hauser, O., O’Brien, J. D., Sherman, E., & Galinsky, A. D. (2018) · Nature Human Behaviour · doi:10.1038/s41562-018-0434-0 · Cited by 169 · see citing works
Brühlmann, F., Petralito, S., Aeschbach, L. F., & Opwis, K. (2020) · Methods in Psychology · doi:10.1016/j.metip.2020.100022 · Cited by 168 · see citing works
Building, Hosting and Recruiting: A Brief Introduction to Running Behavioral Experiments Online
Sauter, M., Draschkow, D., & Mack, W. (2020) · Brain Sciences · doi:10.3390/brainsci10040251 · Cited by 164 · see citing works
Mechanical Turk data collection in addiction research: utility, concerns and best practices
Mellis, A. M., & Bickel, W. K. (2020) · Addiction · doi:10.1111/add.15032 · Cited by 164 · see citing works
Zickar, M. J., & Keith, M. G. (2022) · Annual Review of Organizational Psychology and Organizational Behavior · doi:10.1146/annurev-orgpsych-120920-052946 · Cited by 163 · see citing works
Living “in the zone”: hyperfocus in adult ADHD
Hupfeld, K. E., Abagis, T. R., & Shah, P. (2019) · ADHD Attention Deficit and Hyperactivity Disorders · doi:10.1007/s12402-018-0272-y · Cited by 157 · see citing works
Designing crowdsourced delivery systems: The effect of driver disclosure and ethnic similarity
Ta, H., Esper, T. L., & Hofer, A. R. (2018) · Journal of Operations Management · doi:10.1016/j.jom.2018.06.001 · Cited by 154 · see citing works
Robots should be seen and not heard…sometimes: Anthropomorphism and AI service robot interactions
Letheren, K., Jetten, J., Roberts, J. M., & Donovan, J. W. (2021) · Psychology and Marketing · doi:10.1002/mar.21575 · Cited by 156 · see citing works
Common Concerns with MTurk as a Participant Pool: Evidence and SolutionsPreprint
Hauser, D., Paolacci, G., & Chandler, J. (2018) · doi:10.31234/osf.io/uq45c · Cited by 154 · see citing works
Self-citationReconsidering the voice effect when learning from a virtual human
Craig, S. D., & Schroeder, N. L. (2017) · Computers & Education · doi:10.1016/j.compedu.2017.07.003 · Cited by 151 · see citing works
The racially diverse affective expression (RADIATE) face stimulus set
Conley, M. I., Dellarco, D. V., Rubien-Thomas, E., Cohen, A. O., Cervera, A., Tottenham, N., & Casey, B. J. (2018) · Psychiatry Research · doi:10.1016/j.psychres.2018.04.066 · Cited by 151 · see citing works
Gratz, K. L., Tull, M. T., Richmond, J. R., Edmonds, K. A., Scamaldo, K. M., & Rose, J. P. (2020) · Suicide and Life-Threatening Behavior · doi:10.1111/sltb.12654 · Cited by 150 · see citing works
Yun, H. S., & Bickmore, T. (2025) · Journal of Medical Internet Research · doi:10.2196/68560 · Cited by 151 · see citing works
New rules of social media shopping: Personality differences of U.S. Gen Z versus Gen X market mavens
Goldring, D., & Azab, C. (2020) · Journal of Consumer Behaviour · doi:10.1002/cb.1893 · Cited by 165 · see citing works
Boyraz, G., Legros, D. N., & Tigershtrom, A. (2020) · Journal of Anxiety Disorders · doi:10.1016/j.janxdis.2020.102307 · Cited by 142 · see citing works
A linguistic signature of psychological distancing in emotion regulation.
Nook, E. C., Schleider, J. L., & Somerville, L. H. (2017) · Journal of Experimental Psychology General · doi:10.1037/xge0000263 · Cited by 145 · see citing works
Partisanship, Political Knowledge, and the Dunning‐Kruger Effect
Anson, I. G. (2018) · Political Psychology · doi:10.1111/pops.12490 · Cited by 142 · see citing works
Aesthetic Pleasure versus Aesthetic Interest: The Two Routes to Aesthetic Liking
Graf, L. K. M., & Landwehr, J. R. (2017) · Frontiers in Psychology · doi:10.3389/fpsyg.2017.00015 · Cited by 143 · see citing works
Kwapil, T. R., Gross, G. M., Silvia, P. J., Raulin, M. L., & Barrantes‐Vidal, N. (2017) · Schizophrenia Research · doi:10.1016/j.schres.2017.07.001 · Cited by 142 · see citing works
Plass, J. L., Homer, B. D., MacNamara, A. P., Ober, T. M., Rose, M. C., Pawar, S. S., Hovey, C., & Olsen, A. F. (2019) · Learning and Instruction · doi:10.1016/j.learninstruc.2019.01.005 · Cited by 139 · see citing works
When Victims Help Their Abusive Supervisors: The Role of LMX, Self-Blame, and Guilt
Tröster, C., & Quaquebeke, N. V. (2020) · Academy of Management Journal · doi:10.5465/amj.2019.0559 · Cited by 137 · see citing works
Formalizing Neurath’s ship: approximate algorithms for online causal learning.
Bramley, N. R., Dayan, P., Griffiths, T. L., & Lagnado, D. A. (2017) · Psychological Review · doi:10.1037/rev0000061 · Cited by 136 · see citing works
Using MTurk to Distribute a Survey or Experiment: Methodological Considerations
Hunt, N. C., & Scheetz, A. M. (2018) · Journal of Information Systems · doi:10.2308/isys-52021 · Cited by 136 · see citing works
Distinct Effects of Pride and Gratitude Appeals on Sustainable Luxury Brands
Septianto, F., Seo, Y., & Errmann, A. (2020) · Journal of Business Ethics · doi:10.1007/s10551-020-04484-7 · Cited by 134 · see citing works
Mooijman, M., Meindl, P., Oyserman, D., Monterosso, J. R., Dehghani, M., Doris, J. M., & Graham, J. (2017) · Journal of Personality and Social Psychology · doi:10.1037/pspp0000149 · Cited by 129 · see citing works
Supply chain transparency: Consumer reactions to incongruent signals
Mollenkopf, D. A., Peinkofer, S. T., & Chu, Y. (2022) · Journal of Operations Management · doi:10.1002/joom.1180 · Cited by 139 · see citing works
Studying the Digital: Directions and Challenges for Digital Methods
Hampton, K. N. (2017) · Annual Review of Sociology · doi:10.1146/annurev-soc-060116-053505 · Cited by 128 · see citing works
Sigurðsson, V., Larsen, N. M., Pálsdóttir, R. G., Folwarczny, M., Menon, R. G. V., & Fagerstrøm, A. (2021) · Journal of Business Research · doi:10.1016/j.jbusres.2021.10.052 · Cited by 132 · see citing works
The MTurkification of Social and Personality Psychology
Anderson, C. A., Allen, J. J., Plante, C. N., Quigley‐McBride, A., Lovett, A., & Rokkum, J. N. (2018) · Personality and Social Psychology Bulletin · doi:10.1177/0146167218798821 · Cited by 127 · see citing works
Barends, A. J., & de Vries, R. E. (2019) · Personality and Individual Differences · doi:10.1016/j.paid.2019.02.015 · Cited by 124 · see citing works
Can gamification improve financial behavior? The moderating role of app expertise
Bayuk, J. B., & Nasco, S. A. (2019) · International Journal of Bank Marketing · doi:10.1108/ijbm-04-2018-0086 · Cited by 131 · see citing works
Beyond the Breaking Point? Survey Satisficing in Conjoint Experiments
Bansak, K. C., Hainmueller, J., Hopkins, D. J., & Yamamoto, T. (2017) · Political Science Research and Methods · doi:10.1017/psrm.2019.13 · Cited by 135 · see citing works
Running Behavioral Operations Experiments Using Amazon's Mechanical Turk
Lee, Y. S., Seo, Y. W., & Siemsen, E. (2018) · Production and Operations Management · doi:10.1111/poms.12841 · Cited by 128 · see citing works
Linking decent work with physical and mental health: A psychology of working perspective
Duffy, R. D., Kim, H. J., Gensmer, N. P., Raque‐Bogdan, T. L., Douglass, R. P., England, J. W., & Büyükgöze‐Kavas, A. (2019) · Journal of Vocational Behavior · doi:10.1016/j.jvb.2019.05.002 · Cited by 129 · see citing works
Loss of group memberships predicts depression in postpartum mothers
Seymour‐Smith, M., Cruwys, T., Haslam, S. A., & Brodribb, W. E. (2016) · Social Psychiatry and Psychiatric Epidemiology · doi:10.1007/s00127-016-1315-3 · Cited by 125 · see citing works
Two signals of social rank: Prestige and dominance are associated with distinct nonverbal displays.
Witkower, Z., Tracy, J., Cheng, J. T., & Henrich, J. (2020) · Journal of Personality and Social Psychology · doi:10.1037/pspi0000181 · Cited by 106 · see citing works
Deep learning for post‐hurricane aerial damage assessment of buildings
Cheng, C., Behzadan, A. H., & Noshadravan, A. (2021) · Computer-Aided Civil and Infrastructure Engineering · doi:10.1111/mice.12658 · Cited by 125 · see citing works
LIONESS Lab – A Free Web-Based Platform for Conducting Interactive Experiments Online
Giamattei, M., Yahosseini, K. S., Gächter, S., & Molleman, L. (2019) · Journal of the Economic Science Association · doi:10.1007/s40881-020-00087-0 · Cited by 124 · see citing works
An international survey on the incidence and modulating factors of carsickness
Schmidt, E. A., Kuiper, O. X., Wolter, S. C., Diels, C., & Bos, J. E. (2020) · Transportation Research Part F Traffic Psychology and Behaviour · doi:10.1016/j.trf.2020.03.012 · Cited by 126 · see citing works
Kavanaugh, C. M., & Jong, J. (2020) · Journal for the Study of Religion Nature and Culture · doi:10.1558/jsrnc.39187 · Cited by 124 · see citing works
Amatulli, C., Angelis, M. D., & Stoppani, A. (2020) · Tourism Management · doi:10.1016/j.tourman.2020.104228 · Cited by 124 · see citing works
Ettinger, J., Walton, P., Painter, J., & DiBlasi, T. (2021) · Climatic Change · doi:10.1007/s10584-021-02975-8 · Cited by 131 · see citing works
Cohodes, E. M., McCauley, S., & Gee, D. G. (2021) · Research on Child and Adolescent Psychopathology · doi:10.1007/s10802-020-00732-6 · Cited by 123 · see citing works
- Authors
- Hauser, D. J., & Schwarz, N.
- Affiliations
- University of Michigan (US)
- University of Southern California (US)
- Corresponding author
- Hauser, David J.
- Venue
- Behavior Research Methods
- Published
- 2015-03-12
- Volume
- 48
- Issue
- 1
- Pages
- 400-407
- ISSN
- 1554-351X (ISSN-L)
- 1554-3528
- OpenAlex
- W2076387137
- Cited by
- 1,739
- References
- 23
- Citations by year
- 2026: 60 · 2025: 104 · 2024: 145 · 2023: 180 · 2022: 209 · 2021: 249 · 2020: 248 · 2019: 176 · 2018: 160 · 2017: 122 · 2016: 64 · 2015: 19
- FWCI
- 184.66
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Behavior Research Methods (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Science Applications › Mobile Crowdsensing and Crowdsourcing
- Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Keywords
- Amazon Mechanical Turk, university students
- MeSH terms
- Attention, Data Collection, Female, Humans, Learning, Male, Surveys and Questionnaires, Social Sciences, Internet
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
