Cited by 1,072Mobile Crowdsensing and CrowdsourcingSurvey Methodology and NonresponseEthics in Clinical Research

The DOI matches its Crossref record; 174 of the 177 references it cites were found in a registry or an index.

Conducting Clinical Research Using Crowdsourced Convenience Samples

Chandler, Jesse, Shapiro, Danielle·2016·Annual Review of Clinical Psychology

DOI 10.1146/annurev-clinpsy-021815-093623

Open this paper in CoChat — free account

Abstract

Crowdsourcing has had a dramatic impact on the speed and scale at which scientific research can be conducted. Clinical scientists have particularly benefited from readily available research study participants and streamlined recruiting and payment systems afforded by Amazon Mechanical Turk (MTurk), a popular labor market for crowdsourcing workers. MTurk has been used in this capacity for more than five years. The popularity and novelty of the platform have spurred numerous methodological investigations, making it the most studied nonprobability sample available to researchers. This article summarizes what is known about MTurk sample composition and data quality with an emphasis on findings relevant to clinical psychological research. It then addresses methodological issues with using MTurk--many of which are common to other nonprobability samples but unfamiliar to clinical science researchers--and suggests concrete steps to avoid these issues or minimize their impact.

Integrity signals

Registry

Crossref · 10.1146/annurev-clinpsy-021815-093623

Cites

174 of 174 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Amazon's Mechanical Turk: A New Source of Inexpensive, Yet High-Quality, Data?
Buhrmester, M., Kwang, T., & Gosling, S. D. (2011) · Perspectives on Psychological Science · doi:10.1177/1745691610393980 · Cited by 9,996
Cognitive Reflection and Decision Making
Frederick, S. (2005) · Journal of Economic Perspectives · doi:10.1257/089533005775196732 · Cited by 5,256
Evaluating Online Labor Markets for Experimental Research: Amazon.com's Mechanical Turk
Berinsky, A. J., Huber, G. A., & Lenz, G. S. (2012) · Political Analysis · doi:10.1093/pan/mpr057 · Cited by 4,008
Running experiments on Amazon Mechanical Turk
Paolacci, G., Chandler, J., & Ipeirotis, P. G. (2010) · Judgment and Decision Making · doi:10.1017/s1930297500002205 · Cited by 3,767
Self-citation
Instructional manipulation checks: Detecting satisficing to increase statistical power
Oppenheimer, D. M., Meyvis, T., & Davidenko, N. (2009) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2009.03.009 · Cited by 3,493
Conducting behavioral research on Amazon’s Mechanical Turk
Mason, W., & Suri, S. (2012) · Behavior Research Methods · doi:10.3758/s13428-011-0124-6 · Cited by 3,172
Promoting an open research culture
Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., Buck, S., Chambers, C. D., Chin, G., Christensen, G., et al. (2015) · Science · doi:10.1126/science.aab2374 · Cited by 2,708
Data collection in a flat world: the strengths and weaknesses of Mechanical Turk samples
Goodman, J. K., Cryder, C. E., & Cheema, A. (2013) · Journal of Behavioral Decision Making · doi:10.1002/bdm.1753 · Cited by 2,242
Inside the Turk: Understanding Mechanical Turk as a Participant Pool
Paolacci, G., & Chandler, J. (2014) · Current Directions in Psychological Science · doi:10.1177/0963721414531598 · Cited by 2,136
Self-citation
Crowdsourcing user studies with Mechanical Turk
Kittur, A., Ed, H., & Suh, B. (2008) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1357054.1357127 · Cited by 1,941

Show all 177 references

Research works printed but not linked: 12

Barnhoorn JS, 2014, Behav. Res. Methods, 530, 1
Beramendi P, Duch RM, Matsuo A. 2014. When lab subjects meet real people: comparing different modes of experiments. Presented at Asian Polit. Methodol. Conf., Taipai
Casey L, Chandler J, Levine AS, Proctor A, Strolovitch D. 2015. Demographic characteristics of a large sample of US workers. Mathematica Policy Res. Rep. In press
Downes-Le Guin T, 2006, ESOMAR World Res. Conf. Panel Res.
Higgins C, 2010, Proc. NAACL HLT 2010 Workshop on Creat. Speech and Language Data with Amazon's Mechanical Turk, Los Angeles, 89
10.1007/s10683-011-9273-9 · DOI
10.1145/2676431.2676437 · DOI
Lang ASID, 2011, Code4Lib J., 15, 1
Lebreton P, 2015, Proc. SPIE Conf. Hum. Vis. Electron. Imaging, 20th, San Francisco
10.1026//1618-3169.49.4.243 · DOI
Shao W, 2015, Digit. Cult. Educ., 7
Suri S, 2011, AAAI Workshops, 25th, San Francisco, 61

Sorted by citations

Items 201–300 of 1,001

Social norms and prevention behaviors in the United States early in the COVID-19 pandemic
Latkin, C. A., Dayton, L., Kaufman, M. R., Schneider, K. E., Strickland, J. C., & Konstantopoulos, A. D. (2021) · Psychology Health & Medicine · doi:10.1080/13548506.2021.2004315 · Cited by 37
Kratom Use in the US: Both a Regional Phenomenon and a White Middle-Class Phenomenon? Evidence From NSDUH 2019 and an Online Convenience Sample
Rogers, J. M., Smith, K. E., Strickland, J. C., & Epstein, D. H. (2021) · Frontiers in Pharmacology · doi:10.3389/fphar.2021.789075 · Cited by 37
A closer lens: Cancer survivors' supportive intervention preferences and interventions received
Arch, J. J., Vanderkruik, R. C., Kirk, A., & Carr, A. L. (2017) · Psycho-Oncology · doi:10.1002/pon.4526 · Cited by 38
Identity Fusion in U.S. Military Members
Hart, R., & Lancaster, S. L. (2017) · Armed Forces & Society · doi:10.1177/0095327x17737021 · Cited by 36
Risk and protective factors in problem gambling: an examination of psychological resilience
Mishra, S., Beshai, S., Wuth, A., & Refaie, N. (2018) · International Gambling Studies · doi:10.1080/14459795.2018.1545242 · Cited by 39
Assessing Organizational Information Visibility: Development and Validation of the Information Visibility Scale
ter Hoeven, C. L., Stohl, C., Leonardi, P. M., & Stohl, M. S. (2019) · Communication Research · doi:10.1177/0093650219877093 · Cited by 39
From anxiety to control: Mask‐wearing, perceived marketplace influence, and emotional well‐being during the COVID‐19 pandemic
Schneider, A. B., & Leonard, B. (2021) · Journal of Consumer Affairs · doi:10.1111/joca.12412 · Cited by 36
COVID-19 Pandemic Impacts on Children with Developmental Disabilities: Service Disruption, Transition to Telehealth, and Child Wellbeing
Zhang, S., Hao, Y., Feng, Y., & Lee, N. Y. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph19063259 · Cited by 36
Self-reported Health Status Differs for Amazon’s Mechanical Turk Respondents Compared With Nationally Representative Surveys
Mortensen, K., Alcalá, M. G., French, M. T., & Hu, T. (2018) · Medical Care · doi:10.1097/mlr.0000000000000871 · Cited by 36
Examining the effectiveness of an online program to cultivate mindfulness and self-compassion skills (Mind-OP): Randomized controlled trial on Amazon's Mechanical Turk
Beshai, S., Bueno, C., Yu, M., Feeney, J. R., & Pitariu, A. (2020) · Behaviour Research and Therapy · doi:10.1016/j.brat.2020.103724 · Cited by 43
Beyond Happiness: Minority Stress and Life Meaning Among LGB Individuals
Douglass, R. P., Conlin, S. E., & Duffy, R. D. (2020) · Journal of Homosexuality · doi:10.1080/00918369.2019.1600900 · Cited by 35
Crowdsourced Health Data: Comparability to a US National Survey, 2013–2015
Yank, V. A., Agarwal, S., Loftus, P. D., Asch, S. M., & Rehkopf, D. H. (2017) · American Journal of Public Health · doi:10.2105/ajph.2017.303824 · Cited by 36
Spanish translation and validation of the Interpersonal Needs Questionnaire.
Silva, C., Hurtado, G., Hartley, C. M., Villafaña, J. N. R., Hovey, J. D., Pettit, J. W., Chorot, P., Valiente, R. M., Sandín, B., & Joiner, T. E. (2018) · Psychological Assessment · doi:10.1037/pas0000643 · Cited by 35
An Experimental Investigation of Victim Blaming in Sexual Assault: The Roles of Victim Sexual Orientation, Coercion Type, and Stereotypes About Bisexual Women
Dyar, C., Feinstein, B. A., & Anderson, R. E. (2019) · Journal of Interpersonal Violence · doi:10.1177/0886260519888209 · Cited by 35
Assessing YouTube science news’ credibility: The impact of web-search on the role of video, source, and user attributes
Michalovich, A., & Hershkovitz, A. (2020) · Public Understanding of Science · doi:10.1177/0963662520905466 · Cited by 38
Recognizing the Presidents
Roediger, H. L., III, & DeSoto, K. A. (2016) · Psychological Science · doi:10.1177/0956797616631113 · Cited by 35
“Hard to find”: information barriers among LGBT college students
Stewart, B. J., & Kendrick, K. D. (2019) · Aslib Journal of Information Management · doi:10.1108/ajim-02-2019-0040 · Cited by 34
Exploring Gender Differences in Machiavellianism Using a Measurement Invariance Approach
Collison, K. L., South, S. C., Vize, C. E., Miller, J. D., & Lynam, D. R. (2020) · Journal of Personality Assessment · doi:10.1080/00223891.2020.1729773 · Cited by 34
Thematic Analysis of Parent–Child Conversations About COVID-19: “Playing It Safe”
Tambling, R. R., Tomkunas, A. J., Russell, B. S., Horton, A. L., & Hutchison, M. (2021) · Journal of Child and Family Studies · doi:10.1007/s10826-020-01889-w · Cited by 34
Psychological Distress Among the U.S. General Population During the COVID-19 Pandemic
Guerrini, C. J., Schneider, S. C., Guzick, A. G., Nwankwo, G. N. A., Canfield, I., Fedson, S., Gutierrez, A. M., Sheu, J. C., Song, A. Y., Villagran, A. M., et al. (2021) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2021.642918 · Cited by 34
Short‐term impacts of COVID‐19 on family caregivers: Emotion regulation, coping, and mental health
Russell, B. S., Hutchison, M., Park, C. L., Fendrich, M., & Finkelstein‐Fox, L. (2021) · Journal of Clinical Psychology · doi:10.1002/jclp.23228 · Cited by 35
Parent Expectancies and Preferences for Mental Health Treatment: The Roles of Emotion Mind-Sets and Views of Failure
Schleider, J. L., & Weisz, J. R. (2018) · Journal of Clinical Child & Adolescent Psychology · doi:10.1080/15374416.2017.1405353 · Cited by 34
How caring is “nullified”: Strong racial identity eliminates White participant empathy effects when police shoot an unarmed Black male.
Johnson, J. D., & Lecci, L. B. (2019) · Psychology of Violence · doi:10.1037/vio0000228 · Cited by 33
The use of experimental vignettes in studying police procedural justice: a systematic review
Nivette, A. E., Nägel, C., & Stan, A. (2022) · Journal of Experimental Criminology · doi:10.1007/s11292-022-09529-7 · Cited by 33
Exploring rates of transgender individuals and mental health concerns in an online sample
Dawson, A. E., Wymbs, B. T., Gidycz, C. A., Pride, M., & Figueroa, W. S. (2017) · International Journal of Transgenderism · doi:10.1080/15532739.2017.1314797 · Cited by 37
Aging with Honor: Examining Ambivalent Ageism and Interpersonal Risk-Factors for Suicide as Explanations for the Honor-Suicide Link
Bock, J. E., Brown, R. P., & Green, K. (2019) · Journal of Social and Clinical Psychology · doi:10.1521/jscp.2019.38.9.721 · Cited by 33
Psychopathic boldness: Narcissism, self-esteem, or something in between?
Miller, J. D., Sleep, C. E., Crowe, M. L., & Lynam, D. R. (2019) · Personality and Individual Differences · doi:10.1016/j.paid.2019.109761 · Cited by 36
High-Risk Sexual Behavior, Binge Drinking and Use of Stimulants are Key Experiences on the Pathway to High Perceived HIV Risk Among Men Who Have Sex with Men in Brazil
Luz, P. M., Torres, T. S., Almeida-Brasil, C. C., Marins, L. M. S., Veloso, V. G., Grinsztejn, B. G. J., Cox, J., & Moodie, E. E. M. (2020) · AIDS and Behavior · doi:10.1007/s10461-020-03035-5 · Cited by 33
Associations between maternal technology use, perceptions of infant temperament, and indicators of mother-to-infant attachment quality
Gutierrez, S. A., & Ventura, A. K. (2021) · Early Human Development · doi:10.1016/j.earlhumdev.2021.105305 · Cited by 33
Ego Depletion and Charitable Support: The Moderating Role of Self-Benefit and Other-Benefit Charitable Appeals
Jin, H. S., Kim, H. J., Suh, J., Sheehan, B., & Meeds, R. W. (2021) · Journal of Advertising · doi:10.1080/00913367.2021.1887012 · Cited by 33
Factor Structure and Incremental Validity of the Original and Modified Versions of the Difficulties in Emotion Regulation Scale
Benfer, N., Bardeen, J. R., Fergus, T. A., & Rogers, T. A. (2018) · Journal of Personality Assessment · doi:10.1080/00223891.2018.1492927 · Cited by 33
How Important Is a Guide Who Has Taken Psilocybin in Psilocybin-Assisted Therapy for Depression?
Earleywine, M., Low, F., Altman, B. R., & Leo, J. A. D. (2022) · Journal of Psychoactive Drugs · doi:10.1080/02791072.2022.2047842 · Cited by 32
Please participate in Part 2: Maximizing response rates in longitudinal MTurk designs
Stoycheff, E. (2016) · Methodological Innovations · doi:10.1177/2059799116672879 · Cited by 32
The Feasibility of a Japanese Crowdsourcing Service for Experimental Research in Psychology
Majima, Y. (2017) · SAGE Open · doi:10.1177/2158244017698731 · Cited by 32
Personal relative deprivation associated with functional disorders via stress: An examination of fibromyalgia and gastrointestinal symptoms
Beshai, S., Mishra, S., Mishra, S., & Carleton, R. N. (2017) · PLoS ONE · doi:10.1371/journal.pone.0189666 · Cited by 40
Loving Oneself: The Associations Among Sexually Explicit Media, Body Image, and Perceived Realism
Vogels, E. A. (2018) · The Journal of Sex Research · doi:10.1080/00224499.2018.1475546 · Cited by 32
Psychological Contract Violation or Basic Need Frustration? Psychological Mechanisms Behind the Effects of Workplace Bullying
Sischka, P. E., Melzer, A., Schmidt, A. F., & Steffgen, G. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.627968 · Cited by 34
Obtaining quality data using behavioral measures of impulsivity in gambling research with Amazon’s Mechanical Turk
Schluter, M. G., Kim, H. S. A., & Hodgins, D. C. (2018) · Journal of Behavioral Addictions · doi:10.1556/2006.7.2018.117 · Cited by 32
MTurk participants have substantially lower evaluative subjective well-being than other survey participants
Stone, A. A., Walentynowicz, M., Schneider, S., Junghaenel, D. U., & Wen, C. K. F. (2019) · Computers in Human Behavior · doi:10.1016/j.chb.2018.12.042 · Cited by 32
The impact of naturalistic cannabis use on self-reported opioid withdrawal
Bergeria, C. L., Huhn, A. S., & Dunn, K. E. (2020) · Journal of Substance Abuse Treatment · doi:10.1016/j.jsat.2020.108005 · Cited by 32
The effect of competition and passive avoidant leadership style on the occurrence of workplace bullying
Sischka, P. E., Schmidt, A. F., & Steffgen, G. (2020) · Personnel Review · doi:10.1108/pr-09-2019-0469 · Cited by 37
DEPAC: a Corpus for Depression and Anxiety Detection from Speech
Tasnim, M., Ehghaghi, M., Diep, B., & Novikova, J. (2022) · Workshop on Computational Linguistics and Clinical Psychology (CLPsych) · doi:10.18653/v1/2022.clpsych-1.1 · Cited by 38
Using behavioral economic variables to predict future alcohol use in a crowdsourced sample
Strickland, J. C., Alcorn, J. L., & Stoops, W. W. (2019) · Journal of Psychopharmacology · doi:10.1177/0269881119827800 · Cited by 33
How broad are state physician health program descriptions of physician impairment?
Lawson, N. D., & Boyd, J. W. (2018) · Substance Abuse Treatment Prevention and Policy · doi:10.1186/s13011-018-0168-z · Cited by 31
Diagnostic Delay and Disease Burden in Primary Aldosteronism: An International Patient Survey
Ananda, R. A., Gwini, S. M., Long, K. M., Lai, J. H., Chen, G., Russell, G. M., Stowasser, M., Fuller, P. J., & Yang, J. (2023) · Hypertension · doi:10.1161/hypertensionaha.123.21965 · Cited by 31
Cognitive fusion, experiential avoidance, and their interactive effect: Examining associations with thwarted belongingness and perceived burdensomeness
Hapenny, J. E., & Fergus, T. A. (2016) · Journal of Contextual Behavioral Science · doi:10.1016/j.jcbs.2016.10.004 · Cited by 31
Building Status in an Online Community
Smirnova, I. N., Reitzig, M. G., & Sorenson, O. (2022) · Organization Science · doi:10.1287/orsc.2021.1559 · Cited by 31
Investigating the challenges of teaching sex education to autistic learners: A qualitative exploration of teachers’ experiences
Bloor, D., Ballantyne, C. J., Gillespie‐Smith, K., Wilson, C., & Hendry, G. (2022) · Research in Developmental Disabilities · doi:10.1016/j.ridd.2022.104344 · Cited by 31
Betting to deal: coping motives mediate the relationship between urgency and Problem gambling severity
Kim, H. S. A., Poole, J. C., Hodgins, D. C., McGrath, D. S., & Dobson, K. S. (2018) · Addiction Research & Theory · doi:10.1080/16066359.2018.1455188 · Cited by 31
Pathways to paranoia: Analytic thinking and belief flexibility
Bronstein, M. V., Everaert, J., Castro, A. A., Joormann, J., & Cannon, T. D. (2018) · Behaviour Research and Therapy · doi:10.1016/j.brat.2018.12.006 · Cited by 32
Discovery sustainable servicescape on behavioural intention practices and nationality: The moderating role of parasocial interaction
Liu, C.-H. S., Horng, J., Chou, S.-F., Yu, T., Lee, M.-T., & Lapuz, M. C. B. (2022) · Journal of Retailing and Consumer Services · doi:10.1016/j.jretconser.2022.103213 · Cited by 30
Interplay between uncertainty intolerance, emotion regulation, cognitive flexibility, and psychopathology during the COVID-19 pandemic: a multi-wave study
Godara, M., Everaert, J., Sánchez-López, Á., Joormann, J., & Raedt, R. D. (2023) · Scientific Reports · doi:10.1038/s41598-023-36211-3 · Cited by 35
Perceptions of Cognitive-Behavioral Therapy and Antidepressant Medication for Depression After Brief Psychoeducation: Examining Shifts in Attitudes
Beshai, S., Watson, L. M., Meadows, T. J. S., & Soucy, J. N. (2019) · Behavior Therapy · doi:10.1016/j.beth.2019.01.001 · Cited by 32
Emotion regulation habits related to depression: A longitudinal investigation of stability and change in repetitive negative thinking and positive reappraisal
Everaert, J., & Joormann, J. (2020) · Journal of Affective Disorders · doi:10.1016/j.jad.2020.07.058 · Cited by 30
A five-factor model of perseverative thought.
Hallion, L. S., Wright, A. G. C., Joormann, J., Kusmierski, S. N., Coutanche, M. N., & Caulfield, M. K. (2022) · Journal of Psychopathology and Clinical Science · doi:10.1037/abn0000737 · Cited by 32
Twinks, Jocks, and Bears, Oh My! Differing subcultural appearance identifications among gay men and their associated eating disorder psychopathology
Fogarty, S., & Walker, D. C. (2022) · Body Image · doi:10.1016/j.bodyim.2022.05.010 · Cited by 30
Perceived Barriers to Climate Change Activism Behaviors in the United States Among Individuals Highly Concerned about Climate Change
Latkin, C. A., Dayton, L., Bonneau, H., Bhaktaram, A. V., Ross, J., Pugel, J., & Latshaw, M. W. (2022) · Journal of Prevention · doi:10.1007/s10935-022-00704-0 · Cited by 30
Body dysmorphic disorder symptoms and risk for suicide: The role of depression
Shaw, A. M., Arditte Hall, K. A., Rosenfield, E., & Timpano, K. R. (2016) · Body Image · doi:10.1016/j.bodyim.2016.09.007 · Cited by 29
Policing protests: an experimental evaluation of the impact of protester race on support for police reform
Dunbar, A., & Hanink, P. A. (2022) · Journal of Experimental Criminology · doi:10.1007/s11292-021-09499-2 · Cited by 29
Exploring beyond simple demographic variables: Differences between traditional laboratory samples and crowdsourced online samples on the Big Five personality traits
Colman, D. E., Vineyard, J., & Letzring, T. D. (2017) · Personality and Individual Differences · doi:10.1016/j.paid.2017.06.023 · Cited by 29
Predicting Problem Gambling Severity: Interplay between Emotion Dysregulation and Gambling-related Cognitions
Buen, A., & Flack, M. (2021) · Journal of Gambling Studies · doi:10.1007/s10899-021-10039-w · Cited by 29
A quantitative analysis of the effects of alternative reinforcement rate and magnitude on resurgence
Podlesnik, C. A., Ritchey, C. M., Kuroda, T., & Cowie, S. (2022) · Behavioural Processes · doi:10.1016/j.beproc.2022.104641 · Cited by 29
Biasing and debiasing health decisions with bar graphs: Costs and benefits of graph literacy
Okan, Y., García‐Retamero, R., Cokely, E. T., & Maldonado, A. (2018) · Quarterly Journal of Experimental Psychology · doi:10.1177/1747021817744546 · Cited by 36
Robotic companions and healthy aging: A mixed-methods exploration of older adults' perspectives and insights
Esmaeilzadeh, P., & Maddah, M. (2024) · Technology in Society · doi:10.1016/j.techsoc.2024.102689 · Cited by 29
Turning Around to Look Ahead: Views of Vocational Psychology in 2001 and 2019
Fouad, N. A., & Kozlowski, M. B. (2019) · Journal of Career Assessment · doi:10.1177/1069072719841602 · Cited by 28
Contributions of fixed mindsets and hopelessness to anxiety and depressive symptoms: A commonality analysis approach
Mullarkey, M. C., & Schleider, J. L. (2019) · Journal of Affective Disorders · doi:10.1016/j.jad.2019.10.023 · Cited by 31
Test–retest reliabilities of four tactic-first sexual violence history questionnaires.
Anderson, R. E., Garcia, M., & Delahanty, D. L. (2021) · Psychology of Violence · doi:10.1037/vio0000384 · Cited by 28
Investigating the Accessibility of Crowdwork Tasks on Mechanical Turk
Uzor, S., Jacques, J. T., Dudley, J. J., & Kristensson, P. O. (2021) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445291 · Cited by 28
Clinically significant depression among parents during the COVID-19 pandemic: Examining the protective role of family relationships.
Russell, B. S., Tambling, R. R., Horton, A. L., Hutchison, M., & Tomkunas, A. J. (2021) · Couple and Family Psychology Research and Practice · doi:10.1037/cfp0000175 · Cited by 28
Reliability of web-based affective auditory stimulus presentation
Seow, T. X. F., & Hauser, T. U. (2021) · Behavior Research Methods · doi:10.3758/s13428-021-01643-0 · Cited by 31
PONT: A Protocol for Online Neuropsychological Testing
Saban, W., & Ivry, R. B. (2021) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_01767 · Cited by 29
Threatened-masculinity shame-related responses among straight men: Measurement and relationship to aggression.
Gebhard, K. T., Cattaneo, L. B., Tangney, J. P., Hargrove, S., & Shor, R. (2018) · Psychology of Men & Masculinity · doi:10.1037/men0000177 · Cited by 35
Feasibility, Acceptability, and Initial Efficacy of Delivering Alcohol Use Cognitive Interventions via Crowdsourcing
Strickland, J. C., Hill, J. C., Stoops, W. W., & Rush, C. R. (2019) · Alcoholism Clinical and Experimental Research · doi:10.1111/acer.13987 · Cited by 28
Can “Focused Deterrence” Produce More Effective Ethics Codes? An Experimental Study
Rorie, M. L., & West, M. P. (2020) · Journal of White Collar and Corporate Crime · doi:10.1177/2631309x20940664 · Cited by 27
Vulnerability in research ethics: A call for assessing vulnerability and implementing protections
Findley, M. G., Ghosn, F., & Lowe, S. J. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2322821121 · Cited by 35
Exploring protective decision‐making in the context of impact‐based flood warnings
Landaverde, E. R. M., Werner, M., & Verkade, J. S. (2019) · Journal of Flood Risk Management · doi:10.1111/jfr3.12587 · Cited by 27
Barriers to patient interest in internet-based cognitive behavioral therapy: Informing e-health policies through quantitative analysis
Moskalenko, M. Y., Hadjistavropoulos, H. D., & Katapally, T. R. (2020) · Health Policy and Technology · doi:10.1016/j.hlpt.2020.04.004 · Cited by 35
A preliminary examination of the multiple dimensions of opioid craving
Bergeria, C. L., Strickland, J. C., Huhn, A. S., Strain, E. C., & Dunn, K. E. (2020) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2020.108473 · Cited by 27
General population normative scores for interpreting the BODY‐Q
Dalaei, F., de Vries, C. E. E., Poulsen, L., Kaur, M. N., Pfob, A., Mou, D., Jacobsen, A. L., Repo, J. P., Salzillo, R., Opyrchał, J., et al. (2022) · Clinical Obesity · doi:10.1111/cob.12528 · Cited by 29
Neuroscience from the comfort of your home: Repeated, self-administered wireless dry EEG measures brain function with high fidelity
Barbey, F. M., Farina, F. R., Buick, A. R., Danyeli, L. V., Dyer, J., Islam, M. N., Krylova, M., Murphy, B., Nolan, H., Rueda‐Delgado, L. M., et al. (2022) · Frontiers in Digital Health · doi:10.3389/fdgth.2022.944753 · Cited by 27
Delusion-like beliefs and data quality: Are classic cognitive biases artifacts of carelessness?
Sulik, J., Ross, R. M., Balzan, R. P., & McKay, R. (2023) · Journal of Psychopathology and Clinical Science · doi:10.1037/abn0000844 · Cited by 30
The condom purchase task: A hypothetical demand method for evaluating sexual health decision‐making
Strickland, J. C., Marks, K. R., & Bolin, B. L. (2020) · Journal of the Experimental Analysis of Behavior · doi:10.1002/jeab.585 · Cited by 27
Unemployment and Online Labor: Evidence from Microtasking
Laitenberger, U., Viete, S., Slivko, O., Kummer, M. E., Borchert, K., & Hirth, M. (2023) · MIS Quarterly · doi:10.25300/misq/2022/17347 · Cited by 37
Factor Structure and Incremental Utility of the Distress Tolerance Scale: A Bifactor Analysis
Rogers, T. A., Bardeen, J. R., Fergus, T. A., & Benfer, N. (2018) · Assessment · doi:10.1177/1073191118789496 · Cited by 26
Temporal Self, Psychopathology, and Adaptive Functioning Deficits
Sokol, Y., & Serper, M. R. (2019) · The Journal of Nervous and Mental Disease · doi:10.1097/nmd.0000000000000925 · Cited by 26
Fearless Dominance/Boldness Is Not Strongly Related to Externalizing Behaviors: An Item Response-Based Analysis
Crowe, M. L., Weiss, B., Sleep, C. E., Harris, A. M., Carter, N. T., Lynam, D. R., & Miller, J. D. (2020) · Assessment · doi:10.1177/1073191120907959 · Cited by 26
Neonatal male circumcision is associated with altered adult socio-affective processing
Miani, A., Bernardo, G. A. D., Højgaard, A. D., Earp, B. D., Zak, P. J., Landau, A. M., Hoppe, J., & Winterdahl, M. (2020) · Heliyon · doi:10.1016/j.heliyon.2020.e05566 · Cited by 27
Age impairs mnemonic discrimination of objects more than scenes: A web-based, large-scale approach across the lifespan
Güsten, J., Ziegler, G., Düzel, E., & Berron, D. (2021) · Cortex · doi:10.1016/j.cortex.2020.12.017 · Cited by 26
Hourly Wages in Crowdworking: A Meta-Analysis
Hornuf, L., & Vrankar, D. (2022) · Business & Information Systems Engineering · doi:10.1007/s12599-022-00769-5 · Cited by 26
Tobacco Marketing, E-cigarette Susceptibility, and Perceptions among Adults
Nicksic, N. E., Snell, L. M., Rudy, A. K., Cobb, C. O., & Barnes, A. J. (2017) · American Journal of Health Behavior · doi:10.5993/ajhb.41.5.7 · Cited by 26
Knowledge of congenital cytomegalovirus (cCMV) among physical and occupational therapists in the United States
Muldoon, K. M., Armstrong-Heimsoth, A., & Thomas, J. (2017) · PLoS ONE · doi:10.1371/journal.pone.0185635 · Cited by 26
Who Cares? Supplier Reactions to Buyer Claims after Psychological Contract Over‐Fulfillments
Esslinger, J., Eckerd, S., Kaufmann, L., & Carter, C. R. (2019) · Journal of Supply Chain Management · doi:10.1111/jscm.12210 · Cited by 26
Reducing the number of non-naïve participants in Mechanical Turk samples
Meyers, E. A., Walker, A. C., Fugelsang, J. A., & Koehler, D. J. (2020) · Methods in Psychology · doi:10.1016/j.metip.2020.100032 · Cited by 29
The association between climate change attitudes and COVID-19 attitudes: The link is more than political ideology✰,✰✰,★
Latkin, C. A., Dayton, L., Coyle, C. R., Yi, G. T., Winiker, A. K., & German, D. (2021) · The Journal of Climate Change and Health · doi:10.1016/j.joclim.2021.100099 · Cited by 29
Randomized comparison of two web-based interventions on immediate and 30-day opioid overdose knowledge in three unique risk groups
Bergeria, C. L., Huhn, A. S., & Dunn, K. E. (2019) · Preventive Medicine · doi:10.1016/j.ypmed.2019.05.006 · Cited by 25
Development and validation of the Maternal Distraction Questionnaire
Ventura, A. K., Hupp, M., Gutierrez, S. A., & Almeida, R. (2020) · Heliyon · doi:10.1016/j.heliyon.2020.e03276 · Cited by 25
eMental Health Literacy and Knowledge of Common Child Mental Health Disorders among Parents of Preschoolers
Cormier, E., Park, H., & Schluck, G. (2020) · Issues in Mental Health Nursing · doi:10.1080/01612840.2020.1719247 · Cited by 25
COVID-19 Misinformation Prophylaxis: Protocol for a Randomized Trial of a Brief Informational Intervention
Agley, J. D., Xiao, Y., Thompson, E. E., & Golzarri‐Arroyo, L. (2020) · JMIR Research Protocols · doi:10.2196/24383 · Cited by 26
Correlates of Levels of Willingness to Engage in Climate Change Actions in the United States
Latkin, C. A., Dayton, L., Lee, D., Yi, G. T., & Uzzi, M. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18179204 · Cited by 25
Problematic Smartphone Use Influences the Relationship Between Experiential Avoidance and Anxiety
Gorday, J. Y., & Bardeen, J. R. (2021) · Cyberpsychology Behavior and Social Networking · doi:10.1089/cyber.2021.0062 · Cited by 26

Authors: Chandler, J., & Shapiro, D.
Affiliations: Mathematica Policy Research (US)
University of Michigan (US)
ORCID: Chandler, Jesse
Venue: Annual Review of Clinical Psychology
Publisher: Annual Reviews
Published: 2016-03-28
Volume: 12
Issue: 1
Pages: 53-81
ISSN: 1548-5943 (ISSN-L)
1548-5951
DOI: 10.1146/annurev-clinpsy-021815-093623
OpenAlex: W2126493218
Cited by: 1,072
References: 177
Citations by year: 2026: 27 · 2025: 70 · 2024: 105 · 2023: 103 · 2022: 150 · 2021: 168 · 2020: 122 · 2019: 115 · 2018: 111 · 2017: 77 · 2016: 24
FWCI: 17.58
Type: journal article
Language: en
Open access: closed
Locations: Annual Review of Clinical Psychology (publishedVersion)
PubMed (publishedVersion)
RePEc: Research Papers in Economics (submittedVersion)
Topics: Physical Sciences › Computer Science › Computer Science Applications › Mobile Crowdsensing and Crowdsourcing
Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Ethics in Clinical Research
Keywords: crowdsourcing, Amazon Mechanical Turk, convenience samples, non-probability sampling, sample composition, data quality, participant recruitment, digital labour markets
MeSH terms: Humans, Biomedical Research, Crowdsourcing

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Conducting Clinical Research Using Crowdsourced Convenience Samples

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks