Cited by 1,072Mobile Crowdsensing and CrowdsourcingSurvey Methodology and NonresponseEthics in Clinical Research

The DOI matches its Crossref record; 174 of the 177 references it cites were found in a registry or an index.

Conducting Clinical Research Using Crowdsourced Convenience Samples

Chandler, Jesse, Shapiro, Danielle·2016·Annual Review of Clinical Psychology

DOI 10.1146/annurev-clinpsy-021815-093623

Open this paper in CoChat — free account

Abstract

Crowdsourcing has had a dramatic impact on the speed and scale at which scientific research can be conducted. Clinical scientists have particularly benefited from readily available research study participants and streamlined recruiting and payment systems afforded by Amazon Mechanical Turk (MTurk), a popular labor market for crowdsourcing workers. MTurk has been used in this capacity for more than five years. The popularity and novelty of the platform have spurred numerous methodological investigations, making it the most studied nonprobability sample available to researchers. This article summarizes what is known about MTurk sample composition and data quality with an emphasis on findings relevant to clinical psychological research. It then addresses methodological issues with using MTurk--many of which are common to other nonprobability samples but unfamiliar to clinical science researchers--and suggests concrete steps to avoid these issues or minimize their impact.

Integrity signals

Registry

Crossref · 10.1146/annurev-clinpsy-021815-093623

Cites

174 of 174 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Amazon's Mechanical Turk: A New Source of Inexpensive, Yet High-Quality, Data?
Buhrmester, M., Kwang, T., & Gosling, S. D. (2011) · Perspectives on Psychological Science · doi:10.1177/1745691610393980 · Cited by 9,996
Cognitive Reflection and Decision Making
Frederick, S. (2005) · Journal of Economic Perspectives · doi:10.1257/089533005775196732 · Cited by 5,256
Evaluating Online Labor Markets for Experimental Research: Amazon.com's Mechanical Turk
Berinsky, A. J., Huber, G. A., & Lenz, G. S. (2012) · Political Analysis · doi:10.1093/pan/mpr057 · Cited by 4,008
Running experiments on Amazon Mechanical Turk
Paolacci, G., Chandler, J., & Ipeirotis, P. G. (2010) · Judgment and Decision Making · doi:10.1017/s1930297500002205 · Cited by 3,767
Self-citation
Instructional manipulation checks: Detecting satisficing to increase statistical power
Oppenheimer, D. M., Meyvis, T., & Davidenko, N. (2009) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2009.03.009 · Cited by 3,493
Conducting behavioral research on Amazon’s Mechanical Turk
Mason, W., & Suri, S. (2012) · Behavior Research Methods · doi:10.3758/s13428-011-0124-6 · Cited by 3,172
Promoting an open research culture
Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., Buck, S., Chambers, C. D., Chin, G., Christensen, G., et al. (2015) · Science · doi:10.1126/science.aab2374 · Cited by 2,708
Data collection in a flat world: the strengths and weaknesses of Mechanical Turk samples
Goodman, J. K., Cryder, C. E., & Cheema, A. (2013) · Journal of Behavioral Decision Making · doi:10.1002/bdm.1753 · Cited by 2,242
Inside the Turk: Understanding Mechanical Turk as a Participant Pool
Paolacci, G., & Chandler, J. (2014) · Current Directions in Psychological Science · doi:10.1177/0963721414531598 · Cited by 2,136
Self-citation
Crowdsourcing user studies with Mechanical Turk
Kittur, A., Ed, H., & Suh, B. (2008) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1357054.1357127 · Cited by 1,941

Show all 177 references

Research works printed but not linked: 12

Barnhoorn JS, 2014, Behav. Res. Methods, 530, 1
Beramendi P, Duch RM, Matsuo A. 2014. When lab subjects meet real people: comparing different modes of experiments. Presented at Asian Polit. Methodol. Conf., Taipai
Casey L, Chandler J, Levine AS, Proctor A, Strolovitch D. 2015. Demographic characteristics of a large sample of US workers. Mathematica Policy Res. Rep. In press
Downes-Le Guin T, 2006, ESOMAR World Res. Conf. Panel Res.
Higgins C, 2010, Proc. NAACL HLT 2010 Workshop on Creat. Speech and Language Data with Amazon's Mechanical Turk, Los Angeles, 89
10.1007/s10683-011-9273-9 · DOI
10.1145/2676431.2676437 · DOI
Lang ASID, 2011, Code4Lib J., 15, 1
Lebreton P, 2015, Proc. SPIE Conf. Hum. Vis. Electron. Imaging, 20th, San Francisco
10.1026//1618-3169.49.4.243 · DOI
Shao W, 2015, Digit. Cult. Educ., 7
Suri S, 2011, AAAI Workshops, 25th, San Francisco, 61

Sorted by citations

Items 501–600 of 1,001

Trusted information sources in the early months of the COVID-19 pandemic predict vaccination uptake over one year later
Latkin, C. A., Dayton, L., Miller, J., Eschliman, E. L., Yang, J., Jamison, A., & Kong, X. (2022) · Vaccine · doi:10.1016/j.vaccine.2022.11.076 · Cited by 13
Measuring psychiatric symptoms online: A systematic review of the use of inventories on Amazon Mechanical Turk (mTurk)
Kolc, K. L., Tan, Y. X. K., Lo, A. Z. Y., Shvetcov, A., Mitchell, P. B., & Perkes, I. E. (2023) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2023.05.027 · Cited by 13
Men’s and Women’s Use and Creation of Online Sexually Explicit Materials Including Fandom-Related Works
Anisimowicz, Y., & O’Sullivan, L. F. (2016) · Archives of Sexual Behavior · doi:10.1007/s10508-016-0865-5 · Cited by 13
Assessing State Mental Contamination: Development and Preliminary Validation of the State Mental Contamination Scale
Lorona, R. T., Rowatt, W. C., & Fergus, T. A. (2017) · Journal of Personality Assessment · doi:10.1080/00223891.2017.1303774 · Cited by 13
Ethnic variations in the relationship between multiple stress domains and use of several types of tobacco/nicotine products among a diverse sample of adults
Rogers, C. J., Forster, M., & Unger, J. B. (2018) · Addictive Behaviors Reports · doi:10.1016/j.abrep.2018.03.006 · Cited by 13
A test of the functional avoidance hypothesis in the development of overgeneral autobiographical memory
Hallford, D. J., Austin, D., Raes, F., & Takano, K. (2018) · Memory & Cognition · doi:10.3758/s13421-018-0810-z · Cited by 13
Psychopathy and Ratings of Persuasiveness: Examining Their Relations in Weaker and Stronger Contexts
Weiss, B., Lynam, D. R., & Miller, J. D. (2018) · Clinical Psychological Science · doi:10.1177/2167702618783733 · Cited by 13
Association of exercise with smoking-related symptomatology, smoking behavior and impulsivity in men and women
Tosun, N. L., ALLEN, S. S., Eberly, L. E., Yao, M., Stoops, W. W., Strickland, J. C., Harrison, K. A., Al’Absi, M., & Carroll, M. E. (2018) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2018.07.022 · Cited by 13
The dishonest gambler: Low HEXACO honesty–humility and gambling severity in a community sample of gamblers
Kim, H. S. A., Rash, C. L., & McGrath, D. S. (2018) · Personality and Mental Health · doi:10.1002/pmh.1433 · Cited by 13
Signaling Class: An Examination of the Treatment Validity of Names Used to Signal Race in Bias Experiments with Methodological Recommendations for Name SelectionPreprint
Gaddis, S. M. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3350739 · Cited by 13
Expressive suppression mediates the relation between disordered eating and suicidal ideation
Ortiz, S., Knauft, K., Smith, A., & Kalia, V. (2019) · Journal of Clinical Psychology · doi:10.1002/jclp.22830 · Cited by 12
Self-compassion and pain disability in adults with chronic pain: The mediating role of future self-identification and self-efficacy
Mistretta, E. G., Davis, M. C., Bartsch, E. M., & Olah, M. S. (2023) · Journal of Health Psychology · doi:10.1177/13591053231167275 · Cited by 12
Career decision‐making, career exploration behaviors, and self‐regulated learning
Morgan, M. J., Heo, J., & Osborn, D. S. (2023) · The Career Development Quarterly · doi:10.1002/cdq.12340 · Cited by 12
Online cognitive testing in Parkinson’s disease: advantages and challenges
Binoy, S., Algon, A. L., Adiva, Y. B., Montaser‐Kouhsari, L., & Saban, W. (2024) · Frontiers in Neurology · doi:10.3389/fneur.2024.1363513 · Cited by 12
Psychometric Properties of the Impairment Rating Scale in Samples of Adults With Attention Deficit Hyperactivity Disorder
Dawson, A. E., Egan, T. E., & Wymbs, B. T. (2017) · Assessment · doi:10.1177/1073191117746502 · Cited by 13
Drug Legalization and Decriminalization Beliefs Among Substance-using and Nonusing Individuals
Hammond, A. S., Dunn, K. E., & Strain, E. C. (2019) · Journal of Addiction Medicine · doi:10.1097/adm.0000000000000542 · Cited by 13
Attention-Deficit/Hyperactivity Disorder Symptoms Are Associated with Greater Delay Discounting of Condom-Protected Sex and Money
Berry, M. S., Sweeney, M. M., Dolan, S. B., Johnson, P. S., Pennybaker, S. J., Rosch, K. S., & Johnson, M. W. (2020) · Archives of Sexual Behavior · doi:10.1007/s10508-020-01698-8 · Cited by 13
Customer Online Feedback with an Identity Versus No Identity: The Influence on Review Comments
Jin, D., DiPietro, R. B., & Watanabe, N. M. (2021) · Journal of Hospitality & Tourism Research · doi:10.1177/1096348020988889 · Cited by 12
A Dark Web of Personality: Network Analyses of Dark Personality Features and Pathological Personality Traits
Jordan, D. G., Jonason, P. K., Zeigler‐Hill, V., Winer, E. S., Fletcher, S., & Underhill, D. (2021) · Journal of Psychopathology and Behavioral Assessment · doi:10.1007/s10862-021-09882-3 · Cited by 12
Interdependent self-construal and number of Twitter followers: consumer responses to alcohol industry corporate social responsibility (CSR) campaign on Twitter
Joo, J., Lee, Y.-J., & Yoon, H. J. (2021) · International Journal of Advertising · doi:10.1080/02650487.2021.1933802 · Cited by 14
Lay beliefs about the causes and treatment of depression: Tests of measurement models and associations
Barnwell, P. V., Mann, S., Fedorenko, E. J., Wheeler, C. S., Everett, B., & Contrada, R. J. (2021) · Journal of Affective Disorders · doi:10.1016/j.jad.2021.11.052 · Cited by 12
Surgeon Personality, Time Spent With the Patient, and Quality of Facilities Are the Most Important Factors to Patients in Selecting an Orthopaedic Sports Medicine Surgeon
Engler, I. D., Ahrendt, G. M., Curley, A. J., & Musahl, V. (2022) · Arthroscopy Sports Medicine and Rehabilitation · doi:10.1016/j.asmr.2022.02.006 · Cited by 12
Connecting a health-focused self-concept with orthorexia nervosa symptoms via fear of losing control over eating unhealthy food and disgust for unhealthy food
Tabri, N., Yung, J. J., & Elliott, C. M. (2022) · Eating and Weight Disorders - Studies on Anorexia Bulimia and Obesity · doi:10.1007/s40519-022-01494-4 · Cited by 12
Personal ethical settings for driverless cars and the utility paradox: An ethical analysis of public attitudes in UK and Japan
Takaguchi, K., Kappes, A., Yearsley, J. M., Sawai, T., Wilkinson, D. J. C., & Savulescu, J. (2022) · PLoS ONE · doi:10.1371/journal.pone.0275812 · Cited by 12
Randomised experiment for the effect of ‘Tobacco-Free Nicotine’ messaging on current e-cigarette users’ perceptions, preferences and intentions
Ratnapradipa, K. L., Samson, K. K., & Dai, H. D. (2023) · Tobacco Control · doi:10.1136/tc-2022-057507 · Cited by 12
Threat-Related Interpretation Biases and Intolerance of Uncertainty in Individuals Exposed to Trauma
Hall, K. A. A., & Arditte, S. J. (2022) · Cognitive Therapy and Research · doi:10.1007/s10608-024-10511-y · Cited by 15
Situational Use of Child Restraint Systems and Carpooling Behaviors in Parents and Caregivers
McDonald, C. C., Kennedy, E. E., Fleisher, L. G., & Zonfrillo, M. R. (2018) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph15081788 · Cited by 12
Executive Control Deficits Potentiate the Effect of Maladaptive Metacognitive Beliefs on Posttraumatic Stress Symptoms
Bardeen, J. R., & Fergus, T. A. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.01898 · Cited by 12
Contribution of cannabis-related cues to concurrent reinforcer choice in humans
Strickland, J. C., Lile, J. A., & Stoops, W. W. (2019) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2019.02.022 · Cited by 12
Understanding of evidence‐based mental health care and the perceived importance of scientific information in a sample of U.S. adults
Ringle, V. A. M., Walsh, L., Maxwell, C. A., Smith, A. M., Grossman, R. A., Becker, S. J., & Jensen‐Doss, A. (2019) · Journal of Clinical Psychology · doi:10.1002/jclp.22856 · Cited by 12
The Detailed Assessment of Posttraumatic Stress–Second Edition (DAPS-2): Initial Psychometric Evaluation in an MTurk-Recruited, Trauma-Exposed Community Sample
Petri, J. M., Weathers, F. W., Witte, T. K., & Silverstein, M. W. (2019) · Assessment · doi:10.1177/1073191119880963 · Cited by 12
Causal explanations of depression on perceptions of and likelihood to choose cognitive behavioural therapy and antidepressant medications as depression treatments
Watson, L. M., & Beshai, S. (2020) · Psychology and Psychotherapy Theory Research and Practice · doi:10.1111/papt.12300 · Cited by 12
The impact of financial strain on medication non-adherence: Influence of psychiatric medication use
Strickland, J. C., Stoops, W. W., Kincer, M. A., & Rush, C. R. (2018) · Psychiatry Research · doi:10.1016/j.psychres.2018.11.055 · Cited by 13
Utilizing content-knowledge questionnaires to assess study eligibility and detect deceptive responding
Strickland, J. C., & Stoops, W. W. (2019) · The American Journal of Drug and Alcohol Abuse · doi:10.1080/00952990.2019.1689990 · Cited by 12
Examining impairment and distress from food addiction across demographic and weight groups
Carr, M. M., Lawson, J. L., Wiedemann, A. A., & Barnes, R. D. (2021) · Eating Behaviors · doi:10.1016/j.eatbeh.2021.101574 · Cited by 12
The role of emotional dysregulation in the association between family‐of‐origin conflict and romantic relationship maintenance
Ogan, M. A., Monk, J. K., Thibodeau‐Nielsen, R. B., Vennum, A., & Soloski, K. L. (2023) · Journal of Marital and Family Therapy · doi:10.1111/jmft.12667 · Cited by 11
A Crime by Any Other Name: Gender Differences in Moral Reasoning When Judging the Tax Evasion of Cryptocurrency Traders
Grym, J., Aspara, J., Nandy, M., & Lodh, S. (2024) · Behavioral Sciences · doi:10.3390/bs14030198 · Cited by 11
Identifying as Autistic Without a Formal Diagnosis: Who Self-Identifies as Autistic and Why?
Ahuvia, I. L., Gürbüz, E., Cuda, J., Kapp, S. K., Houck, A. P., Gates, J. A., Schleider, J. L., & Gurba, A. N. (2026) · Autism in Adulthood · doi:10.1177/25739581251409920 · Cited by 13
Trauma Exposure, Post-traumatic Stress Symptoms, and Indices of Parenting
Wamser‐Nanney, R. A., & Sager, J. C. (2020) · Journal of Interpersonal Violence · doi:10.1177/0886260520958642 · Cited by 11
Destigmatization and Its Imbalanced Effects in Labor Markets
Negro, G., Williams, M. J., Pontikes, E., & Lopiano, G. R. (2021) · Management Science · doi:10.1287/mnsc.2020.3881 · Cited by 11
Understanding self‐affirmation effects: The moderating role of self‐esteem
Zhu, X., & Yzer, M. C. (2021) · British Journal of Health Psychology · doi:10.1111/bjhp.12517 · Cited by 11
Perceived Physical Vulnerability Promotes Prosocial Behavior
Motsenok, M., Kogut, T., & Ritov, I. (2021) · Personality and Social Psychology Bulletin · doi:10.1177/01461672211005879 · Cited by 11
Determinants of worry using the SARS‐CoV‐19 worry (CoV‐Wo) scale among United States residents
Dayton, L., Schneider, K. E., Strickland, J. C., & Latkin, C. A. (2021) · Journal of Community Psychology · doi:10.1002/jcop.22577 · Cited by 11
Patient Perspectives on the Cost of Hand Surgery
Wahl, E. P., Huber, J., Richard, M. J., Ruch, D. S., Mithani, S. K., & Pidgeon, T. S. (2021) · Journal of Bone and Joint Surgery · doi:10.2106/jbjs.20.02195 · Cited by 11
Psychological inflexibility as a mediator of the relationship between adverse childhood experiences and dissociation
Parfait, B., Sease, T. B., & Sandoz, E. K. (2021) · Journal of Contextual Behavioral Science · doi:10.1016/j.jcbs.2021.12.005 · Cited by 11
Who will I be when I retire? The role of organizational commitment, group memberships and retirement transition framing on older worker’s anticipated identity change in retirement
Jolles, D., Lamarche, V. M., Rolison, J. J., & Juanchich, M. (2022) · Current Psychology · doi:10.1007/s12144-022-02869-7 · Cited by 20
Examining the Factor Structure and Incremental Utility of the Contrast Avoidance Questionnaires via Bifactor Analysis
Rogers, T. A., Gorday, J. Y., Bardeen, J. R., & Benfer, N. (2022) · Journal of Personality Assessment · doi:10.1080/00223891.2022.2081921 · Cited by 11
Online Crowdsourcing Survey of United States Population Preferences and Perceptions Regarding Outpatient Hip and Knee Arthroplasty
Pagani, N. R., Puzzitiello, R. N., Stambough, J. B., & Saxena, A. (2022) · The Journal of Arthroplasty · doi:10.1016/j.arth.2022.06.011 · Cited by 11
Interpretation inflexibility and negative social impression in paranoia
Deng, W., Joormann, J., & Cannon, T. D. (2022) · Current Psychology · doi:10.1007/s12144-022-04003-z · Cited by 12
Cognitive personalization for online microtask labor platforms: A systematic literature review
Paulino, D., Correia, A., Barroso, J., & Paredes, H. (2023) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-023-09383-w · Cited by 12
Privacy Concerns About Sharing General and Specific Health Information on Twitter: Quantitative Study
Esmaeilzadeh, P. (2024) · JMIR Formative Research · doi:10.2196/45573 · Cited by 12
Enhancing Psychotherapy Outcomes by Encouraging Patients to Regularly Self-Monitor, Reflect on, and Share Their Affective Responses Toward Their Therapist: Protocol for a Randomized Controlled Trial
Stefana, A., Vieta, E., Fusar‐Poli, P., & Youngstrom, E. A. (2024) · JMIR Research Protocols · doi:10.2196/55369 · Cited by 11
Psychometric Properties of ADHD Symptoms in Toddlers
Brown, H. R., & Harvey, E. A. (2018) · Journal of Clinical Child & Adolescent Psychology · doi:10.1080/15374416.2018.1485105 · Cited by 11
Profiles of Alcohol and Marijuana Use Among Simultaneous Alcohol and Marijuana Users: Individual Differences in Demographics and Substance Use
Linden‐Carmichael, A. N., & Allen, H. K. (2020) · Journal of Drug Issues · doi:10.1177/0022042620979617 · Cited by 11
Forgetting faces over a week: investigating self-reported face recognition ability and personality
Kramer, R. S. S. (2021) · PeerJ · doi:10.7717/peerj.11828 · Cited by 11
Indirect effects of negative body talk on eating, exercise, and expectations about steroids in a sample of at-risk adult men.
Velkoff, E. A., Gibler, R. C., Forrest, L. N., & Smith, A. R. (2018) · Psychology of Men & Masculinity · doi:10.1037/men0000189 · Cited by 11
Effects of Two Educational Posters on Contraceptive Knowledge and Intentions
Anderson, S., Frerichs, L., Kaysin, A., Wheeler, S. B., Halpern, C. T., & Lich, K. H. (2018) · Obstetrics and Gynecology · doi:10.1097/aog.0000000000003012 · Cited by 11
Establishing the psychometric properties and construct validity of the Painful and Provocative Events Scale-Revised
Forrest, L. N., Velkoff, E. A., Johnson, C. J., Luebbe, A. M., & Smith, A. R. (2019) · Journal of Affective Disorders · doi:10.1016/j.jad.2019.04.074 · Cited by 11
Development and preliminary validation of a Five Factor Model measure of MachiavellianismPreprint
Collison, K. L., Vize, C. E., Miller, J., & Lynam, D. R. (2018) · doi:10.31234/osf.io/teg8j · Cited by 11
A Randomized Trial of Brief Online Interventions to Facilitate Treatment Seeking for Social Anxiety.
Tobias, M. R., Landy, L. N., Levin, M. E., & Arch, J. J. (2022) · Behavior Modification · doi:10.1177/01454455211040051 · Cited by 10
“One month after Post, all participants who completed the online intervention session were sent a message through MTurk inviting them to participate in the FU session within the next week.”
Influential
Development and validation of the Trauma-Related Cognitions Scale
Valdez, C. E., London, M. J., Gregorich, S. E., & Lilly, M. M. (2021) · PLoS ONE · doi:10.1371/journal.pone.0250221 · Cited by 10
What predicts employer discrimination? The role of implicit and explicit racial attitudes
Silva, F. (2022) · Social Science Research · doi:10.1016/j.ssresearch.2022.102775 · Cited by 10
Gender Stereotypes and Perceptions of Stranger Violence: Attributions of Blame and Motivation
Phillips, A., & de Roos, M. S. (2022) · International Journal of Offender Therapy and Comparative Criminology · doi:10.1177/0306624x221124849 · Cited by 10
Initial motor skill performance predicts future performance, but not learning
Abeles, D., Herszage, J., Shahar, M., & Censor, N. (2023) · Scientific Reports · doi:10.1038/s41598-023-38231-5 · Cited by 11
Perceptions and motivational factors of Chinese coffee consumers towards robot baristas: a Technology Acceptance Model 2 perspective
Chong, K. L., & Zhang, G. (2024) · Journal of Hospitality and Tourism Insights · doi:10.1108/jhti-01-2024-0126 · Cited by 12
Assessing the quality and reliability of the Amazon Mechanical Turk (MTurk) data in 2024
Shimoni, H., & Axelrod, V. (2025) · Royal Society Open Science · doi:10.1098/rsos.250361 · Cited by 10
Trauma exposure, PTSD and indices of fertility
Wamser‐Nanney, R. A. (2019) · Journal of Psychosomatic Obstetrics & Gynecology · doi:10.1080/0167482x.2019.1619691 · Cited by 10
Generalizability of Effect Sizes Within Aviation Research: More Samples Are Needed
Mehta, R., Rice, S. C., Winter, S. R., Spence, T. B., Edwards, M., & Candelaria-Oquendo, K. (2019) · International Journal of Aviation Aeronautics and Aerospace · doi:10.15394/ijaaa.2019.1404 · Cited by 10
Reward-Related Decision-Making in Current and Past Disordered Gambling: Implications for Impulsive Choice and Risk Preference in the Maintenance of Gambling Disorder
Schluter, M. G., & Hodgins, D. C. (2021) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2021.758329 · Cited by 13
Psychometric properties and validation of the English version Giessen Subjective Complaints List (GBB-8)
Petrowski, K., Zenger, M., Schmalbach, B., Bastianon, C. D., & Strauss, B. M. (2022) · BMC Psychology · doi:10.1186/s40359-022-00741-8 · Cited by 10
Predictors of children's emotion regulation outcomes during COVID ‐19: Role of conflict within the family
Horton, A. L., Russell, B. S., Tambling, R. R., Britner, P. A., Hutchison, M., & Tomkunas, A. J. (2022) · Family Relations · doi:10.1111/fare.12717 · Cited by 10
The Advent of Survey Experiments in Politics and International Relations
Thomas, K. (2022) · Government and Opposition · doi:10.1017/gov.2022.36 · Cited by 10
Self-compassion buffers the internalized alcohol stigma and depression link in women sexual assault survivors who drink to cope
Braun, T. D., Green, Z., Meshesha, L. Z., Sillice, M. A., Read, J. P., & Abrantes, A. M. (2022) · Addictive Behaviors · doi:10.1016/j.addbeh.2022.107562 · Cited by 11
How much are patients willing to pay for spine surgery? An evaluation of attitudes toward out-of-pocket expenses and cost-reducing measures
Alsoof, D., Kasthuri, V. S., McDonald, C. L., Cusano, J., Anderson, G., Diebo, B. G., Kuris, E. O., & Daniels, A. H. (2023) · The Spine Journal · doi:10.1016/j.spinee.2023.08.005 · Cited by 10
BODY-Q Normative Scores: Psychometric Validation of the BODY-Q in the General Population in Europe and North America
Dalaei, F., de Vries, C. E. E., Cano, S., Kaur, M. N., Poulsen, L., Pfob, A., Mou, D., Repo, J. P., Salzillo, R., Opyrchał, J., et al. (2023) · Plastic & Reconstructive Surgery Global Open · doi:10.1097/gox.0000000000005401 · Cited by 10
Older Adults’ Perceptions About Using Intelligent Toilet Seats Beyond Traditional Care: Web-Based Interview Survey
Esmaeilzadeh, P. (2023) · JMIR mhealth and uhealth · doi:10.2196/46430 · Cited by 12
Borderline personality disorder features are associated with inflexible social interpretations
Duda, J. M., Fineberg, S. K., Deng, W., Ma, Y., Everaert, J., Cannon, T. D., & Joormann, J. (2023) · Journal of Affective Disorders · doi:10.1016/j.jad.2023.12.036 · Cited by 10
‘They Talk about the Weather, but No One Does Anything about It’: A Mixed-Methods Study of Everyday Climate Change Conversations
Latkin, C. A., Dayton, L., Winiker, A. K., Countess, K., & Hendrickson, Z. M. (2024) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph21030279 · Cited by 10
Navigating the Social World: The Interplay Between Cognitive and Socio-Affective Processes in Depression and Social Anxiety
Vos, L. M. W., Bronstein, M. V., Gendron, M., Joormann, J., & Everaert, J. (2025) · Cognitive Therapy and Research · doi:10.1007/s10608-024-10557-y · Cited by 12
The Cognitive Underpinnings of Continuous Identity: Higher Episodic Memory Recall and Lower Heuristic Usage Predicts Highest Levels of Self-Continuity
Sokol, Y., Conroy, A. K., & Weingartner, K. M. (2017) · Identity · doi:10.1080/15283488.2017.1303384 · Cited by 10
Fast, Cheap, and Unethical? The Interplay of Morality and Methodology in Crowdsourced Survey Research
Haug, M. C. (2017) · Review of Philosophy and Psychology · doi:10.1007/s13164-017-0374-z · Cited by 10
Investigating the Relationship Between Self-Perceived Moral Superiority and Moral Behavior Using Economic Games
Tappin, B. M., & McKay, R. (2018) · Social Psychological and Personality Science · doi:10.1177/1948550617750736 · Cited by 10
Parenting Stress in Fathers of Children With Type 1 Diabetes
Limbers, C. A., & Teasdale, A. E. (2018) · Family & Community Health · doi:10.1097/fch.0000000000000185 · Cited by 13
What Would You Do? Examining Gun Ownership on Safety Planning Activities and Response Intentions to an Armed and Unarmed Assailant by Gender
Logan, T. (2019) · Violence and Gender · doi:10.1089/vio.2019.0044 · Cited by 10
Exploring the Link Between Transdiagnostic Cognitive Risk Factors, Anxiogenic Parenting Behaviors, and Child Anxiety
Casline, E. P., Patel, Z. S., Timpano, K. R., & Jensen‐Doss, A. (2020) · Child Psychiatry & Human Development · doi:10.1007/s10578-020-01078-2 · Cited by 10
Socio-Ecological Predictors of Frequent Bike Share Trips: Do Purposes Matter?
Chen, L., & Hsu, Y. (2020) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph17207640 · Cited by 10
Assessing the validity of white-collar crime definitions using experimental survey data
Galvin, M. A., Logan, M. W., & Snook, D. W. (2021) · Journal of Experimental Criminology · doi:10.1007/s11292-020-09455-6 · Cited by 10
Ride-Share Use and Child Passenger Safety Behaviors: An Online Survey of Parents
Savage, M. F., Kendi, S. A. I., & Macy, M. L. (2021) · Academic Pediatrics · doi:10.1016/j.acap.2021.03.018 · Cited by 11
Childhood Maltreatment and Parental Attitudes Regarding the Use of Corporal Punishment
Walker, H. E., Diemer, M. C., & Wamser‐Nanney, R. A. (2022) · Journal of Child and Family Studies · doi:10.1007/s10826-022-02279-0 · Cited by 10
Applications of operant demand to treatment selection III : Consumer behavior analysis of treatment choice
Gilroy, S. P., & Picardo, R. N. (2022) · Journal of the Experimental Analysis of Behavior · doi:10.1002/jeab.758 · Cited by 10
You never get a chance to undo a negative first impression: Social anxiety is associated with impaired positive updating of social information
Zabag, R., Azoulay, R., Rinck, M., Becker, E. S., Levy‐Gigi, E., & Gilboa‐Schechtman, E. (2022) · Personality and Individual Differences · doi:10.1016/j.paid.2022.111993 · Cited by 10
Insight Regarding the Presence of Antagonism-Related Traits and Their Subsequent Impairment in Relation to Psychopathy, Narcissism, and Machiavellianism
Sleep, C. E., Lynam, D. R., Campbell, W. K., & Miller, J. D. (2019) · Journal of Personality Disorders · doi:10.1521/pedi_2019_33_422 · Cited by 10
Exploring American Parents’ Lived Experiences During the COVID-19 Pandemic: Ramifications for Well-Being
Kaugars, A. S., Holly, L. E., Tait, M., & Oswald, D. L. (2021) · Journal of Pediatric Psychology · doi:10.1093/jpepsy/jsab120 · Cited by 10
Cognitive Behavioral Therapy, Ketamine, and Combination Treatment for Depression: Impressions of Credibility in Participants with Self-Reported Depressive Symptoms
Earleywine, M., Altman, B. R., & De Leo, J. (2022) · Journal of Psychoactive Drugs · doi:10.1080/02791072.2021.1912863 · Cited by 9
The politics of vaccination: a closer look at the beliefs, social norms, and prevention behaviors related to COVID-19 vaccine uptake within two US political parties
Konstantopoulos, A. D., Dayton, L., & Latkin, C. A. (2023) · Psychology Health & Medicine · doi:10.1080/13548506.2023.2283401 · Cited by 9
Augmented Impressions: The role of augmented reality in shaping perceived product value and enhancing purchase intention
Taub, G., Elmalech, A., & Aharony, N. (2025) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2025.100726 · Cited by 9
Sexual Perfectionism and Women’s Sexual Assertiveness: Understanding the Unique Effects of Perfectionistic Expectations about Sex Originating from and Directed toward the Sexual Partner
Kluck, A. S., Hughes, K. S., & Zhuzha, K. (2018) · Sex Roles · doi:10.1007/s11199-018-0901-0 · Cited by 9
Time-of-day and day-of-week variations in Amazon Mechanical Turk survey responses
Binder, C. C. (2021) · Journal of Macroeconomics · doi:10.1016/j.jmacro.2021.103378 · Cited by 9
Blackouts can serve as a contextual feature and enhance resurgence
Martinez‐Perez, C. N., Ritchey, C. M., Kuroda, T., & Podlesnik, C. A. (2022) · Behavioural Processes · doi:10.1016/j.beproc.2022.104587 · Cited by 9
An online experimental test of the compensatory process in hoarding disorder: reducing loneliness and its effects on object attachment
Yap, K., Timpano, K. R., Schmidt, N. B., & Grisham, J. R. (2024) · Current Psychology · doi:10.1007/s12144-024-05895-9 · Cited by 9

Authors: Chandler, J., & Shapiro, D.
Affiliations: Mathematica Policy Research (US)
University of Michigan (US)
ORCID: Chandler, Jesse
Venue: Annual Review of Clinical Psychology
Publisher: Annual Reviews
Published: 2016-03-28
Volume: 12
Issue: 1
Pages: 53-81
ISSN: 1548-5943 (ISSN-L)
1548-5951
DOI: 10.1146/annurev-clinpsy-021815-093623
OpenAlex: W2126493218
Cited by: 1,072
References: 177
Citations by year: 2026: 27 · 2025: 70 · 2024: 105 · 2023: 103 · 2022: 150 · 2021: 168 · 2020: 122 · 2019: 115 · 2018: 111 · 2017: 77 · 2016: 24
FWCI: 17.58
Type: journal article
Language: en
Open access: closed
Locations: Annual Review of Clinical Psychology (publishedVersion)
PubMed (publishedVersion)
RePEc: Research Papers in Economics (submittedVersion)
Topics: Physical Sciences › Computer Science › Computer Science Applications › Mobile Crowdsensing and Crowdsourcing
Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Ethics in Clinical Research
Keywords: crowdsourcing, Amazon Mechanical Turk, convenience samples, non-probability sampling, sample composition, data quality, participant recruitment, digital labour markets
MeSH terms: Humans, Biomedical Research, Crowdsourcing

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Conducting Clinical Research Using Crowdsourced Convenience Samples

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks