Cited by 1,072Mobile Crowdsensing and CrowdsourcingSurvey Methodology and NonresponseEthics in Clinical Research

The DOI matches its Crossref record; 174 of the 177 references it cites were found in a registry or an index.

Conducting Clinical Research Using Crowdsourced Convenience Samples

Chandler, Jesse, Shapiro, Danielle·2016·Annual Review of Clinical Psychology

DOI 10.1146/annurev-clinpsy-021815-093623

Open this paper in CoChat — free account

Abstract

Crowdsourcing has had a dramatic impact on the speed and scale at which scientific research can be conducted. Clinical scientists have particularly benefited from readily available research study participants and streamlined recruiting and payment systems afforded by Amazon Mechanical Turk (MTurk), a popular labor market for crowdsourcing workers. MTurk has been used in this capacity for more than five years. The popularity and novelty of the platform have spurred numerous methodological investigations, making it the most studied nonprobability sample available to researchers. This article summarizes what is known about MTurk sample composition and data quality with an emphasis on findings relevant to clinical psychological research. It then addresses methodological issues with using MTurk--many of which are common to other nonprobability samples but unfamiliar to clinical science researchers--and suggests concrete steps to avoid these issues or minimize their impact.

Integrity signals

Registry

Crossref · 10.1146/annurev-clinpsy-021815-093623

Cites

174 of 174 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Amazon's Mechanical Turk: A New Source of Inexpensive, Yet High-Quality, Data?
Buhrmester, M., Kwang, T., & Gosling, S. D. (2011) · Perspectives on Psychological Science · doi:10.1177/1745691610393980 · Cited by 9,996
Cognitive Reflection and Decision Making
Frederick, S. (2005) · Journal of Economic Perspectives · doi:10.1257/089533005775196732 · Cited by 5,256
Evaluating Online Labor Markets for Experimental Research: Amazon.com's Mechanical Turk
Berinsky, A. J., Huber, G. A., & Lenz, G. S. (2012) · Political Analysis · doi:10.1093/pan/mpr057 · Cited by 4,008
Running experiments on Amazon Mechanical Turk
Paolacci, G., Chandler, J., & Ipeirotis, P. G. (2010) · Judgment and Decision Making · doi:10.1017/s1930297500002205 · Cited by 3,767
Self-citation
Instructional manipulation checks: Detecting satisficing to increase statistical power
Oppenheimer, D. M., Meyvis, T., & Davidenko, N. (2009) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2009.03.009 · Cited by 3,493
Conducting behavioral research on Amazon’s Mechanical Turk
Mason, W., & Suri, S. (2012) · Behavior Research Methods · doi:10.3758/s13428-011-0124-6 · Cited by 3,172
Promoting an open research culture
Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., Buck, S., Chambers, C. D., Chin, G., Christensen, G., et al. (2015) · Science · doi:10.1126/science.aab2374 · Cited by 2,708
Data collection in a flat world: the strengths and weaknesses of Mechanical Turk samples
Goodman, J. K., Cryder, C. E., & Cheema, A. (2013) · Journal of Behavioral Decision Making · doi:10.1002/bdm.1753 · Cited by 2,242
Inside the Turk: Understanding Mechanical Turk as a Participant Pool
Paolacci, G., & Chandler, J. (2014) · Current Directions in Psychological Science · doi:10.1177/0963721414531598 · Cited by 2,136
Self-citation
Crowdsourcing user studies with Mechanical Turk
Kittur, A., Ed, H., & Suh, B. (2008) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1357054.1357127 · Cited by 1,941

Show all 177 references

Research works printed but not linked: 12

Barnhoorn JS, 2014, Behav. Res. Methods, 530, 1
Beramendi P, Duch RM, Matsuo A. 2014. When lab subjects meet real people: comparing different modes of experiments. Presented at Asian Polit. Methodol. Conf., Taipai
Casey L, Chandler J, Levine AS, Proctor A, Strolovitch D. 2015. Demographic characteristics of a large sample of US workers. Mathematica Policy Res. Rep. In press
Downes-Le Guin T, 2006, ESOMAR World Res. Conf. Panel Res.
Higgins C, 2010, Proc. NAACL HLT 2010 Workshop on Creat. Speech and Language Data with Amazon's Mechanical Turk, Los Angeles, 89
10.1007/s10683-011-9273-9 · DOI
10.1145/2676431.2676437 · DOI
Lang ASID, 2011, Code4Lib J., 15, 1
Lebreton P, 2015, Proc. SPIE Conf. Hum. Vis. Electron. Imaging, 20th, San Francisco
10.1026//1618-3169.49.4.243 · DOI
Shao W, 2015, Digit. Cult. Educ., 7
Suri S, 2011, AAAI Workshops, 25th, San Francisco, 61

Sorted by citations

Items 801–900 of 1,001

Using survey experiment pretesting to support future pandemic response
Tappin, B. M., & Hewitt, L. (2024) · PNAS Nexus · doi:10.1093/pnasnexus/pgae469 · Cited by 3
Interpersonal theory of suicide predictability of suicidal behaviour: European Spanish validation of the acquired capability for suicide scale-fearlessness about death
Canal‐Rivero, M., Rider, J., Martínez-López, M. J., Reguera, P., Silva, C., Barrigón, M. L., Baca‐García, E., Joiner, T. E., Crespo‐Facorro, B., & Ruíz-Veguilla, M. (2024) · Current Psychology · doi:10.1007/s12144-024-06986-3 · Cited by 3
Are biased and inflexible updating of interpretations broad or narrow transdiagnostic risk markers for psychopathology? A Hierarchical Taxonomy of Psychopathology (HiTOP) lens
Vos, L. M. W., Lodder, P., Bronstein, M. V., Polack, R. G., Smeets, T., Joormann, J., & Everaert, J. (2025) · Behaviour Research and Therapy · doi:10.1016/j.brat.2025.104845 · Cited by 3
Through your partner’s eyes: Perspective taking tempers optimism in behavior predictions
Peetz, J., Maccosham, A., & May, K. (2017) · Journal of Social and Personal Relationships · doi:10.1177/0265407517744386 · Cited by 3
Porn, Peers, and Performing Oral Sex: The Mediating Role of Peer Norms on Pornography’s Influence Regarding Oral Sex
Vogels, E. A., & O’Sullivan, L. F. (2018) · Media Psychology · doi:10.1080/15213269.2018.1483249 · Cited by 3
When Grandparents Move In
Smith, B. T., & Charlton, K. (2020) · Social Psychology · doi:10.1027/1864-9335/a000415 · Cited by 3
Sexual Assault, Posttraumatic Stress Symptoms, and Indices of Aggression Among Women
Wamser‐Nanney, R. A., Walker, H. E., & Nanney, J. T. (2020) · Journal of Traumatic Stress · doi:10.1002/jts.22572 · Cited by 3
Is it Time to Move Away? How Hurricanes Affect Future Plans
Welch‐Devine, M., & Orland, B. (2020) · International Journal of Mass Emergencies & Disasters · doi:10.1177/028072702003800104 · Cited by 3
Factor Structure and Incremental Utility of the Multidimensional Cognitive Attentional Syndrome Scale (MCASS): A Bifactor Analysis
Bardeen, J. R., Clauss, K. E., & Fergus, T. A. (2022) · Journal of Psychopathology and Behavioral Assessment · doi:10.1007/s10862-022-09955-x · Cited by 3
Parental future orientation and parenting outcomes: Development and validation of an adapted measure of parental decision making
Felton, J. W., Oddo, L. E., Cinader, M. K., Maxwell, T. B., Yi, R., & Chronis-Tuscano, A. M. (2023) · Personality and Individual Differences · doi:10.1016/j.paid.2023.112181 · Cited by 3
Interplay among daily gratitude to God, hassles and uplifts, and positive and negative affect
Park, C. L., & Wilt, J. A. (2023) · The Journal of Positive Psychology · doi:10.1080/17439760.2023.2239784 · Cited by 3
Initial validation of the expectancies for Benzodiazepine Analgesia Scale.
Lape, E. C., Powers, J. M., LaRowe, L. R., & Ditre, J. W. (2023) · Experimental and Clinical Psychopharmacology · doi:10.1037/pha0000692 · Cited by 3
All employees benefit: arguments that help increase support for affirmative action in academic careers
Komlenac, N., Neugebauer, L., Birke, J., & Hochleitner, M. (2023) · Humanities and Social Sciences Communications · doi:10.1057/s41599-023-02508-x · Cited by 3
Implicit Associations in Computer-Mediated Communication
Gill, K. S., & Lippmann, M. (2024) · Journal of Media Psychology Theories Methods and Applications · doi:10.1027/1864-1105/a000404 · Cited by 3
Toward an Integrated Model of Healthy Food Purchase via the Impact of Online Nutrition Information Seeking
Pan, P., Bhandari, M., & Meng, J. (2024) · American Behavioral Scientist · doi:10.1177/00027642241235826 · Cited by 3
Amazon Mechanical Turk as a platform for borderline personality disorder researchPreprint
(2018) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/316356 · Cited by 3
Trait Mindful Nonreactivity and Nonjudgment Prospectively Predict of COVID-19 Health Protective Behaviors Across a Two-Month Interval in a USA SamplePreprint
O’Brien, W. H., Wang, S., Varga, A. V., Lim, C. X., Xu, H., Jarukasemthawee, S., Pisitsungkagarn, K., Suvanbenjakule, P., & Braden, A. L. (2021) · medRxiv · doi:10.1101/2021.07.21.21260971 · Cited by 4
Predicting Pornography Use Over Time: Does Self-Reported "Addiction" Matter?Preprint
Grubbs, J. B., Wilt, J. A., Exline, J. J., & Pargäment, K. I. (2018) · doi:10.31234/osf.io/n96rw · Cited by 3
Development and Validation of the Smoker Self-Stigma Questionnaire
Geist, T., Goldberg, K. L., Burt, K. B., Pomichter, E., & Cepeda‐Benito, A. (2023) · Nicotine & Tobacco Research · doi:10.1093/ntr/ntad038 · Cited by 2
The associations between repetitive negative thinking, insomnia symptoms, and sleep quality in adults with a history of trauma
Hall, K. A. A., McGrory, C. M., Snelson, A. M., & Pineles, S. L. (2024) · Anxiety Stress & Coping · doi:10.1080/10615806.2024.2324266 · Cited by 3
The subjective value of social context in people who use cannabis.
Shellenberg, T., Strickland, J. C., Bergeria, C. L., Regnier, S. D., Stoops, W. W., & Lile, J. A. (2024) · Experimental and Clinical Psychopharmacology · doi:10.1037/pha0000717 · Cited by 2
Structural barriers to the formation of theories about the mind: A proposal for a model-based paradigm for psychology
Kerschke, D. (2025) · Methods in Psychology · doi:10.1016/j.metip.2025.100186 · Cited by 2
Perceived disadvantage: Individual differences in sensitivity to unfavorable social comparison predict poorer mental health
Mishra, S., Beshai, S., Feeney, J. R., Fogg, C., & Iskric, A. (2026) · Personality and Individual Differences · doi:10.1016/j.paid.2026.113677 · Cited by 2
Online Survey Researchers’ Experience With Fraudulent Responders
Schuster, A. M., Agrawal, S., Watson, G. P., & Cotten, S. R. (2026) · SAGE Open · doi:10.1177/21582440261424005 · Cited by 2
Assessment of anxiety in older adults: psychometric properties and relationships with self-reported functional impairment
Vito, A. N. D., Calamia, M. R., Weitzner, D. S., & Bernstein, J. P. K. (2019) · International Psychogeriatrics · doi:10.1017/s1041610219001145 · Cited by 2
Identifying Barriers to Enrollment in Patient Pregnancy Registries: Building Evidence Through Crowdsourcing
Pimenta, J. M., Painter, J. L., Gemzoe, K., Levy, R. A., Powell, M., Meizlik, P., & Powell, G. E. (2022) · JMIR Formative Research · doi:10.2196/30573 · Cited by 2
The Association between Message Framing and Intention to Vaccinate Predictive of Hepatitis A Vaccine Uptake
Satybaldiyeva, N., Martinez, L. S., Cooper, B., & Oren, E. (2024) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph21020207 · Cited by 3
Importance of Certification in the Construction Industry: Case of Value Engineering
Wao, J. O., Kosgei, M., & Healey, J. (2024) · EPiC series in built environment · doi:10.29007/bfsk · Cited by 2
Examining Food-Specific and General Inhibitory Control and Working Memory as Moderators of Relations Between Emotion Regulation Difficulties and Eating Pathology in Adults With Overweight/Obesity: A Preregistered, Cross-Sectional Study
Barnhart, W. R., Braden, A. L., & Buelow, M. T. (2024) · Archives of Clinical Neuropsychology · doi:10.1093/arclin/acae065 · Cited by 2
Revealing the Strength of CSR Endorsement in an Environmental CSR Advertising Campaign on Instagram: The Role of Interdependent Self-Construal, Regulatory-Focused Messages, and the Number of Followers
Joo, J., & Lee, Y. (2024) · Journal of Interactive Advertising · doi:10.1080/15252019.2024.2420637 · Cited by 2
Geospatial Analytics of Pedestrian Safety in Tanah Abang
Yusuf, D. S., Pradana, K. A., Aditya, T., Jong-Gun, L., Amin, I., & Muztahid, M. R. (2018) · International Conference on Science and Technology (ICST) · doi:10.1109/icstc.2018.8528599 · Cited by 2
(Non-) impact of task experience on behavioral economic decision-making.
Strickland, J. C., Bolin, B. L., & Marks, K. R. (2021) · Experimental and Clinical Psychopharmacology · doi:10.1037/pha0000438 · Cited by 3
Experimental Manipulation of the BFI-2, IPIP-NEO-120, and the IPC-5
Smith, M. M., McCabe, G. A., & Widiger, T. A. (2022) · Assessment · doi:10.1177/10731911221107622 · Cited by 2
Investigating the specific domains of the cognitive attentional syndrome that predict posttraumatic stress symptoms: The moderating effect of attentional control
Clauss, K. E., Gorday, J. Y., & Bardeen, J. R. (2023) · Journal of Anxiety Disorders · doi:10.1016/j.janxdis.2023.102670 · Cited by 2
Clinical and Ultrasonographic Characteristics of the Achilles Tendon in Hemodialysis Patients
Tharwat, S., Saleh, M., Elrefaey, R., Nassar, M. K., & Nassar, M. K. (2023) · Medicina · doi:10.3390/medicina59122181 · Cited by 2
Experiences of Interpersonal Emotion Regulation for People with Heightened Emotions: An Examination in People with Bipolar Disorder and Those with High Aggression
Swerdlow, B. A., Berk, L., & Johnson, S. L. (2023) · Journal of Emotion and Psychopathology · doi:10.55913/joep.v1i1.30 · Cited by 2
Moral learning and positive schizotypy: Social cognitive mechanisms in psychosis- proneness
Zhao, W., & Cannon, T. D. (2024) · Schizophrenia Research · doi:10.1016/j.schres.2024.02.023 · Cited by 2
Faking health vulnerabilities to meet eligibility criteria to participate in paid internet-mediated research during the COVID-19 pandemic: three case reports
Hoerger, M. P. (2024) · Ethics & Behavior · doi:10.1080/10508422.2024.2325411 · Cited by 2
The WOUND-Q Function and Symptoms Scales for Chronic Lower Extremity Wounds: A Validation Study
Simonsen, N. V., Klassen, A. F., Rae, C. S., Mundy, L. R., Poulsen, L., Pusic, A. L., Fan, K. L., & Sørensen, J. A. (2024) · Advances in Wound Care · doi:10.1089/wound.2024.0035 · Cited by 2
The Dynamic Impacts of Public Perceptions of Fast-Food Products with Nutrition Facts on Fast-Food Consumption
Pan, P., Bhandari, M., & Zeng, L. (2024) · Sustainability · doi:10.3390/su16229913 · Cited by 2
Examining resurgence following alternating exposures to high- and low-magnitudes of alternative reinforcement
Ritchey, C. M., Kuroda, T., & Podlesnik, C. A. (2025) · Learning and Motivation · doi:10.1016/j.lmot.2025.102133 · Cited by 2
A Survey-Based Assessment of Treatment Preferences for Distal Radius Fractures in Participants Aged 65 Years and Older
Rawi, Z. A., Graesser, E. A., & Wall, L. B. (2025) · The Journal Of Hand Surgery · doi:10.1016/j.jhsa.2025.09.017 · Cited by 2
Tolerance of negative emotion moderates the amplification of mental contamination following an evoking task: A randomized experimental study
Fergus, T. A. (2017) · Journal of Behavior Therapy and Experimental Psychiatry · doi:10.1016/j.jbtep.2017.11.006 · Cited by 2
Community perspectives on difference of sex development (DSD) diagnoses: A crowdsourced survey
Alkazemi, M. H., Johnston, A. W., Meglin, D. E., Adkins, D. W., & Routh, J. C. (2020) · Journal of Pediatric Urology · doi:10.1016/j.jpurol.2020.03.023 · Cited by 2
Differential item functioning analysis for the Eating Disorder Examination Questionnaire among heterosexual and sexual minority men
Donahue, J. M., Simone, M., Scharmer, C., Anderson, L. M., & Anderson, D. A. (2020) · Eating and Weight Disorders - Studies on Anorexia Bulimia and Obesity · doi:10.1007/s40519-020-01035-x · Cited by 2
Understanding the Appropriate and Beneficial Use of Before and After Photos in Breast Surgery: A North American Survey
Datta, S., Valiquette, C. R., & Somogyi, R. B. (2023) · Plastic Surgery · doi:10.1177/22925503221146783 · Cited by 2
Public mental health during and after the SARS-CoV-2 pandemic: Opportunities for intervention via emotional self-efficacy and resilience
Karnaze, M. M., Kious, B. M., Feuerman, L. Z., Classen, S. A., Robinson, J. O., Bloss, C. S., & McGuire, A. L. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1016337 · Cited by 3
Perceived COVID-19 Threat, Psychological Distress, and Increases in Alcohol and Cannabis Use at the Onset of the Pandemic in Racially Diverse Emerging Adults
Marks, L. R., Yang, Y., Miller, H. H., Morgan, M., Yoon, E., & Naar, S. (2023) · Emerging Adulthood · doi:10.1177/21676968231198908 · Cited by 2
Multi-drug Resistant Mannheimia hemolytica and Pasteurella multocida in Pneumonic Sheep in Bangladesh
Akhter, S., Zihadi, M. A. H., Rahman, M. H., & Rahman, M. Z. (2023) · Bangladesh Journal of Infectious Diseases · doi:10.3329/bjid.v10i1.68746 · Cited by 2
Dismantling the Five Factor Form
Rojas, S. L. (2017) · UKnowledge (University of Kentucky) · doi:10.13023/etd.2017.216 · Cited by 2
Day, duration, and start time: are the arts providing what their audiences require?
Rhine, A. S., & Murnin, P. M. (2018) · Arts and the Market · doi:10.1108/aam-12-2017-0027 · Cited by 2
Causal beliefs about depression as predictors of essentialist thinking and in network models
Mann, S. (2018) · doi:10.7282/t3m61pgp · Cited by 2
Exploring potential prejudice toward older adult mobility device users.
MacInnis, C. C., Ulrich, C., & Konnert, C. A. (2018) · Psychology and Aging · doi:10.1037/pag0000325 · Cited by 2
Waking up from Dreamland: Opioid Addiction Precipitance and Support for Redistributive Drug Treatment
Chavanne, D., & Goodyear, K. (2020) · Journal of Drug Policy Analysis · doi:10.1515/jdpa-2019-0010 · Cited by 2
Guilty, Innocent, or Just Not Proven? Bayesian Verdicts in the Case of Inhibitory Deficits
Myerson, J., Featherston, K. G., Flores, C. C., Lilienthal, L., Bui, Y. C., & Hale, S. (2021) · Experimental Aging Research · doi:10.1080/0361073x.2021.1884472 · Cited by 2
Reconstructive and restorative cues improve public perception on the value of plastic and reconstructive surgeries
Xun, H., Lee, E. H., Yesantharao, P. S., Eter, L. E., Kraezlin, F., Persing, S. M., & Sacks, J. M. (2021) · Journal of Plastic Reconstructive & Aesthetic Surgery · doi:10.1016/j.bjps.2021.02.012 · Cited by 2
THE NATURE OF DEPRESSIVE RUMINATION AND ITS CONNECTION WITH DEPRESSIVE SYMPTOMS
Maslej, M. M., Mulsant, B. H., & Andrews, P. W. (2020) · Journal of Social and Clinical Psychology · doi:10.1521/jscp.2020.39.9.761 · Cited by 2
A Web-Based Digital Contact Tracing Strategy Addresses Stigma Concerns Among Individuals Evaluated for COVID-19
Trevino, J., Keswani, M., & Pourmand, A. (2021) · Telemedicine Journal and e-Health · doi:10.1089/tmj.2021.0148 · Cited by 2
Examining associations between concussion history, subjectively experienced memory problems, and general health factors in older men
Patten, R. V., & Iverson, G. L. (2021) · The Clinical Neuropsychologist · doi:10.1080/13854046.2021.1991481 · Cited by 2
Influence of Dispositional Optimism on Ethically Salient Research Perspectives: A Pilot Study
Kim, J. P., Mondal, S., Tsungmey, T., Ryan, K., Dunn, L. B., & Roberts, L. W. (2022) · Ethics & Human Research · doi:10.1002/eahr.500126 · Cited by 2
Uncovering the Potential of Cognitive Personalization for UI Adaptation in Crowd Work
Paulino, D., Correia, A., Guimarães, D., Barroso, J., & Paredes, H. (2022) · International Conference on Computer Supported Cooperative Work in Design (CSCWD) · doi:10.1109/cscwd54268.2022.9776164 · Cited by 2
Adherence to COVID-19 Guidelines among Current, Former, and Never Smokers
Szapary, C. L., Avila, J. C., Monnig, M. A., Sokolovsky, A. W., DeCost, G., & Ahluwalia, J. S. (2022) · American Journal of Health Behavior · doi:10.5993/ajhb.46.4.5 · Cited by 2
Response Consistency of Crowdsourced Web-Based Surveys on Type 1 Diabetes
Lin, Y. K., Newman, S., & Piette, J. D. (2023) · Journal of Medical Internet Research · doi:10.2196/43593 · Cited by 2
Nothing Is Certain Except Taxes and the Other Thing: Searching for Death Anxiety in a Large Online Sample
Storelv, S., & Sætrevik, B. (2023) · Collabra Psychology · doi:10.1525/collabra.87507 · Cited by 2
Re: Characteristics of a Mild Traumatic Brain Injury Sample Recruited Using Amazon's Mechanical Turk
Shapiro, D. N., & Chandler, J. (2018) · PM&R · doi:10.1016/j.pmrj.2017.11.013 · Cited by 2
Self-citation
The Effect of Embedded Lies on the Verbal Quality of Statements and Implications for Credibility Assessment
Verigin, B. L. (2020) · doi:10.26481/dis.20200622bv · Cited by 2
Existence of Differential Belief Profiles of COVID-19 Narratives: The Role of Trust in SciencePreprint
Agley, J. D., & Xiao, Y. (2020) · doi:10.21203/rs.3.rs-35919/v1 · Cited by 2
Boosting model-based control in older adultsPreprint
Patzelt, E. H., Kool, W., & Gershman, S. J. (2019) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/2xbwu · Cited by 2
Investigating the relationship between self-perceived moral superiority and moral behavior using economic gamesPreprint
Tappin, B. M., & McKay, R. (2018) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/kqmuz · Cited by 2
Investigating the Impact of Anti-Vaccine News on SARS-CoV-2 Vaccine IntentionsPreprint
Bronstein, M. V., Kummerfeld, E., MacDonald, A. W., & Vinogradov, S. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.3936927 · Cited by 2
To respond or not to respond: The effect of forced and prefer not to answer options in demographic questionnairesPreprint
Santos, H. C., Mestechkin, R. M., Vogt-Yerem, R. L., Rosica, D., Heck, P. R., Heydari, P., Brown, M. I., Rosenbaum, G. M., Shermohammed, M., Meyer, M. N., et al. (2023) · doi:10.31234/osf.io/ga6b2 · Cited by 2
Are You Sure: Preference and Ambivalence in Delay Discounting
Grunevski, S., Smith, A. P., & Yi, R. (2022) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2021.782991 · Cited by 1
Development and Evaluation of the Clinical Trial HEalth Knowledge and Beliefs Scale (CHEKS)
Chung, A., Donley, T., Hays, R. D., Robbins, R., Seixas, A. A., & Jean‐Louis, G. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph19148660 · Cited by 1
Integrating the interpersonal theory of suicide and the dualistic model of passion among adults at risk for suicide
Petrovic, J., Mills, D. J., & Mitchell, S. M. (2022) · Motivation and Emotion · doi:10.1007/s11031-022-09990-x · Cited by 1
Hydrocarbon Citizens
Mazaheri, N. (2022) · doi:10.1093/oso/9780197636725.001.0001 · Cited by 1
Who said that? Impact of source expertise: A generations focused experiment on the perception of radio news sources’ gender, ethos and expertise.
Mohammed-Baksh, S., Fisher, H. D., & Magee, S. (2022) · KOME · doi:10.17646/kome.75672.92 · Cited by 1
Leveraging Online Treatment to Re-examine the Association Between Alcohol Use and Disinhibition
Anderson, A. C., Giddens, E., Tiego, J., Lubman, D. I., & Verdejo‐García, A. (2024) · Psicothema · doi:10.7334/psicothema2022.516 · Cited by 1
Simple reflection exercises can build efficacy and reduce distress about relationship conflicts
Britton, E. M., Marigold, D. C., & McGregor, I. (2024) · Personal Relationships · doi:10.1111/pere.12545 · Cited by 1
Does parental use of harsh discipline and punishment in response to ambiguous child situations vary by child physical abuse risk?
Milner, J. S., Ammar, J., Crouch, J. L., & Wagner, M. F. (2024) · Child Abuse & Neglect · doi:10.1016/j.chiabu.2024.106829 · Cited by 1
Examining worry and secondary stressors on grief severity using machine learning
Uchimura, K. K., & Papa, A. (2024) · Anxiety Stress & Coping · doi:10.1080/10615806.2024.2391841 · Cited by 1
Exploring the Lived Experiences of Medication for Opioid use Disorder Treatment: A Qualitative Study among a Crowdsourced Convenience Sample
Victor, G. A., Kheibari, A., & Strickland, J. C. (2024) · Community Mental Health Journal · doi:10.1007/s10597-024-01345-9 · Cited by 1
Impaired belief revision yet intact information seeking in positive schizotypy: A modified task of bias against disconfirmatory evidence
Zhao, W., Deng, W., & Cannon, T. D. (2024) · PLOS mental health. · doi:10.1371/journal.pmen.0000017 · Cited by 1
Evaluating the Psychometric Properties of a Physical Activity and Sedentary Behavior Identity Scale: Survey Study With Two Independent Samples of Adults in the United States
Wen, C. K. F., Schneider, S., Junghaenel, D. U., Toledo, M. J. L., Lee, P.-J., Smyth, J. M., & Stone, A. A. (2024) · JMIR Formative Research · doi:10.2196/59950 · Cited by 1
Dropout in a Longitudinal Survey of Amazon Mechanical Turk Workers With Low Back Pain: Observational Study
Qureshi, N., Hays, R. D., & Herman, P. M. (2024) · Interactive Journal of Medical Research · doi:10.2196/58771 · Cited by 1
Factor structure and measurement invariance of the Penn Inventory of Scrupulosity-Revised (PIOS-R) across Christian, Jewish, and Muslim groups
Ball, F. C., White, A. M., Younce, J. A., & Wu, K. D. (2024) · Journal of Obsessive-Compulsive and Related Disorders · doi:10.1016/j.jocrd.2024.100930 · Cited by 1
A Pilot Study to Assess the Impact of a Multifactorial Explanation for Mental Illness on Prejudicial Attitudes Towards People With Mental Illness
Segall, H. A., Dick, D. M., Aeilts, A. M., Shoben, A. B., Allain, D. C., & Austin, J. (2024) · American Journal of Medical Genetics Part B Neuropsychiatric Genetics · doi:10.1002/ajmg.b.33016 · Cited by 1
Social Media Research
Guadagno, R. E., & Olivieri, A. F. (2024) · Cambridge University Press eBooks · doi:10.1017/9781009000796.004 · Cited by 1
Which spine surgery techniques are most appealing to the public? A survey examining public perception of spine surgery techniques and factors associated with procedure preference
Perez‐Albela, A., Hanna, J., Daniels, A. H., & Basques, B. A. (2025) · North American Spine Society Journal (NASSJ) · doi:10.1016/j.xnsj.2025.100588 · Cited by 1
Classism and Gendered Racism as Stressors for African American Women
Cavalhieri, K. E., Jaber, N., & Sievers, B. (2025) · The Counseling Psychologist · doi:10.1177/00110000251320126 · Cited by 3
A pilot test to combat the water crisis through transformative messaging: an application of message framing and psychological reactance theory
Weber, K., Buckley, T. M., Rey, R. T., & Waldeck, J. H. (2025) · Applied Environmental Education & Communication · doi:10.1080/1533015x.2025.2486946 · Cited by 1
A Rasch model based psychometric evaluation of different versions of the Minimal Insomnia Symptom Scale (MISS) among adults
Westergren, A., Hedin, G., Hagell, P., Bäccman, C., & Norell‐Clarke, A. (2025) · Sleep Science and Practice · doi:10.1186/s41606-025-00131-0 · Cited by 1
The Indirect Association Between Alcohol Use and Suicide Ideation Through Perceived Burdensomeness and Thwarted Belonging in a High-Risk USA Sample
Marvin, C., LeDuc, M. K., Mitchell, S. M., & Mills, D. J. (2025) · Crisis · doi:10.1027/0227-5910/a001013 · Cited by 1
Sexual Violence Among People Aged 50 Years and Older: Recommendations for Prevention Provided by Direct Survivors of Sexual Violence Past Age 50
Hand, M. D., Lee, M. Y., Mengo, C. W., Dabelko‐Schoeny, H., & Kaiser, M. L. (2025) · Sage Open Aging · doi:10.1177/30495334251367817 · Cited by 1
Scandal, Associative Stigma, and Sorting in Labor Markets: Archival and Experimental Evidence
Kim, J., Byun, H., & Raffiee, J. (2025) · Management Science · doi:10.1287/mnsc.2023.00075 · Cited by 1
Initial Development of a Multidimensional Computerized Adaptive Test for Intensive Longitudinal Assessment of Suicide Risk: Development and Usability Study
McClure, K. E., Ammerman, B. A., Liu, C., Blacutt, M. E., O’Brien, C., & Jacobucci, R. (2025) · JMIR Formative Research · doi:10.2196/76544 · Cited by 2
Rapid Development and Testing of Behavioral Text Message Reminders for Antidepressant Adherence via Online Panels: Survey Study
Tropea, T. R., Marcus, S. C., Bucher, A., Bowden, C. F., Olfson, M., & Stewart, R. E. (2026) · Journal of Medical Internet Research · doi:10.2196/86605 · Cited by 1
Promotive-mountain and preventive-ocean: The interactive effect of natural environment and regulatory focus on green consumption
Lee, H., Si, S., Lee, K., & Youn, N. (2025) · Journal of Environmental Psychology · doi:10.1016/j.jenvp.2025.102854 · Cited by 1
“I feel good… I knew that I would…”: The role of self in musical reward across cultures
Tang, J. (2026) · PLoS ONE · doi:10.1371/journal.pone.0340597 · Cited by 1
Comparing the Convenience, Data Quality, Generalizability, and Outcomes of Student‐ and MTurk‐Generated Data in an Experimental Vignette Rape Perception Study
George, S. S., & Arhin, E. O. (2026) · Behavioral Sciences & the Law · doi:10.1002/bsl.70039 · Cited by 1
The role of decision making in the negative impact of content moderation.
Lewitzka, S., Bridgland, V. M. E., & Takarangi, M. K. T. (2026) · Current Psychology · doi:10.1007/s12144-025-08958-7 · Cited by 1

Authors: Chandler, J., & Shapiro, D.
Affiliations: Mathematica Policy Research (US)
University of Michigan (US)
ORCID: Chandler, Jesse
Venue: Annual Review of Clinical Psychology
Publisher: Annual Reviews
Published: 2016-03-28
Volume: 12
Issue: 1
Pages: 53-81
ISSN: 1548-5943 (ISSN-L)
1548-5951
DOI: 10.1146/annurev-clinpsy-021815-093623
OpenAlex: W2126493218
Cited by: 1,072
References: 177
Citations by year: 2026: 27 · 2025: 70 · 2024: 105 · 2023: 103 · 2022: 150 · 2021: 168 · 2020: 122 · 2019: 115 · 2018: 111 · 2017: 77 · 2016: 24
FWCI: 17.58
Type: journal article
Language: en
Open access: closed
Locations: Annual Review of Clinical Psychology (publishedVersion)
PubMed (publishedVersion)
RePEc: Research Papers in Economics (submittedVersion)
Topics: Physical Sciences › Computer Science › Computer Science Applications › Mobile Crowdsensing and Crowdsourcing
Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Ethics in Clinical Research
Keywords: crowdsourcing, Amazon Mechanical Turk, convenience samples, non-probability sampling, sample composition, data quality, participant recruitment, digital labour markets
MeSH terms: Humans, Biomedical Research, Crowdsourcing

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Conducting Clinical Research Using Crowdsourced Convenience Samples

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks