Cited by 1,072Mobile Crowdsensing and CrowdsourcingSurvey Methodology and NonresponseEthics in Clinical Research

The DOI matches its Crossref record; 174 of the 177 references it cites were found in a registry or an index.

Conducting Clinical Research Using Crowdsourced Convenience Samples

Chandler, Jesse, Shapiro, Danielle·2016·Annual Review of Clinical Psychology

DOI 10.1146/annurev-clinpsy-021815-093623

Open this paper in CoChat — free account

Abstract

Crowdsourcing has had a dramatic impact on the speed and scale at which scientific research can be conducted. Clinical scientists have particularly benefited from readily available research study participants and streamlined recruiting and payment systems afforded by Amazon Mechanical Turk (MTurk), a popular labor market for crowdsourcing workers. MTurk has been used in this capacity for more than five years. The popularity and novelty of the platform have spurred numerous methodological investigations, making it the most studied nonprobability sample available to researchers. This article summarizes what is known about MTurk sample composition and data quality with an emphasis on findings relevant to clinical psychological research. It then addresses methodological issues with using MTurk--many of which are common to other nonprobability samples but unfamiliar to clinical science researchers--and suggests concrete steps to avoid these issues or minimize their impact.

Integrity signals

Registry

Crossref · 10.1146/annurev-clinpsy-021815-093623

Cites

174 of 174 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Amazon's Mechanical Turk: A New Source of Inexpensive, Yet High-Quality, Data?
Buhrmester, M., Kwang, T., & Gosling, S. D. (2011) · Perspectives on Psychological Science · doi:10.1177/1745691610393980 · Cited by 9,996
Cognitive Reflection and Decision Making
Frederick, S. (2005) · Journal of Economic Perspectives · doi:10.1257/089533005775196732 · Cited by 5,256
Evaluating Online Labor Markets for Experimental Research: Amazon.com's Mechanical Turk
Berinsky, A. J., Huber, G. A., & Lenz, G. S. (2012) · Political Analysis · doi:10.1093/pan/mpr057 · Cited by 4,008
Running experiments on Amazon Mechanical Turk
Paolacci, G., Chandler, J., & Ipeirotis, P. G. (2010) · Judgment and Decision Making · doi:10.1017/s1930297500002205 · Cited by 3,767
Self-citation
Instructional manipulation checks: Detecting satisficing to increase statistical power
Oppenheimer, D. M., Meyvis, T., & Davidenko, N. (2009) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2009.03.009 · Cited by 3,493
Conducting behavioral research on Amazon’s Mechanical Turk
Mason, W., & Suri, S. (2012) · Behavior Research Methods · doi:10.3758/s13428-011-0124-6 · Cited by 3,172
Promoting an open research culture
Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., Buck, S., Chambers, C. D., Chin, G., Christensen, G., et al. (2015) · Science · doi:10.1126/science.aab2374 · Cited by 2,708
Data collection in a flat world: the strengths and weaknesses of Mechanical Turk samples
Goodman, J. K., Cryder, C. E., & Cheema, A. (2013) · Journal of Behavioral Decision Making · doi:10.1002/bdm.1753 · Cited by 2,242
Inside the Turk: Understanding Mechanical Turk as a Participant Pool
Paolacci, G., & Chandler, J. (2014) · Current Directions in Psychological Science · doi:10.1177/0963721414531598 · Cited by 2,136
Self-citation
Crowdsourcing user studies with Mechanical Turk
Kittur, A., Ed, H., & Suh, B. (2008) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1357054.1357127 · Cited by 1,941

Show all 177 references

Research works printed but not linked: 12

Barnhoorn JS, 2014, Behav. Res. Methods, 530, 1
Beramendi P, Duch RM, Matsuo A. 2014. When lab subjects meet real people: comparing different modes of experiments. Presented at Asian Polit. Methodol. Conf., Taipai
Casey L, Chandler J, Levine AS, Proctor A, Strolovitch D. 2015. Demographic characteristics of a large sample of US workers. Mathematica Policy Res. Rep. In press
Downes-Le Guin T, 2006, ESOMAR World Res. Conf. Panel Res.
Higgins C, 2010, Proc. NAACL HLT 2010 Workshop on Creat. Speech and Language Data with Amazon's Mechanical Turk, Los Angeles, 89
10.1007/s10683-011-9273-9 · DOI
10.1145/2676431.2676437 · DOI
Lang ASID, 2011, Code4Lib J., 15, 1
Lebreton P, 2015, Proc. SPIE Conf. Hum. Vis. Electron. Imaging, 20th, San Francisco
10.1026//1618-3169.49.4.243 · DOI
Shao W, 2015, Digit. Cult. Educ., 7
Suri S, 2011, AAAI Workshops, 25th, San Francisco, 61

Sorted by citations

Items 701–800 of 1,001

Conceptualizing problematic use of medicinal Cannabis: Development and preliminary validation of a brief screening questionnaire
Feingold, D., Gliksberg, O., Brill, S., Amit, B. H., Lev‐Ran, S., Kushnir, T., & Sznitman, S. R. (2024) · Addictive Behaviors · doi:10.1016/j.addbeh.2024.108122 · Cited by 5
Weight stigma and binge eating related to poorer perceptions of healthcare provider interaction quality in a community-based sample
Barnes, R. D., & Lawson, J. L. (2024) · Journal of Eating Disorders · doi:10.1186/s40337-024-01093-x · Cited by 5
Social cure model: testing the link between identity centrality and body appreciation in diverse sexual orientation and gender identity groups
Komlenac, N., Stockburger, K., Birke, J., & Hochleitner, M. (2024) · International Journal for Equity in Health · doi:10.1186/s12939-024-02268-3 · Cited by 5
Cross-Cultural Adaptation and Psychometric Validation of the Suicide Cognitions Scale-Revised (SCS-R) in Spanish Adolescents in Residential Care
Muela, A., Bryan, C. J., García‐Ormaza, J., & Salaberría, K. (2024) · The Spanish Journal of Psychology · doi:10.1017/sjp.2024.30 · Cited by 6
Medical mistrust in racial minorities during the COVID-19 pandemic: Attitudes, actions and mental health outcomes
Wang, P., Zhu, Y., Jin, Z., & Deng, W. (2024) · PLOS Global Public Health · doi:10.1371/journal.pgph.0003871 · Cited by 5
Everyday discrimination and cancer metaphor preferences: The mediating effects of needs for personal significance and cognitive closure
Fernandez, J. R., Richmond, J., Nápoles, A. M., Kruglanski, A. W., & Forde, A. T. (2021) · SSM - Population Health · doi:10.1016/j.ssmph.2021.100991 · Cited by 5
The racial stereotype about mental illness
Boysen, G. A., Axtell, E. L., Kishimoto, A. G., & Sampo, B. L. (2022) · The Journal of Social Psychology · doi:10.1080/00224545.2021.2023086 · Cited by 5
Research on Amazon's stock price forecasting based on arbitrage pricing model based on big data
Du, H. (2022) · Ain Shams Engineering Journal · doi:10.1016/j.asej.2022.101793 · Cited by 5
Learning From Mistakes: Impact of Careless Responses on Counseling Research Using Amazon’s Mechanical Turk
Kim, N., & Oh, J. (2022) · Measurement and Evaluation in Counseling and Development · doi:10.1080/07481756.2022.2073895 · Cited by 7
Promoting healthy eating: The intervening role of health and nutrition-related claims in food advertisements
Pan, P., Bhandari, M., & Meng, J. (2022) · Health Education Journal · doi:10.1177/00178969221132210 · Cited by 5
Promoting self-change in cannabis use disorder: Findings from a randomized trial
Schluter, M. G., Hodgins, D. C., Stea, J. N., & Kilborn, M. (2022) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2022.1015443 · Cited by 5
A Model for Cognitive Personalization of Microtask Design
Paulino, D., Guimarães, D., Correia, A., Ribeiro, J., Barroso, J., & Paredes, H. (2023) · Sensors · doi:10.3390/s23073571 · Cited by 5
Recruiting foreign-born individuals who have sought an abortion in the United States: Lessons from a feasibility study
Zuniga, C., Ragosta, S., & Thompson, T. (2023) · Frontiers in Global Women s Health · doi:10.3389/fgwh.2023.1114820 · Cited by 5
A prospective examination of the role of psychological (in)flexibility in posttraumatic stress: A cross-lagged panel design
Gorday, J. Y., Bardeen, J. R., Rogers, T. A., & Benfer, N. (2023) · Journal of Anxiety Disorders · doi:10.1016/j.janxdis.2023.102725 · Cited by 5
Social support mediates the effects of childhood unpredictability on anhedonia: A retrospective investigation in an online adult community sample
Null, K. E., Duda, J. M., & Pizzagalli, D. A. (2024) · Journal of Mood and Anxiety Disorders · doi:10.1016/j.xjmad.2024.100057 · Cited by 5
Does mental contamination mediate the association between childhood trauma and obsessive-compulsive symptoms in adults?
Corkish, B., & Yap, K. (2024) · Child Abuse & Neglect · doi:10.1016/j.chiabu.2024.106789 · Cited by 5
What does it mean when people say that they have received expressions of concern about their drinking or advice to cut down on the AUDIT scale?
Cunningham, J. A., Godinho, A., Kushnir, V., & Bertholet, N. (2017) · BMC Medical Research Methodology · doi:10.1186/s12874-017-0435-0 · Cited by 5
Speaking on Data’s Behalf: What Researchers Say and How Audiences Choose
Chandler, J. J., Martínez, I., Finucane, M., Terziev, J., & Resch, A. M. (2019) · Evaluation review · doi:10.1177/0193841x19834968 · Cited by 1
Self-citation
A Structural Examination of Negative Emotion Vulnerability
Uliaszek, A. A., & Al‐Dajani, N. (2019) · Journal of Psychopathology and Behavioral Assessment · doi:10.1007/s10862-019-09761-y · Cited by 5
When bad gets worse: Negative wording amplifies negative recall in persons with the borderline personality trait
Maráz, A., Nagy, T., & Ziegler, M. D. (2020) · Applied Cognitive Psychology · doi:10.1002/acp.3764 · Cited by 5
Amazon’s MTurk: A Currently Underutilised Resource for Survey Researchers?
Farrell, M., & Sweeney, B. (2021) · Accounting Finance & Governance Review/Accounting finance & governance review · doi:10.52399/001c.22019 · Cited by 5
Differential impacts of economic and demographic variables on substance use patterns during the COVID-19 pandemic
Turner, B. W., Cox, D. H., Shellenberg, T., Katz, N. R., Lundie, B. A., Lile, J. A., Stoops, W. W., & Rush, C. R. (2022) · The American Journal of Drug and Alcohol Abuse · doi:10.1080/00952990.2021.1984492 · Cited by 5
Delay discounting, probability discounting, and interdental cleaning frequency
DeFulio, A., & Rzeszutek, M. J. (2022) · BMC Oral Health · doi:10.1186/s12903-022-02328-6 · Cited by 5
Expectancies for and Pleasure from Simultaneous Alcohol and E-Cigarette Use among Young Adults
Walsh, B. E., Williams, C. M., & Zale, E. L. (2022) · Substance Use & Misuse · doi:10.1080/10826084.2022.2136495 · Cited by 5
Multidimensional Inventory of Hypochondriacal Traits: An Examination of a Bifactor Model and Measurement Invariance Between Those With and Without a Self-Reported Medical Condition
Bardeen, J. R., & Fergus, T. A. (2017) · Assessment · doi:10.1177/1073191117725173 · Cited by 5
Brief versus Thong Hygiene in Obstetrics and Gynecology (B‐THONG): A survey study
Hamlin, A. A., Sheeder, J. L., & Muffly, T. M. (2019) · Journal of obstetrics and gynaecology research · doi:10.1111/jog.13958 · Cited by 5
Posttraumatic Stress Disorder and Expectations of Parenthood and Children's Development
Wamser‐Nanney, R. A. (2019) · Journal of Traumatic Stress · doi:10.1002/jts.22398 · Cited by 6
Contribution of alcohol- and cigarette-related cues to concurrent reinforcer choice in humans
Strickland, J. C., & Bergeria, C. L. (2020) · Behavioural Processes · doi:10.1016/j.beproc.2020.104124 · Cited by 5
Exploring Help Facilities in Game-Making Software
Kao, D. (2020) · International Conference on the Foundations of Digital Games (FDG) · doi:10.1145/3402942.3403014 · Cited by 5
The Coronavirus (COVID-19) Effect on Public Sentiments Regarding Elective Plastic Surgery in the United States
Chen, J., & Ray, E. (2021) · Plastic & Reconstructive Surgery Global Open · doi:10.1097/gox.0000000000003579 · Cited by 5
Assessing Flexibility of Solution Strategy: Strategy Shifting as a Measure of Cognitive Reserve
Barulli, D., Habeck, C. G., & Stern, Y. (2023) · The Journals of Gerontology Series B · doi:10.1093/geronb/gbad024 · Cited by 5
E-cigarette Price Impacts legal and Black-Market Cigarette Purchasing Under a Hypothetical Reduced-Nicotine Cigarette Standard
Dolan, S. B., Bradley, M. K., & Johnson, M. W. (2023) · Nicotine & Tobacco Research · doi:10.1093/ntr/ntad067 · Cited by 5
Rationale and Study Checklist for Ethical Rejection of Participants on Crowdsourcing Research Platforms
Agley, J. D., Mumaw, C., & Johnson, B. (2024) · Ethics & Human Research · doi:10.1002/eahr.500217 · Cited by 4
Examining effects of response-cost punishment and extinction in isolation and in combination on resurgence
Ritchey, C. M., Martinez‐Perez, C. N., Lamperski, M. S., Kuroda, T., & Podlesnik, C. A. (2024) · Learning and Motivation · doi:10.1016/j.lmot.2024.102018 · Cited by 4
Patient Preferences for Utilization of Telemedicine in Aesthetic Surgery Consultations
Long, E., Bustos, V. P., Falcon, D. J., Foster, L., Boustany, A., & Lin, S. J. (2025) · Aesthetic Plastic Surgery · doi:10.1007/s00266-025-04749-4 · Cited by 4
Normative face recognition ability test scores vary across online participant pools
Popovic, B., Dunn, J. D., Towler, A., & White, D. (2025) · Scientific Reports · doi:10.1038/s41598-025-92907-8 · Cited by 4
Evidence of Measurement Invariance in the Working Alliance Inventory Across In-Person and Videoconferencing Psychotherapy
Stefana, A. (2025) · Psychotherapy Research · doi:10.1080/10503307.2025.2495827 · Cited by 4
What makes self-forgiveness so difficult (for some)? Understanding the lived experience of those stuck in self-condemnation
Woodyatt, L., de Vel‐Palumbo, M., Barron, A., Harous, C., Wenzel, M., & de Silva, S. (2025) · Self and Identity · doi:10.1080/15298868.2025.2513878 · Cited by 4
A Love Supreme: Love’s Associations with Well-Being, Anxiety, Depression, and Spirituality
Marcovski, F. C. D. S., & Miller, L. (2025) · Religions · doi:10.3390/rel16060741 · Cited by 4
Citizen Science from the Perspective of Higher Education Professors
Lameira, H. L. N., Guerrero-Moreno, M. A., da Silva, E. C., Santos, P. R. B., Teodósio, M. A., & de Oliveira, J. M. B. (2025) · Education Sciences · doi:10.3390/educsci15060738 · Cited by 4
Fake News and True News Assessment: The Persuasive Effect of Discursive Evidence in Judging Veracity
Baiyere, A., Bauer, J., Constantiou, I., & Hardt, D. (2025) · MIS Quarterly · doi:10.25300/misq/2024/17542 · Cited by 5
Planning for New Threats to Online Research Data Validity: The Issue of Computer-Using Agents
Agley, J. D. (2025) · Evaluation & the Health Professions · doi:10.1177/01632787251367407 · Cited by 4
Variability and stability of autistic traits in the general population: A systematic comparison between online and in-lab samples
Wu, Q., Hong, Q., Kim, N. Y., Adolphs, R., Paul, L. K., & Charpentier, C. J. (2025) · Personality Neuroscience · doi:10.1017/pen.2025.10001 · Cited by 4
Session evaluation scale: psychometric evaluation and development of short versions
Stefana, A., & Hill, C. E. (2025) · Psychotherapy Research · doi:10.1080/10503307.2025.2587694 · Cited by 4
Collecting data from Internet (and other platform) users for mental health research
Lahoud, A. A., Gladstone, T. R., Clark, S. B., & Flessner, C. A. (2021) · Elsevier eBooks · doi:10.1016/b978-0-12-822201-0.00015-0 · Cited by 4
In This Together? Exploring Moderating and Mediating Effects of Shared Racial Fate on the Predictive Role of Racial Socialization and Discrimination on Black Activism
Ajayi, A. A., Grier‐Reed, T., & Houseworth, J. E. (2022) · Journal of Black Psychology · doi:10.1177/00957984211067619 · Cited by 4
Public Perceptions of Opioid Use Following Orthopedic Surgery: A Survey
McIntyre, J. A., Pagani, N. R., Schuyver, P. R. V., Puzzitiello, R. N., Moverman, M. A., Menendez, M. E., & Kavolus, J. J. (2022) · HSS Journal® The Musculoskeletal Journal of Hospital for Special Surgery · doi:10.1177/15563316221097698 · Cited by 4
BRCAShare —Assessment of an animated digital message for intrafamilial communication of pathogenic variant positive test results: A feasibility study
Aeilts, A. M., Carpenter, K. M., Hovick, S. R. A., Byrne, L., Shoben, A. B., & Senter, L. A. (2022) · Journal of Genetic Counseling · doi:10.1002/jgc4.1656 · Cited by 4
Evidence that Mental Contrasting Reduces Health Information Avoidance
Yang, M. Z., King, W. C., Oettingen, G., & Sheeran, P. (2023) · Annals of Behavioral Medicine · doi:10.1093/abm/kaad031 · Cited by 4
Has feminism “gone too far?” A mixed-methods exploration of perceptions of digital feminist activism
Nicolla, S. (2023) · Feminist Media Studies · doi:10.1080/14680777.2023.2285712 · Cited by 4
Diversity in the study of aging and lifespan development
Rolison, J. J. (2024) · Current Opinion in Psychology · doi:10.1016/j.copsyc.2024.101802 · Cited by 4
Methodological considerations for behavioral studies relying on response time outcomes through online crowdsourcing platforms
McConnell, P. A., Finetto, C., & Heise, K.-F. (2024) · Scientific Reports · doi:10.1038/s41598-024-58300-7 · Cited by 4
Psychometric Evaluation of a Measure of Posttraumatic Safety Behaviors
Foulser, A. A., Rubin, M., & Telch, M. J. (2024) · Cognitive and Behavioral Practice · doi:10.1016/j.cbpra.2024.03.003 · Cited by 4
Childhood Victimizing ACEs, Peer Victimization, Cyber Victimization: Gender Differences and Associations with Young Adult Psychological Distress
Miller, H. H., Putzeys, S., & Jenkins, L. N. (2024) · Journal of Child & Adolescent Trauma · doi:10.1007/s40653-024-00671-0 · Cited by 4
Critical thinking about health and treatments in the United States: a cross-sectional assessment of parents and undergraduate college students
Ringle, V. A. M., Dahlgren, A., Rosenbaum, S., & Jensen‐Doss, A. (2025) · BMC Public Health · doi:10.1186/s12889-025-21291-9 · Cited by 4
Criterion A vs. Non-Criterion A Stressors: Relations to Parenting Perceptions, Support, Satisfaction, and Involvement
Wamser‐Nanney, R. A., & Cherry, K. E. (2020) · Journal of Aggression Maltreatment & Trauma · doi:10.1080/10926771.2019.1710640 · Cited by 5
Investigating direct and indirect influences of parent personality on child medication adherence
Durkin, L. K., Bugno, L. T., Feldman, E. C. H., Davies, W. H., & Greenley, R. N. (2020) · Children s Health Care · doi:10.1080/02739615.2020.1835480 · Cited by 4
The relationship between nonsystematic delay discounting and low-quality survey responses in a sample of smokers: ROC curve analysis.
Yeh, Y., Lemos, R. F., Craft, W. H., & Bickel, W. K. (2022) · Experimental and Clinical Psychopharmacology · doi:10.1037/pha0000584 · Cited by 4
“Say Her Name”: Symbolic Racism and Officer Valuing Predict White Americans’ Reactions to the Fatal Police Shooting of a Black Woman
Johnson, J. D., Sattler, D. N., Smith-Galeno, B., Ginther, K., Otton, K., & Dierckx, K. (2023) · Violence Against Women · doi:10.1177/10778012231179210 · Cited by 4
Further psychometric validation and test–retest reproducibility of the WOUND‐Q
Simonsen, N. V., Klassen, A. F., Rae, C. S., Dalaei, F., Cano, S., Poulsen, L., Pusic, A. L., & Sørensen, J. A. (2023) · International Wound Journal · doi:10.1111/iwj.14354 · Cited by 4
Towards Cryptocurrency Adoption in Tanzania: Potential Risks and Challenges to the Financial Ecosystem
Hamidu, K. M., Pastory, D., Massi, J., & Mrindoko, A. E. (2023) · International Journal of Social Science Research and Review · doi:10.47814/ijssrr.v6i7.1355 · Cited by 4
A conceptual dismantling of the Five Factor Form: Lexical support for the bipolarity of maladaptive personality structure
Rojas, S. L., Crego, C., & Widiger, T. A. (2019) · Journal of Research in Personality · doi:10.1016/j.jrp.2019.04.011 · Cited by 4
The particular case of conducting addiction intervention research on Mechanical Turk
Godinho, A., Cunningham, J. A., & Schell, C. (2020) · Addiction · doi:10.1111/add.15097 · Cited by 4
Naturalness, Personality, and Mindfulness Predict EcoWellness: Implications for Counseling Practice
Holden, C. J., Reese, R. F., & Seitz, C. M. (2020) · International Journal for the Advancement of Counselling · doi:10.1007/s10447-020-09414-w · Cited by 4
The adult public’s perception of the utility of play therapy.
Hindman, M., Perryman, K. L., & Robinson, S. (2021) · International Journal of Play Therapy · doi:10.1037/pla0000164 · Cited by 4
A day at the races
Losada, D. E., Elsweiler, D., Harvey, M., & Trattner, C. (2021) · Applied Intelligence · doi:10.1007/s10489-021-02719-2 · Cited by 4
Is all prejudice created equal? The role of modern and aversive racism in mock juror decisions
Estrada-Reynolds, V. C., Freng, S., Schweitzer, K. A., & Leki, E. L. (2022) · Psychiatry Psychology and Law · doi:10.1080/13218719.2022.2073283 · Cited by 4
Influence of Betrayal Trauma on Schizotypal Personality Pathology
Yalch, M. M., Snyder, K., Hujing, C. L., & Torres, A. N. (2022) · Journal of Trauma & Dissociation · doi:10.1080/15299732.2022.2120153 · Cited by 4
Stigma, Post-traumatic Stress, and COVID-19 Vaccination Intent in Mongolia, India, and the United States
Sattler, D. N., Бишхорлоо, Б., Lawley, K. A., Hackler, R., Byambajav, C., Munkhbat, M., & Smith-Galeno, B. (2023) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph20032084 · Cited by 4
Development of the Healthcare Triggering Questionnaire in adult sexual abuse survivors.
Schnur, J. B., Chaplin, W. F., Khurshid, K., Mogavero, J. N., Goldsmith, R. E., Lee, Y., Litman, L., & Montgomery, G. H. (2017) · Psychological Trauma Theory Research Practice and Policy · doi:10.1037/tra0000273 · Cited by 4
Online assessment of problem-solving skills among fathers of young and school-age children with type 1 diabetes: Associations with parent and child outcomes
Teasdale, A. E., & Limbers, C. A. (2020) · Journal of Child Health Care · doi:10.1177/1367493520942711 · Cited by 4
Opioid use, motivation to quit, and treatment status related to COVID-19: a cross-sectional study
Parker, M. A., Agley, J. D., Adams, Z. W., & Villanti, A. C. (2021) · BMC Research Notes · doi:10.1186/s13104-021-05601-z · Cited by 4
Exploring the Structure of Narcissism: Towards an Integrated SolutionPreprint
Crowe, M. L., Lynam, D. R., Campbell, W. K., & Miller, J. (2019) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/wpcjq · Cited by 4
Signaling Class: An Examination of the Treatment Validity of Names Used to Signal Race in Bias Experiments with Methodological Recommendations for Name SelectionPreprint
Gaddis, S. M. (2024) · doi:10.31235/osf.io/cdgju · Cited by 4
Is Crowdsourcing a Puppet Show? Detecting a New Type of Fraud in Online Platforms
Wang, S., Jui, I. J., & Thorpe, J. (2024) · New Security Paradigms Workshop · doi:10.1145/3703465.3703472 · Cited by 3
Effects of Interpretation Flexibility, Community Connectedness, and Perceived Everyday Discrimination on Internalized Homophobia in the US
Deng, W., Xie, H. W., Wang, P., Ortiz, J., Zhou, J. M., Cavanagh, T., & Lin, B. Y. (2025) · Sexuality Research and Social Policy · doi:10.1007/s13178-025-01115-6 · Cited by 3
Cues paired with alternative reinforcement mitigate resurgence in humans
Podlesnik, C. A., Martinez‐Perez, C. N., Ritchey, C. M., & Kuroda, T. (2025) · Learning and Motivation · doi:10.1016/j.lmot.2025.102137 · Cited by 3
It’s more than money: The true costs of spam in online survey research
Agrawal, S., Watson, G. P., Schuster, A. M., Cotten, S. R., Tidwell, M. L., Baker, N. M., & Wang, S. (2025) · Research Ethics · doi:10.1177/17470161251357454 · Cited by 3
Beyond Negative Emotions: The Effects of Positive Emotions and Stealing Thunder on Organizational Outcomes After a Crisis
Ndone, J. (2025) · Journal of Contingencies and Crisis Management · doi:10.1111/1468-5973.70075 · Cited by 3
Empirically Derived Racial Ideologies Among Black Americans: Implications for Psychological Well-Being, Racial Homophily, and Sociopolitical Activism
Ajayi, A. A. (2025) · Journal of Black Psychology · doi:10.1177/00957984251387222 · Cited by 3
Impacts of cognitive forcing and need for cognition on biased AI-assisted decision making about mental health emergencies
Vejandla, S., Ray, A., Sikstrom, L., Ratto, M., Zaheer, J., & Maslej, M. M. (2025) · Scientific Reports · doi:10.1038/s41598-025-30506-3 · Cited by 4
Dissecting Fear and Emotional Pain in Posttraumatic Stress Disorder: From Symptom Networks to Neural Signatures
Ben‐Zion, Z., Basol, E. Z., Simon, A. J., Abargil, M., Samonek, K., Paterson, M., Spiller, T. R., Duek, O. A., Just, S., Preller, K. H., et al. (2025) · Biological Psychiatry · doi:10.1016/j.biopsych.2025.11.016 · Cited by 5
Temporal self-appraisals associated with suicide-related thoughts and behaviors
Sokol, Y., Rosensweig, C., Levin, C., & Glatt, S. (2024) · Journal of Affective Disorders · doi:10.1016/j.jad.2024.01.081 · Cited by 3
Using a standardized sound set to help characterize misophonia: The International Affective Digitized Sounds
Trumbull, J., Lanier, N., Mcmahon, K., Guetta, R. E., & Rosenthal, M. Z. (2024) · PLoS ONE · doi:10.1371/journal.pone.0301105 · Cited by 3
Crowdsourcing in Management Research
Lenart-Gansiniec, R. A. (2024) · Routledge eBooks · doi:10.4324/9781003482253 · Cited by 3
Good for all, good for me: the influences of dedication- and constraint-based switching barriers on user-generated contributions
Chou, E.-Y., & Lin, C. (2024) · Internet Research · doi:10.1108/intr-05-2023-0363 · Cited by 3
“I Can’t Afford the Copays, So Guess I’ll Die?” Using Discounting Methodology to Quantify How Cost Affects Health-Care Utilization
Rzeszutek, M. J., DeFulio, A., & Khan, F. F. (2024) · Behavior and Social Issues · doi:10.1007/s42822-024-00173-6 · Cited by 3
Patient Perception of Allograft Versus Autograft in ACL Reconstruction Using a Validated Online Survey Marketplace
Dean, R. S., Hafen, T. J., Braithwaite, C. L., Farley, K. X., LaPrade, R. F., GUETTLER, J. H., & Bicos, J. (2024) · Orthopaedic Journal of Sports Medicine · doi:10.1177/23259671241263853 · Cited by 3
The causal effect of skin color bias in online dating
Santana, E. (2024) · Social Science Research · doi:10.1016/j.ssresearch.2024.103076 · Cited by 4
Practice the purpose preach! Experimental evidence on the effect of corporate purpose on workers' willingness to go the extra mile
Brosch, N. D. (2025) · Long Range Planning · doi:10.1016/j.lrp.2025.102536 · Cited by 3
Development and Validation of the New Helping Attitude Scale to Assess Propensity for Serving Others
Trzeciak, S., Mazzarelli, A., Roberts, M. B., & Roberts, B. W. (2025) · Researchers Journal of Internal Medicine · doi:10.63495/3092284 · Cited by 3
Do MTurkers exhibit myopic loss aversion?
Schwaiger, R., & Hueber, L. (2021) · Economics Letters · doi:10.1016/j.econlet.2021.110137 · Cited by 3
Can Dining Alone Lead to Healthier Menu Item Decisions than Dining with Others? The Roles of Consumption Orientation and Menu Nutrition Information
Her, E., Behnke, C., & Almanza, B. A. (2021) · 대한지역사회영양학회지 · doi:10.5720/kjcn.2021.26.3.155 · Cited by 3
Attitudes Toward Organ Donation for Persons Who Have a Substance Use Disorder Relative to Other Health Conditions
Durgin, C. J., Bergeria, C. L., Huhn, A. S., & Dunn, K. E. (2021) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2021.698645 · Cited by 3
Violent Assault on a Chinese Man: COVID-19 Psychosocial Resource Loss Diminishes Right Wing Authoritarianism Variability in Societal Reactions
Johnson, J. D., Sattler, D. N., Hiel, A. V., Dierckx, K., Luo, S., & Vezzali, L. (2022) · Journal of Interpersonal Violence · doi:10.1177/08862605221123301 · Cited by 3
Problem Drinking, Perceived Burden, Depression, and Suicide Ideation: A Conceptual Model
Nichols, Z. C., Lee, U., Mills, D. J., & Comiskey, G. E. (2022) · Alcoholism Treatment Quarterly · doi:10.1080/07347324.2022.2149373 · Cited by 3
Exploring the Relationship between Interpersonal Empathy, Social Empathy, and Racial Attitudes
Fisher, A. K., Droubay, B. A., & Bacon, C. (2023) · Journal of Social Service Research · doi:10.1080/01488376.2023.2229405 · Cited by 3
Effects of a Self-Guided Online Mindfulness and Self-Compassion Program (Mind-OP) in Reducing Negative Automatic Thoughts About Self: Randomized Active Controlled Trial
Beshai, S., Bueno, C., & Salimuddin, S. (2023) · Mindfulness · doi:10.1007/s12671-023-02261-x · Cited by 3
Examining the coffee consumption phenomenon in Vietnam through the structured experience lens
Vu, O. T. K., Alonso, A. D., Tran, T. D., Nguyễn, T. T., & Nicholson, G. J. (2024) · Journal of Foodservice Business Research · doi:10.1080/15378020.2024.2336180 · Cited by 3
Shining a Light on Racial Privilege: An Empirical Examination of the Role of Whiteness in Willingness to Commit Elite White-Collar Crime
Rorie, M. L., Sohoni, T., & Reed, S. M. (2024) · Justice Quarterly · doi:10.1080/07418825.2024.2388205 · Cited by 3

Authors: Chandler, J., & Shapiro, D.
Affiliations: Mathematica Policy Research (US)
University of Michigan (US)
ORCID: Chandler, Jesse
Venue: Annual Review of Clinical Psychology
Publisher: Annual Reviews
Published: 2016-03-28
Volume: 12
Issue: 1
Pages: 53-81
ISSN: 1548-5943 (ISSN-L)
1548-5951
DOI: 10.1146/annurev-clinpsy-021815-093623
OpenAlex: W2126493218
Cited by: 1,072
References: 177
Citations by year: 2026: 27 · 2025: 70 · 2024: 105 · 2023: 103 · 2022: 150 · 2021: 168 · 2020: 122 · 2019: 115 · 2018: 111 · 2017: 77 · 2016: 24
FWCI: 17.58
Type: journal article
Language: en
Open access: closed
Locations: Annual Review of Clinical Psychology (publishedVersion)
PubMed (publishedVersion)
RePEc: Research Papers in Economics (submittedVersion)
Topics: Physical Sciences › Computer Science › Computer Science Applications › Mobile Crowdsensing and Crowdsourcing
Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Ethics in Clinical Research
Keywords: crowdsourcing, Amazon Mechanical Turk, convenience samples, non-probability sampling, sample composition, data quality, participant recruitment, digital labour markets
MeSH terms: Humans, Biomedical Research, Crowdsourcing

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Conducting Clinical Research Using Crowdsourced Convenience Samples

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks